KR20230053275A - Method of designating an object area and detecting an object, and a computer program recorded on a recording medium to execute the same - Google Patents

Abstract

Disclosed is a method of designating an object area and detecting an object, which can easily designate an object included in an image. The method may comprise: a step of setting a partial area in an image, which is a target of annotation work for artificial intelligence (AI) learning, as a bounding box according to a worker's control; a step of identifying an object that is completely included in an inner area of the bounding box; a step of detecting a boundary of the identified object; a step of enlarging or reducing a size of the bounding box to have a size corresponding to the boundary of the identified object; and a step of generating, when a position and size of the enlarged or reduced bounding box is determined by the worker's control, a result of the annotation work including a coordinates is according to the position and size of the bounding box.

Description

Method of designating an object area and detecting an object, and a computer program recorded on a recording medium to execute the same}

The present invention relates to data design for artificial intelligence (AI) learning. More specifically, in annotating artificial intelligence (AI) learning data, it relates to an object area designation and object detection method that can easily designate an object included in an image, and a computer program recorded on a recording medium to execute the method. .

Artificial intelligence (AI) refers to a technology that artificially implements some or all of human learning abilities, reasoning abilities, and perception abilities using computer programs. In relation to artificial intelligence (AI), machine learning refers to learning to optimize parameters with given data using a model composed of multiple parameters. Such machine learning is classified into supervised learning, unsupervised learning, and reinforcement learning according to the form of learning data.

In general, the design of artificial intelligence (AI) learning data proceeds in the steps of data structure design, data collection, data refinement, data processing, data expansion, and data verification.

To describe each step in more detail, data structure design is performed through ontology definition, classification system definition, and the like. Data collection is performed by collecting data through direct filming, web crawling, or associations/professional organizations. Data purification is performed by removing redundant data from collected data and de-identifying personal information. Data processing is performed by inputting meta data and performing annotation. Data extension is performed by performing ontology mapping and supplementing or extending the ontology as needed. In addition, data verification is performed by verifying validity according to the set target quality using various verification tools.

In general, annotation in the data processing step is performed by processing a bounding box for an object included in an image and inputting property information of the object included in the image. Such annotations are also referred to as data labeling. And, a dataset corresponding to the work result of the annotation is calculated in the form of a JSON (Java Script Object Notation) file.

Specifically, in the process of processing the bounding box in the annotation, a worker manually processes the bounding box according to the outline of the object for each object to be recognized in the image.

However, the bounding box processing must follow various rules determined in advance to ensure an artificial intelligence (AI) learning effect (ie, recognition rate of an object in an image). For example, the bounding box processing must follow rules such as the minimum size value of an area in which an object can be recognized and the maximum number of pixels away from the object's outline. Therefore, it is not easy for a worker to perfectly comply with all rules determined in advance in the process of manually processing bounding boxes for a plurality of objects.

In addition, the process of inputting attribute information in annotations is also performed by a worker manually inputting information for each bounding box processed object.

However, the attribute information to be entered by the operator includes various information such as the type of annotation, class name (class), classification item (tags), whether the object is truncated (truncated), large category, small category, and instance upper level. should be included Therefore, it is also not easy for an operator to manually and accurately input all attribute information for a plurality of objects one by one.

Korean Patent Laid-open Publication No. 10-2020-0042629, ‘Method and apparatus for generating touch-based annotations and images in mobile devices for artificial intelligence learning’, (published on April 24, 2020)

One object of the present invention is to provide an object area designation and object detection method that can easily designate an object included in an image in annotating artificial intelligence (AI) learning data, and a computer program recorded on a recording medium to execute the method. is to do

Another object of the present invention is to provide an object region designation and object detection method capable of easily and automatically designating an object included in an image, and a computer program recorded on a recording medium to execute the method.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above technical problem, the present invention proposes an annotation method that can easily designate an object included in an image. The method includes the steps of setting a partial region in an image, which is a target of annotation work for artificial intelligence (AI) learning, as a bounding box under the control of an operator; identifying an object completely included in the inner region of the bounding box; detecting a boundary of the identified object; enlarging or reducing the size of the bounding box to have a size corresponding to the boundary of the detected object; and when the position and size of the expanded or reduced bounding box are determined under the operator's control, generating an annotation work result including coordinates according to the determined position and size of the bounding box. .

In the step of detecting the boundary of the object, RGB values are detected in units of pixels of the image, and an outline with a markedly different RGB boundary is extracted to distinguish the boundary from the background, the background is removed through an algorithm, and then the RGB values are calculated again. By restoring, the boundary of the object may be detected.

In addition, in the step of detecting the boundary of the object, the RGBA channel including transparency is changed, and a transparency value is given to the entire image, but a point with a low transparency threshold is treated as a background and removed, and the remaining object The angular portion may be processed through blur processing to detect the boundary of the object.

In addition, in the non-identification processing step, when the transparency value is given, the boundary of the object may be distinguished based on the threshold of the boundary value between a clear point and a non-clear point.

In order to achieve the technical problem as described above, the present invention proposes a computer program recorded on a recording medium to execute the method as described above. The computer program may include a memory; transceiver; and a processor configured to process instructions residing in the memory, so that the processor performs an annotation task for artificial intelligence (AI) learning under the control of an operator. setting a partial area in the image, which is a target of , as a bounding box; identifying an object completely included in the inner region of the bounding box; detecting a boundary of the identified object; enlarging or reducing the size of the bounding box to have a size corresponding to the boundary of the detected object; and when the position and size of the expanded or reduced bounding box are determined under the operator's control, generating an annotation work result including coordinates according to the determined position and size of the bounding box; , it can be a computer program recorded on a recording medium.

Details of other embodiments are included in the detailed description and drawings.

According to the embodiments of the present invention, in performing annotation on each of a plurality of images for artificial intelligence (AI) learning, the size of the bounding box is automatically enlarged or reduced according to the shape of an object included in the image, so that the annotation A worker who performs can more easily set the bounding box, and the annotation work result can have a more uniform quality.

As a result, according to embodiments of the present invention, it is possible to generate artificial intelligence (AI) learning data capable of constantly guaranteeing a recognition rate of an object included in an image.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. In addition, technical terms used in this specification should be interpreted in terms commonly understood by those of ordinary skill in the art to which the present invention belongs, unless specifically defined otherwise in this specification, and are overly inclusive. It should not be interpreted in a positive sense or in an excessively reduced sense. In addition, when the technical terms used in this specification are incorrect technical terms that do not accurately express the spirit of the present invention, they should be replaced with technical terms that those skilled in the art can correctly understand. In addition, general terms used in the present invention should be interpreted as defined in advance or according to context, and should not be interpreted in an excessively reduced sense.

Also, singular expressions used in this specification include plural expressions unless the context clearly indicates otherwise. In this application, terms such as "consisting of" or "having" should not be construed as necessarily including all of the various components or steps described in the specification, and some of the components or steps are included. It should be construed that it may not be, or may further include additional components or steps.

Also, terms including ordinal numbers such as first and second used in this specification may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but other components may exist in the middle. On the other hand, when a component is referred to as “directly connected” or “directly connected” to another component, it should be understood that no other component exists in the middle.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description will be omitted. In addition, it should be noted that the accompanying drawings are only for easily understanding the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the accompanying drawings. The spirit of the present invention should be construed as extending to all changes, equivalents or substitutes other than the accompanying drawings.

The data collected for artificial intelligence learning consists of very large numbers taken in real time, not just a few. There is a lot of burden for the learning data collection device to transmit a lot of raw data to the data collection server for the collected data. In addition, since the collection device generally collects raw data through real-time shooting, among the collected raw data, similar data with low artificial intelligence learning efficiency may be included, thereby highlighting the efficiency of data management.

An artificial intelligence learning system according to an embodiment of the present invention may include one or more annotation devices and an artificial intelligence learning device.

An artificial intelligence learning system according to another embodiment of the present invention may be configured to additionally include a learning data design device in addition to the annotation device and the artificial intelligence learning device.

In addition, in the artificial intelligence learning system according to another embodiment of the present invention, the annotation device, the learning data design device, and the artificial intelligence learning device may be connected to each other through a public network. In this case, a part of the annotation device may be a device that performs annotation by a clouding service.

Since the components of the artificial intelligence learning system according to various embodiments are merely functionally distinct elements, two or more components are integrated and implemented in the actual physical environment, or one component is implemented in the actual physical environment. may be implemented separately from each other.

Each component is described, and the annotation device is a device that can be used to annotate an image provided from a learning data design device or an artificial intelligence learning device.

In particular, the annotation apparatus according to the present invention has a feature of providing various conveniences to the operator in the process of setting a bounding box and inputting object attribute information in relation to annotation work.

Any device capable of transmitting and receiving data to and from a learning data design device or an artificial intelligence learning device and performing calculations using the transmitted and received data may be used as the annotation device.

For example, the annotation device may be any one of a fixed computing device such as a desktop, workstation, or server, but is not limited thereto, and is not limited to, a smart phone or a laptop. ), tablets, phablets, portable multimedia players (PMPs), personal digital assistants (PDAs) or e-book readers. can be any one of them.

With the following configuration, the learning data design device is a device that can be used to design and generate artificial intelligence (AI) learning data. Such a learning data design device is basically a device that is distinguished from an artificial intelligence learning device, but may be implemented by being integrated with an artificial intelligence learning device in a real physical environment.

Specifically, the learning data design device may receive attributes of a project related to artificial intelligence (AI) learning from the artificial intelligence learning device. The learning data design device can design data structures for artificial intelligence (AI) learning, refine collected data, process data, expand data, and verify data based on the user's control and the attributes of the project. there is.

In particular, the learning data design device may transmit an image to be annotated to the annotation device for data processing for artificial intelligence (AI) learning. The learning data design device may receive an annotation work result from the annotation device. In this case, the annotation work result may have a JSON (Java Script Object Notation) file format. Alternatively, the learning data design device may generate a JSON file based on the received result after receiving a result of annotation work in a format different from JSON. In addition, the learning data design device inspects the received or generated JSON file, packages it, and transmits it to the artificial intelligence learning device.

As such, the learning data design device may be any device capable of transmitting and receiving data to and from the annotation device and the artificial intelligence learning device and performing calculations using the transmitted and received data. For example, the learning data design device may be any one of a desktop, a workstation, or a fixed computing device such as a server, but is not limited thereto.

With the following configuration, the artificial intelligence learning device is a device that can be used to perform machine learning of artificial intelligence (AI) based on data for artificial intelligence (AI) learning.

Specifically, the artificial intelligence learning device may receive a JSON file packaged directly from the annotation device or from the learning data design device. And, the artificial intelligence learning device may perform artificial intelligence (AI) machine learning using the received JSON file.

Such an artificial intelligence learning device may be any device capable of transmitting and receiving data to and from an annotation device or learning data design device and performing calculations using the transmitted and received data. For example, the artificial intelligence learning device may be any one of a desktop, a workstation, or a fixed computing device such as a server, but is not limited thereto.

As described above, one or more annotation devices, learning data design devices, and artificial intelligence learning devices may transmit and receive data using a network in which one or more of a security line, a common wired communication network, or a mobile communication network directly connected between devices is combined. there is.

For example, public wired communication networks may include Ethernet, x Digital Subscriber Line (xDSL), Hybrid Fiber Coax (HFC), and Fiber To The Home (FTTH). It may be, but is not limited thereto. In addition, in the mobile communication network, Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), High Speed Packet Access (HSPA), Long Term Evolution, LTE) and 5th generation mobile telecommunication may be included, but is not limited thereto.

In addition, the annotation device may include a communication unit, an input/output unit, a storage unit, a bounding box setting unit, an object property setting unit, and a result generation unit.

Since the components of the annotation device are merely functionally distinct elements, two or more components may be integrated and implemented in an actual physical environment, or one component may be implemented separately from each other in an actual physical environment. There will be.

The communication unit may transmit/receive data with the learning data design device and the artificial intelligence learning device.

Specifically, the communication unit may receive one or more images from the learning data design device or the artificial intelligence learning device.

Here, the image is an image that is a target of annotation work for artificial intelligence (AI) learning. According to the data processing plan designed by the learning data design device or the artificial intelligence learning device, images to be annotated may be individually received or a plurality of images may be collectively received.

The communication unit may transmit the work result of the annotation to the learning data design device or the artificial intelligence learning device.

Here, the work result may include the coordinates of the bounding box set under the operator's control and property information of the object. In addition, the work result may have a JSON file format, but is not limited thereto.

And, the communication unit may receive project properties, image properties, or worker properties from the learning data design device or the artificial intelligence learning device.

Here, the properties of the project may include the learning purpose, learning period, number of images required for learning, object properties to be identified in images, bounding box setting rules, etc. for a project related to artificial intelligence (AI) learning. It is not limited to this.

Image properties include image file name, image size (width, height), resolution, bit level, compression format, shooting device name, exposure time, ISO speed, focal length, aperture value, and shooting location coordinates (GPS latitude, longitude). , shooting time, etc. may be included, but is not limited thereto.

The worker's attributes may include, but are not limited to, the worker's name, identification number, assigned work amount, cost according to the work, work result evaluation, and the like.

In addition, the bounding box setting rule is a rule that a worker must follow in a process of setting a bounding box for an object in an image according to a project property, an image property, or a worker property. These bounding box setting rules may include, but are not limited to, a minimum size value of an area and a maximum number of pixels separated from an object's outline.

With the following configuration, the input/output unit may input a signal from an operator through a user interface (UI) or output calculated results to the outside.

Here, the worker means a person who performs annotation work. Such an operator may be referred to as a user, performer, labeler, or data labeler, but is not limited thereto.

Specifically, the input/output unit may output an image to be an annotation work. The input/output unit may receive a control signal for setting a bounding box from an operator. In addition, the input/output unit may overlay and output the bounding box on the image.

Here, the bounding box is an area for specifying an object to be learned by artificial intelligence (AI) among objects included in the image. Such a bounding box may have a rectangle or polygon shape, but is not limited thereto.

The input/output unit may output a list of recommendation information that can be used as object attribute information. Also, the input/output unit may receive a control signal for setting object attribute information from an operator.

Here, the proposed information list is a list of information that is composed of information that is likely to be included in object attribute information and can be utilized as object attribute information in the process of performing annotation by a worker.

In addition, object attribute information is information for specifying the attribute of an object that is an artificial intelligence (AI) learning target. Such object attribute information includes information on the type of annotation, class name (class), classification items (tags), truncated state of the object, major classification, subclassification, or upper level (instance upper). may be, but is not limited thereto.

With the following configuration, the storage unit can store data required for annotation work.

Specifically, the storage unit may store an image received through the communication unit. The storage unit may store project properties, image properties, or worker properties received through the communication unit.

The storage unit may temporarily store the location and size of the bounding box set according to the control signal input through the input/output unit. The storage unit may temporarily store attributes of objects input through the input/output unit.

With the following configuration, the bounding box setting unit may set a bounding box for specifying an object to be an artificial intelligence (AI) learning target among objects included in an image. In particular, the bounding box setting unit according to the present invention automatically enlarges or reduces the size of the bounding box according to the shape of the object included in the image so that the bounding box primarily set by the operator has a size corresponding to the outline of the object. It has features that can

First of all, the bounding box setting unit may set a partial region in an image, which is an object of annotation work for learning, as a bounding box according to control of an operator input through an input/output unit.

Specifically, the bounding box setting unit receives two coordinates from the operator through the input and output unit, and sets the bounding box based on a rectangle having the input two coordinates as the coordinates of the upper left vertex and the coordinates of the lower right vertex in the image. can In this case, the two coordinates may be set by the operator inputting one type of input signal twice (eg, mouse click) or by the operator inputting two types of input signals once (eg, mouse drag). It may, but is not limited thereto.

Alternatively, the bounding box setting unit may receive three or more line segments from an operator through an input/output unit, and may set a bounding box based on a polygon composed of the input three or more line segments. In this case, three or more line segments may be input by an operator at one time using a polygon tool or by inputting three or more segments by an operator using a straight line tool, but are not limited thereto.

Next, the bounding box setting unit may identify an object that is completely included in the inner region of the bounding box set by the operator's control or spans one or more line segments among a plurality of line segments constituting the bounding box.

Specifically, the bounding box setting unit may identify all or some of the objects included in the inner region of the bounding box set under the control of the operator. To this end, the bounding box setting unit may perform image processing on an area inside the bounding box.

For example, the bounding box setting unit may divide the inner region of the bounding box into three images according to RGB (Red, Green, Blue). The bounding box setting unit may extract an edge of each image divided into three images. More specifically, the bounding box setting unit may use either a Laplacian of Gaussian (LoG) algorithm or a Difference of Gaussian (DoG) algorithm to extract an edge of each image.

When using the LoG algorithm, the bounding box setting unit may remove noise existing in the image using a Gaussian filter. The bounding box setting unit may apply a Laplacian filter to the noise-removed image. The bounding box setting unit may extract an edge by detecting zerocrossing in the image to which the Laplacian filter is applied.

When using the DoG algorithm, the bounding box setting unit creates two Gaussian masks with different variances from the image. The bounding box setting unit subtracts another mask from one created mask. The bounding box setting unit may extract an edge by applying the subtracted mask to the image.

The bounding box setting unit may identify one or more enclosures by edges extracted from each image. In this case, the bounding box setting unit may first process binarization on each image in order to clarify whether the edge area is closed. And, the bounding box setting unit may identify the identified closed area as an object.

If there are a plurality of objects identified in the inner region of the bounding box, the bounding box setting unit may select only one object from among the identified plurality of objects according to a preset priority. In this case, the priority may be a criterion set according to the size, position or shape of the object occupying the image.

Next, the bounding box setting unit determines whether the identified object spans one or more line segments among a plurality of line segments constituting the bounding box, based on continuity between pixels included in the inner area of the bounding box and pixels in the outer area of the bounding box. can determine whether Here, continuity may be determined based on whether a difference in color, saturation, or brightness between two adjacent pixels is within a preset range.

The bounding box setting unit may determine that the object is completely included in the inner region of the bounding box when the identified object does not cross any line segment among a plurality of line segments constituting the bounding box.

Characteristically, according to an embodiment of the present invention, when it is determined that the object is completely included in the inner region of the bounding box, the bounding box setting unit sets the size of the bounding box so that the object closely fits the inner region of the bounding box. can be scaled down

In one embodiment, the bounding box setting unit sets the bounding box until the distance from the nearest point existing on the outline of the object is within the preset maximum number of pixels for each of the plurality of line segments constituting the bounding box. The size of the bounding box can be reduced by moving the segment toward the center.

Characteristically, according to one embodiment of the present invention, when the bounding box setting unit determines that the object spans one or more line segments among a plurality of line segments constituting the bounding box, the size of the bounding box is such that the object is completely included in the bounding box. can be enlarged. After the size expansion of the bounding box is performed, the bounding box setting unit may reduce the size of the bounding box so that the object completely fits within the inner region of the bounding box.

In one embodiment, when the object spans one or more line segments among a plurality of line segments constituting the bounding box, the bounding box setting unit moves the object towards the outside of the bounding box until the object is completely included in the inner area of the bounding box. The size of the bounding box can be enlarged by moving the spanned line segment.

In another embodiment, the bounding box setting unit, when the object spans one or more line segments among a plurality of line segments constituting the bounding box, sets the center of the bounding box to the line segment the object spans until the object is completely included in the inner region of the bounding box. By moving in the direction of , the size of the bounding box can be enlarged in all directions.

In another embodiment, the bounding box setting unit divides the bounding box into a plurality of detailed areas when the object spans one or more line segments among a plurality of line segments constituting the bounding box, and then the object is completely included in the inner area of the bounding box. , it is possible to enlarge only the size of the detailed region over which the object spans among the divided detailed regions. In this case, the size of the detailed region may be determined in advance according to the resolution of the image or the resolution of the object to be identified. Also, the bounding box enlarged according to the shape of the detailed region may have a polygonal shape.

In another embodiment, the bounding box setting unit, when the object spans one or more line segments among a plurality of line segments constituting the bounding box, until the object is completely included in the inner region of the merged bounding box, outside the line segment the object spans A process of newly setting an additional area in the area and merging the newly set additional area and the bounding box may be repeatedly performed.

With the following configuration, the object property setting unit may set the property of the object specified by the bounding box. In particular, the object property setting unit according to the present invention may preemptively provide a list of recommended information that can be used as object property information before a worker directly inputs property information.

First of all, the object property setting unit may create a list of recommended information that can be used as property information of an object corresponding to the bounding box set by the bounding box setting unit.

In one embodiment, the object property setting unit includes information that is likely to be used as object property information according to the properties of a project related to artificial intelligence (AI) learning, the property of an image, or the property of a worker set in advance to provide recommended information. can make a list of

As another embodiment, the object property setting unit may calculate the average color, saturation, or brightness of pixels included in the bounding box set by setting the bounding box. The object property setting unit may select one recommended object type according to the calculated average color, saturation, or brightness based on a probability distribution of color, saturation, or brightness configured for each type of object. In addition, the object property setting unit may create a list of recommendation information by including information related to the type of the selected recommendation object.

As another embodiment, the object property setting unit may identify the input frequency of the property information previously input by the operator from the operator's log stored in the local storage of the annotation device. And, based on the identified input frequency, the object property setting unit may create a list of recommended information by including property information with a high input frequency.

Next, the object property setting unit may filter a pre-created list of recommended information.

Specifically, the object property setting unit includes information related to the type of object that cannot be set by the work tool used by the operator in the process of setting the bounding box (ie, object irrelevant information) can be removed.

As an embodiment, when a worker uses a tool capable of designating a rectangular area in the process of setting a bounding box, the object property setting unit removes information related to the type of object corresponding to a living organism from a list of previously created recommendation information. can do.

In another embodiment, when a worker uses a tool capable of designating a skeleton structure of an object in the process of setting a bounding box, the object property setting unit is related to the type of object that does not have a skeleton from a list of pre-created recommended information. Information can also be removed.

The object property setting unit may remove information irrelevant to the object from the list of recommendation information and sort remaining information.

As an embodiment, the object property setting unit may arrange the information included in the list of recommended information according to a pre-set sorting criterion based on the position or size occupied by the bounding box in the image.

In another embodiment, the object property setting unit compares the shape of the object included in the bounding box with the basic shape provided in advance for each type of object, and sorts the information included in the list of recommended information based on the similarity of the two shapes. may be

Next, the object property setting unit may output the list of recommended information prepared and filtered to the input/output unit. The object property setting unit may receive a control signal for setting object property information from an operator through an input/output unit.

When one piece of information is selected from the list of recommended information under the control of an operator, the object property setting unit may provide feedback according to the type of object corresponding to the selected information.

In one embodiment, the object property setting unit reflects the color or transparency set differently according to the type of object corresponding to the selected information, and sets a user interface (UI) related to the area inside the bounding box or the area inside the object. can be changed

With the following configuration, the output generation unit may generate annotation work results and transmit them to the learning data design device or the artificial intelligence learning device.

Specifically, the result generation unit determines the location and size of the expanded or reduced bounding box under the control of the operator, and when the property information of the object is determined under the control of the operator based on the list of recommended information, the location of the determined bounding box is determined. And it is possible to create an annotation work result including coordinates according to the size and determined attribute information. Such a work result may have a JSON file format, but is not limited thereto.

In addition, the output generation unit may transmit the generated annotation work result to the learning data design device or the artificial intelligence learning device through the communication unit.

Hereinafter, hardware for implementing the above-described logical components of the annotation device will be described in more detail.

The annotation device may include a processor, a memory, a transceiver, an input/output device, a data bus, and a storage.

The processor may implement operations and functions of the annotation device based on instructions according to software in which the annotation method resides in memory. Software in which the annotation method is implemented may be loaded in the memory. The transceiver may transmit and receive data to and from the learning data design device and the artificial intelligence learning device. The input/output device may receive data necessary for the operation of the annotation device and output a list of images, bounding boxes, and recommendation information. The data bus is connected to the processor, memory, transceiver, input/output device, and storage, and may serve as a movement path for transferring data between each component.

The storage may store an application programming interface (API), a library file, a resource file, and the like required for execution of software in which the annotation method is implemented. The storage may store software 180b in which the annotation method is implemented. Also, the storage may store information necessary for performing the annotation method. In particular, the storage may include a database 185 for storing an image to be annotated, a project attribute, an image attribute, a worker attribute, and a worker's work log.

According to the first embodiment of the present invention, software (180a, 180b) for implementing an annotation method resident in a memory or stored in a storage enables a processor to learn artificial intelligence (AI) under the control of an operator input through an input/output device. Setting a partial region in the image that is a target of annotation work for the bounding box as a bounding box, the processor identifying an object completely included in the inner region of the bounding box or spanning one or more line segments among a plurality of line segments constituting the bounding box. step of expanding or reducing the size of the bounding box so that the processor has a size corresponding to the outline of the identified object, and the position of the bounding box expanded or reduced by the operator's control input through the input/output device by the processor and if the size is confirmed, it may be a computer program recorded on a recording medium to execute a step of generating an annotation work result including coordinates according to the location and size of the determined bounding box.

According to the second embodiment of the present invention, the software (180a, 180b) for implementing the annotation method resident in the memory or stored in the storage enables the processor to learn artificial intelligence (AI) according to the operator's control input through the input/output device. Setting a region in the image that is the target of annotation work for the bounding box, creating a list of recommendation information that can be used as attribute information of an object corresponding to the bounding box set by the processor, and When the property information of an object is determined under the control of an operator based on the list of recommended information, records are recorded to execute the step of generating the work result of the annotation including coordinates according to the location and size of the bounding box and the determined property information. It may be a computer program recorded on a medium.

More specifically, the processor may include an Application-Specific Integrated Circuit (ASIC), other chipsets, logic circuits, and/or data processing devices. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and/or other storage devices. The transceiver may include a baseband circuit for processing wired/wireless signals. Input/output devices include input devices such as keyboards, mice, and/or joysticks, liquid crystal displays (LCDs), organic LEDs (OLEDs), and/or active organic An image output device such as an active matrix OLED (AMOLED) or the like may include a printing device such as a printer or a plotter.

When the embodiments included in this specification are implemented as software, the above-described method may be implemented as a module (process, function, etc.) that performs the above-described functions. A module resides in memory and can be executed by a processor. The memory may be internal or external to the processor and may be coupled with the processor in a variety of well-known means.

Each component may be implemented by various means, eg, hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one embodiment of the present invention includes one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), FPGAs ( Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.

In addition, in the case of implementation by firmware or software, an embodiment of the present invention is implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above, and is stored on a recording medium readable through various computer means. can be recorded. Here, the recording medium may include program commands, data files, data structures, etc. alone or in combination. Program instructions recorded on the recording medium may be those specially designed and configured for the present invention, or those known and usable to those skilled in computer software. For example, recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs (Compact Disk Read Only Memory) and DVDs (Digital Video Disks), floptical It includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, such as a floptical disk, and ROM, RAM, flash memory, and the like. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler. These hardware devices may be configured to operate as one or more pieces of software to perform the operations of the present invention, and vice versa.

An annotation device of an artificial intelligence learning system according to an embodiment of the present invention transmits a control signal for specifying one object included in an image from an operator when an image to be an annotation work for AI learning is loaded. can be input.

The annotation device may set a partial region in the image as a bounding box according to a control signal input from a person in charge.

A rectangular shape having two coordinates input from a vendor as an upper left vertex and a lower right vertex can be set as a bounding box, but the bounding box that can be set according to the present invention may be various, such as polygons and skeletons, in addition to rectangles.

The annotation device may identify an object that is completely included in the inner region of the bounding box or spans one or more line segments among a plurality of line segments constituting the bounding box.

And, when it is determined that the identified object spans one or more line segments among a plurality of line segments constituting the bounding box, the annotation device may enlarge the size of the bounding box so that the object is included in the bounding box.

As an embodiment, the annotation device may enlarge the size of the bounding box by moving a line segment spanning the object toward the outside of the bounding box until the object is completely included in the inner region of the bounding box.

In another embodiment, the annotation device may move the center of the bounding box in the direction of the line segment where the object spans and expand the size of the bounding box in all directions until the object is completely included in the inner region of the bounding box.

As another embodiment, after dividing the bounding box into a plurality of detailed regions, the annotation device may enlarge only the size of the detailed region spanning the object among the divided detailed regions until the object is completely included in the inner region of the bounding box. .

In another embodiment, the annotation device repeatedly sets an additional area outside the line segment where the object spans and merges the newly set additional area with the bounding box until the object is completely included in the inner area of the bounding box. You may.

Referring to FIG. 9 , when the object does not cross any of a plurality of line segments constituting the bounding box, the annotation device determines that the object is completely contained in the inner area of the bounding box, and the object is closely related to the inner area of the bounding box. You can reduce the size of the bounding box so that it fits better.

In one embodiment, the annotator is the center of the bounding box until the distance from the nearest point existing on the outline of the object is within a preset maximum number of pixels for each of a plurality of line segments constituting the bounding box. You can reduce the size of the bounding box by moving the line towards .

According to the embodiment of the present invention as described above, in performing annotation on each of a plurality of images for artificial intelligence (AI) learning, the size of the bounding box is automatically enlarged or enlarged according to the shape of an object included in the image. By being reduced, the operator can more easily set the bounding box, and the work result of the annotation can have a more uniform quality.

An annotation device according to an embodiment of the present invention may set a bounding box based on coordinates input by an operator.

As an embodiment, the annotation device may set a bounding box based on a rectangle having two coordinates input from an operator as coordinates of an upper left vertex and a coordinate of a lower right vertex in an image. As another embodiment, the annotation device may set a bounding box based on a polygon composed of three or more line segments input from an operator.

The annotation device may identify all or part of objects included in the area inside the set bounding box. To this end, the annotation device may perform image processing such as edge extraction, binarization, and closed region identification on the inner region of the bounding box.

The annotation device may determine whether there are a plurality of objects included in the inner region of the bounding box. As a result of the determination, if there are a plurality of objects identified in the inner region of the bounding box, the annotation device may select only one object from among the identified plurality of objects according to a preset priority.

The annotation device may determine whether the identified object is completely included in the inner region of the bounding box. To this end, the annotation device determines whether the identified object spans one or more line segments among a plurality of line segments constituting the bounding box, based on continuity between pixels included in the inner area of the bounding box and pixels in the outer area of the bounding box. can judge

As a result of the determination, when it is determined that the object spans one or more line segments among a plurality of line segments constituting the bounding box, the annotation device may enlarge the size of the bounding box so that the object is completely included in the bounding box.

As an embodiment, the annotation device may enlarge the size of the bounding box by moving a line segment spanning the object toward the outside of the bounding box until the object is completely included in the inner region of the bounding box.

In another embodiment, the annotation device may move the center of the bounding box in the direction of the line segment where the object spans and expand the size of the bounding box in all directions until the object is completely included in the inner region of the bounding box.

As another embodiment, after dividing the bounding box into a plurality of detailed regions, the annotation device may enlarge only the size of the detailed region spanning the object among the divided detailed regions until the object is completely included in the inner region of the bounding box. .

In another embodiment, the annotation device repeatedly sets an additional area outside the line segment where the object spans and merges the newly set additional area with the bounding box until the object is completely included in the inner area of the bounding box. You may.

As a result of the determination, when it is determined that the object is completely included in the inner region of the bounding box, the annotation device may reduce the size of the bounding box so that the object closely fits the inner region of the bounding box.

In one embodiment, the annotator is the center of the bounding box until the distance from the nearest point existing on the outline of the object is within a preset maximum number of pixels for each of a plurality of line segments constituting the bounding box. By moving the line segment toward , the size of the bounding box can be reduced.

Hereinafter, the method for specifying an object area and detecting an object according to various embodiments of the present invention as described above and a computer program recorded on a recording medium for executing the method will be described in detail.

Objects can be detected using RGB and transparency.

Object detection using RGB detects RGB values in pixel units, extracts the outskirts with markedly different RGB boundaries, separates the outskirts from the background (by RGB values), applies an algorithm to remove the background, and redraws the RGB values. Restoration can be done.

Object detection using transparency changes to a channel called RGBA that includes transparency, assigns a transparency value to the entire image, processes the point where the transparency threshold is low as a background and removes it, and blurs the remaining angular parts of the object. ) can be performed by gently processing with the treatment.

In addition, when transparency is gradually given, the outline of the object can be distinguished based on the threshold of the boundary value between a clear point and a non-clear point.

The final result of the bounding box is provided in the form of a box, but in designating the initial box-shaped area, it is possible to designate a box in the form of 2-point coordinates, and it is possible to provide it in the form of a box in the form of 4-point coordinates.

As described above, although preferred embodiments of the present invention have been disclosed in the present specification and drawings, it is in the technical field to which the present invention belongs that other modified examples based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein. It is self-evident to those skilled in the art. In addition, although specific terms have been used in the present specification and drawings, they are only used in a general sense to easily explain the technical content of the present invention and help understanding of the present invention, but are not intended to limit the scope of the present invention. Accordingly, the foregoing detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be selected by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (5)

Setting a partial region in an image, which is a target of annotation for artificial intelligence (AI) learning, as a bounding box under operator control;
identifying an object completely included in the inner region of the bounding box;
detecting a boundary of the identified object;
enlarging or reducing the size of the bounding box to have a size corresponding to the boundary of the detected object; and
When the position and size of the expanded or reduced bounding box are determined under the operator's control, generating an annotation work result including coordinates according to the determined position and size of the bounding box, the object area Designation and object detection methods.
According to claim 1,
Detecting the boundary of the object,
Detecting RGB values in units of pixels of the image, extracting an outline with a markedly different RGB boundary, distinguishing the outline from a background, removing the background through an algorithm, and restoring the RGB values to detect the boundary of the object Characterized in, object area designation and object detection method.
According to claim 1,
Detecting the boundary of the object,
It is changed to an RGBA channel that includes transparency, and a transparency value is given to the entire image, but the point where the transparency threshold is low is processed as a background and removed, and the angled part of the remaining object is processed by blur processing, An object region designation and object detection method, characterized in that the boundary of the object is detected.
According to claim 3,
The de-identification processing step,
Characterized in that, when the transparency value is given, the boundary of the object is divided based on the limit of the boundary value of the clear point and the non-clear point, object area designation and object detection method.
memory;
transceiver; and
In combination with a computing device configured to include a processor for processing instructions resident in the memory,
Setting, by the processor, a partial region in an image, which is a target of an annotation task for artificial intelligence (AI) learning, as a bounding box under control of an operator;
identifying an object completely included in the inner region of the bounding box;
detecting a boundary of the identified object;
enlarging or reducing the size of the bounding box to have a size corresponding to the boundary of the detected object; and
When the position and size of the expanded or reduced bounding box are determined under the operator's control, generating an annotation work result including coordinates according to the determined position and size of the bounding box; To execute, A computer program recorded on a recording medium.