KR20220081520A - Method and apparatus for configuring learning data set in object recognition - Google Patents

Abstract

In a method for constructing a training dataset for object recognition according to an embodiment of the present invention, the method comprising: (a) calling an object image stored in a first dataset; (b) randomly calling an image from the second dataset; (c) automatically arranging the object image of the called first dataset in the random image of the called second dataset in an arbitrary position according to a preset method; and (d) storing an image in which the object image of the first dataset is automatically placed in the random image of the second dataset.

Description

{Method and apparatus for configuring learning data set in object recognition}

The present invention relates to deep learning learning, and more particularly, to a method and apparatus for configuring a learning dataset when the training dataset for object recognition is insufficient.

The content described in this section merely provides background information for the present embodiment and does not constitute the prior art.

Recently, object recognition technology using large-scale deep learning techniques such as deep learning has been greatly developed. Accordingly, big data is required for each class, but the number of open datasets available for learning is still insufficient and the types are not diverse.

Therefore, a user such as a researcher cannot obtain a sufficient data set necessary for a study only by using only a data set published on the Internet, and there is a high possibility of errors occurring when a study is conducted using the data set. Therefore, the reliability of the research results through the above process is also low.

On the other hand, in order to solve this problem, manually constructing a data set or constructing such a data set itself takes a lot of time and effort, and there is a problem in that the initial cost and the labeling cost cannot be ignored.

It is an object of the present invention to define a framework for a method of constructing a training dataset for resolving the problem of imbalance of classes that are markedly insufficient in number in a training dataset for object recognition.

Another task of the present invention is to automate the configuration of the training dataset through the framework defined above.

In a method for constructing a training dataset for object recognition according to an embodiment of the present invention, the method comprising: (a) calling an object image stored in a first dataset; (b) randomly calling an image from the second dataset; (c) automatically arranging the object image of the called first dataset in the random image of the called second dataset in an arbitrary position according to a preset method; and (d) storing an image in which the object image of the first dataset is automatically placed in the random image of the second dataset.

According to one embodiment of the present invention, Steps (a), (c) and (d) may be repeatedly performed for each image of the second dataset called in step (b) until all object images stored in the first dataset are used. can

According to an embodiment of the present invention, calling an image randomly from the first dataset; designating an area for the called image and removing the background; removing the background and extracting an object from the designated area; and storing the extracted object in a specified size.

According to an embodiment of the present invention, the step (c) may include: extracting four major image features for an object image extracted or stored in the first dataset; dividing an image randomly called from the second dataset into a plane; extracting four major image features for each divided plane image; determining whether a reference value is exceeded by comparing the feature amount of the four image features of the extracted or stored object image from the extracted first dataset with the feature amounts of the four image features of each flat image divided from the extracted second dataset; selecting one divided planar image from the second dataset when a reference value is exceeded in all of the planar images divided in the second dataset as a result of the determination; and inserting the object image of the first dataset into the divided planar image of the selected second dataset.

According to an embodiment of the present invention, the inserting includes: performing smoothing on a boundary region of the object image of the first data set to be inserted; and determining a final insertion position in the divided planar image of the selected second dataset after determining the object label-based situation.

An apparatus for configuring a learning dataset for object recognition according to an embodiment of the present invention includes: a memory; and a processor, wherein the processor (a) calls a stored object image from a first dataset, (b) calls an image randomly from a second dataset, and (c) calls The object image of the first dataset called in the random image is automatically placed in an arbitrary position according to a preset method, and (d) the object image of the first dataset is automatically placed in the random image of the second dataset. Save the image.

According to one embodiment of the present invention, the processor (a) until all the object images stored in the first dataset are used for each image of the second dataset called in step (b). (c) and (d) may be repeated.

According to an embodiment of the present invention, the processor randomly calls an image from the first dataset, designates an area for the called image, removes the background, and removes the background and extracts an object from the designated area and the extracted object may be stored in the memory in a specified size.

According to an embodiment of the present invention, the processor, in the process (c), Extracts four image features for an object image extracted or stored in the first dataset, plane-segments a randomly called image from the second dataset, and extracts four image features for each divided plane image and comparing the feature amounts of the four image features of the extracted or stored object image from the extracted first dataset with the four image features for each flat image divided from the extracted second dataset to determine whether the reference value is exceeded, As a result of the determination, if the reference value is exceeded in all the plane images divided in the second dataset, one plane image divided in the second dataset is selected, and the divided plane image of the selected second dataset is added to the An object image of the first dataset may be inserted.

According to an embodiment of the present invention, the processor performs smoothing on the boundary region of the object image of the first dataset to be inserted during the insertion process, and after determining the object label-based situation, the selected second It is possible to determine the final insertion position within the segmented flat image of the dataset.

According to various embodiments of the present invention, the following effects are obtained.

First, it has the effect of constructing a variety of new learning datasets for necessary research by combining publicly available learning datasets.

Second, there is an effect that can reduce the initial cost and labeling cost required when building your own dataset for learning.

1 is a block diagram of a system for configuring a learning dataset for object recognition according to an embodiment of the present invention.
2 is a block diagram of a computing device according to an embodiment of the present invention.
3 is a flowchart illustrating a data set configuration method in a computing device according to an embodiment of the present invention.
4 is a flowchart illustrating an object arrangement automation method in a computing device according to an embodiment of the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term 'and/or' includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. It should be understood that terms such as “comprise” or “have” in the present application do not preclude the possibility of addition or existence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification in advance. .

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In the present specification, one task of the present invention according to the present invention is to define a framework for a method of constructing a training dataset for resolving the problem of the imbalance of classes with a markedly insufficient number in the training dataset for object recognition. and discloses various embodiments of automating the configuration of a training dataset through the framework defined above.

For better understanding of the present invention and for convenience of explanation, two data sets have been described below as an example, but the present invention is not limited thereto, and three or more data sets may be used. Through this, it is possible to construct and secure a sufficient training dataset and use it for necessary research.

In the above-described embodiment according to the present invention, a description of a convolutional neural network (CNN), etc. in relation to deep learning, refers to a known technology, and a detailed description is omitted in this specification.

1 is a block diagram of a learning dataset configuration system 10 for object recognition according to an embodiment of the present invention.

2 is a block diagram of a computing device 120 according to an embodiment of the present invention.

Referring to FIG. 1 , a system 10 for configuring a learning dataset for object recognition according to an embodiment of the present invention may include a source 110 and a computing device 120 .

The source 110 stores an open data set for deep learning learning for object recognition. According to an embodiment, the source 110 may be various, such as a public database, the Internet, and the like.

Although only one source is illustrated in FIG. 1 , the present invention is not limited thereto, and a plurality of sources may be used for object recognition according to the present invention.

The computing device 120 may acquire and store the open data set for deep learning learning for object recognition from the source 110 , and configure a new data set for learning based on the stored data set for learning.

According to an embodiment, the computing device 120 may be a terminal or a component included in the terminal.

According to an embodiment, the computing device 120 may be a server that transmits a training dataset newly constructed based on the published training dataset to the terminal.

The computing device 120 may configure and store a new dataset for deep learning learning for object recognition by combining the dataset for learning that is configured and stored based on the dataset for public learning obtained and stored from the source 110 . According to an embodiment, the computing device 120 may secure a sufficient dataset for research by repeatedly performing the above-described process on a publicly and/or stored learning dataset, and for self-deep learning learning for such object recognition. By building a dataset, not only the initial cost but also the labeling cost can be reduced.

The computing device 120 defines a framework to automatically execute the learning dataset configuration process for object recognition according to the present invention to be described later in FIGS. 3 and 4, and provides various and sufficient object recognition through the defined framework. You can build datasets for deep learning training.

Referring to FIG. 2 , a computing device 120 according to an embodiment of the present invention may include a memory 207 and a processor. According to an embodiment, the processor includes a communication interface 201 , a region designation/background removal module 202 , an object extraction module 203 , an object arrangement module 204 , a learning module 205 , and a control module 206 . ) may be included. However, the present invention is not limited thereto, and at least one component not shown may be further included or vice versa. According to an embodiment, two or more of the above components may be implemented as one component or vice versa.

According to an embodiment, the memory 207 illustrated in FIG. 2 may be an internal memory of the computing device 120 , an external memory, or a virtual memory. According to an embodiment, although one memory is illustrated in FIG. 2 , a plurality of memories may be used in the present invention. In this case, it is not necessary that all the memories are built-in or external. On the other hand, when a plurality of memories are used in the present invention, the type, capacity, etc. of each memory do not have to be the same.

The communication interface 201 may provide a communication environment with the source 110 and support data communication.

The processor according to the present invention solves the problem of imbalance of classes, which are markedly insufficient in number in the dataset for deep learning for object recognition, after removing the background from other images containing the class, and in some of the other training datasets possessed. We want to solve it through a framework defined to be inserted.

In the above, the other training dataset may be, for example, a dataset held after acquiring the corresponding class in the above.

A detailed description of each configuration of FIG. 2 will be described together with the flowcharts of FIGS. 3 and 4 .

3 is a flowchart illustrating a data set configuration method in the computing device 120 according to an embodiment of the present invention.

4 is a flowchart illustrating an object arrangement automation method in the computing device 120 according to an embodiment of the present invention.

A method of configuring a learning dataset for object recognition according to an embodiment of the present invention will be described with reference to FIGS. 2 and 3 .

In this case, the processor may prepare a training dataset folder for recognizing at least two objects in advance. However, the present invention is not limited thereto, and three or more training dataset folders may be used.

First, the following process can be performed with respect to the prepared dataset A folder.

The region designation/background removal module 202 of the processor may select (or call) one image from the dataset A folder, designate a region in the selected image, and remove the background of the designated region (S102). According to an embodiment, the image may be randomly selected from a folder or selected according to a preset criterion. According to an embodiment, the region designation and the background removal of the designated region may be performed based on the graph cut algorithm, but the present invention is not limited thereto.

The object extraction module 203 of the processor may extract an object from a designated area of the selected image from which the background is removed (S104).

The memory 207 may store the extracted object image in a specified size (S106).

The processor may call the object image stored in the specified size through the process S106 with respect to the training dataset A (S108).

The processor may call one image from the training dataset B (S110). According to an embodiment, the call may be made randomly or according to a preset setting.

The object placement module 204 of the processor may arrange the object image of the extracted learning dataset A in the called random image of the called learning dataset B (S112). In the arrangement process of S112, the determination of the arrangement area may be performed automatically, and a detailed description thereof will be described in detail with reference to FIG. 4 , and a detailed description thereof will be omitted.

The control module 206 of the processor may store an image in which the object image of the dataset A is automatically placed in the random image of the dataset B in the memory 207 (S114).

The processor determines whether all of the training dataset A has been used with respect to the image called from the training dataset B (S116), and repeats the above process until all are used.

According to an embodiment, the processor may repeat the above-described process for all images that can be called from the training dataset B in units of the called images.

The learning module 205 of the processor may use the learning dataset newly constructed through the above-described process for deep learning learning.

In FIG. 4 , step S112 of FIG. 3 , that is, a method for automating object placement in the computing device 120 will be described in more detail.

Referring to FIG. 4 , in step S108 of FIG. 3 , the processor may extract an image feature of an object image extracted or stored in the training dataset A ( S202 ). According to an embodiment, the extracted image features may be four major image features for points, lines, planes, and textures.

Meanwhile, the processor may divide the image called from the training dataset B in the process S110 of FIG. 3 ( S204 ). According to an embodiment, the processor may divide the called image into four planes. However, the present invention is not limited thereto.

The processor may extract image features from each of the divided planar images (S206). In this case, as will be described later, four image features of points, lines, planes, and textures may be extracted so that the extracted image features correspond to the aforementioned training dataset A for comparison of image feature amounts.

The processor may determine whether the reference value is exceeded by comparing the feature amount of the image feature extracted from the object image of the training dataset A with the image feature of the divided flat image extracted from the training dataset B (S210) (S212) .

If it is determined in step S212 that the reference value is exceeded in the plane image divided in the training dataset B, the processor may select one plane image divided in the training dataset B. According to an embodiment, the selection may be sequentially performed for all divided planes. The processor may insert the object image of the training dataset A into the divided plane image of the selected training dataset B (S216).

The processor may perform smoothing on the boundary region of the object image of the inserted training dataset A (S218).

The processor may determine the final insertion position of the object image of the training dataset A in the divided plane image of the selected training dataset B after determining the object label-based situation and complete the insertion (S220).

Through the above process, an object arrangement automation process can be made.

According to an embodiment, the subsequent steps of S212 may be performed in units of each divided planar image.

If all the flat images divided in step S212 do not exceed the reference value, the processor may re-segment the image divided in step S204 ( S214 ). If the plane 4 division is performed in step S204, in this case, the plane is divided into n divisions, but the n division may mean more than 4 divisions (eg, 8 divisions, 16 divisions, etc.). According to an embodiment, the processor determines whether the image plane re-segmentation in the process S214 is more than a certain number of repetitions (S208), and if it is greater than or equal to a certain number of repetitions according to the determination result, the image newly randomly called from the training dataset B is described above You can do one process.

Although it is described that each process is sequentially executed in FIGS. 3 to 4 , this is merely illustrative of the technical idea of an embodiment of the present invention. In other words, those of ordinary skill in the art to which an embodiment of the present invention pertain may change the order described in FIGS. Since it will be possible to apply various modifications and variations by executing the process in parallel, FIGS. 3 to 4 are not limited to a time-series order.

Meanwhile, the processes illustrated in FIGS. 3 to 4 can be implemented as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. That is, the computer-readable recording medium includes a magnetic storage medium (eg, a ROM, a floppy disk, a hard disk, etc.), an optically readable medium (eg, a CD-ROM, a DVD, etc.) and a carrier wave (eg, the Internet). storage media such as transmission via In addition, the computer-readable recording medium is distributed in a network-connected computer system so that the computer-readable code can be stored and executed in a distributed manner.

The above description is merely illustrative of the technical idea of this embodiment, and a person skilled in the art to which this embodiment belongs may make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain rather than limit the technical spirit of the present embodiment, and the scope of the technical spirit of the present embodiment is not limited by these embodiments. The protection scope of this embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present embodiment.

10: Learning dataset configuration system for object recognition
110: source
120: computing device
201: communication interface
202: area designation / background removal module
203: object extraction module
204: object placement module
205: learning module
206: control module
207: memory

Claims (10)

In the method of constructing a training dataset for object recognition,
(a) calling the stored object image in the first dataset;
(b) randomly calling an image from the second dataset;
(c) automatically arranging the object image of the called first dataset in the random image of the called second dataset in an arbitrary position according to a preset method; and
(d) storing an image in which the object image of the first dataset is automatically placed in a random image of a second dataset; According to claim 1,
Steps (a), (c) and (d) are,
For each image of the second dataset called in step (b), iteratively performed until all object images stored in the first dataset are used. The method of claim 1,
randomly calling an image from the first dataset;
designating an area for the called image and removing the background;
removing the background and extracting an object from the designated area; and
The method of constructing training data, further comprising the step of storing the extracted object in a specified size. The method of claim 1,
Step (c) is,
extracting four major image features of the extracted or stored object image from the first dataset;
dividing an image randomly called from the second dataset into a plane;
extracting four major image features for each divided plane image;
determining whether or not a reference value is exceeded by comparing the feature amounts of the four image features of the extracted or stored object images from the extracted first dataset with the feature amounts of the four image features of each flat image divided from the extracted second dataset;
selecting one divided planar image from the second dataset when a reference value is exceeded in all of the planar images divided in the second dataset as a result of the determination; and
Inserting the object image of the first dataset into the divided planar image of the selected second dataset; further comprising, learning data configuration method. 5. The method of claim 4,
The inserting step is
performing smoothing on a boundary region of the object image of the inserted first dataset; and
After determining the object label-based situation, determining the final insertion position in the divided planar image of the selected second dataset; further comprising, a method of constructing learning data. In an apparatus for configuring a learning dataset for object recognition,
Memory; and
including a processor;
The processor is
(a) calling the stored object image in the first dataset,
(b) randomly calling an image from the second dataset,
(c) automatically placing the object image of the called first dataset in the random image of the called second dataset at an arbitrary position according to a preset method, and (d) the object image of the first dataset is 2 A training dataset configuration device that stores images automatically placed in random images of the dataset. 7. The method of claim 6,
The processor is
Repeating steps (a), (c) and (d) until all object images stored in the first dataset are used for each image of the second dataset called in step (b), Learning data organization device. 7. The method of claim 6,
The processor is
Calling an image randomly from the first dataset,
Designate an area for the called image and remove the background,
The background is removed and the object is extracted from the specified area,
Storing the extracted object in the memory in a specified size, learning data configuration device. 7. The method of claim 6,
The processor, in the process (c),
Extracting the four major image features of the extracted or stored object image from the first dataset,
Splitting the image randomly called from the second dataset in a plane,
Extracting four image features for each segmented flat image,
Comparing the feature amounts of the four image features of the extracted or stored object image from the extracted first dataset with the feature amounts of the four image features for each flat image divided from the extracted second dataset, it is determined whether the reference value is exceeded, the determination As a result, if the reference value is exceeded in all the flat images divided in the second dataset, one flat image divided in the second dataset is selected,
An apparatus for constructing learning data that inserts the object image of the first dataset into the divided planar image of the selected second dataset. 10. The method of claim 9,
The processor, in the process of inserting,
Smoothing is performed on the boundary area of the object image of the inserted first dataset,
After determining the object label-based situation, the learning data configuration apparatus determines the final insertion position in the divided flat image of the selected second dataset.

Images