KR102615510B1 - Device for detecting the localization of object, method for detecting the localization of object and computer program stored in a recording medium to execute the method - Google Patents

Abstract

An unsupervised learning-based object location detection method according to an embodiment of the disclosed invention includes the steps of: (a) converting a learning image by a conversion image generator to generate a conversion image; (b) extracting a first object area from the learning image and extracting a second object area from the converted image by an object extractor; and (c) learning an artificial intelligence model by comparing the first object area and the second object area by an artificial intelligence learning unit.

Description

A computer program stored in a recording medium to execute an object location detection device, an object location detection method, and an object location detection method THE METHOD}

The present invention relates to an unsupervised learning-based object location detection method and object location detection device that can improve the performance of detecting the location of an object based on additional location-related information of the object in the image.

As a method of machine learning, supervised learning uses a convolution neural network (CNN) to detect object locations in images by using bounding boxes and segmentation through human annotation. It is a method of learning through a network. However, there is a problem that a large amount of human labor is required to obtain accurate annotations.

Unsupervised Learning, another machine learning method, applies self-supervised learning to train an artificial intelligence model using a classification loss function to classify artificial labels. However, in unsupervised learning, since the artificial intelligence model learns information related to the target, the background may be activated, lowering the learning performance.

In addition, existing contrast expression learning encodes images into feature vectors through a neural network, but since feature vectors do not contain spatial information, the loss function does not cover only the object, but focuses on discriminatory parts or excessively covers the background. there is a problem. Therefore, a method that can detect the location of an object more efficiently using a new contrast loss function is needed.

The present invention can cover the entire area of the object based on a converted image that additionally contains information related to the location of the object in the learning image, and does not focus on only the discriminatory part of the learning image or excessively cover the background area. The purpose is to provide an object recognition system, object recognition method, and computer program that can accurately detect the location of an object.

In addition, the present invention provides an unsupervised learning-based object location detection device, object location detection method, and computer program that can more accurately detect the object location on an image than conventional object location detection technology even if the number of learning times for the artificial intelligence model is small. It is intended to provide.

An unsupervised learning-based object location detection method according to one aspect of the disclosed invention includes the steps of: (a) converting a learning image by a conversion image generator to generate a conversion image; (b) extracting a first object area from the learning image and extracting a second object area from the converted image by an object extractor; and (c) learning an artificial intelligence model by comparing the first object area and the second object area by an artificial intelligence learning unit.

In addition, step (c) is: comparing, by the artificial intelligence learning unit, a first background area excluding the first object area in the learning image and a second background area excluding the second object area in the converted image. It may include a step of learning the artificial intelligence model.

In addition, step (c) includes: analyzing the similarity between the first object area and the second object area, analyzing the difference between the first background area and the first object area, and calculating a loss function; and learning the artificial intelligence model to reduce the loss function.

In addition, step (b) is: (b1) extracting the first object area based on the first channel, which is a channel containing relatively more information about the recognized object among the plurality of channels constituting the learning image. ; and (b2) extracting the second object area based on a second channel, which is a channel containing relatively more information about the recognized object among the plurality of channels constituting the converted image.

In addition, step (b1) includes: determining whether a pixel constituting a channel of the learning image is a pixel containing information about the recognized object; determining the number of pixels containing information about the object for a channel of the training image; and determining, as the first channel, a channel containing relatively more information about the recognized object among the channels of the plurality of training images based on the number of pixels. Step (b2) includes: determining whether a pixel constituting a channel of the converted image contains information about the recognized object; determining the number of pixels containing information about the object for a channel of the converted image; and determining, as the second channel, a channel containing relatively more information about the recognized object among channels of the plurality of converted images based on the number of pixels.

Additionally, extracting the first object area from the input image using the artificial intelligence model by an object extractor; and determining an object location in the input image based on the extracted first object area.

In addition, step (a) may include: generating the converted image by rotating the learning image at a predetermined angle, by the converted image generator.

Additionally, step (a) may include: generating the converted image by enlarging or reducing the learning image by the converted image generator.

In addition, step (a) may include: generating the converted image by symmetrically converting the learning image by the converted image generator.

A computer program according to one aspect of the disclosed invention may be stored in a computer-readable recording medium to execute the unsupervised learning-based object location detection method.

An unsupervised learning-based object location detection device according to one aspect of the disclosed invention includes a conversion image generator configured to convert a learning image to generate a conversion image; an object extraction unit configured to extract a first object area from the learning image and a second object area from the converted image; and an artificial intelligence learning unit configured to learn an artificial intelligence model by comparing the first object area and the second object area.

In addition, the artificial intelligence learning unit is configured to learn the artificial intelligence model by comparing a first background area excluding the first object area in the learning image and a second background area excluding the second object area in the converted image. It can be.

In addition, the artificial intelligence learning unit: analyzes the similarity between the first object area and the second object area, analyzes the difference between the first background area and the first object area, and calculates a loss function; And it may be configured to learn the artificial intelligence model so that the loss function decreases.

In addition, the object extractor: extracts the first object area based on a first channel, which is a channel containing relatively more information about the recognized object among the plurality of channels constituting the learning image; Additionally, the second object area may be extracted based on a second channel, which is a channel containing relatively more information about the recognized object among the plurality of channels constituting the converted image.

In addition, the object extractor: determines whether a pixel constituting a channel of the learning image is a pixel containing information about the recognized object; determining the number of pixels containing information about the object for a channel of the training image; determining a channel containing relatively more information about a recognized object among channels of the plurality of training images as the first channel based on the number of pixels; determining whether a pixel constituting a channel of the converted image is a pixel containing information about the recognized object; determine the number of pixels containing information about the object for a channel of the converted image; And, based on the number of pixels, a channel containing relatively more information about the recognized object among channels of the plurality of converted images may be determined as the second channel.

In addition, it further includes a processor, wherein the object extraction unit is configured to extract the first object area from the input image using the artificial intelligence model, and the processor is configured to extract the first object area based on the extracted first object area. It may be configured to determine an object location in an input image.

Additionally, the converted image generator may be configured to generate the converted image by rotating the learning image at a predetermined angle.

Additionally, the converted image generator may be configured to generate the converted image by enlarging or reducing the learning image.

Additionally, the converted image generator may be configured to symmetrically transform the learning image to generate the converted image.

According to one aspect of the disclosed invention, an artificial intelligence model can cover the entire area of the object in the image based on the training image and the converted image, and does not excessively cover the background area while not focusing only on the discriminatory part of the training image. By learning, the location of the object can be accurately detected.

In addition, according to an embodiment of the present invention, even if the number of training sessions for the artificial intelligence model is small, object location detection performance can be improved by detecting the object location on the image more accurately than conventional object location detection technology.

1 is a configuration diagram of an object location detection device according to an embodiment.
Figure 2 is a diagram for explaining an unsupervised learning method according to an embodiment.
Figure 3 is a diagram illustrating a method of calculating a loss function according to one embodiment.
FIG. 4 is a diagram illustrating extracting an object area based on some channels among a plurality of channels according to an embodiment.
FIG. 5 is another diagram illustrating extracting an object area based on some channels among a plurality of channels according to an embodiment.
FIG. 6 is a diagram illustrating a process for determining an object location in an input image according to an embodiment.
Figure 7 is a diagram showing object position detection results according to one embodiment and conventional object position detection results.
Figure 8 is a flowchart of an object location detection method according to an embodiment.
Figure 9 is a table showing the degree of improvement of the object position detection method according to an embodiment compared to the conventional object position detection method.
FIG. 10 is a diagram for explaining criteria for determining some channels among a plurality of channels according to an embodiment.
FIG. 11 is another diagram for explaining criteria for determining some channels among a plurality of channels according to an embodiment.
Figure 12 is a table showing the degree to which the position detection method according to an embodiment is improved compared to the conventional position detection method for various input image transformations.

Like reference numerals refer to like elements throughout the specification. This specification does not describe all elements of the embodiments, and general content or overlapping content between the embodiments in the technical field to which the disclosed invention pertains is omitted. The term '~unit' used in the specification may be implemented as software or hardware, and depending on the embodiments, multiple '~units' may be implemented as one component, or one '~unit' may be implemented as a plurality of components. It is also possible to include elements.

Additionally, when a part "includes" a certain component, this means that it may further include other components rather than excluding other components, unless specifically stated to the contrary.

Terms such as first and second are used to distinguish one component from another component, and the components are not limited by the above-mentioned terms.

Singular expressions include plural expressions unless the context clearly makes an exception.

The identification code for each step is used for convenience of explanation. The identification code does not explain the order of each step, and each step may be performed differently from the specified order unless a specific order is clearly stated in the context. there is.

Hereinafter, the operating principle and embodiments of the disclosed invention will be described with reference to the attached drawings.

FIG. 1 is a configuration diagram of an object location detection device according to an embodiment, and FIG. 2 is a diagram for explaining an unsupervised learning method according to an embodiment.

Referring to FIG. 1, the object detection device 100 according to an embodiment of the present invention includes a conversion image generation unit 110, an object extraction unit 120, an artificial intelligence learning unit 130, a processor 140, and a memory ( 150) may be included.

The object location detection device 100 may recognize an object in the learning image 200 or the input image 300 including the object and determine the location of the recognized object in the image.

The training image 200 may be input to the object detection device 100 in the form of image data. The training image 200 may be an image used for unsupervised learning of an object detection method according to an embodiment.

Unsupervised learning is a type of machine learning and can be a type of learning that finds out how data is structured. Unlike supervised learning or reinforcement learning, unsupervised learning may not be given a target value for the input value.

The converted image generator 110 may be configured to generate the converted image 400 by converting the learning image 200.

At this time, the converted image generator 110 may be configured to generate the converted image 400 by rotating the learning image 200 at a predetermined angle.

For example, the converted image generator 110 rotates the learning image 200 90 degrees clockwise to generate the converted image 400, or rotates the learning image 200 90 degrees counterclockwise to create the converted image 400. 400 can be generated, but the angle at which the learning image 200 is rotated is not limited to 90 degrees.

The converted image generator 110 may be configured to generate the converted image 400 by enlarging or reducing the learning image 200.

For example, the converted image generator 110 generates the converted image 400 by enlarging the size of the learning image 200 by two times, or reduces the size of the learning image 200 by half to create the converted image 400. However, the scale for enlarging or reducing the learning image 200 is not limited to this.

The converted image generator 110 may be configured to symmetrically transform the learning image 200 to generate the converted image 400.

As described above, the conversion image generator 110 may generate the conversion image 400 by converting the learning image 200 by rotating, enlarging, reducing, and symmetrically transforming the learning image 200. The method of generating the converted image 400 is not limited to this, and various conversion methods may be used.

The object extractor 120 may be configured to extract a first object area 201 from the training image 200 and a second object area 401 from the converted image 400.

The object extractor 120 may be an encoder that recognizes an object area from an image based on deep learning-based object detection technology.

Deep learning-based object detection technology learns the features of object areas extracted from images based on data. At this time, a CNN (Convolutional Neural Networks) structure that stacks several stages of convolution layers can be used to learn how to extract features from images, but is not limited to this.

Referring to FIG. 2, the artificial intelligence learning unit 130 may be configured to learn an artificial intelligence model by comparing the first object area 201 and the second object area 401.

That is, the artificial intelligence learning unit 130 can learn an artificial intelligence model by comparing the object area extracted from the training image 200 before conversion and the object area extracted from the converted image 400 after conversion.

The object extractor 120 may individually extract objects from the learning image 200 and the conversion image 400. Accordingly, the first object area 201 and the second object area 401 may not necessarily coincide. However, the second object area 401 may tend to match the first object area 201 to some extent because the learning image 200 is extracted from the converted image 400 generated through a certain conversion process. . Specifically, the area created by inversely transforming the second object area 401 may be an object area that matches the first object area 201 to some extent.

For example, if the converted image 400 is an image created by rotating the learning image 200 90 degrees clockwise, the area created by rotating the second object area 401 90 degrees counterclockwise is the second object area 401. 1 It may be an object area that matches the object area 201 to some extent.

If the performance of the object position detection device 100 is excellent, an almost identical area will be generated whether the object is recognized in the training image 200, which is the image before conversion, or the converted image 400, which is the image after conversion. It will be extracted into the first object area (201) and the second object area (401). Therefore, the artificial intelligence learning unit 130 according to one embodiment can learn the artificial intelligence model so that the first object area 201 and the second object area 401 become similar areas as learning is repeated.

The artificial intelligence learning unit 130 may be configured to learn an artificial intelligence model by comparing the first background area 202 and the second background area 402.

The first background area 202 may be an area in which the first object area 201 is excluded from the learning image 200. Additionally, the second background area 402 may be an area in which the second object area 401 is excluded from the converted image 400.

That is, the artificial intelligence learning unit 130 can learn an artificial intelligence model by comparing the background area extracted from the training image 200 before conversion with the background area extracted from the converted image 400 after conversion.

Since the second background area 402 is extracted from the converted image 400 generated through a certain conversion process of the learning image 200, it may tend to match the first background area 202 to some extent. Specifically, the area created by inversely transforming the second background area 402 may be an object area that matches the first background area 202 to some extent.

For example, if the converted image 400 is an image created by rotating the learning image 200 90 degrees clockwise, the area created by rotating the second background area 402 90 degrees counterclockwise is the second background area 402. 1 It may be an object area that matches the background area 202 to some extent.

If the object location detection apparatus 100 has excellent performance, almost identical areas will be extracted as the first background area 202 and the second background area 402. Therefore, the artificial intelligence learning unit 130 according to one embodiment can learn the artificial intelligence model so that the first background area 202 and the second background area 402 become similar areas as learning is repeated.

The artificial intelligence learning unit 130 may be configured to learn an artificial intelligence model by setting the characteristics of the learning image 200 as input variables and the position of the object in the image as an output variable through learning data.

The characteristics of the learning image 200 may be information representing various characteristics of the corresponding learning image 200. For example, the characteristics of the learning image 200 may be information about color, brightness, etc. in each pixel unit of the learning image 200.

Learning an artificial intelligence model can be done through machine learning. Machine learning can mean using a model composed of multiple parameters and optimizing the parameters with given data. The artificial intelligence learning unit 130 can learn an artificial intelligence model using outputs that are the final results obtained through the artificial intelligence model according to the input. The artificial intelligence model may be stored in memory 150.

The conversion image generator 110, the object extraction unit 120, and the artificial intelligence learning unit 130 may include one processor 140 among the plurality of processors 140 included in the object location detection device 100. You can. Additionally, the object recognition method according to the embodiments of the present invention described so far and the embodiments to be described in the future may be implemented in the form of a program that can be driven by the processor 140.

Here, the program may include program instructions, data files, and data structures, etc., singly or in combination. Programs may be designed and produced using machine code or high-level language code. The program may be specially designed to implement the above-described method for modifying the code, or may be implemented using various functions or definitions known and available to those skilled in the art in the field of computer software. A program for implementing the above-described information display method may be recorded on a recording medium readable by the processor 140. At this time, the recording medium may be the memory 150.

The memory 150 can store programs that perform the operations described above and the operations described later, and the memory 150 can execute the stored programs. When the processor 140 and the memory 150 are plural, they may be integrated into one chip or may be provided in physically separate locations. The memory 150 may include volatile memory such as Static Random Access Memory (S-RAM) or Dynamic Random Access Memory (D-Lab) for temporarily storing data. In addition, the memory 150 includes read only memory (ROM), erasable programmable read only memory (EPROM), and electrically erasable programmable read only memory (EEPROM) for long-term storage of control programs and control data. May include non-volatile memory.

The processor 140 may include various logic circuits and operation circuits, process data according to a program provided from the memory 150, and generate a control signal according to the processing results.

Figure 3 is a diagram illustrating a method of calculating a loss function according to one embodiment.

Referring to FIG. 3, the artificial intelligence learning unit 130 analyzes the similarity between the first object area 201 and the second object area 401, and determines the similarity between the first background area 202 and the first object area 201. The loss function can be calculated by analyzing the difference.

[Equation 1]

Specifically, referring to [Equation 1], Mfg is a factor used to mark the object area to calculate the background area of the image, and A is an attention map, an embedding that indicates the degree to which each pixel in the image is recognized as an object area. It can be a value. Mfg can be set to 1 if the value obtained by subtracting A from 1 for each pixel (1-A) exceeds a predetermined threshold value θbg, and the value obtained by subtracting A from 1 for each pixel is 1-A. If A) is less than or equal to a predetermined threshold value (θbg), it may be set to 0.

[Equation 2]

Referring to [Equation 2], Abg is a background attention map and may be an embedding factor corresponding to an image in which the object area is marked. Abg can be calculated by subtracting A from 1 for each pixel (1-A) and multiplying the Mfg value for each pixel. As a result, Abg may be a map representing the background area by excluding the object area from the image.

[Equation 3]

Referring to [Equation 3], may be a loss function according to one embodiment. Specifically, May be a function representing triplet loss.

Triplet loss can be a loss function for an artificial neural network that compares an anchor with positive and negative inputs. At this time, it may be desirable to minimize the distance between the anchor input and the positive input, and to maximize the distance between the anchor input and the negative input. Triplet loss can be used to learn similarity for the purpose of embedding learning, such as word embeddings, vector learning, and matrix learning. The triplet loss function may take the form of a Euclidean distance function such as [Equation 3].

Aorig, which is an embedding value for the training image 200, and Atf, an embedding value for the converted image 400, may be an embedding value corresponding to a positive input.

Aorig2tf may be an embedding value generated when the same transformation as the transformation applied to the training image 200 is performed on Aorig. Atf2orig may be an embedding value generated when the transformation applied to the training image 200 is inversely transformed to Atf. These Atf2orig and Aorig2tf may be embedding values corresponding to anchor input. Abg may be an embedding value corresponding to negative inpout.

The artificial intelligence learning unit 130 can learn an artificial intelligence model so that the loss function is reduced. That is, the artificial intelligence learning unit 130 uses a triplet loss function ( ) can be performed so that learning is minimized.

That is, the artificial intelligence learning unit 130 can repeatedly perform learning so that the distance between Atf2orig and Aorig is minimized and the distance between Atf2orig and Abg is maximized. Additionally, the artificial intelligence learning unit 130 may repeatedly perform learning so that the distance between Aorig2tf and Atf is minimized and the distance between Aorig2tf and Abg is maximized.

As a result, the artificial intelligence learning unit 130 minimizes the distance between the anchor input and the positive input and maximizes the distance between the anchor input and the negative input to create a loss function. You can learn an artificial intelligence model to reduce .

FIG. 4 is a diagram illustrating extracting an object area based on some of a plurality of channels according to an embodiment, and FIG. 5 is a diagram illustrating extracting an object area based on some of a plurality of channels according to an embodiment. This is another drawing to explain what to do.

Referring to FIG. 4, the existing object location detection method extracts the object area by performing pooling on all of the plurality of channels constituting the training image 200. That is, the conventional method ultimately extracted the object area by simply averaging the object extraction results for each channel, regardless of the object information included in each channel.

This existing method extracts the object area by giving the same weight to other channels even for channels that contain slightly less object information than other channels, so there is a problem that it may perform incorrect recognition of the surrounding background of the object.

Referring to FIGS. 4 and 5, the object extractor 120 according to one embodiment extracts the first object area 201 based on the first channel, which is a partial channel among the plurality of channels constituting the learning image 200. can be extracted. The first channel may be a channel that contains relatively more information about the recognized object among the plurality of channels.

Additionally, the object extractor 120 may extract the second object area 401 based on the second channel, which is a partial channel among the plurality of channels constituting the converted image 400. The second channel may be a channel that contains relatively more information about the recognized object among the plurality of channels.

That is, the object extractor 120 assigns a relatively large weight to a channel containing relatively much information about the recognized object among the plurality of channels, and to a channel containing relatively little information about the recognized object. The area of an object can be extracted by assigning a relatively small weight and averaging.

As a result, the object location detection method according to one embodiment can extract the object area by placing a higher weight on the channel containing relatively more information about the object, resulting in an error in recognizing the surrounding background as an object compared to the conventional technology. This has the effect of reducing the occurrence of .

The learning image 200 and the converted image 400 may be composed of a plurality of pixels. At this time, some pixels may be pixels containing information about the object included in the image, and some pixels may be pixels for the surrounding background that do not contain any information about the object.

The object extractor 120 determines whether the pixel constituting the channel of the training image 200 is a pixel containing information on the recognized object, and determines whether the pixel containing information on the object for the channel of the training image 200 is selected. The number can be determined.

The object extractor 120 may determine a channel containing relatively more information about the recognized object among the channels of the plurality of training images 200 as the first channel based on the number of pixels.

At this time, a method of determining the first channel may be a method of determining the top few percent of channels with a large number of pixels containing object information among a plurality of channels as the first channel.

For example, if the number of channels is 4 and the preset ratio value is 50%, the object extractor 120 selects the channel with the largest number of pixels containing object information among the four channels and the channel with the second largest number, that is, Two channels can be determined as first channels.

Meanwhile, the above-described method can be used not only in the process of extracting an object from the learning image 200 but also in the process of extracting an object from the conversion image 400.

The object extractor 120 determines whether the pixel constituting the channel of the converted image 400 is a pixel containing information about the recognized object, and calculates the number of pixels containing information about the object for the channel of the converted image 400. can be decided.

Additionally, the object extractor 120 may determine a channel containing relatively more information about the recognized object among the channels of the plurality of converted images 400 as the second channel based on the number of pixels.

Meanwhile, the method of determining the first channel and the second channel is limited to determining the top few percent of channels with the largest number of pixels containing object information among a plurality of channels as the first channel and the second channel. It does not matter which method is used as long as the area of the object can be extracted by assigning a relatively large weight to the channel containing relatively more information about the recognized object among the plurality of channels and averaging it. .

FIG. 6 is a diagram illustrating a process for determining an object location in an input image according to an embodiment.

Referring to FIG. 6, the object extraction unit 120 may be configured to extract the first object area 201 from the input image 300 using an artificial intelligence model. That is, the object extractor 120 can extract the object area from the input image 300, where the user wants to know the location of the object, through an artificial intelligence model learned based on the training image 200.

The processor 140 may determine the object location in the input image 300 based on the extracted first object area 201. At this time, the processor 140 creates a bounding box on the input image 300 based on the first object area 201 and determines the object location in the input image 300 based on the coordinates of the bounding box. can be decided.

Figure 7 is a diagram showing object position detection results according to one embodiment and conventional object position detection results.

Referring to Figure 7, object location detection for each state in which the artificial intelligence model was not trained (0 epoch), the artificial intelligence model was trained 10 times (10 epochs), and the artificial intelligence model was trained 40 times (40 epochs). When performing, you can check the object position detection results according to the conventional object position detection method (Baseline) and the object position detection method of the present invention (Ours).

A red box may be a bounding box indicating the ground truth of a predetermined object location.

The green box may be a bounding box indicating the location of the object detected according to the conventional object location detection method and the object location detection method of the present invention.

Referring to Figure 7, it can be seen that the object location detection method (Ours) of the present invention detects an object location that is closer to the correct answer than the conventional method (Baseline) even without learning an artificial intelligence model. In other words, it can be confirmed that the object location detection method (Ours) of the present invention causes fewer errors in misrecognizing the surrounding background of an object as an object.

In the case of the object location detection method of the present invention, unlike the conventional method, the final object area is not extracted by simply averaging the object extraction results for each channel, but rather a higher weight is assigned to the channel containing relatively more object information. This may be the result because two object areas were extracted.

It can be seen that the object location detection method (Ours) of the present invention more accurately detected the object location close to the correct answer in the state where the artificial intelligence model was learned 10 times and the artificial intelligence model was learned 40 times.

At least one component may be added or deleted in response to the performance of the components described above. Additionally, it will be easily understood by those skilled in the art that the mutual positions of the components may be changed in response to the performance or structure of the system.

Figure 8 is a flowchart of an object location detection method according to an embodiment. This is only a preferred embodiment for achieving the purpose of the present invention, and of course, some components may be added or deleted as needed.

Referring to FIG. 8, the converted image generator 110 may generate a converted image 400 by converting the learning image 200 (1001). At this time, the converted image generator 110 may generate the converted image 400 by rotating the learning image 200 at a predetermined angle, enlarging it, reducing it, or symmetrically converting it.

The object extractor 120 may extract the first object area 201 based on the first channel, which is a channel containing relatively more information about the recognized object among the plurality of channels constituting the learning image 200. (1002).

At this time, the object extractor 120 determines whether the pixel constituting the channel of the learning image 200 is a pixel containing information about the recognized object, and the pixel containing information about the object for the channel of the learning image 200 The number of may be determined, and based on the number of pixels, a channel containing relatively more information about the recognized object among the channels of the plurality of training images 200 may be determined as the first channel.

The object extractor 120 may extract the second object area 401 based on the second channel, which is a channel containing relatively more information about the recognized object among the plurality of channels constituting the converted image 400. (1003).

At this time, the object extractor 120 determines whether the pixel constituting the channel of the converted image 400 is a pixel containing information about the recognized object, and determines whether the pixel containing information about the object for the channel of the converted image 400 The number of may be determined, and based on the number of pixels, a channel containing relatively more information about the recognized object among the channels of the plurality of converted images 400 may be determined as the second channel.

The artificial intelligence learning unit 130 may analyze the similarity between the first object area 201 and the second object area 401 (1004). Additionally, the artificial intelligence learning unit 130 may analyze the difference between the first background area 202 and the first object area 201 (1005).

The artificial intelligence learning unit 130 may calculate a loss function based on the above-described analysis results (1006). At this time, the loss function calculated by the artificial function learning unit may be a triplet loss function.

The artificial intelligence learning unit 130 may learn the artificial intelligence model so that the loss function is reduced (1007). At this time, the artificial intelligence learning unit 130 may repeatedly learn the artificial intelligence model so that the triplet loss function decreases.

In order to verify the performance of the unsupervised learning-based object location detection method according to an embodiment of the present invention, an object location detection experiment was conducted.

Figure 9 is a table showing the degree of improvement of the object position detection method according to an embodiment compared to the conventional object position detection method.

Referring to FIG. 9, when the position of the object is detected by performing rotation transformation on the input image 300, the loss function ( Rather than calculating only ) and reducing the loss function (Baseline), the triplet loss function according to the present invention ( ) and a method of reducing the triplet loss function and a method of assigning different weights to each channel of the training image 200 ( It can be seen that the method that uses all (Ours(full)) detects the location of the object better.

Specifically, when the overlap ratio between the bounding box representing the ground truth of the object location and the bounding box representing the location of the detected object is more than 30%, the conventional method (Baseline) was 96.75% of the total number of experiments, but this In the case of invention, it can be confirmed that it is 97.30% of the total number of experiments.

In addition, when the overlap ratio between the bounding box representing the ground truth of the object location and the bounding box representing the location of the detected object is more than 70%, the conventional method (Baseline) only accounts for 28.66% of the total number of experiments. In the case of the present invention, it can be confirmed that it is 38.64% of the total number of experiments.

Referring to Figure 9, simply the triplet loss function according to the present invention ( The loss function ( ) and a method of assigning different weights to each channel of the learning image 200 ( ) It can be seen that the performance of the object location detection method that uses both overlapping methods is excellent.

FIG. 10 is a diagram for explaining criteria for determining some channels among a plurality of channels according to an embodiment, and FIG. 11 is another diagram for explaining criteria for determining some channels among a plurality of channels according to an embodiment. am.

Referring to FIGS. 10 and 11 , among a plurality of channels, it is possible to set in advance what percentage of channels with the highest number of pixels containing object information will be determined as the first channel or the second channel.

At this time, according to the experimental results, it can be confirmed that setting the top 70% of channels with the largest number of pixels containing object information to be determined as the first channel and the second channel most effectively detects the location of the object.

Figure 12 is a table showing the degree to which the position detection method according to an embodiment is improved compared to the conventional position detection method for various input image transformations.

Referring to FIG. 12, regardless of the conversion method used when generating the converted image 400, the object location detection method (Ours) according to the present invention has better performance in detecting the location of the object than the conventional method (Base). You can see that it is excellent.

Specifically, use any of the following transformation methods: rotation, enlargement or reduction (Scale), pixel-wise movement (Translation), horizontal symmetry transformation (Hflip), or vertical symmetry transformation (Vflip). Even when the converted image 400 is generated, it can be confirmed that the object position detection method (Ours) according to the present invention has better performance in detecting the position of the object than the conventional method (Base).

As described above, the disclosed embodiments have been described with reference to the attached drawings. A person skilled in the art to which the present invention pertains will understand that the present invention can be practiced in forms different from the disclosed embodiments without changing the technical idea or essential features of the present invention. The disclosed embodiments are illustrative and should not be construed as limiting.

100: Object location detection device
110: Conversion image creation unit
120: Object extraction unit
130: Artificial Intelligence Learning Department
140: processor
150: memory
200: Image for training
201: first object area
202: first background area
300: input image
400: converted image
401: Second object area
402: Second background area

Claims (20)

(a) converting the learning image to generate a converted image by a converted image generator;
(b) extracting a first object area from the learning image and extracting a second object area from the converted image by an object extractor; and
(c) learning an artificial intelligence model by comparing the first object area and the second object area by an artificial intelligence learning unit,
Step (c) above is:
Learning the artificial intelligence model by comparing, by the artificial intelligence learning unit, a first background area excluding the first object area in the learning image and a second background area excluding the second object area in the converted image. ;
calculating a loss function by analyzing similarity between the first object area and the second object area and analyzing a difference between the first background area and the first object area; and
Including; learning the artificial intelligence model to reduce the loss function,
The steps for learning the artificial intelligence model are:
The loss function is reduced by ensuring that the distance between the embedding value generated when the same transformation as the transformation applied to the training image is performed on the embedding value for the training image and the embedding value corresponding to the image of the first background area is maximized. learning the artificial intelligence model to do so; and
The loss function is reduced by maximizing the distance between the embedding value generated when the transformation applied to the learning image is inversely transformed to the embedding value for the transformed image and the embedding value corresponding to the image of the second background area. An unsupervised learning-based object location detection method, including the step of learning the artificial intelligence model. delete delete According to paragraph 1,
Step (b) above is:
(b1) extracting the first object area based on a first channel, which is a channel containing relatively more information about the recognized object among the plurality of channels constituting the learning image; and
(b2) extracting the second object area based on a second channel, which is a channel containing relatively more information about the recognized object among the plurality of channels constituting the converted image; Object location detection method. According to paragraph 4,
Step (b1) above is:
determining whether a pixel constituting a channel of the learning image is a pixel containing information about the recognized object;
determining the number of pixels containing information about the object for a channel of the training image; and
and determining, as the first channel, a channel containing relatively more information about the recognized object among the channels of the plurality of training images based on the number of pixels,
Step (b2) above is:
determining whether a pixel constituting a channel of the converted image is a pixel containing information about the recognized object;
determining the number of pixels containing information about the object for a channel of the converted image; and
Based on the number of pixels, determining a channel containing relatively more information about the recognized object among the channels of the plurality of converted images as the second channel. According to paragraph 1,
Extracting, by an object extraction unit, the first object area from the input image using the artificial intelligence model; and
An unsupervised learning-based object location detection method further comprising: determining an object location in the input image based on the extracted first object area. According to paragraph 1,
Step (a) above is:
An unsupervised learning-based object location detection method comprising: generating the transformed image by rotating the learning image at a predetermined angle, by the transformed image generator. According to paragraph 1,
Step (a) above is:
An unsupervised learning-based object location detection method comprising: generating the converted image by enlarging or reducing the learning image by the converted image generator. According to paragraph 1,
Step (a) above is:
An unsupervised learning-based object location detection method comprising: generating the transformed image by symmetrically transforming the learning image by the transformed image generator. delete A computer program stored in a computer-readable recording medium to execute the unsupervised learning-based object location detection method of any one of claims 1, 4 to 9. a conversion image generator configured to generate a conversion image by converting the learning image;
an object extraction unit configured to extract a first object area from the learning image and a second object area from the converted image; and
It includes an artificial intelligence learning unit configured to learn an artificial intelligence model by comparing the first object area and the second object area,
The artificial intelligence learning department:
learning the artificial intelligence model by comparing a first background area excluding the first object area in the training image with a second background area excluding the second object area in the converted image;
Analyzing the similarity between the first object area and the second object area, analyzing the difference between the first background area and the first object area, and calculating a loss function;
configured to learn the artificial intelligence model so that the loss function decreases;
The loss function is reduced by ensuring that the distance between the embedding value generated when the same transformation as the transformation applied to the training image is performed on the embedding value for the training image and the embedding value corresponding to the image of the first background area is maximized. learning the artificial intelligence model to do so; and
The loss function is reduced by maximizing the distance between the embedding value generated when the transformation applied to the training image is inversely transformed to the embedding value for the transformed image and the embedding value corresponding to the image of the second background area. An unsupervised learning-based object location detection device that learns the artificial intelligence model. delete delete According to clause 12,
The object extraction unit:
extracting the first object area based on a first channel, which is a channel containing relatively more information about the recognized object among the plurality of channels constituting the learning image; and
An unsupervised learning-based object location detection device configured to extract the second object area based on a second channel, which is a channel containing relatively more information about the recognized object among the plurality of channels constituting the converted image. According to clause 15,
The object extraction unit:
Determine whether a pixel constituting a channel of the learning image is a pixel containing information about the recognized object;
determining the number of pixels containing information about the object for a channel of the training image;
determining a channel containing relatively more information about a recognized object among channels of the plurality of training images as the first channel based on the number of pixels;
determining whether a pixel constituting a channel of the converted image is a pixel containing information about the recognized object;
determine the number of pixels containing information about the object for a channel of the converted image; and
An unsupervised learning-based object location detection device configured to determine a channel containing relatively more information about a recognized object among channels of the plurality of converted images as the second channel based on the number of pixels. According to clause 12,
It further includes a processor;
The object extraction unit,
Configured to extract the first object area from the input image using the artificial intelligence model,
The processor,
An unsupervised learning-based object location detection device configured to determine an object location in the input image based on the extracted first object area. According to clause 12,
The converted image generator,
An unsupervised learning-based object location detection device configured to generate the transformed image by rotating the learning image at a predetermined angle. According to clause 12,
The converted image generator,
An unsupervised learning-based object location detection device configured to generate the converted image by enlarging or reducing the learning image. According to clause 12,
The converted image generator,
An unsupervised learning-based object location detection device configured to generate the transformed image by symmetrically transforming the learning image.

Images