KR20220019432A - Method of image data augmentation for learning of objects detecting, computer-readable storage medium and computer program - Google Patents

Abstract

According to an embodiment of the present invention, an image data augmentation method for object recognition learning comprises: a step of selecting an image; a step of selecting an object region in the image; a step of generating augmented data in any area that obscures at least a portion of the object area; and a step of adjusting a bounding box based on probability distribution. The embodiment has an effect of obtaining a dataset capable of learning an object recognizer capable of recognizing a partially occluded object through data augmentation with high accuracy.

Description

Image data augmentation method for object recognition learning, computer readable recording medium and computer program

The embodiment relates to an image data augmentation method for object recognition learning for acquiring data for recognizing an object with high accuracy.

In general, the speed and accuracy of object recognition algorithms using deep convolutional neural networks are increasing.

When an object recognition algorithm using a deep convolutional neural network is actually applied, when data that is not included in the training data is input, the accuracy is inevitably lowered, and accordingly, it is important to secure as much training data as possible.

However, since it is impossible to secure training data indefinitely, a method of increasing the amount of data through data augmentation when there is given training data is common.

In general, various methods have been proposed to apply image classification, but a specific method has not been proposed for a method for augmenting object detection data.

One of the biggest problems in the practical application of object recognition data is that it is difficult to respond to a case where a part of an object is obscured by another object. If most of the datasets are collected in a situation where all the target objects are visible and labeled, the recognition accuracy of objects obscured by other objects is very poor when applied in practice.

In order to solve the above-mentioned problem, the embodiment aims to provide an image data augmentation method for object recognition learning for improving object recognition accuracy of an object recognizer.

An image data augmentation method for object recognition learning according to an embodiment comprises the steps of selecting an image, selecting an object area from the image, and generating augmented data in an arbitrary area covering at least a part of the object area; adjusting the bounding box based on the probability distribution.

In the generating of the augmented data in the arbitrary region, the augmented data may be generated by filling the arbitrary region with an image of a black color.

The generating of the augmented data in the arbitrary region may include generating the augmented data by filling the arbitrary region with an image corresponding to a background region of the image.

The generating of the augmented data in the arbitrary region may include generating the augmented data by filling a random noise value in the arbitrary region.

In the generating of the augmented data in the arbitrary region, the augmented data may be generated by filling the arbitrary region with a noise value in consideration of a color distribution of the image.

In the step of adjusting the bounding box based on the probability distribution, when the left, right, upper, and lower bounding boxes are adjusted, the cumulative probability value of an area included in the bounding box may be greater than or equal to a specific threshold.

A probability distribution for adjusting the bounding box may be performed based on a bivariate normal distribution.

The probability distribution for adjusting the bounding box may be performed based on the object probability distribution of the actual data set.

The embodiment has the effect of obtaining a dataset capable of learning an object recognizer that can be robustly recognized with high accuracy even when recognizing a partially occluded object through data augmentation.

1 is a flowchart illustrating an image data augmentation method for object recognition learning according to an embodiment.
2 to 11 are diagrams for explaining an image data augmentation method for object recognition learning according to an embodiment.

Hereinafter, the embodiment will be described in detail with reference to the drawings.

1 is a flowchart illustrating an image data augmentation method for object recognition learning according to an embodiment, and FIGS. 2 to 11 are diagrams for explaining an image data augmentation method for object recognition learning according to an embodiment.

1 , the image data augmentation method according to the embodiment includes the steps of selecting an image ( S100 ), selecting an object area from the image ( S200 ), and an arbitrary area covering at least a part of the object area. It may include the step of generating the augmented data (S300) and adjusting the bounding box based on the probability distribution (S400).

In the step of selecting an image ( S100 ), first, a plurality of images may be collected. As shown in FIG. 2 , an image 100 to be used for augmentation may be selected from among a plurality of collected images.

When the step of selecting the image ( S100 ) is finished, the step of selecting the object region from the image ( S200 ) may be performed. The image 10 may include one or more object regions. Accordingly, in step S100 , an object area including a target object to be recognized may be selected.

After selecting the object region from the image ( S200 ), generating augmented data in an arbitrary region that covers at least a part of the object region ( S300 ) may be performed.

The object region may include a specific object region to be recognized and other object regions that partially or entirely cover the specific object region. When other object areas cover a specific object area, incorrect data may be included and recognition accuracy may be reduced.

Accordingly, data processing is required for another object area (hereinafter referred to as an 'arbitrary area') that covers a specific object area.

As shown in FIG. 3 , first, an arbitrary region 20 may be selected. Augmented data may be generated in an arbitrary region 20 as shown in FIGS. 4 to 7 .

As shown in FIG. 4 , according to an embodiment, augmented data may be generated by filling an arbitrary region 20 with an image 22 of a black color on the image. Here, the black color may be a value of '0'.

5 , as another embodiment, augmented data may be generated by filling an arbitrary area 20 with an image 24 corresponding to a background area of the image.

As shown in FIG. 6 , in another embodiment, the augmented data may be generated by filling the random region 20 with a noise value 26 . The noise may be a random value.

As shown in FIG. 7 , in another embodiment, the augmented data may be generated by filling the random region 20 with a noise value 28 . The noise may be a value in consideration of color distribution of an image.

After the step of generating the augmented data in an arbitrary area (S300) is finished, the step of adjusting the bounding box based on the probability distribution (S400) may be performed.

As described above, when an arbitrary area is labeled with a real object, there is a possibility that the object recognizer may learn including incorrect data, so it is necessary to adjust the bounding box suitable for the real area. Accordingly, the region corresponding to the bounding box can be expressed as a probability distribution for the probability that an object exists in the corresponding space. If a standard bivariate normal distribution is assumed, it can be expressed as Equation (1).

[Equation 1]

를 같은 크기의 박스에서의 가려짐이 적용된 영역으로 정의하고, 이 영역을 0으로 치환한 새로운

을 수학식 2와 같이 정의할 수 있다.In this case, the region of the image may correspond to a region that fits a specific probability distribution. For example, the probability distribution of a square region can be assumed for the region from -3 to +3 of x and y in the bivariate normal distribution. this time the hidden area

is defined as an area to which occlusion is applied in a box of the same size, and a new

can be defined as in Equation 2.

[Equation 2]

At this time, the variable corresponding to the hidden area is adjusted. Adjust ymin if the occluded area includes the top side of the object, ymax if it includes the bottom side, xmin if it includes the left side, and xmax if it includes the right side. The adjustment method is such that the cumulative probability value on the probability distribution is greater than or equal to a specific threshold for the remaining regions in which each axis is reduced, and the adjustment equation is as shown in Equation (3).

[Equation 3]

If necessary, as shown in FIG. 10 , a probability distribution-based bounding box adjustment may be performed by masking an arbitrary area and then resizing to the original image size.

In other words, as shown in FIG. 8, for example, resizing in a 1:1 ratio, as shown in FIG. 9, adjusting the probability distribution-based bounding box, and as shown in FIG. 10, the adjusted bounding box 11 , resizing may be performed to the original picture size.

Among the distributions, the standard bivariate normal distribution can be replaced with another distribution depending on the distribution of the actual data set. For example, if it is a data set for detecting a pedestrian in a car position, a probability distribution according to the shape of a person's general walking can be used.

New data generated by adjusting the bounding box based on the probability distribution ( S400 ) may be utilized as learning data of the object recognizer.

As described above, by using the augmented data as learning data, there is an effect of improving the accuracy of object recognition.

The image data augmentation method for object recognition learning according to an embodiment may be performed by an image data augmentation apparatus for object recognition learning, and the image data augmentation apparatus for object recognition learning includes an image selection unit, an object region selection unit, an augmented data generation unit, and a bounding box. It may include a coordinator.

Image selector You can select an image. The object region selector may select an object region from the image. The augmented data generating unit may generate augmented data in an arbitrary area that covers at least a part of the object area. Augmented data may be performed in the same manner as in FIGS. 4 to 7 . The bounding box adjuster may adjust the bounding box based on the probability distribution.

Various embodiments of the present document provide software (eg, a machine-readable storage media) (eg, a memory (internal memory or external memory)) including instructions stored in a machine-readable storage medium (eg, a computer). : program) can be implemented. The device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include the electronic device according to the disclosed embodiments. When the command is executed by the control unit, the control unit may perform a function corresponding to the command directly or by using other components under the control of the control unit. Instructions may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, non-transitory means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

According to an embodiment, the method according to various embodiments disclosed in the present document may be included and provided in a computer program product.

According to one embodiment, there is provided a computer-readable recording medium storing a computer program, comprising the steps of: selecting an image; selecting an object area from the image; and instructions for causing the processor to perform a method comprising generating data and performing operations for performing the steps of adjusting a bounding box based on the probability distribution.

According to an embodiment, there is provided a computer program stored in a computer-readable recording medium, comprising the steps of: selecting an image; selecting an object area from the image; and instructions for causing the processor to perform a method comprising generating data and performing operations for performing the steps of adjusting a bounding box based on the probability distribution.

Although the above has been described with reference to the drawings and embodiments, it will be understood by those skilled in the art that various modifications and changes can be made to the embodiments without departing from the spirit of the embodiments described in the claims below. will be able

10: image
20: random area

Claims (10)

selecting an image;
selecting an object region in the image;
generating augmented data in an arbitrary area that obscures at least a portion of the object area; and
adjusting the bounding box based on the probability distribution;
Image data augmentation method for object recognition learning, including. According to claim 1,
The step of generating augmented data in the arbitrary area comprises:
An image data augmentation method for object recognition learning to generate the augmented data by filling the arbitrary region with an image of a black color. According to claim 1,
The step of generating augmented data in the arbitrary area comprises:
An image data augmentation method for object recognition learning to generate the augmented data by filling the arbitrary region with an image corresponding to a background region of the image. According to claim 1,
The step of generating augmented data in the arbitrary area comprises:
An image data augmentation method for object recognition learning to generate the augmented data by filling the random region with a random noise value. According to claim 1,
The step of generating augmented data in the arbitrary area comprises:
An image data augmentation method for object recognition learning to generate the augmented data by filling the arbitrary region with a noise value in consideration of the color distribution of the image. According to claim 1,
In the step of adjusting the bounding box based on the probability distribution,
When adjusting the left, right, upper and lower bounding boxes, the cumulative probability value of the region included in the bounding box is greater than or equal to a threshold value. 7. The method of claim 6,
An image data augmentation method for object recognition learning performed based on a bivariate normal distribution on a probability distribution for adjusting the bounding box. 7. The method of claim 6,
An image data augmentation method for object recognition learning in which the probability distribution for adjusting the bounding box is performed based on the object probability distribution of an actual data set. As a computer-readable recording medium storing a computer program,
The computer program is
selecting an image;
selecting an object region in the image;
generating augmented data in an arbitrary area that obscures at least a portion of the object area; and
adjusting the bounding box based on the probability distribution;
A computer-readable recording medium containing instructions for causing a processor to perform As a computer program stored in a computer-readable recording medium,
The computer program is
selecting an image;
selecting an object region in the image;
generating augmented data in an arbitrary area that obscures at least a portion of the object area; and
adjusting the bounding box based on the probability distribution;
A computer program comprising instructions for causing a processor to perform

Images