KR20210051722A - Deep learning based object detection model training method and an object detection apparatus to execute the object detection model - Google Patents

Abstract

Disclosed are a method for training an objection detection model and a device for detecting an object, to execute the objection detection model. The method for training the object detection model includes: dividing a target image to be trained into a plurality of cells having a specific size; generating encoding feature maps having mutually different resolutions corresponding to the plurality of convolution layers, from the target image through a plurality of convolution layers included in the object detection module; generating a decoding feature map by fusing an encoding feature map having a low resolution, which generated by a first convolution layer, which is positioned at the last end, of the plurality of convolution layers through a single deconvolution layer included in the object detection module and an encoding feature map having a higher resolution, which is generated by a second convolution layer positioned at a previous stage of the first convolution; and detecting object information predicted in each of the plurality of cells by using the generated decoding feature map through the object detection layer included in the object detection module. The single deconvolution layer may additionally include a convolution layer for increasing the receptive field.

Description

Object detection model learning method and object detection device in which the object detection model is executed {DEEP LEARNING BASED OBJECT DETECTION MODEL TRAINING METHOD AND AN OBJECT DETECTION APPARATUS TO EXECUTE THE OBJECT DETECTION MODEL}

The present invention relates to a learning method of an object detection model and an object detection apparatus in which the object detection model is executed, and more specifically, to an efficient learning method of an object detection model through simplified feature fusion based on deconvolution. .

The conventional object detection method provides an anchor-based object detection method, a cell-based object detection method, and an anchorless-based object detection method. First, in the anchor-based object detection method, anchors are placed in each cell constituting the feature map of the object detection network, and objectness, class score, and object location are based on the placed anchors. ) And shape can be learned. This anchor-based object detection method requires object classification and region estimation in proportion to the number of anchors, resulting in a large amount of computation, and in particular, a problem in that the difference in performance increases depending on how anchors are set.

The cell-based object detection method may divide an image into a plurality of cells, and predict the existence of an object and a class probability for each cell through predetermined bounding boxes. In such a cell-based object detection method, the shape of the bounding boxes is dependent on learning, so in the case of a new type of bounding box due to occlusion, size change, view change, etc., there is a problem in that the accuracy of object detection is poor.

Finally, the anchorless-based object detection method converts the ground truth (GT) region based on the input image into a region based on the feature map size for object detection, and then within a certain standard from the center of the converted GT region. You can learn the domain. In such an anchorless-based object detection method, since the area in which the learning loss is calculated varies according to the size of the object, there is a problem that a learning deviation occurs according to the size of the object.

Accordingly, there is a need for a new type of object detection method capable of overcoming the problems occurring in each of these object detection methods.

The present invention provides a method and apparatus for calculating a learning loss (Loss) of an object detection model that is efficient for changing the size or shape of an object in a deep learning feature map and occlusion by providing a new type of distance-based GT-cell encoding method. .

In addition, in the present invention, a plurality of deconvolution layers designed according to characteristics of different objects are arranged in parallel at the rear ends of a plurality of convolution layers constituting an object detection model, A method and apparatus for securing the effectiveness of annotation and refining cost of training data by sequentially iterative learning of layers and deconvolutional layers are provided.

A method of learning an object detection model according to an embodiment of the present invention includes the steps of dividing a target image to be trained into a plurality of cells having a predetermined size; Generating encoding feature maps having different resolutions corresponding to each of the plurality of convolution layers from the target image through a plurality of convolution layers included in the object detection module; A low-resolution encoding feature map and the first convolution generated by a first convolution layer located at the last end of the plurality of convolution layers through a single deconvolution layer included in the object detection module. Generating a decoding feature map by fusing the high-resolution encoding feature maps generated by the second convolution layer located at the previous end of the solution; And detecting object information predicted in each of the plurality of cells by using the generated decoding feature map through an object detection layer included in the object detection module, wherein the single deconvolution layer is an accommodation region A convolution layer may be added to increase the (Receptive field).

The object information predicted in each of the plurality of cells corresponds to objectness information, class information of the predicted object, and a region in which the predicted object is expected to exist in each of the plurality of cells. It may include at least one of bounding box information.

The method may further include determining a final detection area of an object by removing an area of the overlapping bounding box from among the object information predicted in each of the plurality of cells.

The method may further include calculating a learning loss (Loss) of the object detection model using distance information between each of the plurality of cells and a preset ground truth (GT) box.

The calculating of the learning loss of the object detection model may include performing matching between each of the plurality of cells and the GT box based on a distance between each of the plurality of cells and the GT box; And calculating a loss function using object information predicted from a cell in which matching with the GT box is performed and object information corresponding to the GT box, thereby calculating a learning loss of the object detection model.

The step of performing the matching between each of the plurality of cells and the GT box includes the cell size of each of the plurality of cells and the relative distance reference ratio to the size of the GT box, the minimum distance reference ratio, and the maximum distance reference ratio. Determining a distance between a center point of each of the plurality of cells and a center point of the GT box; And matching cells whose determined distance is less than or equal to a preset reference to the GT box.

The method may further include adjusting a parameter value of the object detection model to minimize the learning loss of the calculated object detection model.

An object detection apparatus for executing an object detection model according to an embodiment of the present invention includes a processor, and the object detection model is an encoding feature map having different resolutions for a target image divided into a plurality of cells having a predetermined size. A plurality of convolution layers for generating a; A low-resolution encoding feature map generated by a first convolutional layer positioned at the last of the plurality of convolutional layers and a high-resolution encoding feature generated by a second convolutional layer positioned before the first convolutional layer A single deconvolution layer for generating a decoding feature map by fusing the maps; And an object detection layer for detecting object information predicted in each of the plurality of cells using the generated decoding feature map, wherein the single deconvolution layer is a convolution for increasing a receptive field. An object detection device that can be added to a layer.

The object information predicted in each of the plurality of cells corresponds to objectness information, class information of the predicted object, and a region in which the predicted object is expected to exist in each of the plurality of cells. It may include at least one of bounding box information.

The processor may determine a final object detection area by removing an area of an overlapping bounding box among object information of each of the plurality of cells predicted through the object detection layer.

The processor may calculate a learning loss (Loss) of the object detection model by using distance information between each of the plurality of cells and a preset ground truth (GT) box.

The processor performs matching between each of the plurality of cells and the GT box based on a distance between each of the plurality of cells and the GT box, and predicts object information and the GT box in a cell in which the matching with the GT box is performed. The learning loss of the object detection model can be calculated by calculating a loss function using object information corresponding to the box.

The processor uses a cell size of each of the plurality of cells and a relative distance reference ratio to the size of the GT box, a minimum distance reference ratio, and a maximum distance reference ratio between a center point of each of the plurality of cells and a center point of the GT box. A distance may be determined, and cells having the determined distance equal to or less than a preset reference may be matched to the GT box.

The processor may adjust a parameter value of the object detection model to minimize the learning loss of the calculated object detection model.

In the object detection model, a plurality of deconvolution layers designed according to characteristics of different objects may be arranged in parallel at the rear ends of the plurality of convolution layers.

The processor may sequentially and repeatedly learn the plurality of convolutional layers and the plurality of deconvolutional layers using training data corresponding to each of the plurality of deconvolutional layers arranged in parallel.

The present invention provides a new type of distance-based GT-cell encoding method, so that it is possible to calculate the learning loss of an object detection model that is effective for changing the size or shape of an object and occlusion in a deep learning feature map.

In addition, the present invention arranges a plurality of deconvolution layers designed according to characteristics of different objects in parallel at the rear end of a plurality of convolution layers constituting an object detection model, so that respective convolution layers and deconvolution layers are arranged. By sequentially iterative learning, it is possible to secure the effectiveness of the cost of annotation and refining of the learning data.

In addition, the present invention can improve the performance of an object detection model by providing a structure of a modified deconvolution layer capable of increasing a receptive field and capacity in an object detection model.

1 is a diagram showing an object detection system according to an embodiment of the present invention.
2 is a diagram illustrating a structure diagram of an object detection model based on deconvolution feature fusion according to an embodiment of the present invention.
3 is a diagram illustrating a multi-task based object detection apparatus according to an embodiment of the present invention.
4 is a diagram illustrating a method of calculating a learning loss (Loss) of an object detection model according to an embodiment of the present invention.
5 is a flowchart illustrating an object detection method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing an object detection system according to an embodiment of the present invention.

When the target image 120 is received, the object detection device 110 constituting the object detection system 100 of the present invention uses the deep learning-based object detection model 111 to exist in the target image 120. Various objects can be detected. In addition, the object detection device 110 identifies the location information and class information of the detected objects, and then provides a result image 130 displaying the location information and class information of the identified objects through a separate display (not shown) to the user. Can provide.

For example, the object detection model 111 of the present invention may be configured based on a Fully Convolutional Network (FCN). The FCN-based object detection model uses upsampling and a method of combining convolutional features of several latter layers for dense prediction on a pixel-by-pixel basis. This FCN-based object detection model reduces the resolution through several stages of convolution and pooling, and uses a method of restoring the reduced resolution through upsampling, so the details of the target image 120 Object detection may have inaccurate results due to information disappearing or excessive smoothing effect.

In order to solve this problem, the present invention adds a symmetrical deconvolution network to a convolution network constituting the FCN-based object detection model 111 so that the upsampled target image 120 ) Resolution problem can be solved. Hereinafter, the object detection model 111 of the present invention is defined as an object detection model based on deconvolution feature fusion.

A more detailed operation of the object detection model 111 will be described in detail with reference to the following drawings.

2 is a diagram illustrating a structure diagram of an object detection model based on deconvolution feature fusion according to an embodiment of the present invention.

The conventional deep learning-based object detection model uses anchor and multi-scale fusion information to detect objects that are robust to shape change and size change of various objects. However, in such a conventional deep learning-based object detection model, the complexity may increase as the number of anchors increases and the number of multi-scale fusion information increases. For this reason, a conventional deep learning-based object detection model requires a large number of learning parameters, and as the number of necessary learning parameters increases, the amount of computation increases.

The object detection model 111 provided by the present invention can provide a method of reducing the amount of computation by using a smaller number of learning parameters through a simplified feature fusion method without needing anchors.

More specifically, the object detection model 111 of the present invention includes (1) a plurality of convolution layers that generate encoding feature maps having different resolutions for a target image divided into a plurality of cells having a constant size. S 210, (2) by a low-resolution encoding feature map generated by a first convolution layer located at the last of the plurality of convolution layers and a second convolution layer located before the first convolution layer. Object information predicted in each of a plurality of cells using a single deconvolution layer 220 that generates a decoding feature map by fusing the generated high-resolution encoding feature map and (3) the generated decoding feature map It may be composed of an object detection layer 230 that detects.

The operation of the object detection model 111 executed by the object detection device 110 according to an embodiment is as follows. First, the object detection model 111 generates encoding feature maps having different resolutions through a plurality of convolutional layers 210 when a target image 120 divided into a plurality of cells having a constant size is input. I can. Referring to FIG. 2, when a target image 120 of 1248x384x3 is input into the object detection model 111, encoding feature maps having different resolutions through a plurality of convolutional layers 210 corresponding to Conv_1 to Conv_5 It can be seen that (ex, Conv_5: 2048, Conv_4: 1024) are created.

In this case, in FIG. 2, the object detection model 111 provides a ReseNet50 model as a type of encoder corresponding to a plurality of convolutional layers 210, but is not limited thereto, and various models such as VGG, XceptionNet, ResnetXT, and SuffleNet Can be applied.

Thereafter, the object detection model 111 includes encoding feature maps generated by the latter convolution layers among the encoding feature maps generated by the plurality of convolution layers 210 through a single deconvolution layer 220. By fusing, a decoding feature map can be generated. Referring to FIG. 2, it can be seen that encoding feature maps generated by Conv_5 and Conv_4 corresponding to the second convolution layer among the plurality of convolution layers 210 are input to the deconvolution layer 220. At this time, since Conv_5 is at a later stage than Conv_4, the encoding feature map of Conv_5 may have a lower resolution than the encoding feature map of Conv_4.

Specifically, the single deconvolution layer 220 can convert the low-resolution encoding feature map to high resolution by sequentially performing 1x1 convolution, upsampling, and 3x3 convolution on the low-resolution encoding feature map. Here, the 3x3 convolution can be viewed as an element that determines a receptive field when generating a decoding feature map for object detection. The object detection model 111 of the present invention increases the receiving area by adding a 3x3 convolution layer 221 to the deconvolution layer 220 as shown in FIG. 2 in order to consider a wider receiving area, and the deconvolution layer By increasing the depth of (220), a method of increasing the expressive power and capacity of an object detection model is provided.

Thereafter, the single deconvolution layer 220 concatenates the low-resolution encoding feature map converted to high resolution and the high-resolution encoding feature map input from the convolution layers 210, and then performs 1x1 convolution. As a result, a decoding feature map having high-resolution fusion features can be generated. Referring to FIG. 2, it can be seen that a decoding feature map of 78x24x512 for a 512 channel is generated by using encoding feature maps corresponding to Conv_5 and Conv_4 in the deconvolution layer 220.

Thereafter, the object detection layer 230 may generate an object detection feature map including object information predicted from each of a plurality of cells constituting the target image 120 by using the decoding feature map. As an example, referring to FIG. 2, it can be seen that a 74x24x8 object detection feature map is generated by applying a 1x1x8 convolution to a decoding feature map in the object detection layer 230. In this case, the object detection feature map includes information about the presence or absence of an object predicted in each of the plurality of cells (1 channel), class information of the predicted object (3 channels), and bounding box information corresponding to a region where the predicted object is expected to exist 4 channels). In this case, although the class information of the object is limited to 3 channels in FIG. 2, the number of channels of the class information may be changed according to the type of detectable object.

Finally, the object detection apparatus 110 may determine a final detection area of an object by removing an area of the bounding box overlapping with the object detection feature map generated through the object detection layer 230. For example, referring to FIG. 2, the object detection device 110 may determine a final object detection area in the target image 120 by applying Non-Maximum Suppression (NMS) to the generated object detection feature map. .

3 is a diagram illustrating a multi-task based object detection apparatus according to an embodiment of the present invention.

The basic structure of the multi-task model can be largely classified into structures of hard parameter sharing and soft parameter sharing. Unlike the soft parameter sharing structure, the hard parameter sharing structure is a structure that obtains common features from a sharing layer, converts the acquired common features into features of each task domain, and then performs each task result. This can reduce the number of parameters by minimizing redundant operations between tasks, thereby reducing computational complexity. In addition, the multi-task model can learn a lot of diversified data according to the learning of several tasks during model training, effectively preventing overfitting of the model, and improving the performance of the learning model by increasing the feature expression of the sharing layer. I can make it.

The object detection apparatus 110 of the present invention applies a multi-task structure to an object detection model as shown in FIG. 3. First, decoders 320 to 350 each designed according to characteristics/types/purposes may be connected in parallel to a backbone 310 which is a sharing layer of an object detection model. In this case, the backbone 310 may correspond to the convolutional layers 210 of the object detection model 111 of FIG. 2, and the decoders 320 to 350 may correspond to the deconvolutional layer 220.

Here, the backbone 310 may be configured with an encoder such as VGG or ResNet, or may be configured with a structure such as Feature Pyramid Network or Atrous Spatial Pyramid Pooling configured based on such an encoder. Further, the decoders 320 to 350 connected to the backbone 310 may be configured in a structure capable of being plug and play (PnP) according to the purpose of use of the user.

On the other hand, the object detection apparatus 110 having a multi-task structure of the present invention provides the entire object detection model through a sequential learning method of a single object detection model corresponding to the backbone 310 + each of the decoders 320 to 350. I can learn. As an example, the object detection device 110 learns the backbone 310 and the first decoder 320, which are share layers, using the learning data of the first decoder 320, and uses the learning data of the second decoder 330. By using the share layer, the backbone 310 and the second decoder 330 may be learned. Likewise, the object detection apparatus 110 may sequentially repeatedly apply the above-described learning method to the learning data of the third decoder 340 and the learning data of the fourth decoder 350.

This learning method is kind of similar to the ensemble learning method, and since the backbone 310, which is a share layer, can learn a lot of diversified data, overfitting can be prevented. In addition, this learning method improves the ability to express learning data of various angles as feature data having high correlation, thereby improving the performance of each of the single decoders 320 to 350.

In addition, the object detection apparatus 110 having a multi-task structure of the present invention supplements the learning data of other similar object detection models even if the training data set of the single object detection model designed according to each characteristic/type/purpose is small. Learning can be done so that the problem of lack of learning data can be solved. Accordingly, the object detection apparatus 110 having a multi-task structure of the present invention is designed to annotate and refine learning data according to the existing characteristics/types/purposes, and to design an object detection model according to the characteristics and types of each learning data. There is an advantage of reducing the cost as well as the cost of learning.

For example, referring to FIG. 3, the first decoder 320 may be an object detection model for a CoCo data set, and the second decoder 330 may be an object detection model for a Kitti data set. Here, the CoCo data set is a data set that detects 80 types of objects, which is an OpenData set in which several situations are aggregated, and annotated ground truth (GT) without predicting the occluded area of the object. Have. On the other hand, the KITTI data set is a data set that detects three objects in a road driving situation, predicts an occluded area of an object, and has an annotated GT. Accordingly, the characteristics and types of the two data sets and the object detection purpose are different, and an object detection model corresponding to the decoder must be separately configured.

The object detection apparatus 110 having a multi-task structure according to the present invention uses a sequential iterative learning method of separately configured object detection models according to the characteristics/types/purposes of the data set. It is possible to reduce the learning cost as well as the design cost of the object detection model according to the characteristics and types of.

4 is a diagram illustrating a method of calculating a learning loss (Loss) of an object detection model according to an embodiment of the present invention.

In the conventional object detection method, a plurality of anchors are placed in one cell, and learning is performed by dividing by size and shape, and a learning loss is calculated for the performed result. In contrast, the object detection apparatus 110 of the present invention provides a distance-based GT-Cell encoding method, and information about the existence of an object predicted in one cell without a separate anchor, and the predicted object's class. It is possible to provide a method of simultaneously learning information and bounding box information corresponding to a region in which the predicted object is expected to exist.

,

는 셀과 GT 박스 사이의 거리를 나타낸다. 여기서

,

는 복수의 셀들 각각의 중심점과 GT 박스의 중심점 간 거리에 관한 것으로 아래의 식 1과 같이 표현할 수 있다.First, the object detection apparatus 110 may perform matching between the cell and the GT box based on the distance between the cell and the GT box. Referring to Figure 4,

,

Represents the distance between the cell and the GT box. here

,

Is about the distance between the center point of each of the plurality of cells and the center point of the GT box, and can be expressed as Equation 1 below.

<Equation 1>

,

= box_center - cell_center (image plane)

,

= box_center-cell_center (image plane)

는 상대적 거리 기준 비율, b는 최소 거리 기준 비율, c는 최대 거리 기준 비율을 각각 나타낸다.In addition, S represents the stride size (cell size) of the feature map, and w and h represent the width and height of the GT box.

Denotes the relative distance reference ratio, b denotes the minimum distance reference ratio, and c denotes the maximum distance reference ratio.

,

가 각각 최대 거리 기준 cS보다 작으면서 aw, ah 보다 작거나, 최소 거리 기준 bS보다 작으면 해당 셀과 GT 박스를 매칭할 수 있다. 즉, 객체 검출 장치(110)는 아래의 식 2를 만족하는 경우, 셀과 GT 박스 간 매칭을 수행할 수 있다.At this time, the object detection device 110

,

If is smaller than the maximum distance reference cS , smaller than aw , ah , or smaller than the minimum distance reference bS , the corresponding cell and the GT box can be matched. That is, the object detection apparatus 110 may perform matching between the cell and the GT box when Equation 2 below is satisfied.

<Equation 2>

The object detection apparatus 110 may not match when the distance between the GT box and the cell becomes more than a certain distance based on the maximum distance criterion, thereby reducing variation in cell selection according to the size of the object. As described above, the object detection apparatus 110 may enable generalized learning by reducing a deviation in a learning degree with respect to a shape change of an object to be detected.

In addition, the object detection apparatus 110 may calculate the learning loss using information on the class and bounding box of the GT only for the matched cell, and the class and bounding box information predicted from the matched cell. Since this distance-based GT-Cell encoding method learns all GTs for one cell instead of k anchors, the number of output channels is C+4. Here, C denotes the number of classes, and 4 denotes region information of the predicted object.

5 is a flowchart illustrating an object detection method according to an embodiment of the present invention.

In step 510, the object detection apparatus 100 may divide the target image to be learned into a plurality of cells having a predetermined size.

In step 520, the object detection apparatus 100 includes encoding features having different resolutions corresponding to each of the plurality of convolution layers from the target image through a plurality of convolution layers included in the object detection module. You can create a map. In this case, various models such as ReseNet50, VGG, XceptionNet, ResnetXT, and SuffleNet may be applied as encoders corresponding to the plurality of convolutional layers.

In step 530, the object detection device 100 is generated by the first convolution layer located at the last end of the plurality of convolution layers through a single deconvolution layer included in the object detection module. The decoded feature map may be generated by fusing the low-resolution encoding feature map and the high-resolution encoding feature map generated by the second convolution layer located at the previous stage of the first convolution.

Specifically, a single deconvolution layer can convert low-resolution encoding feature maps to high resolution by sequentially performing 1x1 convolution, upsampling, and 3x3 convolution on low-resolution encoding feature maps. The object detection apparatus 110 of the present invention increases the receiving area by adding a 3x3 convolution layer to a single deconvolution layer in order to consider a wider receiving area, and increases the depth of the deconvolution layer to detect an object model. Provides a way to increase the expressive and receptive capacity of the person.

Thereafter, the object detection device 110 performs 1x1 convolution after concatenating the low-resolution encoding feature map converted to high resolution through a single deconvolution layer and the high-resolution encoding feature map input from the convolution layers. By doing so, it is possible to finally generate a decoding feature map having a high-resolution fusion feature.

In step 540, the object detection apparatus 100 may detect object information predicted in each of a plurality of cells by using a decoding feature map through an object detection layer included in the object detection module. The object information predicted in each of the plurality of cells includes information about the existence of an object predicted in each of the plurality of cells, information about the class of the predicted object, and a bounding box corresponding to an area in which the predicted object is expected to exist ( bounding box) information.

In operation 550, the object detection apparatus 100 may determine a final object detection area by removing an area of the overlapping bounding box from among object information predicted from each of the plurality of cells.

In step 560, the object detection apparatus 100 may calculate a learning loss (Loss) of the object detection model by using distance information between each of the plurality of cells and a preset GT box. More specifically, the object detection apparatus 100 performs matching between each of the plurality of cells and the GT box based on the distance between each of the plurality of cells and the GT box, and the predicted object information from the cell in which the matching with the GT box is performed. The learning loss of the object detection model can be calculated by calculating a loss function using object information corresponding to the GT box.

In this case, the object detection apparatus 100 uses the relative distance reference ratio, minimum distance reference ratio, and maximum distance reference ratio to the cell size of each of the plurality of cells and the size of the GT box. The distance between the center points may be determined, and cells having the determined distance equal to or less than a preset reference may be matched to the GT box.

Finally, in step 570, the object detection apparatus 110 may optimize the performance of the object detection model by adjusting the parameter value of the object detection model so that the learning loss of the calculated object detection model is minimized.

Meanwhile, the method according to the present invention is written as a program that can be executed on a computer and can be implemented in various recording media, such as a magnetic storage medium, an optical reading medium, and a digital storage medium.

Implementations of the various techniques described herein may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or combinations thereof. Implementations include a data processing device, e.g., a programmable processor, a computer, or a computer program product, i.e. an information carrier, e.g., machine-readable storage, for processing by or controlling the operation of a number of computers. It may be implemented as a computer program tangibly embodied in an apparatus (computer readable medium) or a radio signal. Computer programs such as the above-described computer program(s) may be recorded in any type of programming language, including compiled or interpreted languages, and as a standalone program or in a module, component, subroutine, or computing environment. It can be deployed in any form, including as other units suitable for the use of. A computer program can be deployed to be processed on one computer or multiple computers at one site or to be distributed across multiple sites and interconnected by a communication network.

Processors suitable for processing a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. In general, the processor will receive instructions and data from read-only memory or random access memory or both. Elements of the computer may include at least one processor that executes instructions and one or more memory devices that store instructions and data. In general, a computer may include one or more mass storage devices that store data, such as magnetic, magnetic-optical disks, or optical disks, or receive data from or transmit data to them, or both. It can also be combined so as to be. Information carriers suitable for embodying computer program instructions and data are, for example, semiconductor memory devices, for example, magnetic media such as hard disks, floppy disks and magnetic tapes, Compact Disk Read Only Memory (CD-ROM). ), Optical Media such as DVD (Digital Video Disk), Magnetic-Optical Media such as Floptical Disk, ROM (Read Only Memory), RAM (RAM) , Random Access Memory), flash memory, EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), and the like. The processor and memory may be supplemented by or included in a special purpose logic circuit structure.

Further, the computer-readable medium may be any available medium that can be accessed by a computer, and may include both a computer storage medium and a transmission medium.

While this specification includes details of a number of specific implementations, these should not be construed as limiting to the scope of any invention or claimable, but rather as a description of features that may be peculiar to a particular embodiment of a particular invention. It must be understood. Certain features described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination. Furthermore, although features operate in a particular combination and may be initially described as so claimed, one or more features from a claimed combination may in some cases be excluded from the combination, and the claimed combination may be a sub-combination. Or sub-combination variations.

Likewise, although operations are depicted in the drawings in a specific order, it should not be understood that such operations must be performed in that particular order or sequential order shown, or that all illustrated operations must be performed in order to obtain a desired result. In certain cases, multitasking and parallel processing can be advantageous. In addition, separation of the various device components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and the program components and devices described are generally integrated together into a single software product or packaged in multiple software products. It should be understood that you can.

On the other hand, the embodiments of the present invention disclosed in the specification and drawings are merely presented specific examples to aid understanding, and are not intended to limit the scope of the present invention. It is apparent to those of ordinary skill in the art that other modified examples based on the technical idea of the present invention may be implemented in addition to the embodiments disclosed herein.

100: object detection system
110: object detection device
111: object detection model
120: target image
130: resulting image

Claims (17)

In the learning method of the object detection model,
Dividing the target image to be learned into a plurality of cells having a predetermined size;
Generating encoding feature maps having different resolutions corresponding to each of the plurality of convolution layers from the target image through a plurality of convolution layers included in the object detection module;
A low-resolution encoding feature map and the first convolution generated by a first convolution layer located at the last end of the plurality of convolution layers through a single deconvolution layer included in the object detection module. Generating a decoding feature map by fusing the high-resolution encoding feature maps generated by the second convolution layer located at the previous end of the solution; And
Detecting object information predicted in each of the plurality of cells using the generated decoding feature map through an object detection layer included in the object detection module
Including,
The single deconvolution layer,
An object detection model learning method in which a convolutional layer for increasing a receptive field is added. The method of claim 1,
The object information predicted in each of the plurality of cells,
At least one of information about the existence of an object predicted in each of the plurality of cells, information about the class of the predicted object, and information about a bounding box corresponding to an area where the predicted object is expected to exist Learning method of an object detection model comprising a. The method of claim 1,
Determining a final detection area of an object by removing an area of the overlapping bounding box from among the object information predicted in each of the plurality of cells
Learning method of an object detection model further comprising. The method of claim 1,
Calculating a learning loss (Loss) of the object detection model using distance information between each of the plurality of cells and a preset ground truth (GT) box
Learning method of an object detection model further comprising. The method of claim 4,
Calculating the learning loss of the object detection model,
Performing matching between each of the plurality of cells and the GT box based on a distance between each of the plurality of cells and the GT box; And
Calculating a learning loss of the object detection model by calculating a loss function using object information predicted from the cell in which matching with the GT box is performed and object information corresponding to the GT box
Learning method of an object detection model comprising a. The method of claim 5,
The step of performing matching between each of the plurality of cells and the GT box,
The distance between the center point of each of the plurality of cells and the center point of the GT box is determined using a cell size of each of the plurality of cells and a relative distance reference ratio to the size of the GT box, a minimum distance reference ratio, and a maximum distance reference ratio The step of doing; And
Matching cells whose determined distance is less than or equal to a preset reference to the GT box
Learning method of an object detection model comprising a. The method of claim 4,
Adjusting the parameter value of the object detection model so that the learning loss of the calculated object detection model is minimized
Learning method of an object detection model further comprising. A computer-readable recording medium on which a program for executing the method of any one of claims 1 to 7 is recorded. In the object detection device in which the object detection model is executed,
Processor;
Including,
The object detection model,
A plurality of convolution layers for generating encoding feature maps having different resolutions for the target image divided into a plurality of cells having a constant size;
A low-resolution encoding feature map generated by a first convolutional layer positioned at the last of the plurality of convolutional layers and a high-resolution encoding feature generated by a second convolutional layer positioned before the first convolutional layer A single deconvolution layer for generating a decoding feature map by fusing the maps; And
An object detection layer that detects object information predicted in each of the plurality of cells using the generated decoding feature map
Including,
The single deconvolution layer,
An object detection device to which a convolution layer for increasing a receptive field is added. The method of claim 9,
The object information predicted in each of the plurality of cells,
At least one of information about the existence of an object predicted in each of the plurality of cells, information about the class of the predicted object, and information about a bounding box corresponding to an area where the predicted object is expected to exist Object detection model comprising a. The method of claim 9,
The processor,
An object detection apparatus configured to determine a final object detection region by removing an overlapping bounding box region among object information of each of the plurality of cells predicted through the object detection layer. The method of claim 9,
The processor,
An object detection apparatus for calculating a learning loss (Loss) of the object detection model by using distance information between each of the plurality of cells and a preset ground truth (GT) box. The method of claim 12,
The processor,
Matching is performed between each of the plurality of cells and the GT box based on the distance between each of the plurality of cells and the GT box, and corresponding object information predicted in the cell in which the matching with the GT box is performed and the GT box An object detection device that calculates a learning loss of the object detection model by calculating a loss function using the object information. The method of claim 13,
The processor,
The distance between the center point of each of the plurality of cells and the center point of the GT box is determined using a cell size of each of the plurality of cells and a relative distance reference ratio to the size of the GT box, a minimum distance reference ratio, and a maximum distance reference ratio And matching cells whose determined distance is less than or equal to a preset reference to the GT box. The method of claim 12,
The processor,
An object detection device that adjusts a parameter value of the object detection model to minimize the learning loss of the calculated object detection model. The method of claim 9,
The object detection model,
An object detection device in which a plurality of deconvolution layers designed according to characteristics of different objects are arranged in parallel at the rear ends of the plurality of convolution layers. The method of claim 16,
The processor,
An object detection device that sequentially and repeatedly learns the plurality of convolutional layers and the plurality of deconvolutional layers using training data corresponding to each of the plurality of deconvolutional layers arranged in parallel.

Images