KR20230053347A - Object Detection Network using Coordinate Information - Google Patents

Abstract

Disclosed are a YOLO-based object detection method and apparatus, which may accurately mark a position of a bounding box. The YOLO-based object detection apparatus disclosed in the present invention comprises: a backbone network which extracts an image feature in an input image; a neck network which includes regression information of a bounding box for the extracted image feature, and outputs the regression information of the bounding box to which a coordinate convolution module having coordinate information on a position of the bounding box is applied for increasing the accuracy of the position of the bounding box from among the regression information of the bounding box; and a head network which detects an object by using a YOLO layer for the output of the neck network, and performs regression learning for the detected object.

Description

Object detection network technique using coordinate information {Object Detection Network using Coordinate Information}

The present invention relates to a method and apparatus for detecting an object using coordinate information.

Object detection is required in many fields such as medical image analysis, autonomous driving, face recognition, and human recognition through CCTV. In order to apply the above application service, high detection performance and high speed are required. However, it was difficult to apply to real-time application services with the existing object detection method, but the development of deep learning technology showed the possibility of accuracy and speed of object detection technology. Before the development of deep learning, object detection technology typically used a sliding window. The sliding window technique finds objects by moving blocks of various sizes from the upper left to the lower right of the image. However, it is inefficient in terms of speed because it has to be calculated repeatedly in one image. To solve this problem, DPM (Deformable Part Model) has been proposed [1]. Object detectors using deep learning can be largely divided into two-stage object detectors and one-stage object detectors. Two-stage object detectors were proposed in R-CNN [2], fast R-CNN [3] and faster R-CNN [4]. To improve the inefficiency of the sliding window technique, the two-stage object detector uses a region proposal technique that quickly finds a region where an object exists and classifies the object. Because of these two processes, there is a disadvantage that it is slow in terms of speed. On the other hand, one-stage object detectors have been studied in CenterNet [5], EfficientDet [6], and YOLO [7-9]. Since the one-stage object detector detects the bounding box and the object type in a single process, the accuracy is lower than the two-stage object detector, but the detection speed is fast.

One-stage object detectors can be divided into object detectors that do not use anchor boxes and object detectors that use anchor boxes. The anchor box provides information about the bounding box by averaging the size of objects in the data set by group during initial training. Therefore, learning converges quickly when an anchor box is used. Representatively, an object detector that does not use an anchor box has been proposed in CenterNet. Object detectors using anchor boxes have been proposed in EfficientDet, YOLOv3 [7], YOLOv4 [8], and PP-YOLO [9] and have relatively high accuracy.

A technical problem to be achieved by the present invention is to provide a method and apparatus for improving detection performance by applying various techniques to a one-stage object detector based on YOLO. The proposed object detection network using coordinate information uses CIOU Loss, Swish, and EMA to improve learning performance, and uses Focus module, SPP module, and PANet to widen the receptive field of the network. In addition, to accurately display the location of the bounding box, we will add a coordinate convolution (CoordConv) module that has coordinate information and reduce the model parameters and computation amount by using the CSP technique.

In one aspect, the YOLO-based object detection apparatus proposed in the present invention includes a backbone network for extracting image features in an input image, regression information of a bounding box for the extracted image features, and In order to increase the accuracy of the location of the bounding box among the regression information, a neck network that outputs the regression information of the bounding box to which a coordinate convolution module having coordinate information on the location of the bounding box is applied and a YOLO layer is used for the output of the neck network. and a head network that detects objects and performs regression learning on the detected objects.

The backbone network divides the input image into grid cells using a focus module, and then adds the divided image to channel information to widen the acceptance field.

The neck network adds the X and Y coordinates of the bounding box to a channel by applying a coordinate convolution module to increase the accuracy of the bounding box, and then performs convolution.

The neck network applies a Cross Stage Partial (CSP) technique to reduce model parameters and computational complexity, and uses a Path Aggregation Network (PANet) that adds a bottom-up method to a feature pyramid net (FPN). In order to increase the receptive field of the network, SPP (Spatial Pyramid Pooling) module is used to apply maxpooling of multiple sizes and connect each channel.

The head network performs regression learning according to the midpoint position of the bounding box and Intersection Over Union (IOU), and uses CIOU (Complete Intersection Over Union) loss to obtain the aspect ratio using arctangent to impose aspect ratio consistency. .

In another aspect, the YOLO-based object detection method proposed in the present invention includes the steps of extracting image features in an input image through a backbone network, and the bounding box among the regression information of the bounding box for the extracted image features Outputting regression information of the bounding box through a neck network to which a coordinate convolution module having coordinate information on the location of the bounding box is applied in order to increase location accuracy, and output of the neck network in the head network and detecting an object using the YOLO layer and performing regression learning on the detected object.

According to embodiments of the present invention, detection performance can be improved by applying various techniques to a one-stage object detector based on YOLO. The object detection network using the proposed coordinate information can improve learning performance by using CIOU Loss, Swish, and EMA, and can widen the acceptance field of the network by using Focus module, SPP module, and PANet. In addition, the position of the bounding box can be accurately displayed by adding a coordinate convolution (CoordConv) module that has coordinate information, and the parameters and calculation amount of the model can be reduced by using the CSP technique.

1 is a diagram for explaining an output structure of a YOLO layer according to the prior art.
2 is a diagram for explaining a Focus module according to the prior art.
3 is a diagram for explaining the configuration of an object detection network using coordinate information according to an embodiment of the present invention.
4 is a diagram for explaining a coordinate convolution (CoordConv) module according to an embodiment of the present invention.
5 is a diagram for explaining an SPP module according to an embodiment of the present invention.
6 is a diagram for explaining a CSP module according to an embodiment of the present invention.
7 is a flowchart illustrating a method of detecting an object using coordinate information according to an embodiment of the present invention.

The main purpose of object detection is to find the type and location of an object in an image. The present invention proposes a method for improving detection performance by applying various techniques to a one-stage object detector based on YOLO. The proposed algorithm uses CIOU Loss, Swish, and EMA to increase learning performance, and uses Focus module, SPP module, and PANet to widen the receptive field of the network. In addition, to accurately display the location of the bounding box, a coordinate convolution (CoordConv) module with coordinate information can be added, and the model parameters can be reduced by about 23% and the amount of computation by about 27% by using the CSP technique. The performance of the detector was evaluated with the MS COCO 2017 test dataset, and the proposed detector (CSP-Coords YOLO) showed high accuracy (46.3% mAP) and high speed (96.2 FPS) compared to YOLOv4. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining an output structure of a YOLO layer according to the prior art.

으로 나타내고, k개의

의 집합으로 구성된다. n은 모델 내에 스트라이드(stride)가 2인 컨볼루션을 지날 때마다 증가하며, n이 커질 때마다 입력되는 입력 값의 크기 는 2n 만큼 줄어든다. 이때

의 집합 요소 중 하나인

는 텐서(Tensor)의 형태로 이루어져 있으며 식(1)과 같다: The YOLO layer according to the prior art is an algorithm proposed by YOLOv3 [7]. The method of detecting an object is the input value coming into the YOLO layer.

Represented by, k

consists of a set of n increases each time a convolution with a stride of 2 passes through the model, and whenever n increases, the size of the input value decreases by 2n. At this time

is one of the set elements of

is in the form of a Tensor and is represented by Equation (1):

식(1)

Equation (1)

로 k개의

의 집합이 된다. 이때

의 집합 요소 중 하나인

의 출력 텐서는 식(2)와 같다: In equation (1), b, c, W _S , and H _S are the batch size, the number of object types, and the horizontal and vertical sizes of images input from the backbone, respectively. After passing the YOLO layer, the tensor shape is changed and regression learning is performed on the bounding box. The output of the YOLO layer is

by k

becomes a set of At this time

is one of the set elements of

The output tensor of is equal to equation (2):

식(2)

Equation (2)

In Equation (2), the last channel 5+c may be configured as shown in FIG. Each of X _B and Y _B represents the position of the midpoint of the bounding box, and W _B and H _B are the width and height of the bounding box. C represents the probability that an object exists in the bounding box, and P _C represents the probability for the object type. When the size of one image in the YOLO layer is W _S Х H _S , the number of bounding boxes that can be extracted is as shown in Equation (3):

식(3)

Equation (3)

In FIG. 1, when C is greater than the confidence threshold, C is multiplied by each P _C . If C Х P _C is greater than the reliability threshold, it is assumed that an object of that type exists within the bounding box, and P _C is converted to P. The upper left coordinates (X ₁ , Y ₁ ) and lower right coordinates (X ₂ , Y ₂ ) of the bounding box are calculated using X _B , Y _B , W _B , and H _B . A bounding box is created by concatenating six pieces of information X ₁ , Y ₁ , X ₂ , Y ₂ , C, and P for each bounding box. A bounding box with the highest probability of having an object is extracted through NMS (Non Maximum Suppression).

2 is a diagram for explaining a Focus module according to the prior art.

The Focus module is a module used in the YOLOv5 backbone [10] and divides the input image as shown in Figure 2 and adds the divided image to the channel information. Although RGB image has 3 channels, the acceptance field is widened because the channel value is widened by the focus module.

(4)

(4)

In Equation (4), b, c, w, and h mean the batch size, channel, width, and height, respectively, and the output channel value of the focus module increases by 4 times compared to the input channel value, and the width and height are reduced by half.

3 is a diagram for explaining the configuration of an object detection network using coordinate information according to an embodiment of the present invention.

In the present invention, we propose a one-stage object detection algorithm using an anchor box to improve object detection speed and performance. The proposed object detector includes a backbone network 310 that extracts image features from an image, a neck network 320 that has regression information about a bounding box, and a head network that detects objects ( 330). The proposed detector uses Focus [10] module, SPP [11] module, and PANet [12] to widen the receptive field of the network, and CIOU Loss [13] and Swish [14] to improve learning performance. ], using EMA [9]. In addition, in order to accurately display the bounding box in the neck (320) part of the model, a coordinate convolution (CoordConv) [15] module with coordinate information is added, and the CSP [16] technique is used to reduce the parameters and operation amount of the model. apply

An object detection network using the coordinate information of the proposed YOLO-based object detection device includes a backbone network 310, a neck network 320, and a head network 330.

The backbone network 310 according to an embodiment of the present invention extracts image features from an input image.

The backbone network 310 divides the input image into grid cells using the focus module, and then adds the divided image to channel information to widen the acceptance field.

According to an embodiment of the present invention, the focus module may use a module used in the YOLOv5 backbone 311 [10]. The input image is divided as shown in FIG. 2 and the divided image is added to the channel information. Although RGB image has 3 channels, the acceptance field is widened because the channel value is widened by the focus module. As explained above, the value of the output channel of the Focus module is increased by 4 times compared to the value of the input channel, and the width and height are reduced by half.

The structure of the backbone network 310 according to an embodiment of the present invention is shown in Table 1.

<Table 1>

CC-YOLO according to an embodiment of the present invention uses the backbone structure used in YOLOv5 instead of the Darknet53 structure used in YOLOv3. The output values extracted from the backbone network 310 are represented by P ₃ , P ₄ , and P ₅ as shown in FIG. 3 , and P ₃ , P ₄ , and P ₅ are reduced by 1/8, 1/16, and 1/32 of the input image. have a size

The neck network 320 according to an embodiment of the present invention includes bounding box regression information for the extracted image feature, and coordinates for the bounding box position to increase the accuracy of the bounding box position among the bounding box regression information. Outputs the regression information of the bounding box to which the informational coordinate convolution module is applied.

The neck network 320 applies a coordinate convolution module to increase the accuracy of the bounding box, adds the X and Y coordinates of the bounding box to the channel, and then performs convolution.

In the existing YOLOv3, the FPN [18] structure that connects image features in the backbone network was used. CC-YOLO according to an embodiment of the present invention uses PANet 321 instead of FPN structure. PANet 321 can obtain more information about the location of the bounding box by adding a bottom-up method to the existing top-down FPN. The image features P ₃ , P ₄ , and P ₅ obtained from the backbone network are used as inputs of PANet. The result value output from PANet 321 is used as an input of the head network 330, and the detector (Detect) using P ₅ as an input detects a relatively large object and the detector (Detect) using P ₃ as an input Detect relatively small objects. The detailed structure of PANet 321 is shown in FIG.

The neck network 320 according to an embodiment of the present invention may apply a Cross Stage Partial (CSP) technique to reduce model parameters and computational complexity. In addition, information on the location of the bounding box may be additionally acquired using a path aggregation network (PANet) that adds a bottom-up scheme to a feature pyramid net (FPN). In addition, in order to increase the receptive field of the network, maxpooling of a plurality of sizes may be applied using a Spatial Pyramid Pooling (SPP) module and connected for each channel.

The head network 330 according to an embodiment of the present invention detects an object using the YOLO layer 331 for the output of the neck network, and performs regression learning on the detected object.

The head network 330 performs regression learning according to the midpoint position and Intersection Over Union (IOU) of the bounding box, and completes Intersection Over Union (CIOU) loss to obtain the aspect ratio using the arctangent to impose aspect ratio consistency. use.

CC-YOLO according to an embodiment of the present invention uses the YOLO layer 331 to detect objects in YOLOv3. The input values P ₃ , P ₄ , and P ₅ used in the detector (Detect) are converted by the YOLO layer method described above, and regression learning is performed on the output values output from each detector. When object detection is performed instead of model learning, objects are detected by the previously described bounding box extraction method for the three detectors.

4 is a diagram for explaining a coordinate convolution (CoordConv) module according to an embodiment of the present invention.

Existing CNNs had a problem in that an object was not located at the part corresponding to the coordinates when coordinate information was given and a convolution was performed. However, in the present invention, by adding the X and Y coordinates to the channel through the coordinate convolution (CoordConv) module, the position of the object can be predicted at the exact coordinates. Since the object detection problem is also a problem of predicting the position of the bounding box, the position of the bounding box can be accurately located by adding coordinate information to the neck network portion having information on the bounding box as shown in FIG. 4 .

5 is a diagram for explaining an SPP module according to an embodiment of the present invention.

As shown in FIG. 5, the SPP module according to an embodiment of the present invention has a structure in which maxpooling of various sizes is applied and connected for each channel. In the embodiment of the present invention, kernel sizes of 1, 5, 9, and 13 are used, and the stride is equal to 1. Batch normal and Swish are applied whenever it passes through the convolution (Conv) 1x1. In this way, the receptive field of the network can be effectively increased. In the embodiment of the present invention, as shown in FIG. 3 , it was applied to feature P ₅ extracted at the top.

6 is a diagram for explaining a CSP module according to an embodiment of the present invention.

CSPNet according to an embodiment of the present invention has the advantage of being applicable to any network. As shown in FIG. 6, CSPNet separates the input channels from the existing module into two and inputs them to convolution (Conv) 1x1, one passes through the Bottleneck module, and the other skips the Bottleneck module and creates a channel in the middle. put it back together Since only half of the channels are initially used for the module, the amount of computation is reduced. In CC-YOLO according to an embodiment of the present invention, CSPNet is applied to all bottlenecks existing in the backbone network and the neck network.

In the existing YOLOv3, MSE loss [7] was used for bounding box regression, but CIOU loss [13] and GIOU loss [19] point out this problem because regression learning is performed only on the part where the boxes overlap, and IOU loss [20] I solved the problem by using . According to the embodiment of the present invention, the CIOU loss has the advantage of fast regression because regression is performed according to the position of the midpoint of the box, the aspect ratio, and the intersection over union (IOU). Therefore, the proposed algorithm uses the CIOU loss considering all requirements.

According to an embodiment of the present invention, the activation function used in Table 1 uses Swish [22] instead of ReLU [20] and Mish [21]. In the proposed detector, Swish is used because Swish is faster in operation than Mish and reaches the target value quickly.

EMA according to an embodiment of the present invention obtains a moving average for the learned parameters when training a model. Since it is more efficient to use the average of the learned parameters than to write the last learned parameter, the EMA method is applied in the embodiment of the present invention.

7 is a flowchart illustrating a method of detecting an object using coordinate information according to an embodiment of the present invention.

The object detection method using the proposed coordinate information includes extracting image features in an input image through a backbone network (710), and increasing the accuracy of the bounding box location among regression information of the bounding box for the extracted image features. Step 720 of outputting regression information of the bounding box through a neck network to which a coordinate convolution module having coordinate information on the location of the bounding box is applied, and a YOLO layer for output of the neck network in the head network and detecting an object using , and performing regression learning on the detected object (730).

In step 710, image features in the input image are extracted through the backbone network.

The backbone network according to an embodiment of the present invention divides an input image into grid cells using a focus module, and then adds the divided images to channel information to widen the acceptance field.

According to an embodiment of the present invention, the Focus module may use a module used in the YOLOv5 backbone. The input image is divided as shown in FIG. 2 and the divided image is added to the channel information. Although RGB image has 3 channels, the acceptance field is widened because the channel value is widened by the focus module. As explained above, the value of the output channel of the Focus module is increased by 4 times compared to the value of the input channel, and the width and height are reduced by half.

In step 720, in order to increase the accuracy of the bounding box position among the regression information of the bounding box for the extracted image feature, bounding is performed through a neck network to which a coordinate convolution module having coordinate information on the bounding box position is applied. Output the regression information of the box.

In order to increase the accuracy of the bounding box, the neck network according to an embodiment of the present invention applies a coordinate convolution module to add the X and Y coordinates of the bounding box to a channel, and then performs convolution.

A neck network according to an embodiment of the present invention may apply a Cross Stage Partial (CSP) technique to reduce model parameters and computational complexity. In addition, information on the location of the bounding box may be additionally acquired using a path aggregation network (PANet) that adds a bottom-up scheme to a feature pyramid net (FPN). In addition, in order to increase the receptive field of the network, maxpooling of a plurality of sizes may be applied using a Spatial Pyramid Pooling (SPP) module and connected for each channel.

In step 730, an object is detected using the YOLO layer for the output of the neck network in the head network, and regression learning is performed on the detected object.

The head network according to an embodiment of the present invention performs regression learning according to the position of the midpoint of the bounding box and Intersection Over Union (IOU), and CIOU (Complete Intersection Over Union), which obtains the aspect ratio by using the arctangent to impose consistency of the aspect ratio. Union) loss.

In order to verify the efficiency of the object detection network structure using coordinate information according to an embodiment of the present invention, object detection performance was compared based on the MS COCO 2017 data set, and an ablation test was performed for each of the proposed algorithms as shown in Table 2. ) was carried out.

<Table 2>

, no bias decay, cosine learning rate decay)을 적용하였다. 네트워크의 학습 방법은 SGD(Stochastic Gradient Descent)방식을 사용하였고 초기 학습 비율(learning rate)은 0.01로 설정하였다. SGD의 모멘텀(momentum) 값은 0.937, 가중치 감쇠(weight decay) 값은 0.0005로 설정하였다. 제안하는 검출기의 백본 네트워크는 초기 값을 사전 학습된 가중치(Pretrained weight)로 이용한다. 그리고 학습 중에 백본 네트워크는 업데이트되지 않도록 설정하였다. 넥 네트워크에 존재하는 PANet과 헤드 네트워크는 사전 학습을 하지 않고 초기 값을 무작위로 설정하였다. mAP(mean Average Precision)는 객체 검출 성능의 평가 지표이며 실측 자료(ground truth)의 바운딩 박스와 예측한 바운딩 박스와 비교하여 모델의 성능을 평가한다. mAP50는 실측 자료의 바운딩 박스와 예측한 바운딩 박스가 50%이상 겹친 경우 정답으로 간주하는 평가 방식이다. 파라미터는 모델 크기를 나타내며, GFLOPS(Giga Floating Point Operations Per Second)는 컴퓨터가 1초동안 계산할 수 있는 연산량을 나타낸다. FPS(Frames Per Second)는 객체 검출기가 1초에 계산할 수 있는 프레임 수를 의미한다. In each experiment, one RTX 2080ti GPU was used and the default image size 640Х640 was used. The present invention is a learning method (learning rate warmup, zero

, no bias decay, cosine learning rate decay) was applied. The learning method of the network used SGD (Stochastic Gradient Descent) method, and the initial learning rate was set to 0.01. The momentum value of SGD was set to 0.937, and the weight decay value was set to 0.0005. The proposed detector's backbone network uses initial values as pretrained weights. Also, during training, the backbone network was set not to be updated. The PANet and the head network existing in the neck network were randomly set to initial values without pre-learning. mAP (mean average precision) is an evaluation index of object detection performance and compares the bounding box of the ground truth with the predicted bounding box to evaluate the performance of the model. mAP50 is an evaluation method in which the bounding box of the measured data and the predicted bounding box overlap by more than 50%, which is considered correct. The parameter represents the size of the model, and GFLOPS (Giga Floating Point Operations Per Second) represents the amount of computation a computer can calculate in one second. FPS (Frames Per Second) means the number of frames that an object detector can calculate in one second.

Referring to Table 2, comparing A and B, in the existing YOLOv3, leaky ReLU was used in the model structure and MSE loss was used for the bounding box, but the loss was reduced by using swish and CIOU loss, and by applying the EMA method, the overall mAP improved.

Comparing B and C with reference to Table 2, the parameter increased by about 2% and the amount of calculation increased by 0.5% by adding the SPP module. However, the network became more aware of the image and the mAP increased by about 0.4.

Referring to Table 2, comparing C and D, the parameter decreased by about 23% from 62.99M to 48.51M by applying CSPNet, and the amount of computation also decreased from 157.1G to 115.7G. The mAP of the C model was higher until 40 epoch, but after that, the mAP gradually reversed and the mAP increased by about 0.2.

Referring to Table 2, comparing D and E, darknet53, which was the backbone network of the existing YOLOv3, was replaced with the YOLOv5 backbone network. The YOLOv5 backbone network has increased the overall acceptance field by adding the Focus module and building the backbone network after the SPP block. Although some parameters were increased, mAP increased by about 0.3.

Comparing E and F with reference to Table 2, PANet was used instead of FPN for the neck network of the model, and the accuracy of the object type and location in the bounding box was improved because the deep feature values for the bounding box increased. Although slightly lowered in mAP50, mAP increased by about 0.8.

Comparing F and G with reference to Table 2, the coordinate convolution (CoordConv) with coordinate information was added to PANet with bounding box information to predict the location of the bounding box more accurately. Although the parameter increased by about 0.02M and the amount of calculation by about 0.1G, mAP increased by about 0.7 and mAP50 also increased by about 0.9.

Comparing G and H with reference to Table 2, experiments were conducted [23,24] to improve performance by learning the backbone network as a transfer-learned backbone network. mAP increased by about 1.3 and mAP50 also increased significantly by about 1.2.

In this way, according to an embodiment of the present invention, detection performance can be improved by applying various techniques to the one-stage object detector based on YOLO. The object detection network using the proposed coordinate information can improve learning performance by using CIOU Loss, Swish, and EMA, and can widen the acceptance field of the network by using Focus module, SPP module, and PANet. In addition, the location of the bounding box can be accurately displayed by adding a coordinate convolution (CoordConv) module having coordinate information, and the parameters and calculation amount of the model can be reduced by using the CSP technique.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. can be embodied in Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

<References>

[1] P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, p. 1627-1645, 2010.

[2] R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," arXiv [cs.CV], pp. 580- 587, 2013.

[3] R. Girshick, "Fast R-CNN," in 2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1440-1448.

[4] S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, p. 1137-1149, 2017.

[5] K. Duan, S. Bai, L. Xie, H. Qi, Q. Huang, and Q. Tian, "CenterNet: Keypoint triplets for object detection," in 2019 IEEE/CVF International Conference on Computer Vision (ICCV ), pp. 6569-6578, 2019.

[6] M. Tan, R. Pang, and Q. V. Le, "EfficientDet: Scalable and Efficient Object Detection," arXiv [cs.CV], pp. 10781-10790, 2019.

[7] J. Redmon and A. Farhadi, "YOLOv3: An Incremental Improvement," arXiv [cs.CV], 2018.

[8] A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, "YOLOv4: Optimal speed and accuracy of object detection," arXiv [cs.CV], 2020.

[9] X. Long, K. Deng, G. Wang, Y. Zhang, Q. Dang, Y. Gao, H. Shen, J. Ren, S. Han, E. Ding, and S. Wen, "PP -YOLO: An effective and efficient implementation of object detector," arXiv [cs.CV], 2020.

[10] YOLO-v5. https://github.com/ultralytics/yolov5

[11] K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 9, p. 1904-1916, Sep. 2015.

[12] S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia, "Path aggregation network for instance segmentation," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8759-8768, 2018.

[13] Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, "Distance-IoU loss: Faster and better learning for bounding box regression," arXiv [cs.CV] , 2019.

[14] P. Ramachandran, B. Zoph, and Q. V. Le, "Searching for Activation Functions," arXiv [cs.NE], 2017.

[15] R. Liu, J. Lehman, P. Molino, F. P. Such, E. Frank, A. Sergeev, J. Yosinski, "An intriguing failing of convolutional neural networks and the CoordConv solution," arXiv [cs.CV] , 2018.

[16] C. Y. Wang, H. Y. Mark Liao, Y. H. Wu, P. Y. Chen, J. W. Hsieh, and I. H. Yeh, "CSPNet: A New Backbone that can Enhance Learning Capability of CNN," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 390-391, 2020.

[17] T.-Y. Lin, M. mayor, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, "Microsoft COCO: Common objects in context," in Computer Vision - ECCV 2014, Cham: Springer International Publishing, pp. 740-755, 2014.

[18] T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, "Feature pyramid networks for object detection," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2117- 2125, 2017.

[19] H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese, "Generalized intersection over Union: A metric and A loss for bounding box regression," arXiv [cs.CV ], pp. 658-666, 2019.

[20] J. Yu, Y. Jiang, Z. Wang, Z. Cao, and T. Huang, "UnitBox: An advanced object detection network," in Proceedings of the 2016 ACM on Multimedia Conference - MM '16, 2016.

[21] A. L. Maas, A. Y. Hannun, and A. Y. Ng, "Rectifier nonlinearities improve neural network acoustic models," Stanford.edu, 2013.

[22] D. Misra, "Mish: A self regularized non-monotonic activation function," arXiv [cs.LG], 2019.

[23] T. He, Z. Zhang, H. Zhang, Z. Zhang, J. Xie, and M. Li, "Bag of tricks for image classification with convolutional neural networks," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 558-567, 2019.

[24] Z. Zhang, T. He, H. Zhang, Z. Zhang, J. Xie, and M. Li, "Bag of freebies for training object detection neural networks," arXiv [cs.CV], 2019.

Claims (10)

a backbone network for extracting image features in an input image;
Regression of a bounding box including regression information of the bounding box for the extracted image feature and applying a coordinate convolution module having coordinate information on the bounding box position to increase the accuracy of the bounding box position among the regression information of the bounding box A neck network that outputs information; and
Head network that detects objects using the YOLO layer on the output of the neck network and performs regression learning on the detected objects
YOLO-based object detection device comprising a. According to claim 1,
The backbone network,
After dividing the input image into grid cells using the focus module, the divided image is added to the channel information to widen the receptive field.
YOLO-based object detection device. According to claim 1,
The neck network,
In order to increase the accuracy of the bounding box, a coordinate convolution module is applied to add the X and Y coordinates of the bounding box to the channel and then perform convolution.
YOLO-based object detection device. According to claim 3,
The neck network,
Apply CSP (Cross Stage Partial) technique to reduce model parameters and calculation amount,
Information on the location of the bounding box is additionally acquired using PANet (Path Aggregation Network), which adds a bottom-up method to FPN (feature Pyramid Net),
In order to increase the receptive field of the network, SPP (Spatial Pyramid Pooling) module is used to apply maxpooling of multiple sizes and connect each channel.
YOLO-based object detection device. According to claim 1,
The head network,
Perform regression learning according to the midpoint position and Intersection Over Union (IOU) of the bounding box, and use CIOU (Complete Intersection Over Union) loss to obtain the aspect ratio using the arctangent to impose aspect ratio consistency.
YOLO-based object detection device. extracting image features in an input image through a backbone network;
In order to increase the accuracy of the bounding box position among the regression information of the bounding box for the extracted image feature, the bounding box regression information is output through a neck network to which a coordinate convolution module having coordinate information on the bounding box position is applied. doing; and
Detecting an object using a YOLO layer on the output of the neck network in the head network, and performing regression learning on the detected object.
YOLO-based object detection method comprising a. According to claim 6,
The step of extracting image features in the input image through the backbone network,
After dividing the input image into grid cells using the focus module, the divided image is added to the channel information to widen the receptive field.
YOLO-based object detection method. According to claim 6,
Outputting the regression information of the bounding box through a neck network to which a coordinate convolution module having coordinate information on the bounding box position is applied in order to increase the accuracy of the bounding box position among the regression information of the bounding box for the extracted image feature includes: ,
In order to increase the accuracy of the bounding box, a coordinate convolution module is applied to add the X and Y coordinates of the bounding box to the channel and then perform convolution.
YOLO-based object detection method. According to claim 8,
The neck network applies a CSP (Cross Stage Partial) technique to reduce the model parameters and calculation amount,
Information on the location of the bounding box is additionally acquired using PANet (Path Aggregation Network), which adds a bottom-up method to FPN (feature Pyramid Net),
In order to increase the receptive field of the network, SPP (Spatial Pyramid Pooling) module is used to apply maxpooling of multiple sizes and connect each channel.
YOLO-based object detection method. According to claim 6,
The step of detecting an object using a YOLO layer for the output of the neck network in the head network and performing regression learning on the detected object,
Perform regression learning according to the midpoint position and Intersection Over Union (IOU) of the bounding box, and use CIOU (Complete Intersection Over Union) loss to obtain the aspect ratio using the arctangent to impose aspect ratio consistency.
YOLO-based object detection method.

Images