KR102261894B1 - Apparatus and method for object detection - Google Patents

Abstract

The present application relates to an object recognition apparatus and an object recognition method, wherein the object recognition apparatus according to an embodiment of the present invention receives a target image, and performs a feature extraction operation to generate a feature image. block); and a backbone network unit that repeatedly performs a feature map extraction operation on the feature image to generate a plurality of first feature maps each having different resolutions depending on the number of times the feature map extraction operation is applied. have.

Description

Object recognition apparatus and method for object recognition {Apparatus and method for object detection}

The present application relates to an object recognition apparatus and an object recognition method capable of recognizing an object included in a target image.

Recently, as various applications using face information appear, interest in a practical face detection method is increasing. The face recognition system is used to protect personal privacy in a security system that permits access of a specific person and a surveillance environment. In addition, facial expression recognition is being used in the field of analyzing facial expression changes in the face region and interpreting human emotions from external facial expression changes. As the field of applications utilizing such face information is expanded and the number of applications increases, research on a highly practical face detection method capable of accurately extracting a face area in various environments is being actively conducted.

Recently, a convolutional neural network (CNN) method based on learning has achieved great results in various fields of computer vision. Although CNN's face detection method has made great strides in detection performance, the increased complexity of the system has cast doubt on its practicality. The number of windows that can be extracted from a 320×240 image reaches billions. For numerous patches, feature information is extracted based on CNN and classified into faces and non-face regions. This well represents the trade-off relationship between face detection performance and system complexity. In addition, since a convolution operation is repeatedly performed on the intersection region between adjacent windows, unnecessary computational processes are included, and the input and output of the fully-connected layer of the convolutional neural network are fixed. All input data passing through the neural network is accompanied by a process of resizing the size of the input data to a fixed size, thereby increasing the computational complexity of the system.

An object of the present application is to provide an object recognition apparatus and an object recognition method capable of realizing high object recognition performance for objects of various sizes with a relatively small capacity.

An object of the present application is to provide an object recognition apparatus and an object recognition method capable of generating a plurality of feature maps using the iterative reuse of a backbone network.

An object recognition apparatus according to an embodiment of the present invention includes: a feature extraction block for generating a feature image by performing a feature extraction operation upon receiving a target image; and a backbone network unit that repeatedly performs a feature map extraction operation on the feature image to generate a plurality of first feature maps each having different resolutions depending on the number of times the feature map extraction operation is applied. have.

Object recognition method according to an embodiment of the present invention, generating a feature image by performing a feature extraction operation on the input target image; repeatedly performing a feature map extraction operation on the feature image, and generating a plurality of first feature maps each having different resolutions according to the number of times the feature map extraction operation is applied; and using each bounding box moving within the plurality of first feature maps, it is determined whether an object is included in the bounding box, and when the object is included, the bounding box in the first feature map is set to the target image It may include the step of extracting the location information of the object by returning to the location in the.

Object recognition method according to another embodiment of the present invention, generating a feature image by performing a feature extraction operation on the input target image; repeatedly performing a feature map extraction operation on the feature image, and generating a plurality of first feature maps each having different resolutions according to the number of times the feature map extraction operation is applied; setting a first feature map having the lowest resolution among a plurality of first feature maps as an initial second feature map; generating a second feature map by upsampling the previously generated second feature map and connecting the first feature map having the same resolution as the upsampled result using a skip connection; and using each bounding box moving within the plurality of second feature maps, it is determined whether an object is included in the bounding box, and when the object is included, the bounding box in the second feature map is set to the target image It may include the step of extracting the location information of the object by returning to the location in the.

Incidentally, the means for solving the above problems do not enumerate all the features of the present invention. Various features of the present invention and its advantages and effects may be understood in more detail with reference to the following specific embodiments.

The object recognition apparatus and object recognition method according to an embodiment of the present invention can generate a plurality of feature maps using the iterative reuse of a backbone network, so that the number of parameters required for the implementation of the object recognition apparatus can be dramatically reduced. can be reduced

According to the object recognition apparatus and the object recognition method according to an embodiment of the present invention, since iterative learning is performed on objects of various sizes, it is possible to increase the object recognition rate for objects of relatively small sizes.

However, the effects that can be achieved by the object recognition apparatus and the object recognition method according to the embodiments of the present invention are not limited to those mentioned above, and other effects not mentioned are from the description below. It will be clearly understood by those of ordinary skill in the art.

1 is a block diagram illustrating an object recognition apparatus according to an embodiment of the present invention.
2 is a block diagram showing a feature extraction unit according to an embodiment of the present invention.
3 is a block diagram illustrating a reverse residual module according to an embodiment of the present invention.
4 is a block diagram illustrating an up-sampling module, a classification unit, and a position sensing unit according to an embodiment of the present invention.
5 is a block diagram showing the structure of a backbone network according to an embodiment of the present invention.
6 is a block diagram illustrating an object recognition apparatus according to another embodiment of the present invention.
7 is a flowchart illustrating an object recognition method according to an embodiment of the present invention.
8 is a flowchart illustrating an object recognition method according to another embodiment of the present invention.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes 'module' and 'part' for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. That is, the term 'unit' used in the present invention means a hardware component such as software, FPGA, or ASIC, and 'unit' performs certain roles. However, 'part' is not limited to software or hardware. The 'unit' may be configured to reside on an addressable storage medium or may be configured to refresh one or more processors. Thus, as an example, 'part' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Functions provided within components and 'units' may be combined into a smaller number of components and 'units' or further divided into additional components and 'units'.

In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

1 is a block diagram illustrating an object recognition apparatus according to an embodiment of the present invention.

Referring to FIG. 1 , the object recognition apparatus according to an embodiment of the present invention includes a feature extraction unit 110 , a backbone network unit 120 , an upsampling unit 130 , a classification unit 140 , and a position sensing unit ( 150) may be included.

Hereinafter, an object recognition apparatus according to an embodiment of the present invention will be described with reference to FIG. 1 .

When receiving a target image, the feature extraction unit 110 may generate a feature image by performing a feature extraction operation on the target image. As illustrated in FIG. 2 , the feature extraction unit 110 may include a 3×3 convolution layer, a batch normalization layer, and an activation function layer, and sequentially apply each layer. Through this, feature extraction operation can be performed. Here, s included in the 3×3 convolutional layer is the stride of the convolutional layer, p is whether padding is performed, g is the group, c _in is the width of the input channel, c _out Each corresponds to the width of the output channel. That is, according to FIG. 2 , the feature extraction unit 110 performs a 3×3 convolution filter operation with a stride of 2, padding, and a width of an input channel of 3 and an output channel of h. can be configured.

For example, when the target image is a color image, the input channel may include three channels corresponding to three colors of RGB, and the 3×3 convolutional layer of the feature extraction unit 110 has three input channels. can have Here, since the output channel is h, the 3×3 convolution layer can extend the channel of the feature image to h, and since the stride is 2, the resolution of the feature image can be reduced to half that of the target image. For example, when the resolution of the target image is 640×640, the feature image may be reduced to 320×320.

The backbone network unit 120 may repeatedly perform a feature map extraction operation on the feature image, and may generate first feature maps having different resolutions according to the number of times the feature map extraction operation is performed.

Referring to FIG. 1 , the backbone network unit 120 may include a plurality of inverted residual modules 121 , and performs an operation on one set including all of the plurality of inverted residual modules 121 . When completed, it corresponds to performing the feature map extraction operation once.

That is, if the feature map extraction operation is performed once, a first feature map f1 can be generated as a result of the operation, and then the first feature map f1 is inputted to the backbone network unit 120 again to perform the feature map extraction operation a second time. can make it In this case, the backbone network unit 120 may generate the first feature map f2 corresponding to the feature map extraction operation twice. Thereafter, the first feature maps f3, f4, f5, and f6 may be sequentially generated by repeating the same method. Here, an embodiment in which six first feature maps are generated is presented, but the present invention is not limited thereto, and the number of generated first feature maps may be variously modified according to embodiments.

In the case of a conventional single shot detector (SSD), a plurality of convolution filters are sequentially applied to a target image to generate feature maps corresponding to each convolution filter. That is, since the conventional SSD generates feature maps by sequentially passing through different convolutional filters, it is necessary to design parameters of six different convolutional filters in order to generate six feature maps.

On the other hand, in the present invention, since the same feature map extraction operation is repeatedly reused, it is possible to extract a plurality of feature maps even by designing only parameters for the feature map extraction operation. Therefore, according to the present invention, the number of necessary parameters can be remarkably reduced compared to the conventional SSD.

Meanwhile, the backbone network unit 120 may be represented as follows.

Here, {f ₁ , f ₂ , ... , f _N } is each first feature map, N is an integer greater than or equal to 1, and F(·) corresponds to the feature map extraction operation of the backbone network unit 120 . . In addition, E(·) corresponds to the feature extraction operation of the feature extraction unit 110 , x corresponds to the target image, and f ₀ corresponds to the feature image. That is, the plurality of first feature maps may be generated by repeating the output value as the input value again.

Additionally, the reverse residual modules 121 included in the backbone network unit 120 may be divided into a first residual residual module and a second residual residual module. Here, the first inverse residual module may be calculated first in the backbone network unit 120 , and the second inverse residual module may be sequentially calculated after the first inverse residual module. Specifically, as shown in FIG. 5 , the backbone network unit 120 may be implemented to include one first residual residual module and a plurality of second residual residual modules, respectively. 3 shows an example of a first inverse residual module and a second inverse residual module, where FIG. 3(a) corresponds to a first inverse residual module, and FIG. 3(b) corresponds to an example of a second inverse residual module. .

Here, each inverse residual module may perform a depth-wise separable convolution operation, thereby reducing the amount of computation required for feature map extraction.

Specifically, in the case of the first inverse residual module, as shown in FIG. 3(a), a 3×3 convolution layer, a batch normalization layer, an activation layer, a 1×1 convolution layer, and a batch normalization layer may be included. have. Here, since the feature extraction unit 110 extends the channel width of the feature image from 3 to h, a depth-wise convolution operation is performed in the 3×3 convolution layer, and then 1×1 convolution is performed. The channel width can be reduced from h to c again by performing a point-wise convolution operation through the convolution layer.

In addition, as shown in FIG. 3(b), the second inverse residual module includes a 1×1 convolution layer, a batch normalization layer, an activation layer, a 3×3 convolution layer, a batch normalization layer, an activation layer, and a 1×1 convolutional layer. A depth-based separation convolution operation may be performed by sequentially including a 1 convolution layer and a batch normalization layer. That is, the channel width is extended from c to h by performing a point-based convolution operation in the first 1×1 convolution layer, and then a depth-based convolution operation is performed on the expanded channel width in the 3×3 convolution layer. After that, a point-based convolution operation that reduces the channel width from h to c may be performed again in the 1×1 convolution layer.

Meanwhile, in the case of each activation layer included in the first inverse residual module and the second inverse residual module, a Parametric Rectified Linear Unit (PReLU) or Leaky-ReLU may be used as an activation function. Conventionally, a Rectified Linear Unit (ReLU) is used as an activation function. However, since ReLU sets a negative value to 0 due to its characteristics, there may be problems such as loss of information during repeated feature map extraction operations. Therefore, here, by using a Parametric Rectified Linear Unit (PReLU) or Leaky-ReLU capable of reflecting a negative value as an activation function, more accurate object recognition can be made possible.

Additionally, the backbone network unit 120 sets the stride of the inverse residual module 121 calculated last among the plurality of inverse residual modules to 2, and the stride of the remaining inverse residual modules 121 is 1 can be set. In this case, the resolution of the newly generated first feature map may be reduced by half. For example, when the resolution of the feature image is 320×320, the resolution of the first feature map f1 is reduced to 160×160, and then the resolution of the first feature maps f2, f3, f4, f5, and f6 is 80×80, respectively. , 40×40, 20×20, 10×10, 5×5. The number of these feature maps can be adjusted more or less depending on whether to detect smaller faces at the expense of speed and computational amount, or to secure fast speed and little computational amount.

Here, when the resolution of the feature map is reduced by half, it is possible to recognize a relatively large object from the feature map. When detecting an object, a method of determining whether an object is included in a preset bounding box may be utilized. In this case, if the resolution of the feature map is reduced by half, the area covered by the bounding box may be doubled. Accordingly, since all objects not previously included in the bounding box are all included in the bounding box, it is possible to detect an object of a relatively large size. That is, an object having a relatively small size may be detected in a feature map having a high resolution, and an object having a relatively large size may be detected in a feature map having a low resolution.

In this way, the backbone network unit 120 may recognize objects of various sizes included in the target image by changing the resolution of each of the generated first feature maps.

According to an embodiment, a 3×3 convolution filter having a stride of 2 is added to the end of a plurality of inverse residual modules included in the backbone network unit 120 to increase the resolution of the feature maps generated by the backbone network unit 120 . It is also possible to cut it in half.

Meanwhile, as shown in FIG. 6 , it is also possible to detect an object using the first feature maps generated by the backbone network unit 120 . However, when the first feature map is used, it may be difficult to detect a relatively small object. That is, relatively small objects are detected from the low-level first feature map, and the low-level first feature maps may be generated in a state in which the inverse residual modules are not formed to a sufficient depth.

To solve this problem, the object recognition apparatus according to an embodiment of the present invention may further include an upsampling unit 130 . That is, by introducing a Feature Pyramid Network (FPN) structure, it is possible to form a depth of sufficient inverse residual modules even in a low-level feature map.

Specifically, the upsampling unit 130 upsampling the previously generated second feature map, and connects the first feature map having the same resolution as the upsampled result by a skip connection technique. In this way, the second feature map may be generated. In this case, the upsampling unit 130 may set the first feature map having the lowest resolution among the plurality of first feature maps as the first second feature map.

That is, as shown in FIG. 1 , the last generated first feature map f6 may be set as the first second feature map g1, and then the second feature map g1 is up-sampled, and the up-sampled result is transferred to the previous one. In addition to the first feature map f5, a second feature map g2 may be generated. Here, the second feature map g2 may have the same resolution as the first feature map f5.

In addition, the upsampling unit 130 may upsample the second feature map g2 and add it to the first feature map f4 having the same resolution to generate the second feature map g3, and then repeat the same method in the same way to generate the remaining second feature map f4. 2 You can create feature maps.

Here, the operation of the upsampling unit 130 may be expressed by the following equation, where {g ₁ , g ₂ , ... , g _N } is each of the second feature maps, {f ₁ , f ₂ , . .. , f _N } is each first feature map, N is an integer greater than or equal to 1, and U _i (·) corresponds to an upsampling function.

Meanwhile, as shown in FIG. 1 , the upsampling unit 130 may include an upsampling module 131 , and the upsampling module 131 performs upsampling on each of the second feature maps. can Referring to FIG. 4( a ), the upsampling module 131 may include a bilinear upsample layer, a 3×3 convolution layer, a 1×1 convolution layer, a batch normalization layer, and an activation function layer. can

The classification unit 140 may determine whether an object is included in the boundary box by using each boundary box moving within the plurality of second feature maps. Here, the object detected by the classification unit 140 may be a face.

In addition, when the classifying unit 140 detects an object, the position detection unit 150 may extract the position information of the object by returning the bounding box in the second feature map to the position in the target image. That is, location information of an object included in the target image may be provided, and the location of the object may be displayed in the target image using the location information.

Meanwhile, FIGS. 4(b) and 4(c) correspond to the classification unit 140 and the position sensing unit 150, respectively, and the classification unit 140 and the position sensing unit 150 are 3×3 convolutions. It can act as a filter. In the case of the classification unit 140, since two cases are displayed, a case in which an object is included in the bounding box and a case in which an object is not included in the bounding box, there may be two output channels. Here, when there are four output channels, two channels can be selected from among the four channels by using Maxout, thereby reducing the false positive rate for relatively small objects. .

In addition, in the case of the position sensing unit 150, the output channel may include four dimensions, each of which may be the width, height, center point coordinates of the bounding box, and the like.

Meanwhile, the object recognition apparatus according to an embodiment of the present invention may be simultaneously learned using a multitask loss function. That is, the configuration of the feature extraction unit 110 , the backbone network unit 120 , the up-sampling unit 130 , the classification unit 140 , and the position sensing unit 150 can be simultaneously learned using a multi-task loss function. have.

Specifically, the multitask loss function is

can be Here, l _c is the classification loss, l _r is the regression loss, j is the index of the anchor box, and r _j ^* is the ground truth corresponding to the bounding box. . In addition, c _j ^* is set to 0 or 1, if the Jaccard overlap of the bounding box is greater than the reference value t, it may be set to 1, and if it is less than the reference value t, it may be set to 0. In addition, N _cls is the total number of positive and negative samples used during training, and N _reg = ∑ _j c _j ^* , λ may be an arbitrary variable. Here, since the regression loss is calculated only for similar samples, N _reg can be calculated using ∑ _j c _j ^*. In addition, the classification loss may be set as a cross-entropy loss, and the regression loss may be set as a smooth l1 loss.

Thereafter, by using the input learning data, parameters of the respective components of the object recognition apparatus may be set so that the classification loss and the regression loss of the multi-task loss function are minimized.

Here, since the object recognition apparatus according to an embodiment of the present invention generates a feature map by using the repeated reuse of the backbone network unit 120, the number of necessary parameters can be dramatically reduced compared to the conventional object recognition apparatus. . In addition, since a reverse residual module for extracting a feature map can be added as much as the number of parameters is reduced, more accurate object recognition can be implemented.

In addition, since the backbone network 120 according to an embodiment of the present invention is generated by repeatedly learning objects of various sizes, it is possible to improve the recognition rate of relatively small objects.

5 is a block diagram showing the structure of a backbone network according to an embodiment of the present invention. Three embodiments of the backbone network are shown in Fig. 5, and each embodiment corresponds to Figs. 5(a), 5(b) and 5(c).

In the first embodiment, the number of output channels is 32, in the second embodiment, the number of output channels is 48, and in the third embodiment, the number of output channels is 64. On the other hand, in the first embodiment, seven inverse residual modules are included, and in the second and third embodiments, each of five inverse residual modules is included. In this case, the first embodiment may include 60,000 parameters, the second embodiment may include 100,000 parameters, and the third embodiment may include 160,000 parameters. Here, in the case of the second and third embodiments, the number of parameters is reduced by reducing the number of inverse residual modules instead of a large number of output channels.

As for the object recognition performance for each of the embodiments, the third embodiment showed the best performance, followed by the second embodiment and the first embodiment in order. This indicates that the number of channels corresponds to a more important factor than depth, such as the number of inverse residual modules.

7 is a flowchart illustrating an object recognition method according to an embodiment of the present invention.

7, the object recognition method according to an embodiment of the present invention may include a feature image generating step (S110), a first feature map generating step (S120) and an object recognition step (S130), each The steps may be performed by the object recognition device.

Hereinafter, an object recognition method according to an embodiment of the present invention will be described with reference to FIG. 7 .

In the feature image generation step ( S110 ), a feature image may be generated by performing a feature extraction operation on the received target image. Here, the feature image may be generated through a 3×3 convolution operation, and in this case, the resolution of the feature image may be reduced by half by setting the stride of the 3×3 convolution operation to 2. Also, it is possible to generate a feature image by extending three RGB channels included in the target image to h.

In the first feature map generation step S120, a feature map extraction operation is repeatedly performed on the feature image, and a plurality of first feature maps each having different resolutions may be generated according to the number of times the feature map extraction operation is applied. .

Here, the feature map extraction operation may correspond to an operation for one set including all of a plurality of preset inverse residual modules. That is, if the feature map extraction operation is performed once, the first feature map f1 can be generated as a result of the operation, and the first feature map f2 can be generated by performing the feature map extraction operation again on the first feature map f1. . Thereafter, the first feature maps f3, f4, f5, and f6 may be sequentially generated by repeating the same method. In this case, since the same feature map extraction operation is repeatedly reused, it is possible to extract a plurality of feature maps even by designing only parameters for the feature map extraction operation.

Here, each inverse residual module may perform a depth-wise separable convolution operation, thereby reducing the amount of computation required for feature map extraction. Meanwhile, the inverse residual modules may use a Parametric Rectified Linear Unit (PReLU) or Leaky-ReLU as an activation function.

Additionally, the stride may be set to 2 for the inverse residual module that is calculated last among the plurality of inverse residual modules, and the stride of the remaining inverse residual modules may be set to 1. That is, the resolution of the first feature map may be reduced by half for each feature map extraction operation. As such, by changing the resolution of each of the first feature maps, it is possible to detect objects of various sizes included in the target image.

In the object recognition step ( S130 ), it is possible to determine whether an object is included in the bounding box by using each bounding box moving within the plurality of first feature maps. In addition, when the object is included, the location information of the object may be extracted by returning the bounding box in the first feature map to the location in the target image. Here, the object recognition step S130 may be implemented using a 3×3 convolution filter, and the location information may include the width, height, and center point location coordinates of the bounding box.

8 is a flowchart illustrating an object recognition method according to another embodiment of the present invention.

Referring to FIG. 8 , the object recognition method according to another embodiment of the present invention may further include an initial setting step ( S130 ) and a second feature map generation step ( S140 ) when compared with the object recognition method of FIG. 7 . have.

That is, as shown in FIG. 7 , although it is possible to detect an object using the first feature maps, it may be difficult to detect an object having a relatively small size when the first feature map is used.

In order to solve this problem, the object recognition method according to another embodiment of the present invention may further include an initial setting step (S130) and a second feature map generation step (S140) to introduce a Feature Pyramid Network (FPN) structure. .

Specifically, in the initial setting step (S130), the first feature map having the lowest resolution among the plurality of first feature maps may be set as the first second feature map, and thereafter, in the second feature map generating step (S130), immediately before The second feature map may be generated by upsampling the generated second feature map and connecting the first feature map having the same resolution as the up-sampled result using a skip connection technique.

That is, the last generated first feature map f6 may be set as the first second feature map g1, then the second feature map g1 is up-sampled, and the up-sampled result is added to the previous first feature map f5 to obtain the first feature map f5. 2 A feature map g2 can be generated. Here, the second feature map g2 may have the same resolution as the first feature map f5.

In addition, the second feature map g2 may be upsampled and added to the first feature map f4 having the same resolution to generate a second feature map g3, and then the remaining second feature maps may be generated by repeating the same method. have.

The present invention described above can be implemented as computer-readable code on a medium in which a program is recorded. The computer-readable medium may continuously store a computer-executable program, or may be temporarily stored for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or several hardware combined, it is not limited to a medium directly connected to any computer system, and may exist distributed on a network. Examples of the medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and those configured to store program instructions, including ROM, RAM, flash memory, and the like. In addition, examples of other media may include recording media or storage media managed by an app store that distributes applications, sites that supply or distribute other various software, and servers. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

The present invention is not limited by the above embodiments and the accompanying drawings. For those of ordinary skill in the art to which the present invention pertains, it will be apparent that the components according to the present invention can be substituted, modified and changed without departing from the technical spirit of the present invention.

100: object recognition device 110: feature extraction unit
120: backbone network unit 121: reverse residual module
130: up-sampling unit 131: up-sampling module
140: classification unit 150: position detection unit

Claims (11)

a feature extraction block for generating a feature image by performing a feature extraction operation upon receiving a target image; and
A backbone network unit that repeatedly performs a feature map extraction operation on the feature image to generate a plurality of first feature maps each having a different resolution depending on the number of times the feature map extraction operation is applied,
The backbone network unit

to generate the first feature map {f ₁ , f ₂ , ... , f _N } respectively, where N is an integer greater than or equal to 1, F(·) is the feature map extraction operation of the backbone network unit, E (·) is a feature extraction operation of the feature extraction unit, x is the target image, and f ₀ is the feature image.
delete The method of claim 1, wherein the backbone network unit
An object recognition apparatus comprising a plurality of inverted residual modules, and sequentially applying the feature image to the plurality of inverted residual modules to perform the feature map extraction operation.
The method of claim 3, wherein the inverse residual module
An object recognition apparatus comprising a depth-wise separable convolution operation and using a Parametric Rectified Linear Unit (PReLU) or Leaky-ReLU as an activation function.
The method of claim 3, wherein the backbone network unit
Among the plurality of inverse residual modules, the stride of the last calculated inverse residual module is set to 2, and the stride of the remaining inverse residual modules is set to 1.
According to claim 1,
a classification head for determining whether an object is included in the bounding box by using each bounding box moving within the plurality of first feature maps; and
When the object is included, object recognition further comprising a regression head for extracting location information of the object by returning the bounding box in the first feature map to a location in the target image Device.
7. The method of claim 6,
The object recognition apparatus, characterized in that the feature extraction unit, the backbone network unit, the classification unit and the position detection unit are simultaneously learned using a multitask loss function.
According to claim 1,
Upsampling for generating a second feature map by upsampling a second feature map generated just before, and connecting a first feature map having the same resolution as the up-sampled result using a skip connection technique By further including wealth,
The up-sampling unit
An object recognition apparatus, characterized in that the first feature map having the lowest resolution among the plurality of first feature maps is set as the first second feature map.
generating a feature image by performing a feature extraction operation on the input target image;
repeatedly performing a feature map extraction operation on the feature image, and generating a plurality of first feature maps each having different resolutions according to the number of times the feature map extraction operation is applied; and
Using each bounding box moving within the plurality of first feature maps, it is determined whether an object is included in the bounding box, and when the object is included, a bounding box in the first feature map is set within the target image. To include the step of extracting the location information of the object by returning to the location,
The step of generating the plurality of first feature maps includes:

to generate the first feature map {f ₁ , f ₂ , ... , f _N } respectively, where N is an integer greater than or equal to 1, F(·) is a feature map extraction operation of the backbone network unit, E( ·) is the feature extraction operation of the feature extraction unit, x is the target image, and f ₀ is the feature image.
generating a feature image by performing a feature extraction operation on the input target image;
repeatedly performing a feature map extraction operation on the feature image, and generating a plurality of first feature maps each having different resolutions according to the number of times the feature map extraction operation is applied;
setting a first feature map having the lowest resolution among a plurality of first feature maps as an initial second feature map;
generating a second feature map by upsampling the previously generated second feature map and connecting the first feature map having the same resolution as the upsampled result using a skip connection; and
Using each bounding box moving within the plurality of second feature maps, it is determined whether an object is included in the bounding box, and when the object is included, a bounding box in the second feature map is placed within the target image. To include the step of extracting the location information of the object by returning to the location,
The step of generating the plurality of first feature maps includes:

to generate the first feature map {f ₁ , f ₂ , ... , f _N } respectively, where N is an integer greater than or equal to 1, F(·) is a feature map extraction operation of the backbone network unit, E( ·) is the feature extraction operation of the feature extraction unit, x is the target image, and f ₀ is the feature image.
A computer program stored in a medium in combination with hardware to execute the object recognition method of any one of claims 9 to 10.

Images