KR102333545B1 - Method for learnig images using convolution neural network and apparatus for executing the method - Google Patents

Abstract

Disclosed are an image learning method based on a convolutional neural network and an apparatus for performing the same. A method for image learning based on a convolutional neural network and an apparatus for performing the same according to an embodiment of the present invention include one or more processors, and a computing having a memory for storing one or more programs executed by the one or more processors A method performed by an apparatus, the method comprising: generating image data by normalizing a frame of an input image; extracting a feature map for the image data through a convolution layer with respect to the generated image data; performing sub-sampling on the feature map through a pooling layer on the extracted feature map; extracting one or more final feature maps for the image data based on the convolutional layer and the pooling layer; and generating and outputting classification information for the image data based on the extracted final feature map and a preset activation function.

Description

Image learning method based on convolutional neural network and apparatus for performing the same

Embodiments of the present invention relate to an image learning technique based on a convolutional neural network.

Recently, various types and vast amounts of image data are being generated, and various technologies for automating and processing such image data are being developed. In particular, as artificial neural network technology develops, processing for automatically learning and classifying image data using this technology is being applied to the industry.

In the case of a convolutional neural network (CNN) technology, which is relatively widely used among these artificial neural network technologies, one or several convolutional layers, a pooling layer, and a fully connected layer ), and is known to be mainly used for video and audio analysis, which are two-dimensional input data.

However, the image recognition technology using an artificial neural network has a limitation in that it cannot simultaneously learn spatial or temporal information. In addition, the image recognition technology has a problem in that it is difficult to consider the passage of time in analyzing the relationship between objects including people and objects.

Republic of Korea Patent Publication No. 10-1735365 (2017.05.16.)

Embodiments of the present invention are for analyzing the behavior of an object in a video using a convolutional neural network.

An image learning method according to an embodiment disclosed herein is a method performed in a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, wherein the frame of an input image is generating image data by normalizing extracting a feature map for the image data through a convolution layer with respect to the generated image data; performing sub-sampling on the feature map through a pooling layer on the extracted feature map; extracting one or more final feature maps for the image data based on the convolutional layer and the pooling layer; and generating and outputting classification information for the image data based on the extracted final feature map and a preset activation function.

The generating of the image data may include: extracting each image size included in the input image, and resizing the extracted image size to the same pixel; and normalizing the image frame adjusted to the same size and generating the image data as the image data.

The extracting of the feature map may include sequentially applying a first filter to each pixel of the image data to reset each pixel value of the image data according to a weight in the pixel.

The extracting of the feature map may include performing a transformation operation on the plurality of feature maps using a nonlinear activation function.

The performing of the subsampling may include performing an operation of concatenating each pooling result calculated through a pooling box including a plurality of pooling layers with respect to the extracted feature map through a concatenated layer. can

The performing of the subsampling includes sequentially performing the first pooling, the second pooling, and the third pooling through the first pooling layer, the second pooling layer, and the third pooling layer on the extracted feature map. The first pooling layer, the second pooling layer, and the third pooling layer may use the same pooling method, but may have second filters of different sizes.

The performing of the sub-sampling may include inputting the first pooling layer, the second pooling layer, and the third pooling layer having the different sizes of the second filters to yield pooling results of the same size. adjusting a padding width of the feature map; and performing a concatenation operation of adding each pooling result calculated with the same size in a channel direction.

The performing of the subsampling includes simultaneously performing maximum pooling and average pooling through a maximum pooling layer and an average pooling layer on the extracted feature map, wherein the maximum pooling layer and the average pooling layer are The second filter may have the same size.

The performing of the sub-sampling may include performing a concatenation operation of adding each pooling result calculated with the same size in the maximum pooling layer and the average pooling layer having the second filter of the same size in the channel direction. have.

The generating and outputting the classification information may include generating the classification information of the image data through a nonlinear transformation of a weighted sum of the feature maps through a fully connected layer.

According to embodiments of the present invention, a behavior of an object in a video may be analyzed using a convolutional neural network.

In addition, according to embodiments of the present invention, by having a smaller number of parameters compared to a general 3D convolutional neural network method, there is an effect of more efficiently learning a video and image learning.

In addition, according to the embodiments of the present invention, since the number of parameters is very large, it occupies a lot of memory, takes a long time to learn, and can solve the problems of a general 3D convolutional neural network model that takes a long time to calculate when using the trained model. have.

1 is a block diagram illustrating and describing a computing environment including a computing device suitable for use in example embodiments;
2 is a block diagram illustrating the operation of a synthetic neural network (CNN) according to an embodiment of the present invention.
3 is a flowchart illustrating an image learning method according to an embodiment of the present invention;
4 is a diagram for explaining maximum pooling and average pooling performed in a pooling block according to an embodiment of the present invention;
5 is a block diagram illustrating an operation of a pooling block according to an embodiment of the present invention;
6 is a block diagram illustrating an operation of a pooling block according to another embodiment of the present invention;

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to provide a comprehensive understanding of the methods, apparatus, and/or systems described herein. However, this is merely an example and the present invention is not limited thereto.

In describing the embodiments of the present invention, if it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the content throughout this specification. The terminology used in the detailed description is for the purpose of describing embodiments of the present invention only, and should in no way be limiting. Unless explicitly used otherwise, expressions in the singular include the meaning of the plural. In this description, expressions such as “comprising” or “comprising” are intended to indicate certain features, numbers, steps, acts, elements, some or a combination thereof, one or more other than those described. It should not be construed to exclude the presence or possibility of other features, numbers, steps, acts, elements, or any part or combination thereof.

1 is a block diagram illustrating and describing a computing environment 10 including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components in addition to those described below.

The illustrated computing environment 10 includes a computing device 12 . In one embodiment, the computing device 12 may be a device for performing image learning according to an embodiment of the present invention.

Computing device 12 includes at least one processor 14 , computer readable storage medium 16 , and communication bus 18 . The processor 14 may cause the computing device 12 to operate in accordance with the exemplary embodiments discussed above. For example, the processor 14 may execute one or more programs stored in the computer-readable storage medium 16 . The one or more programs may include one or more computer-executable instructions that, when executed by the processor 14, configure the computing device 12 to perform operations in accordance with the exemplary embodiment. can be

Computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. The program 20 stored in the computer readable storage medium 16 includes a set of instructions executable by the processor 14 . In one embodiment, computer-readable storage medium 16 includes memory (volatile memory, such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash It may be memory devices, other forms of storage medium accessed by computing device 12 and capable of storing desired information, or a suitable combination thereof.

Communication bus 18 interconnects various other components of computing device 12 , including processor 14 and computer readable storage medium 16 .

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24 . The input/output interface 22 and the network communication interface 26 are coupled to the communication bus 18 . Input/output device 24 may be coupled to other components of computing device 12 via input/output interface 22 . Exemplary input/output device 24 may include a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices, and/or imaging devices. input devices, and/or output devices such as display devices, printers, speakers and/or network cards. The exemplary input/output device 24 may be included in the computing device 12 as a component constituting the computing device 12 , and may be connected to the computing device 12 as a separate device distinct from the computing device 12 . may be

2 is a block diagram illustrating an operation of a convolutional neural network (CNN) according to an embodiment of the present invention.

A convolutional neural network according to an embodiment of the present invention may include an input layer 100 , a convolutional layer 210 , a pooling layer 220 , a fully connected layer 300 , and an output layer 400 . The convolutional neural network extracts a feature map for the input image data through the convolution layer 210 and the pooling layer 220 with respect to the input image data, and identifies the input image data through the feature map Or classify. An image learning method operated by a convolutional neural network according to an embodiment of the present invention will be described in detail below.

3 is a flowchart illustrating an image learning method according to an embodiment of the present invention. As described above, the image learning method according to an embodiment of the present invention may be performed in a computing device 12 having one or more processors, and a memory for storing one or more programs executed by the one or more processors. can To this end, the image learning method may be implemented in the form of a program or software including one or more computer-executable instructions and stored in the memory.

In addition, in the illustrated flowchart, the method is described by dividing the method into a plurality of steps, but at least some of the steps are performed in a different order, are performed in combination with other steps, are omitted, are performed in separate steps, or are shown One or more steps not included may be added and performed.

Meanwhile, an image input to the image learning method according to an embodiment of the present invention may be an image corresponding to a photographed object or an activity field of the object, but is not limited thereto.

In step 302, the computing device 12 normalizes the frame of the input image and generates it as image data.

Specifically, each of the extracted image frames is readjusted and resized to the same size (eg, 224 pixels * 224 pixels). The computing device 12 may use a resizing algorithm to rescale and resize the extracted image frame, for example, a downsampling algorithm may be used. The extracted image frame may minimize loss of image information by using a down-sampling algorithm so that the aspect ratio of each image frame is maintained.

Thereafter, the computing device 12 normalizes the image frame adjusted to the same size and generates the image data as image data. Normalization is for all input dimensions to share a similar uniform distribution, and the mean-standard deviation normalization method can be used. The computing device 12 may use a mean and standard deviation estimation algorithm to perform a mean-standard deviation normalization method.

In operation 304, the computing device 12 extracts a feature map for the input image data through a convolution layer 210 for the image data. The feature map includes feature information on image data. In order to extract the feature map, the convolutional layer 210 and the pooling layer 220 are repeated, and the number of repetitions may be variously determined according to embodiments.

Specifically, when the size of a first filter used for convolution is determined, convolution is performed through a weighted sum of a weight assigned to each pixel of the first filter and a pixel value of image data. As a result of the convolution, a feature map for the image data is generated. That is, the computing device 12 may generate a plurality of feature maps by applying a plurality of different first filters to one image data, respectively. The computing device 12 sequentially applies the first filter to each pixel of the image data (that is, applies the first filter while moving it pixel by pixel) to reset each pixel value of the image data according to the weight of the first filter, You can create a feature map.

In this case, the computing device 12 performs padding (eg, zero padding) along the edge of the image data to prevent the size of the feature map from becoming smaller than the image data, and then Filter 1 may be sequentially applied to each pixel For example, the computing device 12 performs a padding operation of adding pixels having a value of zero along the edge of the image data, and then Filters can be applied sequentially to each pixel.

Here, the first filter may have a size of 3x3, but is not limited thereto. In the first filter, a weight (eg, 0 or 1) may be assigned to each pixel of 3x3. As a method of performing the convolution, for example, the computing device 12 multiplies a pixel value of pixels in the vicinity of the pixel with each weight of the first filter with respect to a predetermined pixel of image data, and then adds the sum to the pixel value of the pixel. It can be reset to a new pixel value. However, the present invention is not limited thereto, and the computing device 12 may reset the maximum value among the pixel values of neighboring pixels included in the first filter to a new pixel value of the predetermined pixel of the image data.

The computing device 12 may generate a plurality of feature maps by applying a plurality of first filters to which each pixel weight of 3x3 is different from each other to one image data. Through this process, robust features can be extracted from the image data.

Meanwhile, the computing device 12 may perform a transformation operation on the feature map generated through the convolutional layer 210 using a non-linear activation function. In this case, the computing device 12 may perform a transformation operation on the feature map using a Rectified Linear Unit (ReLU) activation function. The RuLU function is a function that outputs a value less than 0 as 0, and outputs a value greater than 0 as an input value. Here, the activation function is necessary to readjust the signal strength of neurons, and a Rectified Linear Unit (ReLU) function may be used as the activation function, but is not limited thereto.

In operation 306 , the computing device 12 may sub-sampling the feature map through a pooling layer 220 . Here, the pooling layer 220 may be applied in a manner that reduces the size of the feature map.

Specifically, when the size of the second filter used for pooling is determined, one pixel value may be extracted using pixel values of image data included in the size of the second filter. That is, the computing device 12 moves the second filter non-overlapping in the feature map (that is, moves in a unit of the size of the second filter in the feature map) using the pixel value of the feature map included in the second filter to generate one You can extract pixel values. For example, the pooling layer 220 may use Max pooling, Min pooling, or Average pooling. The maximum pooling may extract a maximum value among pixel values of pixels of the feature map included in the second filter. The minimum pooling may extract a minimum value among pixel values of pixels of the feature map included in the second filter. The average pooling may extract an average value of pixel values of pixels of the feature map included in the second filter.

4 is a diagram for explaining maximum pooling and average pooling performed in a pooling block according to an embodiment of the present invention.

While max pooling extracts the most important features for each region, average pooling takes into account all values in the region and produces an average value, so very smooth features are extracted. Therefore, max pooling is useful in capturing image structures such as edges and textures, and is therefore widely used in image recognition. Max pooling, which is often used in convolutional neural networks, applies a filter of a certain size to each pixel of the input layer and extracts the largest value among the values in the pixel. Average pooling extracts an average value of pixel values in a corresponding filter. As shown, the result of the pooling layer is different according to the method of pooling. For example, when maximum pooling is applied to a 4X4 feature map with a 2x2 second filter compressed to a size of 1/4, 25, 9, 41, and 3 are output for each 2X2 region marked differently, whereas average pooling is applied. When , it can be seen that 14, 5, 20, and 1 are output for each 2X2 area displayed differently. Since the pooling operation is not multiplied by the filter value, there are no parameters to learn, and there is no change in the channel because the number of inputs is output as it is.

Meanwhile, in general, in a convolutional neural network, the pooling layer 220 uses only one pooling method and one filter size. Select either max pooling, min pooling, or average pooling and specify the filter size. In this case, the performance of average pooling can be degraded when sharp features need to be detected, since it produces a smoothly downsampled output when applied to average pooling in a convolutional model.

Accordingly, in the present invention, a pooling layer most suitable for image data may be applied using a pooling block including a plurality of pooling layers. Pooling blocks will be described in detail below.

5 is a block diagram for explaining the operation of the pooling block 500 according to an embodiment of the present invention.

Referring to FIG. 5 , the computing device 12 may perform subsampling on the feature map through a pooling block 500 including a plurality of pooling layers 510 , 520 , and 530 with respect to the feature map.

Specifically, the first pooling, the second pooling, and the third pooling may be sequentially performed on the feature map through the first pooling layer 510 , the second pooling layer 520 , and the third pooling layer 530 . . Here, the first pooling, the second pooling, and the third pooling may use the same pooling method (eg, maximum pooling, average pooling, minimum pooling, etc.), and may each have different filter sizes. For example, the first pool may be a maximum pool of 1X1 size, the second pool may be a maximum pool of 3X3 size, and the third pool may be a maximum pool of 5X5 size. The computing device 12 connects each pooling result calculated through the first pooling layer 510, the second pooling layer 520, and the third pooling layer 530 through the concatenation layer 540 (Concatenation layer) ( Concatenation) can be performed. Through this, the output data of the layers separated from each other can be collected through one single layer.

In an exemplary embodiment, the concatenation layer 540 may expand the pooling results calculated through the pooling block 500 by merging them into one layer. At this time, the computing device 12 performs padding (padding) along the edges of the input feature maps so that output data of the same size is output from the first pooling, the second pooling, and the third pooling having different filter sizes. For example, zero padding may be performed The computing device 12 may calculate the size of output data from which the feature map is output through the pooling layer through the following equation.

(Equation)

Hi, Wi: Size of the input feature map (H X W)

Ho, Wo : Size of output data (H X W)

K: the size of the second filter (K X K)

P: padding width

S: stride

That is, according to Equation 1, the first pooling (1X1), the second pooling (3X3), and the third pooling (5X5) each having different filter sizes adjust the padding width of the input feature map (the border of the feature map) to adjust the number of pixels having a value of zero (0)) so that output data of the same size can be output. The computing device 12 may collect output data of layers separated from each other through one single layer by performing a concatenation operation of adding output data output with the same size in the channel direction (stacking along the channel axis).

6 is a block diagram for explaining the operation of the pooling block 600 according to another embodiment of the present invention.

Referring to FIG. 6 , the computing device 12 may perform subsampling on the feature map through a pooling block 600 including a plurality of pooling layers 610 and 620 with respect to the feature map.

Specifically, the maximum pooling layer 610 and the average pooling layer 620 may simultaneously perform maximum pooling and average pooling in parallel with respect to the feature map. The computing device 12 may perform an operation of concatenating each pooling result calculated through the maximum pooling layer 610 and the average pooling layer 620 through a concatenation layer 630 . Through this, the output data of the layers separated from each other can be collected through one single layer.

In an exemplary embodiment, the concatenation layer 630 may expand the pooling result calculated through the pooling block 600 by combining them into one layer. That is, the computing device 12 performs a concatenation operation of adding output data output with the same size in the maximum pooling and average pooling in the channel direction (stacking in the channel axis) to combine the output data of separate layers into one single layer. can be collected through

Therefore, in the image learning method according to an embodiment of the present invention, various features are derived from input image data by using a plurality of pooling methods or different filter sizes through a pooling block including a plurality of pooling layers at the same time. It can improve the learning accuracy. That is, there is an effect of more efficiently learning an image and image learning while having a smaller number of parameters compared to a general convolutional neural network.

In operation 308 , the computing device 12 may extract a final feature map for the image data resulting from the convolutional layer 210 and the pooling layer 220 . For example, the computing device 12 may extract final feature maps of a preset number of one-dimensional matrices by repeatedly performing the convolutional layer 210 and the pooling layer 220 on the image data.

In operation 310, the computing device 12 may generate and output classification information of the image data based on a preset number of final feature maps and a preset activation function.

Specifically, classification information of image data is generated by multiplying preset weighting coefficients for a preset number of final feature maps through a fully connected layer, and then using the summed value as an input of a preset activation function. Thus, it can be transmitted to the output layer. Here, as the activation function, a Softmax function or the like may be used.

Although representative embodiments of the present invention have been described in detail above, those of ordinary skill in the art will understand that various modifications are possible without departing from the scope of the present invention with respect to the above-described embodiments. . Therefore, the scope of the present invention should not be limited to the described embodiments and should be defined by the claims described below as well as the claims and equivalents.

10: Computing Environment
12: computing device
14: Processor
16: computer readable storage medium
18: communication bus
20: Program
22: input/output interface
24: input/output device
26: network communication interface

Claims (30)

one or more processors, and
A method performed in a computing device having a memory storing one or more programs to be executed by the one or more processors, the method comprising:
generating image data by normalizing a frame of an input image;
extracting a feature map for the image data through a convolution layer with respect to the generated image data;
performing sub-sampling on the feature map through a pooling layer on the extracted feature map;
extracting one or more final feature maps for the image data based on the convolutional layer and the pooling layer; and
and generating and outputting classification information for the image data based on the extracted final feature map and a preset activation function,
The step of performing the sub-sampling comprises:
The method further comprises performing an operation of concatenating each pooling result calculated through a pooling box including a plurality of pooling layers with respect to the extracted feature map through a concatenated layer,
The step of performing the sub-sampling comprises:
The method further comprises sequentially performing a first pooling, a second pooling, and a third pooling on the extracted feature map through a first pooling layer, a second pooling layer, and a third pooling layer,
The first pooling layer, the second pooling layer, and the third pooling layer use the same pooling method, but have second filters of different sizes,
The step of performing the sub-sampling comprises:
The padding width of the input feature maps is adjusted so that pooling results of the same size are calculated in the first pooling layer, the second pooling layer, and the third pooling layer having the different sizes of the second filters. to do; and
The method further comprising the step of performing a concatenation operation of adding each pooling result calculated with the same size in a channel direction.
The method according to claim 1,
The step of generating the image data includes:
extracting a size of each image included in the input image, and resizing the extracted image size to the same pixel; and
and generating the image data by normalizing the image frame adjusted to the same size.
The method according to claim 1,
The step of extracting the feature map,
and resetting each pixel value of the image data according to a weight in the pixel by sequentially applying a first filter to each pixel of the image data.
The method according to claim 1,
The step of extracting the feature map,
An image learning method comprising the step of performing a transformation operation on a plurality of feature maps using a non-linear activation function.
delete delete delete delete one or more processors, and
A method performed in a computing device having a memory storing one or more programs to be executed by the one or more processors, the method comprising:
generating image data by normalizing a frame of an input image;
extracting a feature map for the image data through a convolution layer with respect to the generated image data;
performing sub-sampling on the feature map through a pooling layer on the extracted feature map;
extracting one or more final feature maps for the image data based on the convolutional layer and the pooling layer; and
and generating and outputting classification information for the image data based on the extracted final feature map and a preset activation function,
The step of performing the sub-sampling comprises:
The method further comprises performing an operation of concatenating each pooling result calculated through a pooling box including a plurality of pooling layers with respect to the extracted feature map through a concatenated layer,
The step of performing the sub-sampling comprises:
The method further comprises simultaneously performing maximum pooling and average pooling on the extracted feature map through a maximum pooling layer and an average pooling layer,
The maximum pooling layer and the average pooling layer have a second filter of the same size,
The step of performing the sub-sampling comprises:
The method further comprising the step of performing a concatenation operation of adding each pooling result calculated with the same size in the maximum pooling layer and the average pooling layer having the second filter of the same size in the channel direction.
The method according to claim 1,
The step of generating and outputting the classification information includes:
and generating the classification information of the image data through nonlinear transformation of the weighted sum of the feature maps through a fully connected layer.
one or more processors, and
A method performed in a computing device having a memory storing one or more programs to be executed by the one or more processors, the method comprising:
a command for generating image data by normalizing a frame of an input image;
instructions for extracting a feature map for the image data through a convolution layer with respect to the generated image data;
instructions for performing sub-sampling on the feature map through a pooling layer on the extracted feature map;
instructions for extracting one or more final feature maps for the image data based on the convolutional layer and the pooling layer; and
and a command for generating and outputting classification information for the image data based on the extracted final feature map and a preset activation function,
The command for performing the subsampling is:
Further comprising a command for performing an operation of concatenating each pooling result calculated through a pooling box including a plurality of pooling layers with respect to the extracted feature map through a concatenated layer,
The command for performing the subsampling is:
Further comprising instructions for sequentially performing a first pooling, a second pooling, and a third pooling through a first pooling layer, a second pooling layer, and a third pooling layer on the extracted feature map,
The first pooling layer, the second pooling layer, and the third pooling layer use the same pooling method, but have second filters of different sizes;
The command for performing the subsampling is:
The padding width of the input feature maps is adjusted so that pooling results of the same size are calculated in the first pooling layer, the second pooling layer, and the third pooling layer having the different sizes of the second filters. order to do; and
The computing device further comprising an instruction for performing a concatenation operation of adding each pooling result calculated with the same size in a channel direction.
12. The method of claim 11,
The command for generating the image data is
a command for extracting a size of each image included in the input image, and resizing the extracted image size to the same pixel; and
and a command for generating the image data by normalizing the image frame adjusted to the same size.
12. The method of claim 11,
The command for extracting the feature map is
and a command for sequentially applying a first filter to each pixel of the image data to reset each pixel value of the image data according to a weight in the pixel.
12. The method of claim 11,
The command for extracting the feature map is
A computing device comprising: instructions for performing a transform operation on the plurality of feature maps using a non-linear activation function.
delete delete delete delete one or more processors, and
A method performed in a computing device having a memory storing one or more programs to be executed by the one or more processors, the method comprising:
a command for generating image data by normalizing a frame of an input image;
instructions for extracting a feature map for the image data through a convolution layer with respect to the generated image data;
instructions for performing sub-sampling on the feature map through a pooling layer on the extracted feature map;
instructions for extracting one or more final feature maps for the image data based on the convolutional layer and the pooling layer; and
and a command for generating and outputting classification information for the image data based on the extracted final feature map and a preset activation function,
The command for performing the subsampling is:
Further comprising a command for performing an operation of concatenating each pooling result calculated through a pooling box including a plurality of pooling layers with respect to the extracted feature map through a concatenated layer,
The command for performing the subsampling is:
Further comprising instructions for simultaneously performing maximum pooling and average pooling through a maximum pooling layer and an average pooling layer on the extracted feature map,
The maximum pooling layer and the average pooling layer have a second filter of the same size,
The command for performing the subsampling is:
The computing device further comprising an instruction for performing a concatenation operation of adding each pooling result calculated to have the same size in the maximum pooling layer and the average pooling layer having the second filter of the same size in a channel direction.
12. The method of claim 11,
A command for generating and outputting the classification information is
and an instruction for generating the classification information of the image data through nonlinear transformation of the weighted sum of the feature maps through a fully connected layer.
As a computer program stored in a non-transitory computer readable storage medium,
The computer program includes one or more instructions, which, when executed by a computing device having one or more processors, cause the computing device to:
generating image data by normalizing a frame of an input image;
extracting a feature map for the image data through a convolution layer with respect to the generated image data;
performing sub-sampling on the feature map through a pooling layer on the extracted feature map;
extracting one or more final feature maps for the image data based on the convolutional layer and the pooling layer; and
to generate and output classification information for the image data based on the extracted final feature map and a preset activation function;
The computer program causes the computing device to perform the sub-sampling,
To perform an operation of concatenating each pooling result calculated through a pooling box including a plurality of pooling layers with respect to the extracted feature map through a concatenated layer,
The computer program causes the computing device to perform the sub-sampling,
to sequentially perform first pooling, second pooling, and third pooling on the extracted feature map through a first pooling layer, a second pooling layer, and a third pooling layer;
The first pooling, the second pooling, and the third pooling use the same pooling method, but have second filters of different sizes,
The computer program causes the computing device to perform the sub-sampling,
The padding width of the input feature maps is adjusted so that pooling results of the same size are calculated in the first pooling layer, the second pooling layer, and the third pooling layer having the different sizes of the second filters. to do, and
A computer program for performing a concatenation operation of adding each pooling result calculated with the same size in a channel direction.
22. The method of claim 21,
The computer program causes the computing device to generate the image data,
extracting each image size included in the input image, and resizing the extracted image size to the same pixel, and
A computer program for generating the image data by normalizing the image frame adjusted to the same size.
22. The method of claim 21,
The computer program causes the computing device to extract the feature map,
and sequentially applying a first filter to each pixel of the image data to reset each pixel value of the image data according to a weight within the pixel.
22. The method of claim 21,
The computer program causes the computing device to extract the feature map,
A computer program for performing a transformation operation on a plurality of feature maps using a non-linear activation function.
delete delete delete delete As a computer program stored in a non-transitory computer readable storage medium,
The computer program includes one or more instructions, which, when executed by a computing device having one or more processors, cause the computing device to:
generating image data by normalizing a frame of an input image;
extracting a feature map for the image data through a convolution layer with respect to the generated image data;
performing sub-sampling on the feature map through a pooling layer on the extracted feature map;
extracting one or more final feature maps for the image data based on the convolutional layer and the pooling layer; and
to generate and output classification information for the image data based on the extracted final feature map and a preset activation function;
The computer program causes the computing device to perform the sub-sampling,
To perform an operation of concatenating each pooling result calculated through a pooling box including a plurality of pooling layers with respect to the extracted feature map through a concatenated layer,
The computer program causes the computing device to perform the sub-sampling,
Maximum pooling and average pooling are simultaneously performed on the extracted feature map through a maximum pooling layer and an average pooling layer,
The maximum pooling layer and the average pooling layer have a second filter of the same size,
The computer program causes the computing device to perform the sub-sampling,
A concatenation operation of adding each pooling result calculated with the same size in the maximum pooling layer and the average pooling layer having the second filter of the same size in the channel direction is performed.
22. The method of claim 21,
The computer program causes the computing device to generate and output the classification information,
A computer program for generating classification information of the image data through nonlinear transformation of the weighted sum of the feature maps through a fully connected layer.

Images