JP2020119518A - Method and device for transforming cnn layers to optimize cnn parameter quantization to be used for mobile devices or compact networks with high precision via hardware optimization - Google Patents

Abstract

To provide a method for transforming CNN layers to flatten values included in at least one feature map in order to properly reflect respective values of specific channels including small values on output values.SOLUTION: A method for transforming convolutional layers of a CNN including m convolutional blocks includes steps of: generating k-th quantization loss values by referring to k-th initial weight values of a k-th initial convolutional layer included in a k-th convolutional block, a (k-1)-th feature map outputted from the (k-1)-th convolutional block, and each of k-th scaling parameters; determining each of k-th optimized scaling parameters by referring to the k-th quantization loss values; generating a k-th scaling layer and a k-th inverse scaling layer by referring to the k-th optimized scaling parameters; and transforming the k-th initial convolutional layer into a k-th integrated convolutional layer by using the k-th scaling layer and the (k-1)-th inverse scaling layer.SELECTED DRAWING: Figure 3b

Description

The present invention provides a method for transforming a convolution layer of a CNN containing m convolution blocks, wherein (a) when a computing device obtains an input image used to determine a scaling parameter, i) one or more k-th initial weight values of the k-th initial convolution layer included in the k-th convolution block, and (ii) (ii-1) if k is 1, the input image, (ii) -2) When k is a constant from 2 to m, the (k-1)th feature map corresponding to the input image output from the (k-1)th convolution block, and (iii)(iii- 1) When k is 1, each of the k-th scaling parameters corresponding to each channel included in the input image, and (iii-2) when k is a constant from 2 to m, the (k- 1) A step of generating one or more kth quantization loss values with reference to each kth scaling parameter corresponding to each channel included in the feature map (k is a constant from 1 to m) ); (b) The computing device refers to the kth quantization loss value and corresponds to each of the channels included in the (k-1)th feature map among the kth scaling parameters. Determining each of the kth optimal scaling parameters; (c) the computing device generating the kth scaling layer and the kth inverse scaling layer with reference to the kth optimal scaling parameter; (d) the computing device. And (i) k is 1, the k-th scaling layer is used to transform the k-th initial convolution layer into a k-th integrated convolution layer, and (ii) k is 2 to m. Transforming the k-th initial convolution layer into the k-th integrated convolution layer using the k-th scaling layer and the (k-1)-th inverse scaling layer. And a method and device characterized by the above.

Deep Convolution Neural Networks (Deep CNN) are at the heart of the amazing developments that have taken place in the field of deep learning. CNN was also used in the 90's to solve the problem of character recognition, but in recent years, it has become widely used in the field of machine learning. For example, CNN won the image recognition contest (ImageNet Large Scale Visual Recognition Challenge) in 2012 over other competitors. After that, CNN came to be used as a very useful tool in the field of machine learning.

However, due to the prejudice that deep learning algorithms require 32-bit floating point arithmetic, mobile devices were considered to be incapable of executing programs containing deep learning algorithms.

However, some experiments have demonstrated that 10-bit fixed point arithmetic, which requires less computing performance than 32-bit floating point arithmetic, is sufficient for deep learning algorithms. Therefore, there have been many attempts to provide a method of using 10-bit fixed point arithmetic for deep learning algorithms in resource limited devices, ie mobile devices.

Although some successful methods of quantizing numbers represented in 32-bit floating point to 10-bit fixed point have been presented, there have been significant problems. When the values included in a plurality of channels are significantly changed, values of one or more specific channels including a small value among the channels may be ignored. It can be seen in FIG.

FIG. 5 exemplarily shows values of various channels that are significantly different.

Referring to FIG. 5, each value included in the first channel of the first feature map output from the first convolution block 210-1 is (0.64, 0.65, 0.63), It can be seen that the values included in the second channel are (0.002, 0.001, 0.0019). According to the prior art, when the value of the first channel and the value of the second channel are quantized, the unit value used for the quantization is determined by the first channel or the second channel.

If the unit value is determined by the first channel, the unit value is increased to indicate the value included in the first channel. And, since the unit value is too large as compared with the value included in the second channel, the value included in the second channel may be quantized to zero. On the contrary, when the unit value is determined by the second channel, the unit value becomes smaller to indicate the value included in the second channel. Then, the unit value is too small to correctly quantize the value included in the first channel.

If the particular channel values are ignored, or if the particular channel values are not properly quantized as described above, the CNN output may be distorted.

The present invention aims to solve the above-mentioned problems.

The present invention is a method of transforming a CNN layer so that the values included in at least one feature map can be flattened to properly reflect each value of a specific channel including a small value in an output value. The purpose is to provide.

The characteristic constitution of the present invention for achieving the above-mentioned object of the present invention and realizing the characteristic effect of the present invention described later is as follows.

According to one aspect of the invention, in a method of transforming a convolution layer of a CNN containing m convolution blocks, (a) a computing device obtains an input image used to determine a scaling parameter. Then, if (i) one or more kth initial weighting values of the kth initial convolution layer included in the kth convolution block and (ii)(ii-1)k is 1, the input The image, (ii-2) where k is a constant from 2 to m, the (k-1)th feature map corresponding to the input image output from the (k-1)th convolution block; and (iii) )(Iii-1)k is 1, each of the k-th scaling parameters corresponding to each channel included in the input image, and (iii-2)k is a constant from 2 to m, A step of generating one or more k-th quantization loss values with reference to each of the k-th scaling parameters corresponding to each channel included in the (k-1)th feature map (k is from 1 to m) (B) the computing device refers to the kth quantization loss value, and the channel included in the (k-1)th feature map among the kth scaling parameters. Determining each of the k-th optimal scaling parameters corresponding thereto; (c) the computing device generating the k-th scaling layer and the k-th inverse scaling layer with reference to the k-th optimal scaling parameter; d) the computing device transforms the kth initial convolutional layer into akth integrated convolutional layer using the kth scaling layer if (i)k is 1, and (ii)k Is a constant from 2 to m, transforming the k th initial convolution layer into the k th integrated convolution layer using the k th scaling layer and the (k−1) inverse scaling layer. A method is provided.

In one embodiment, in the step (a), the computing device expresses (iv) a BW value (wherein the BW value is a weight value included in the CNN and a value included in a feature map in binary numbers). And (v) the kth FL value (the kth FL value is (i) the kth initial weighting value of the kth initial convolution layer, and (ii)k). Is a constant from 2 to m, it is a value included in the (k-1)th feature map, and when k is 1, an index of the number indicated by the LSB of the value included in the input image. , Which is the absolute value of), and generating the k-th quantization loss value.

In one embodiment, in the step (a), the k-th quantization loss value is generated according to the equation, and θ _p is (i) k is a constant from 2 to m. k-1) the value of the k-th initial weighting value of the feature map and the k-th initial convolution feature map, (ii) when k is 1, the input image and the k-th initial convolution feature map C _k is a specific kth scaling parameter of the kth scaling parameters, FL and BW are the FL value and the BW value, respectively, and the Q operation is the a calculation for generating the difference between the quantized value of the FL values and C _ki theta _i generated by referring to the BW values and C _ki theta _i, step (b), the computing device, by selecting the C _ki be minimized the △ L _k, wherein determining a respective said first k optimal scaling parameters are provided.

In one embodiment, a method is provided in which the computing device uses the Nesterov Accelerated Gradient method to select the C _ki to determine the k-th optimal scaling parameter. It

In one embodiment, in the step (c), the computing device generates the kth scaling layer, each of which is determined as a constituent of the kth optimum scaling parameter, and a reciprocal of the kth optimum scaling parameter. A method is provided, characterized in that it produces said k-th inverse scaling layer, each of which is determined as its component.

In one embodiment, in the step (d), when the computing device (1) k is 1, (i) the operation of the k-th initial convolution layer and the k-th scaling layer are input values. The k-th initial con- struct so that the difference between the result generated by applying the result and (ii) the result generated by applying the operation of the k-th integrated convolution layer to the input value is smaller than a threshold value. When the volume layer is converted into the kth integrated convolution layer, and (2) k is a constant of 2 or more and m or less, (i) the (k-1) inverse scaling layer and the kth initial convolution layer. And (ii) a result generated by applying the operation with the k-th scaling layer to an input value, and (ii) a result generated by applying the operation of the k-th integrated convolution layer to the input value. A method is provided, which comprises transforming the k th initial convolution layer into the k th integrated convolution layer such that the difference is less than the threshold.

In one embodiment, (e) the computing device quantizes each weighting value of the kth integrated convolution layer included in the kth convolution block and is performed by the kth convolution block. Providing a k-th quantized weighting value as an optimized weighting value for the CNN operation.

According to another aspect of the present invention, in a computing device for transforming a convolution layer of a CNN including m convolution blocks, at least one memory storing each instruction, and (I)(i) kth concatenation. One or more kth initial weighting values of the kth initial convolution layer included in the volume block, and (ii)(ii-1) if k is 1, the input used to determine the scaling parameter Image, (ii-2) where k is a constant from 2 to m, the (k-1)th feature map corresponding to the input image output from the (k-1)th convolution block; and (iii) )(Iii-1)k is 1, each of the k-th scaling parameters corresponding to each channel included in the input image, and (iii-2)k is a constant from 2 to m, A process of generating one or more k-th quantization loss values with reference to each of the k-th scaling parameters corresponding to each channel included in the (k-1)th feature map (k is from 1 to m). (II) referring to the k-th quantization loss value, the k-th corresponding to each of the channels included in the (k-1)th feature map among the k-th scaling parameters. A process of determining each optimum scaling parameter; (III) a process of generating a kth scaling layer and akth inverse scaling layer with reference to the kth optimum scaling parameter; (IV)(i) when k is 1 , The k-th scaling layer is used to transform the k-th initial convolution layer into a k-th integrated convolution layer, and (ii) if k is a constant from 2 to m, then the k-th scaling layer and Converting the kth initial convolutional layer to the kth integrated convolutional layer using the (k-1)th inverse scaling layer; at least configured to perform the instructions. A computing device is provided that includes a processor.

In one embodiment, the process (I) is performed by the processor to express (iv) a BW value (wherein the BW value is a weight value included in the CNN and a value included in a feature map in binary numbers). The number of bits used in the above), and (v) the kth FL value (the kth FL value is (1) the kth initial weighting value of the kth initial convolution layer, and (2) k is 2). Is a value included in the (k−1)th feature map when it is a constant from m to m, and when k is 1, the absolute value of the exponent of the number indicated by the LSB of the value included in the input image Value) is further provided to provide the k-th quantization loss value.

In one embodiment, in the process (I), the processor generates the k-th quantization loss value according to the formula, and θ _p is (i) k is a constant from 2 to m. The value of the kth initial weighting value of the (k-1)th feature map and the kth initial convolution feature map, (ii) when k is 1, the input image and the kth initial convolution feature C _k is a particular kth scaling parameter of the kth scaling parameters, FL and BW are the FL value and the BW value, respectively. calculation is a calculation for generating the difference between the FL values and C _ki are generated by referring to the BW value theta _i quantized values and C _ki theta _i of the (II) process , wherein the processor is the △ by selecting the C _ki a L _k be the smallest, the computing apparatus characterized by determining each of the first k optimal scaling parameters are provided.

In one embodiment, a computing device is provided, characterized in that the processor uses the Nesterov Accelerated Gradient method to select the C _ki to determine the k-th optimal scaling parameter. It

In one embodiment, in the (III) process, the processor generates the k-th scaling layer for which each of the k-th optimum scaling parameters is determined as its constituents, and each of the reciprocals of the k-th optimum scaling parameter is A computing device is provided that is characterized in that it generates the k-th inverse scaling layer that is determined as its component.

In one embodiment, the (IV) process includes: (1) applying (i) an operation of the k-th initial convolution layer and the k-th scaling layer to an input value when k is 1. The kth initial convolution layer such that the difference between the result generated by (ii) applying the operation of the kth integrated convolution layer to the input value is smaller than a threshold value. To the kth integrated convolution layer, and (2) k is a constant not less than 2 and not more than m, (i) the (k-1) inverse scaling layer, the kth initial convolution layer, The difference between the result generated by applying the operation with the kth scaling layer to the input value and (ii) the result generated by applying the operation of the kth integrated convolution layer to the input value is A computing device is provided, which transforms the k-th initial convolution layer into the k-th integrated convolution layer so as to be smaller than the threshold value.

In one embodiment, the processor quantizes (V) a weight value of the kth integrated convolution layer included in the kth convolution block to perform a CNN operation performed by the kth convolution block. A computing device is provided, further comprising: a process of generating a k-th quantization weighting value as an optimization weighting value.

The present invention is a method of transforming a CNN layer so that the values included in at least one feature map can be flattened to properly reflect each value of a specific channel including a small value in an output value. And can be used by optimizing hardware that can be applied to mobile devices or high-precision small networks.

The following drawings, which are attached to explain the embodiments of the present invention, are only a part of the embodiments of the present invention, and are common knowledge in the technical field to which the present invention belongs. For those who have it (hereinafter "regular technician"), other drawings can be obtained based on this drawing without any inventive work.
FIG. 1 is a diagram illustrating a configuration of a computing device for performing a method of transforming a CNN layer for optimizing CNN parameter quantization according to an exemplary embodiment of the present invention. FIG. 2 is a diagram illustrating a configuration of a CNN including a scaling layer and an inverse scaling layer according to an exemplary embodiment of the present invention. FIG. 3a illustrates a process of generating an integrated convolution layer by switching positions of a scaling layer and an inverse scaling layer according to an exemplary embodiment of the present invention. FIG. 3b is a diagram illustrating a process of generating an integrated convolution layer by switching positions of a scaling layer and an inverse scaling layer according to an embodiment of the present invention. FIG. 4 is a view illustrating values of several different channels whose values do not change significantly according to a scaling method according to an exemplary embodiment of the present invention. FIG. 5 is an exemplary diagram showing the values of several different channels whose values change significantly according to the related art.

DETAILED DESCRIPTION The following detailed description of the invention refers to the accompanying drawings that illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These examples are described in sufficient detail to enable one of ordinary skill in the art to practice the invention. It should be understood that the various embodiments of the invention differ from each other, but need not be mutually exclusive. For example, the particular shape, structure, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention in connection with one embodiment. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention is, if properly explained, accompanied by all the scope equivalent to what the claims claim. It is limited only by the appended claims. Like reference symbols in the various drawings indicate identical or similar functions across various aspects.

Also, the word "comprising" and variations thereof throughout the detailed description of the invention and the claims are not intended to exclude other technical features, additions, components or steps. .. Other objects, advantages and features of the invention will be apparent to one of ordinary skill in the art, in part, from the specification and in part from the practice of the invention. The following illustrations and drawings are provided by way of illustration and are not intended to limit the invention.

The various images referred to in the present invention may include paved or unpaved road-related images, in which case objects that may appear in the road environment (eg, cars, people, animals, plants, objects, buildings, airplanes or A flying object such as a drone and other obstacles may be envisioned, but the present invention is not limited to this. The various images referred to in the present invention are images not related to roads (for example, unpaved roads). Roads, alleys, vacant lots, seas, lakes, rivers, mountains, forests, deserts, sky, indoors and related images), in which case unpaved roads, alleys, vacant lots, seas, lakes, rivers, mountains, forests , Deserts, sky, objects that can appear in indoor environments (eg, cars, people, animals, plants, objects, buildings, air vehicles such as airplanes and drones, and other obstacles), but not necessarily It is not limited.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily practice the present invention.

FIG. 1 illustrates a configuration of a computing device 100 for performing a method of transforming a CNN layer for optimizing CNN parameter quantization according to an exemplary embodiment of the present invention. In addition, FIG. 2 is a diagram illustrating a configuration of a CNN including a scaling layer and an inverse scaling layer according to an exemplary embodiment of the present invention.

With reference to FIG. 1, the computing device 100 may include a CNN 200. The process of inputting and outputting various data and calculating various data by the CNN 200 may be performed by the communication unit 110 and the processor 120, respectively. However, a detailed description of how the communication unit 110 and the processor 120 are connected in FIG. 1 will be omitted. In addition, the computing device may further include a memory 115 that may store computer readable instructions for performing the following processes. By way of example, a processor, memory, medium, etc. may be integrated with an integrated processor.

CNN 200 may include one or more convolution blocks. Hereinafter, for convenience, the CNN 200 includes m convolution blocks, and k is used as a variable for indicating a constant number of 1 to m. Here, the kth convolution block may include the kth initial convolution layer 211_k, the kth activation layer 212_k, and the kth pooling layer 213_k, as shown in FIG.

As described above, the configuration of the computing device 100 of the present invention and the CNN 200 included in the computing device 100 have been examined, and a method of converting the k-th initial convolution layer 211_k according to an embodiment of the present invention will be briefly described.

First, the input image used to determine the scaling parameter may be obtained by the communication unit 110. Thereafter, the computing device 100 uses (i) one or more kth initial weighting values of the kth initial convolution layer 211_k included in the kth convolution block 210_k, and (ii) the kth convolution block 210_k. With reference to the (k-1)th feature map to be processed and (iii) each kth scaling parameter of the kth scaling layer 214_k corresponding to each channel included in the (k-1)th feature map. , One or more k-th quantization loss values can be generated. Here, when k is 1, the (k−1)th feature map can indicate the input image, and is the same as the following.

The computing device 100 may further include (iv) a BW value (the BW value is a number of bits used to represent a weight value included in the CNN and a value included in the feature map in binary numbers). And (v) the kth FL value (the kth FL value is a number indicated by the LSB of the kth initial weighting value of the kth initial convolution layer and the value included in the (k-1)th feature map. (Which is the absolute value of the exponent of ), the k-th quantization loss value can be generated.

In addition, the computing device 100 may generate the kth quantization loss value according to the following formula.

Here, the mathematical formula represents a process of differentiating the quantization loss to generate the k-th quantization loss value.

In the equation, θ _p may include the (k−1)th feature map and the kth initial weighting value of the kth initial convolution layer. C _ki may be a specific k-th scaling parameter corresponding to the i-th channel included in the (k-1)th feature map among the k-th scaling parameters. FL and BW are each the FL value and the BW values, Q calculation, and the FL value and the quantized value of the _{C ki} theta _i generated by referring to the BW values and _{C ki} theta _i Can be an operation that produces the difference between.

As described above, after the kth quantization loss value is generated, the computing device 100 corresponds to each of the channels included in the (k−1)th feature map among the kth scaling parameter. Each k-th optimal scaling parameter can be determined. Specifically, the computing device 100, by selecting the C _ki be minimized the △ L _k, to determine each of the k-th best scaling parameter. A Nesterov optimization algorithm may be used to select the C _ki , but is not limited thereto.

In order to apply the Nesterov optimization algorithm, the constraint between the kth scaling parameters must be determined. Therefore, the computing device 100 may topologically sort the layers included in the CNN 200. Thereafter, the constraint condition for the kth scaling parameter corresponding to the type of each layer can be determined. However, there may be unnecessary constraints, such as duplicate constraints, among the constraints. Therefore, some constraints may be removed. Here, the connection state information between layers can be referred to in the process of removing the constraint condition.

After that, the computing apparatus 100 repeats forward passing and backward passing by the CNN 200 several times to generate 2D histograms corresponding to the weighting values included in each layer. It is possible to obtain and generate a graph of the k-th quantization loss value. The computing device 100 may determine the smallest k-th best scaling parameter corresponding to the k quantization loss value while changing the C _ki is a k-th scaling parameter, respectively. The process of changing the parameters may be according to the vector movement technique proposed by the Nesterov optimization algorithm.

When the kth optimum scaling parameter is determined by the above method, the computing device 100 determines each of the kth optimum scaling parameter corresponding to each channel included in the (k-1)th feature map. Generates the k-th scaling layer 214_k determined as a constituent element thereof, and the reciprocal of the k-th optimum scaling parameter corresponding to each channel included in the (k-1)th feature map is a constituent element thereof. The determined kth inverse scaling layer 215_k may be generated.

Hereinafter, it will be described with reference to FIG. 2 how the kth scaling layer 214_k and the kth inverse scaling layer 215_k are inserted into the kth convolution block 210_k.

Referring to FIG. 2, the kth scaling layer 214_k and the kth inverse scaling layer 215_k may be inserted at the front end and the rear end of the activation layer 212_k, respectively. This is because the exchange law is established in the operations performed by the kth activation layer 212_k, the kth scaling layer 214_k, and the kth inverse scaling layer 215_k. Here, the operation performed by the kth activation layer 212_k may be a ReLU operation, but is not limited thereto.

Mathematically,

Referring to the above formula, Sc*I. Sc can originally be added to the formula. Because Sc, that is, the scaling layer and I.S. This is because Sc and the inverse scaling layer have an inverse function relationship with each other. The Sc item and the I.D. The Sc item can be transferred to both the activation layer because the activation law and the exchange law are established.

On the other hand, when the kth scaling layer 214_k and the kth inverse scaling layer 215_k are added to the CNN 200, more computer resources are required, which is inefficient. Accordingly, the present invention presents a method of integrating a scaling layer, an initial convolution layer, and an inverse scaling layer, which will be described with reference to Figures 3a and 3b.

3a and 3b are views showing a process of generating an integrated convolution layer by switching positions of a scaling layer and an inverse scaling layer according to an embodiment of the present invention.

Referring to FIGS. 3A and 3B, the (k−1)th inverse scaling layer 215_(k−1) included in the (k−1)th convolution block 210_(k−1) is a kth convolution block. 210_k. This is because the (k-1)th pooling layer 213_(k-1) itself is not associated with a change in value.

Referring to FIG. 3b, the (k−1)th inverse scaling layer 215_(k−1), the kth initial convolution layer 211_k, and the kth scaling layer 214_k may generate the kth integrated convolution layer 216_k. Can be integrated. The computing device 100 includes (i) a (k-1)th inverse scaling layer, a result generated by applying an operation of the kth initial convolution layer and the kth scaling layer to an input value, (Ii) Parameters of the kth integrated convolution layer 216_k may be determined such that a difference between the result generated by applying the operation of the kth integrated convolution layer to an input value is smaller than a threshold value. Here, the integration process described so far multiplies the components corresponding to the (k−1)th inverse scaling layer 215_(k−1), the kth initial convolution layer 211_k, and the kth scaling layer 214_k. Processes may be included, but are not limited to.

The case where k is 1 is not shown in Fig. 3b, but since there is of course no descaling layer moved from the block before it, only the first initial convolution layer 211_1 and the first scaling layer 214_1 are integrated. It is used to generate the evolution layer 216_1.

The process described above is for generating the parameters of the k-th integrated convolutional layer 216_k optimized for quantization. Here, the quantization process is described independently of the process of generating the parameters of the kth integrated convolution layer 216_k. Therefore, the computing device 100 quantizes the weight value included in the kth convolution block 210_k, and determines the kth quantization weight as an optimized weight value for the CNN operation performed by the kth convolution block 210_k. A value can be generated. This may be performed before, during, or after the process of generating the kth integrated convolution layer 216_k.

The advantages of the optimized quantized CNN weighting value will be described with reference to FIG.

FIG. 4 is a view illustrating values of several different channels whose values do not change significantly according to a scaling method according to an exemplary embodiment of the present invention.

First, referring to FIG. 5 referred to when describing the prior art, when the method provided by the present invention is not applied, the value of the second channel included in the first feature map is converted into the first feature map. It was confirmed that it was much smaller than the value of the included first channel. On the contrary, referring to FIG. 4, it can be seen that the value of the first channel and the value of the second channel are similar. This is due to the first scaling parameter reflected in the weighting value of the first integrated convolution layer 216_1, and since the difference between the first value and the second value is not large, the first convolution block 210_2 After the operation performed, the first value and the second value can be properly quantized.

As can be understood by those of ordinary skill in the art of the present invention, the transmission and reception of the image data such as the images described above, for example, the original image, the original label, and the additional label is performed by the learning device and the test device. Data that may be performed by the unit and may be retained/maintained by the processor (and/or memory) of the learning device and the test device to perform the feature map and the operation, and the process of convolution operation, deconvolution operation, loss value operation. Can be accomplished primarily by the processor of the learning and testing devices, but the invention will not be so limited.

The embodiments of the present invention described above may be embodied in the form of program command words that can be executed through various computer components and stored in a computer-readable recording medium. The computer-readable recording medium may include program command words, data files, data structures, etc., alone or in combination. The program command stored in the computer-readable recording medium may be specially designed and constructed for the present invention, or may be known and used by those skilled in the computer software field. possible. Examples of the computer-readable recording medium include a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, and a magnetic-optical medium such as a floppy disk. (Magneto-Optical Media), and hardware devices specially configured to store and execute program command words, such as ROM, RAM, flash memory, and the like. Examples of program instruction words include not only machine language code such as that produced by a compiler, but also high level language code executed by a computer using an interpreter or the like. Said hardware device may be arranged to operate as one or more software modules for carrying out the processes according to the invention and vice versa.

The present invention has been described above with reference to specific examples such as specific components and limited examples and drawings, which are provided to help a more general understanding of the present invention. However, the present invention is not limited to the above-described embodiments, and various modifications and variations can be made from the above description by a person having ordinary knowledge in the technical field to which the present invention belongs.

Therefore, the idea of the present invention should not be limited to the above-described embodiments, and should not be limited to the scope of the claims to be described later, but may be modified equivalently or equivalently to the scope of the claims. All can be said to belong to the scope of the idea of the present invention.

Claims (14)

In a method of transforming a convolution layer of a CNN containing m convolution blocks,
(A) When the computing device obtains the input image used to determine the scaling parameter, (i) one or more th of the k th initial convolution layers included in the k th convolution block. The k initial weighting value and (ii)(ii-1)k is 1 if the input image, (ii-2)k is a constant from 2 to m, the (k-1) convolution If the (k−1)th feature map corresponding to the input image output from the block is (iii)(iii−1)k is 1, then the kth feature map corresponding to each channel included in the input image. When each of the scaling parameters and (iii-2)k is a constant from 2 to m, each of the kth scaling parameters corresponding to each of the channels included in the (k-1)th feature map is referred to, Generating one or more k th quantization loss values, where k is a constant from 1 to m;
(B) The computing device refers to the k-th quantization loss value, and among the k-th scaling parameters, the k-th corresponding to each of the channels included in the (k-1)th feature map. Determining each optimal scaling parameter;
(C) referring to the k-th optimal scaling parameter, the computing device generates a k-th scaling layer and a k-th inverse scaling layer;
(D) the computing device transforms the kth initial convolutional layer into akth integrated convolutional layer using the kth scaling layer if (i)k is 1, and (ii) When k is a constant from 2 to m, the kth initial convolution layer is converted to the kth integrated convolution layer using the kth scaling layer and the (k-1)th inverse scaling layer. Stage of doing;
A method comprising: The step (a) includes
(Iv) BW value (the BW value is the number of bits used to represent the weight value included in the CNN and the value included in the feature map in binary number) by the computing device. And (v) a kFL value (wherein the kFL value is (i) the kth initial weighting value of the kth initial convolution layer, and (ii) k is a constant from 2 to m, (K-1) value included in the feature map, where k is 1, it is the absolute value of the exponent of the number indicated by the LSB of the value included in the input image). The method of claim 1, comprising generating the kth quantization loss value. The step (a) includes
Wherein is generated the first k quantization loss value by a formula, when in theta _p is the equation is a constant to m 2 is (i) k, wherein the (k-1), wherein the map and the first k initial con If (ii) k is 1, the value of the k-th initial weighting value of the convolution feature map includes the input image and the value of the k-th initial weighting value of the k-th initial convolution feature map, and C _ki is , A specific k-th scaling parameter among the k-th scaling parameters, FL and BW are the FL value and the BW value, respectively, and a Q operation is generated by referring to the FL value and the BW value. and a calculation for generating the difference between the C _ki theta _i quantized values and C _ki theta _i of
Step (b), the computing device, wherein the △ by selecting the C _ki a L _k be minimized, according to claim 2, wherein determining a respective said first k optimal scaling parameters the method of. 4. The method of claim 3, wherein the computing device uses the Nesterov Accelerated Gradient method to select the C _ki to determine the k-th optimal scaling parameter. The step (c) includes
The computing device is
Each of the k-th optimal scaling parameters produces the k-th scaling layer whose constituents are determined, and each of the reciprocals of the k-th optimal scaling parameter produces the k-th inverse scaling layer whose constituents are determined. The method of claim 1, wherein: The step (d) includes
(1) when k is 1, (i) a result generated by applying (i) an operation of the kth initial convolution layer and the kth scaling layer to an input value; and (ii) Transforming the kth initial convolution layer into the kth integrated convolution layer such that a difference between the result generated by applying the operation of the kth integrated convolution layer to the input value is smaller than a threshold value; If (2) k is a constant of 2 or more and m or less, (i) the (k-1) inverse scaling layer, the kth initial convolution layer, and the kth scaling layer are input. The k th th such that a difference between a result generated by applying the value to the value and (ii) a result generated by applying the operation of the k th integrated convolution layer to the input value is smaller than the threshold. The method of claim 1, converting an initial convolutional layer to the kth integrated convolutional layer. (E) The computing device quantizes each weighting value of the kth integrated convolution layer included in the kth convolution block to optimize the CNN operation performed by the kth convolution block. Generating a k-th quantization weighting value as a weighting value;
The method of claim 1, further comprising: In a computing device for transforming a CNN convolution layer containing m convolution blocks,
At least one memory for storing each instruction,
(I) (i) one or more k-th initial weighting values of the k-th initial convolution layer included in the k-th convolution block, and (ii) (ii-1) k is 1, a scaling parameter The input image used to determine (ii-2)k is a constant from 2 to m, the (k-1)th corresponding to the input image output from the (k-1) convolution block. -1) If the feature map and (iii)(iii-1)k are 1, the k-th scaling parameter corresponding to each channel included in the input image, and (iii-2)k from 2 If it is a constant up to m, one or more k-th quantization loss values are generated by referring to the k-th scaling parameter corresponding to each channel included in the (k-1)th feature map. (K) is a constant number from 1 to m; (II) is included in the (k-1)th feature map among the kth scaling parameters with reference to the kth quantization loss value. And (IV) generating a kth scaling layer and a kth inverse scaling layer by referring to the kth optimal scaling parameter. (I) When k is 1, the k-th scaling layer is used to transform the k-th initial convolution layer into a k-th integrated convolution layer, and (ii) k is a constant from 2 to m. A step of converting the kth initial convolutional layer to the kth integrated convolutional layer using the kth scaling layer and the (k-1)th inverse scaling layer, if any; At least one processor configured to execute
A computing device comprising:. The process (I) is
The processor is (iv) a BW value (the BW value is the number of bits used to represent the weight value included in the CNN and the value included in the feature map in binary); and (V) kth FL value (the kth FL value is (1) the kth initial weighting value of the kth initial convolution layer, and (2) when k is a constant from 2 to m, the k-1) a value included in the feature map, where k is 1, it is the absolute value of the exponent of the number indicated by the LSB of the value included in the input image). The computing device of claim 8 including generating a k-quantization loss value. The process (I) is
The processor is
Wherein is generated the first k quantization loss value by a formula, when in theta _p is the equation is a constant to m 2 is (i) k, wherein the (k-1), wherein the map and the first k initial con If (ii) k is 1, the value of the k-th initial weighting value of the convolution feature map includes the input image and the value of the k-th initial weighting value of the k-th initial convolution feature map, and C _ki is , A specific k-th scaling parameter among the k-th scaling parameters, FL and BW are the FL value and the BW value, respectively, and a Q operation is generated by referring to the FL value and the BW value. and a calculation for generating the difference between the C _ki theta _i quantized values and C _ki theta _i of
The process (II) is
The processor is
The △ L by selecting the smallest to the C _ki and _k, computing device of claim 9, wherein determining a respective said first k optimal scaling parameter. 11. The computing device of claim 10, wherein the processor uses the Nesterov Accelerated Gradient method to select the C _ki to determine the k-th optimal scaling parameter. The process (III) is
The processor generates the kth scaling layer, each of the kth optimal scaling parameters being determined as its constituents, and each of the kth inverse scalings, wherein each reciprocal of the kth optimal scaling parameter is determined as its constituents. The computing device of claim 8, wherein the computing device generates a layer. The process (IV) is
(1) when k is 1, (i) a result generated by applying the operation of the k-th initial convolution layer and the k-th scaling layer to an input value; and (ii) the converting the k-th initial convolution layer to the k-th integrated convolution layer such that the difference between the k-integrated convolution layer operation and the result generated by applying the input value to the input value is smaller than a threshold value; ) When k is a constant of 2 or more and m or less, (i) the (k-1) inverse scaling layer, the kth initial convolution layer, and the kth scaling layer are applied to input values. The kth initial convolution layer such that the difference between the result generated by (ii) the result generated by applying the operation of the kth integrated convolution layer to the input value is smaller than the threshold value. 9. The computing device of claim 8, wherein is converted to the kth integrated convolution layer. The processor is
(V) The weighting value of the kth integrated convolution layer included in the kth convolution block is quantized, and the kth quantum is used as an optimized weighting value for the CNN operation performed by the kth convolution block. 9. The computing device of claim 8, further comprising the step of: generating a quantization weighting value.