KR20210111014A - Electronic apparatus and method for controlling thereof - Google Patents

Abstract

Disclosed is an electronic device that performs a computation of a neural network model. The present electronic device comprises: a memory in which weight data including quantized weight values of a neural network model is stored; and a processor that obtains computation data based on binary data and input data having at least one bit value different from each other, generates a lookup table that the computation data is matched to the binary data, obtains the computation data corresponding to the weight data from the lookup table, and performs computation of the neural network model based on the obtained computation data. Therefore, the present invention is capable of accurately deriving an output value for an input value within a short time.

Description

ELECTRONIC APPARATUS AND METHOD FOR CONTROLLING THEREOF

The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device operating based on artificial intelligence technology and a control method thereof.

Recently, artificial intelligence systems that implement human-level intelligence are being developed. Unlike the existing rule-based system, the artificial intelligence system is a system in which a machine learns and judges by itself, and is used in various fields such as voice recognition, image recognition, and future prediction.

In particular, recently, an artificial intelligence system for solving a given problem through a deep neural network based on deep learning has been developed.

A deep neural network is a neural network that includes a number of hidden layers between an input layer and an output layer. it means.

As such, a deep neural network generally includes a plurality of neurons in order to derive an accurate result value.

However, when there are a large amount of neurons, there is a problem in that it takes a lot of time to calculate the output value, apart from increasing the accuracy of the output value with respect to the input value.

In addition, due to a large amount of neurons, there is a problem in that the deep neural network cannot be used in a mobile device such as a smart phone having a limited memory due to a capacity problem.

The present disclosure has been devised to solve the above problems, and an object of the present disclosure is to accurately derive an output value for an input value within a short time, and to realize artificial intelligence technology in a mobile device with limited memory, etc. to provide the device.

An electronic device for performing an operation of a neural network model according to an embodiment of the present disclosure includes a memory in which weight data including quantized weight values of the neural network model is stored, and binary data and input data having at least one bit value different from each other. to obtain operation data based on the calculation data, generate a lookup table in which the operation data is matched to the binary data, obtain operation data corresponding to the weight data from the lookup table, and based on the obtained operation data, the neural network It includes a processor that performs calculations on the model.

wherein each of the binary data consists of n bit values, the input data includes a plurality of input values of a matrix, and the processor obtains n input values from each column of the matrix, The operation data may be obtained for each binary data based on the binary data and the n input values.

And, the weight data includes a plurality of weight values of a matrix, and the processor identifies n weight values corresponding to the n input values in each row of the matrix, and the identified weight values among the binary data. Binary data corresponding to n weight values may be identified, operation data corresponding to the identified binary data may be obtained from the lookup table, and the neural network model may be calculated based on the obtained operation data.

In addition, the processor determines, from among a plurality of lookup tables generated based on input values of each column of the matrix, a lookup table corresponding to each column of an output matrix for the input data, and each of the lookup tables Output values of each column of the output matrix may be obtained from

Then, the processor divides the matrix including the plurality of input values into a first matrix and a second matrix based on a predetermined row, and divides the matrix including the plurality of weight values based on a predetermined column. It is divided into a third matrix and a fourth matrix, a plurality of lookup tables are generated based on input values of each column of the first matrix, and operation data corresponding to each row of the third matrix are generated from the plurality of lookup tables. obtained, a plurality of lookup tables may be generated based on input values of each column of the second matrix, and operation data corresponding to each row of the fourth matrix may be obtained from the plurality of lookup tables.

The processor may obtain eight input values from each column of the matrix, and obtain the operation data for each binary data based on the binary data and the eight input values.

And, in the plurality of arithmetic expressions based on the binary data and the n input values, the processor may be configured to, when there is a first arithmetic expression and a second arithmetic expression having the same intermediate arithmetic expression, the operation of the second arithmetic expression is performed in the first 1 It can be performed based on the calculation value of the calculation expression.

According to an embodiment of the present disclosure, a control method of an electronic device performing an operation of a neural network model includes: acquiring operation data based on binary data and input data having at least one bit value different from each other; generating a lookup table to which data is matched; obtaining computation data corresponding to weight data including quantized weight values of the neural network model from the lookup table; performing an operation.

Here, each of the binary data consists of n bit values, the input data includes a plurality of input values of a matrix, and the obtaining of the operation data includes n input values in each column of the matrix. values may be obtained, and the operation data may be obtained for each binary data based on the binary data and the n input values.

In addition, the weight data includes a plurality of weight values of a matrix, and the operation of the neural network model includes identifying n weight values corresponding to the n input values in each row of the matrix, Identifies binary data corresponding to the identified n weight values among the binary data, obtains computational data corresponding to the identified binary data from the lookup table, and based on the obtained computational data, the neural network model operation can be performed.

And, in the operation of the neural network model, a lookup table corresponding to each column of the output matrix for the input data is determined from among a plurality of lookup tables generated based on input values of each column of the matrix. and obtaining output values of each column of the output matrix from the respective lookup tables.

The obtaining of the operation data may include dividing the matrix including the plurality of input values into a first matrix and a second matrix based on a predetermined row, and dividing the matrix including the plurality of weight values into a predetermined matrix. It is divided into a third matrix and a fourth matrix based on a column, and a plurality of lookup tables are generated based on input values of each column of the first matrix, and from the plurality of lookup tables, each row of the third matrix is Obtaining corresponding operation data, generating a plurality of lookup tables based on input values of each column of the second matrix, and obtaining operation data corresponding to each row of the fourth matrix from the plurality of lookup tables can

In addition, the obtaining of the operation data may include obtaining eight input values from each column of the matrix, and obtaining the operation data for each binary data based on the binary data and the eight input values.

In addition, the generating of the lookup table may include, in a plurality of arithmetic expressions based on the binary data and the n input values, when there is a first arithmetic expression and a second arithmetic expression having the same intermediate arithmetic expression, the second arithmetic expression The operation of may include performing the operation based on the operation value of the first expression.

According to various embodiments of the present disclosure as described above, it is possible to accurately derive an output value for an input value within a short time, and it is possible to realize artificial intelligence technology even in a mobile device having a limited memory.

1 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.
2A is a diagram illustrating a matrix corresponding to input data according to an embodiment of the present disclosure.
2B is a diagram illustrating a lookup table according to an embodiment of the present disclosure.
3A is a diagram for explaining the operation of a neural network model using a lookup table according to an embodiment of the present disclosure.
3B is a diagram for describing a lookup table used for each column of a matrix corresponding to output data according to an embodiment of the present disclosure.
3C is a diagram for explaining an embodiment of obtaining an output value of a first column of a matrix corresponding to output data according to an embodiment of the present disclosure;
3D is a diagram for explaining an embodiment of obtaining an output value of a second column of a matrix corresponding to output data according to an embodiment of the present disclosure.
3E is a diagram for describing an example of obtaining an output value of a third column of a matrix corresponding to output data according to an embodiment of the present disclosure;
4A is a diagram for explaining an arithmetic expression used to generate a lookup table according to an embodiment of the present disclosure.
4B is a diagram for explaining the same intermediate expression included in a plurality of expression expressions according to an embodiment of the present disclosure.
4C is a diagram for explaining the same intermediate expression included in a plurality of expression expressions according to an embodiment of the present disclosure.
4D is a diagram for explaining the same intermediate expression included in a plurality of expression expressions according to an embodiment of the present disclosure.
4E is a diagram for explaining the same intermediate expression included in a plurality of expression expressions according to an embodiment of the present disclosure.
4F is a diagram for explaining an embodiment of obtaining an operation value based on the same intermediate operation expression according to an embodiment of the present disclosure.
5 is a diagram for explaining a method of calculating a neural network model according to an embodiment of the present disclosure.
6 is a detailed block diagram illustrating an electronic device according to an embodiment of the present disclosure.
7 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

Hereinafter, various embodiments of the present disclosure will be described with reference to the accompanying drawings. However, this is not intended to limit the technology described in the present disclosure to specific embodiments, and it should be understood that various modifications, equivalents, and/or alternatives of the embodiments of the present disclosure are included. . In connection with the description of the drawings, like reference numerals may be used for like components.

In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

In this disclosure, expressions such as “A or B,” “at least one of A and/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, "A or B," "at least one of A and B," or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.

As used in the present disclosure, expressions such as "first," "second," "first," or "second," may modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component) When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element). On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.

The expression "configured to (or configured to)" as used in this disclosure depends on the context, for example, "suitable for," "having the capacity to" ," "designed to," "adapted to," "made to," or "capable of." The term “configured (or configured to)” may not necessarily mean only “specifically designed to” in hardware. Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase “a processor configured (or configured to perform) A, B, and C” refers to a dedicated processor (eg, an embedded processor) for performing the operations, or by executing one or more software programs stored in a memory device. , may mean a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

In the present disclosure, a “module” or “unit” performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of “modules” or a plurality of “units” are integrated into at least one module and implemented with at least one processor (not shown) except for “modules” or “units” that need to be implemented with specific hardware. can be

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 1 , an electronic device 100 according to an embodiment of the present disclosure includes a memory 110 and a processor 120 .

The electronic device 100 according to an embodiment of the present disclosure is a device for obtaining output data for input data using a neural network model (or artificial intelligence model), and for example, the electronic device 100 is a desktop PC. , a laptop, a smart phone, a tablet PC, a server, and the like. Alternatively, the electronic device 100 may be a system itself in which a cloud computing environment is built. However, the present invention is not limited thereto, and the electronic device 100 may be any device as long as it is capable of calculating a neural network model.

The memory 110 may be implemented as a hard disk, non-volatile memory, or volatile memory. Here, the non-volatile memory may be one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, etc. The memory may be a dynamic RAM (DRAM), a static RAM (SRAM), or a synchronous dynamic RAM (SDRAM).

Meanwhile, although the memory 110 is illustrated as a separate configuration from the processor 120 in FIG. 1 , the memory 110 may be included in the processor 120 . That is, the memory 110 may be implemented as an off-chip memory as well as an on-chip memory.

Also, although one memory 110 is illustrated in FIG. 1 , a plurality of memories 110 may be implemented according to an embodiment.

The memory 110 may store weight data of the neural network model. Here, the weight data is data used for calculation of the neural network model, and the memory 110 may store a plurality of weight data corresponding to a plurality of layers constituting the neural network model.

In particular, the memory 110 may store weight data including quantized weight values. Here, the quantized weight value may be -1 or 1, and the weight data may be expressed as an m×n matrix composed of -1 or 1. Also, the weight value -1 may be replaced with 0 and stored in the memory 110 . That is, the memory 110 may store weight data composed of 0 or 1. According to an embodiment, weight data including weight values of -1 or 1 is stored in a first memory (eg, a hard disk), and weight data including weight values of 0 or 1 is stored in a second memory (eg, SDRAM) ) can be stored in Here, weight values of 0 or 1 may be used for calculation of a neural network model, which will be described later.

Meanwhile, the quantization of the neural network model may be performed by the processor 120 of the electronic device 100 as well as by an external device (eg, a server). When the quantization of the neural network model is performed by the external device, the processor 120 may receive weight data including the quantized weight value from the external device and store it in the memory 110 . The quantization method of the neural network model will be described later.

Such a neural network model may be a model based on a neural network. As an example, the neural network model may be a model based on a recurrent neural network (RNN). Here, RNN means a recurrent neural network, and is a kind of deep learning model for learning data that changes over time, such as time series data.

However, the present invention is not limited thereto, and the neural network model includes a variety of models such as Convolutional Neural Network (CNN), Deep Neural Network (DNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), or Bidirectional Recurrent Deep Neural Network (BRDNN). It can be a model based on the network. Alternatively, the memory 110 may store a model generated based on a rule rather than a model learned through an artificial intelligence algorithm, and there is no particular limitation on the model stored in the memory 110 .

The processor 120 controls the overall operation of the electronic device 100 . To this end, the processor 120 may be composed of one or a plurality of processors. Here, one or more processors may be general-purpose processors such as a central processing unit (CPU), etc., as well as a graphics-only processor or neural network processing unit such as a graphic processing unit (GPU). It can be a processor dedicated to artificial intelligence, such as a neural network processing unit (NPU). In addition, the processor 120 may be a system on chip (SoC) (eg, an on-device AI chip), a large scale integration (LSI), or a field programmable gate array (FPGA).

The processor 120 may quantize a weight value of the neural network model. Specifically, when the processor 120 quantizes the weight value with k bits, the processor 120 may quantize the weight value of the neural network model through various quantization algorithms satisfying Equation 1 below.

[Equation 1]

(Here, w is a weight value before quantization, a is a scaling factor, and b is a quantized weight value, which may be -1 or +1.)

For example, the processor 120 may quantize the weight value through a greedy algorithm. In this case, the processor 120 may obtain a scaling factor and a quantized weight value when k=1 in Equation 1 above based on Equation 2 below.

[Equation 2]

(where w is the weight value before quantization, a* is a scaling factor in the case of k=1, b* is the quantized weight value in the case of k=1, which is -1 or +1, and n is 1 or more It can be an integer.)

In addition, the processor 120 may obtain a scaling factor and a quantized weight value when k=i (1 < i ≤ k) by repeatedly calculating Equation 3 below. That is, the processor 120 uses r that is the difference between the weight value before quantization and the quantized weight value when k=1, and the scaling factor and the quantized weight value when k=i (1 < i ≤ k). can be obtained.

[Equation 3]

(Where w is the weight value before quantization, a is the scaling factor, b is the quantized weight value, which is -1 or +1, and r is the weight value before quantization and the quantized weight value when k=1. It can make a difference.)

Accordingly, the electronic device 100 may store weight data including a scaling factor and weight values quantized to -1 or 1 in the memory 110 . Meanwhile, although an embodiment of quantizing through a greedy algorithm has been described above, there is no particular limitation on a method of quantizing a weight value. For example, quantization may be performed through various algorithms, such as unitary quantization, adaptive quantization, uniform quantization, or supervised iterative quantization.

The processor 120 may obtain output data for the input data based on the quantized weight values of the neural network model. Here, the input data may be text, an image, or a user's voice. For example, the text may be text input through an input unit (not shown) such as a keyboard or a touchpad of the electronic device 100 , and the image may be an image captured by the camera of the electronic device 100 . . Also, the user's voice may be a user's voice input into the microphone of the electronic device 100 .

Meanwhile, the output data may be different according to the type of the input data and/or the neural network model. That is, the output data may be different depending on which input data is input to which neural network model. For example, when the neural network model of the present disclosure is a model for language translation, the processor 120 may obtain output data expressed in a second language with respect to input data expressed in a first language. Alternatively, when the neural network model of the present disclosure is a model for image analysis, the processor 120 may input an image as input data of the neural network model, and obtain information about an object detected in the image as output data. Also, when the neural network model of the present disclosure is a model for voice recognition, the processor 120 may acquire a user voice as input data and a text corresponding to the user voice as output data. Meanwhile, the above-described output data is an example, and the type of output data of the present disclosure is not limited thereto.

To this end, when input data is input, the processor 120 may express the input data as a matrix (or a vector or a tensor) including a plurality of input values. Here, a method of expressing the input data as a matrix (or a vector or a tensor) may be different depending on the type and type of the input data. For example, when text (or text obtained by converting a user's voice) is input as input data, the processor 120 expresses the text as a vector through one hot encoding or word embedding. The text can be expressed as a vector. Here, the one-hot encoding is a method in which only the index value of a specific word is expressed as 1 and the values of the remaining indices are expressed as 0, and word embedding is a method in which words are mistakenly expressed as a dimension of a vector set by the user (eg, 128 dimensions). way of expressing it. As a word embedding method, for example, Word2Vec, FastText, Glove, etc. may be used. Meanwhile, when an image is input as input data, the processor 120 may represent each pixel of the image as a matrix. For example, the processor 120 expresses each pixel of the image as a value of 0 to 255 for each RGB color, or expresses the image as a matrix by dividing a value expressed by 0 to 255 by a preset value (eg, 255). can

The processor 120 may obtain at least one intermediate data of the input data and obtain output data of the at least one intermediate data based on the quantized weight values and the input value of the input data.

Here, in the present disclosure, unlike the conventional electronic device in which a matmul operation is performed on a plurality of quantized weight values and a plurality of input values to obtain output data on the input data, a lookup table is used for the input data. Output data can be obtained.

This is to prevent a latency problem and memory overload phenomenon occurring during multiple matmul operations. First, a lookup table of the present disclosure will be described in detail with reference to FIGS. 2A and 2B .

2A and 2B are diagrams for explaining a lookup table according to an embodiment of the present disclosure.

As described above, the processor 120 may obtain input data in the form of a matrix (or a vector or a tensor) including a plurality of input values. As an example, the processor 120 may obtain a 4×3 matrix including a plurality of input values as shown in FIG. 2A . Hereinafter, a matrix including a plurality of input values is referred to as an input matrix (or a matrix corresponding to input data).

The processor 120 may generate a lookup table based on input values of the input data and binary data. Here, the binary data may be data composed of n bit values having a value of 0 or 1. And, the number of binary data may be 2^n. For example, 2-bit binary data is data composed of two-bit values having a value of 0 or 1, and may be one of 00, 01, 10, and 11. As another example, 4-bit binary data is data composed of 4 bit values having a value of 0 or 1, and is 0000, 0001, 0010, 0011, 0100, 0101, 0110, 0111, 1000, 1001, 1010, 1011, 1100. , 1101, 1110, or 1111.

Specifically, the processor 120 may obtain n input values from each column of the input matrix, and may obtain operation data based on bit values of binary data and the obtained n input values. Here, the bit value of the binary data may be 0 or 1 as described above, and when the bit value of the binary data is 0, the processor 120 calculates -1 to the input value, and the bit value of the binary data If this is 1, 1 can be calculated on the input value. In addition, the processor 120 may generate the lookup table by matching the operation data obtained for each binary data.

For example, referring to FIG. 2 , when n = 2, the processor 120 may obtain input values 0.03 and −0.17 of the first and second rows from the first column of the input matrix. Then, the processor 120 may calculate -(0.03)-(-0.17) to obtain an operation value of 0.14, and match the operation value 0.14 to the binary data 00. Similarly, the processor 120 may calculate -(0.03)+(-0.17) to obtain the operation value -0.20, match the operation value -0.20 to the binary data 01, and (0.03)-(-0.17). ) to obtain the operation value 0.20, the operation value 0.20 can be matched to binary data 10, (0.03)+(-0.17) is calculated to obtain the operation value -0.14, and the operation value -0.14 is binary data 11 can be matched. In addition, the processor 120 may obtain input values 0.20 and −0.17 of the third and fourth rows in the first column. Then, the processor 120 may calculate -(0.20)-(0.17) to obtain an operation value -0.37, and match the operation value -0.37 to the binary data 00. Similarly, the processor 120 may calculate -(0.20)+(0.17) to obtain an operation value of -0.03, match the operation value -0.03 to binary data 01, and obtain (0.20)-(0.17) Calculate to obtain an operation value of 0.03, match the operation value 0.03 to binary data 10, calculate (0.20)+(0.17) to obtain an operation value 0.37, and match the operation value 0.37 to binary data 11 have. In a similar manner, the processor 120 may obtain operation values for the input values of the second column and the input values of the third column of the input matrix, and match the operation values to the binary data. Meanwhile, the above-described case where n = 2 is an embodiment, and n may be changed according to a user setting.

Accordingly, the processor 120 may generate at least one lookup table for each column of the input matrix. As described above, if the input matrix is a 4×3 matrix and n = 2, the processor 120 generates two lookup tables for each column of the input matrix, that is, a total of six lookup tables as shown in FIG. 2B. can If n = 4, the processor 120 may generate one lookup table for each column of the input matrix, that is, a total of three lookup tables.

3A to 3E are diagrams for explaining the operation of a neural network model using a lookup table according to an embodiment of the present disclosure.

As described above, the processor 120 may obtain the output data based on the quantized weight values and the input values of the input data. Specifically, the processor 120 is configured to obtain output data for the input data X based on the calculation of the weight data W (which may include a scaling factor A and a quantized weight value B) and the input data X. can For example, if the weight values of the weight data are quantized to 3 bits, the processor 120 may obtain output data for the input data X based on Equation 4 below.

[Equation 4]

Equation 4 is expressed in a matrix form as follows.

For example, when k=1, the scaling factor Ao, the quantized weight Bo, and the input data X may have values as shown in FIG. 3A . Here, unlike the conventional electronic device that obtains the calculated value of B*X through the matmul operation, the present disclosure may obtain the calculated value of B*X through reference to a lookup table. Hereinafter, for convenience of explanation, a method of obtaining the operation value of Bo*X will be described, but the following technical idea can be applied even when obtaining an operation value of Bn*X such as B1*X, B2*X, etc. will see

Referring to FIG. 3B , the output matrix including the operation values of Bo*X may have output values of y1 to y109. Here, in order to obtain an output value, the processor 120 may determine a lookup table corresponding to each column of the output matrix from among a plurality of lookup tables generated for each column of the input matrix. Specifically, the processor 120 may determine the lookup table generated based on the input matrix of the same column as the column of the output matrix as the lookup table for obtaining the output value of the output matrix. As an example, referring to FIG. 3B , the processor 120 generates a first lookup table 311 and a second lookup table 312 generated based on input values of a first column of an input matrix among a plurality of lookup tables, It may be determined as a lookup table for obtaining output values of the first column of the output matrix.

In addition, the processor 120 may identify n weight values corresponding to n input values in each row of a weight matrix including a weight value of 0 or 1. That is, when the lookup table is generated by obtaining n input values from each column of the input matrix, the processor 120 may identify n weight values corresponding to the n input values in each row of the weight matrix. . For example, as described above, when n=2, the processor 120 receives two input values for each row in a matrix including weight values (hereinafter, referred to as a weight matrix or a matrix corresponding to weight data). Two corresponding weight values may be identified. That is, in FIG. 3C , the processor 120 identifies 1 and 0 and, 0 and 1 in the first row of the weight matrix, and 0 and 1 and 1 and 0 in the second row, and similarly the rest Two weight values can be identified in the row.

Then, the processor 120 may identify binary data corresponding to the identified weight values from among the binary data. Here, the identified binary data is data including the same bit value as the weight value. For example, as shown in FIG. 3C , if the identified weight values are 1 and 0, the binary data corresponding to the weight values is may be 10, and if the identified weight values are 0 and 1, binary data corresponding to the weight values may be 10.

Then, the processor 120 may obtain an operation value corresponding to the binary data identified from the lookup table.

To this end, the processor 120 may determine a lookup table including operation values corresponding to the identified weight values from among the plurality of lookup tables (eg, the above-described first and second lookup tables 311 and 312 ). have. Specifically, if the identified weight values are values included in the k column of the weight matrix, the processor 120 uses a lookup table generated based on input values of k rows of the input matrix from among the plurality of lookup tables as the identified weight values. It can be determined as a lookup table including operation values corresponding to . For example, referring to FIG. 3C , if the identified weight values are values included in the first and second columns of the weight matrix, the processor 120 selects the input matrix from among the first and second lookup tables 311 and 312 . The first lookup table 311 generated based on the input values of the first and second rows of R is determined as a lookup table including operation values corresponding to the identified weight values, and the identified weight values are used as the first lookup table of the weight matrix. If the values are included in the third and fourth columns, the second lookup table 312 generated based on the input values of the fourth and fourth rows of the input matrix among the first and second lookup tables 311 and 312 is used. It may be determined as a lookup table including operation values corresponding to the identified weight values.

Then, the processor 120 obtains an operation value of 0.20 matched to the binary data 10 identified from the first lookup table 311 , and an operation value -0.37 matched to the binary data 01 identified from the second lookup table 312 . , and the y1 value of the output matrix can be obtained by summing the operation values 0.20 and 0.37. Similarly, for the second row of the weight matrix, the processor 120 obtains an operation value −0.20 matched to the binary data 01 identified from the first lookup table 311 , and obtains an operation value −0.20 identified from the second lookup table 312 . An operation value of 0.37 matched to binary data 10 is obtained, and the y4 value of the output matrix can be obtained by summing the operation values -0.20 and 0.37, and in the case of the nth row of the weight matrix, the output value is similarly obtained can be obtained.

In addition, in the case of output values of the second column of the output matrix, as shown in FIG. 3D , it may be obtained from the third and fourth lookup tables 313 and 314 , and output values of the third column of the output matrix In this case, as shown in FIG. 3E , they may be obtained from the fifth and sixth lookup tables 315 and 316 . Here, since the above-described technical idea can be applied, a detailed description will be omitted.

Thereafter, when the output values of the output matrix are obtained, the processor 120 may calculate the output matrix and the scaling factor, thereby obtaining the result value of Equation 4 described above. Then, the processor 120 may output the final output data using the obtained result value. Here, as described above, if the neural network model is a model for language translation, the output data may be text in a language different from the text input as input data, and if the neural network model is a model for image analysis, it is included in the image. It may be data including information about the object that has been used, but is not necessarily limited thereto.

As such, the present disclosure obtains output values through a lookup table without matmul operation of quantized weight values and input values of input data, thereby preventing latency problems and memory overload due to a huge amount of computation.

4A to 4F are diagrams for explaining an arithmetic expression used to generate a lookup table according to an embodiment of the present disclosure.

As described above, the processor 120 may generate the lookup table by obtaining operation values through a plurality of operation expressions based on binary data and n input values, and matching the operation values for each binary data.

Here, a plurality of arithmetic expressions based on binary data and n input values may include the same intermediate arithmetic expression. For example, when n=8, a plurality of arithmetic expressions based on 8-bit binary data and 8 input values (x0 to x7) may be as shown in FIG. 4A . That is, when n=8, the formula for obtaining the operation value Ro corresponding to the binary data 00000000 is -x0-x1-x2-x3-x4-x5-x6-x7, and the operation value R1 corresponding to the binary data 00000001 An arithmetic expression for obtaining is -x0-x1-x2-x3-x4-x5-x6+x7, and similarly, there may be a plurality of arithmetic expressions for obtaining a plurality of operation values corresponding to each of R2 to R255. In this case, the processor 120 may use the result value of the intermediate arithmetic expression equally included in the plurality of arithmetic expressions when obtaining operation values based on the plurality of arithmetic expressions.

Specifically, referring to FIG. 4B , an equation for obtaining Ro -x0-x1-x2-x3-x4-x5-x6-x7 and an equation for obtaining R1 -x0-x1-x2-x3-x4 -x5-x6-x7 contains the same intermediate expression -x0-x1-x2-x3-x4-x5-x6. And, referring to FIG. 4C , an arithmetic expression -x0-x1-x2-x3-x4-x5-x6-x7 for obtaining Ro and an arithmetic expression -x0-x1-x2-x3-x4- for obtaining R2 x5+x6-x7 includes the same intermediate expression -x0-x1-x2-x3-x4-x5-x7, and the expression -x0-x1-x2-x3-x4-x5-x6- for obtaining R1 The arithmetic expressions -x0-x1-x2-x3-x4-x5+x6+x7 for obtaining x7 and R3 may include the same intermediate arithmetic expression x0-x1-x2-x3-x4-x5+x7. Also, as shown in FIGS. 4D and 4E , a plurality of arithmetic expressions for obtaining arithmetic values may include a plurality of arithmetic expressions including the same intermediate arithmetic expression.

In this case, the processor 120 may perform the operation of any one of the plurality of arithmetic expressions having the same intermediate arithmetic expression based on the operation value of the other arithmetic expression. That is, as in the above-described embodiment, the arithmetic expression -x0-x1-x2-x3-x4-x5-x6-x7 for obtaining Ro and the arithmetic expression -x0-x1-x2-x3-x4 for obtaining R1 When -x5-x6-x7 includes the same intermediate expression -x0-x1-x2-x3-x4-x5-x6, the processor 120 determines that Ro is the expression -x0-x1-x2-x3-x4- It is obtained through x5-x6-x7, and R1 can be obtained by adding 2*x7 to the calculated value of R0 (ie, R1=R0+2x7). In a similar way, the processor 120 may obtain values of R2 to R255, and consequently, the processor 120 may obtain operation values through a plurality of equations shown in FIG. 4F . Meanwhile, here, the case where n is 8 has been described as an embodiment, but it will be seen that the above-described technical idea can be applied as it is even when n is another integer.

As such, in the case of including the same arithmetic expression, the present disclosure can significantly reduce the amount of computation of the processor for generating the lookup table by performing the operation of one arithmetic expression using the operation value of the other arithmetic expression. there is an effect

5 is a diagram for explaining a method of calculating a neural network model according to an embodiment of the present disclosure.

As described above, the processor 120 may generate a look-up table and perform an operation of the neural network model based on the look-up table. Here, the processor 120 may generate a plurality of lookup tables for all input values of the input data, and may perform an operation of the neural network model based on the plurality of lookup tables, as well as some input values of the input data. After generating some lookup tables for , some lookup tables for the neural network model are performed based on some lookup tables, some lookup tables for the remaining input values of the input data are generated, and the rest of the neural network model is calculated based on some lookup tables. can also be performed.

Specifically, the processor 120 divides the input matrix into a first input matrix and a second input matrix based on a predetermined row, and divides the weight matrix into a third weight matrix and a fourth weight matrix based on a predetermined column. can do. Here, the preset row may be an n/2 row if the number of rows in the input matrix is n, and the preset column may be an n/2 column if the number of columns in the weight matrix is n. no.

Then, the processor 120 generates a plurality of lookup tables based on input values of each column of the first input matrix, and obtains operation data corresponding to each row of the third weight matrix from the plurality of lookup tables, A plurality of lookup tables may be generated based on input values of each column of the second input matrix, and operation data corresponding to each row of the fourth weight matrix may be obtained from the plurality of lookup tables. Here, the above-described technical idea may be applied to a method of generating a lookup table and obtaining operation data based on the lookup table, and thus, detailed omissions will be omitted.

For example, referring to FIG. 5 , when the number of rows of the input matrix and the number of columns of the weight matrix are 512, the processor 120 divides the input matrix into an input matrix X1 and an input matrix X2 based on the row 256, and weights The matrix may be partitioned into a weight matrix W1 and a weight matrix W2 based on column 256 . Then, the processor 120 generates a lookup table based on the input values of the input matrix X1, obtains operation values corresponding to the weight matrix W1 from the lookup table, and creates a lookup table based on the input values of the input matrix X2. generated, and calculated values corresponding to the weight matrix W2 may be obtained from the lookup table. As described above, by partially acquiring a plurality of operation values through a divided matrix, the present disclosure can prevent a memory overhead problem and efficiently use a memory. In addition, according to an embodiment, when a plurality of processors 120 are implemented, calculation values corresponding to the weight matrix W1 from the lookup table based on the input matrix X1 and the weight matrix W2 from the lookup table based on the input matrix X2 By parallelly acquiring corresponding operation values through a plurality of processors, it is possible to reduce the time required for the operation.

6 is a detailed block diagram illustrating an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 6 , a first memory 110 , a processor 120 , a lookup table generator 130 , a second memory 140 and a multiplier according to an embodiment of the present disclosure ( 150) may be included.

The first memory 110 may store an input matrix, a scaling factor for calculation of a neural network model, and a weight matrix. Here, the input matrix may include a plurality of input values as described above, and the weight matrix may include weight values quantized to 0 or 1.

The lookup table generator 130 may load the input matrix from the first memory 110 . In addition, the lookup table generator 130 may obtain operation values for the input values of the input matrix for each binary data. Specifically, when the lookup table generator 130 generates a lookup table of n-bit binary data, it obtains n input values from each column of the input matrix, and each binary data based on the binary data and the n input values. arithmetic values can be obtained. In addition, the lookup table generator 130 may match and store information on columns and rows of the input matrix, which is a basis for generating the lookup table, in the lookup table. Here, the column information may be used when determining a lookup table corresponding to each column of the output matrix from among a plurality of lookup tables generated for each column of the input matrix. Also, the row information may be used when determining a lookup table including an operation value corresponding to each column of the weight matrix from among a plurality of lookup tables corresponding to each column of the output matrix.

Meanwhile, according to an embodiment, the lookup table generator 130 may generate a lookup table based on 8-bit binary data. Specifically, the lookup table generator 130 may obtain 8 input values from each column of the input matrix, and may obtain 8-bit binary data and operation data for each binary data based on the 8 input values. This is in consideration that the processor 120 such as a CPU processes data in units of bytes, and accordingly, the present disclosure does not separately perform a shift operation for processing bit-level data as data in units of bytes. overload can be prevented.

The second memory 140 may store at least one lookup table. Here, the second memory 140 may be a scratchpad memory (SPM) that temporarily stores data such as a lookup table, but is not limited thereto.

The processor 120 may load the weight matrix from the first memory 110 and load the lookup table from the second memory 140 . Then, the processor 120 obtains an operation value of the weight values of the weight matrix from the lookup table, accumulates it in an accumulator, and an output value of the output matrix (that is, based on the summation of the operation values accumulated in the accumulator) weight matrices B * input matrices X) can be obtained. In addition, the processor 120 may store information on the output values of the output matrix in the first memory 110 . Thereafter, the multiplier 150 may load the output values and the scaling factor stored in the first memory 110 , and perform a multiplication operation of the output values and the scaling factor.

Meanwhile, although the first memory 110 and the second memory 120 are illustrated as separate components in FIG. 6 , the first memory 110 and the second memory 120 may be one memory. In addition, the first memory 110 and the second memory 120 may be configured separately from the processor 120 , but may also be included in the processor 120 . In addition, the lookup table generator 130 may be implemented as a separate operator for generating the lookup table, and the function of the lookup table generator may be mounted in the processor 120 . In addition, the multiplier 150 may be implemented as a separate operator that multiplies an operation value obtained from a lookup table by a scaling factor, as well as a function of the multiplier 150 may be mounted in the processor 120 .

Meanwhile, an embodiment of generating a lookup table based on an input value of input data has been described above. However, the electronic device 100 according to an embodiment of the present disclosure may generate a lookup table based on a weight value of weight data by applying the above-described method in reverse. That is, the processor 120 may quantize the input values of the input data while leaving the weight value as it is (ie, processing it as a real value). Then, the processor 120 may generate a lookup table in which an operation value is matched for each binary data based on the weight values and n-bit binary data, and obtain operation data corresponding to the input data from the lookup table. The lookup table based on such weight data may be used for calculation of a language model having a small size of input data and a large size of weight data. Meanwhile, the lookup table based on the above-described input data in FIG. 2 and the like may be used to calculate an image model having a large input data size and a small weight data size.

7 is a diagram for explaining a method of controlling an electronic device according to an embodiment of the present disclosure.

The electronic device 100 may obtain operation data based on binary data and input data having at least one bit value different from each other ( S710 ). Here, the binary data may be data configured with a bit value of 0 or a bit value of 1, the input data may be data including a plurality of input values, and the operation data may be data including a plurality of operation values. In addition, each of the input data and the operation data may be expressed as a matrix. Specifically, the electronic device 100 may obtain n input values from each column of the input matrix, and may obtain binary data and operation data for each binary data based on the n input values.

Then, the electronic device 100 may generate a lookup table in which operation data is matched to binary data ( S720 ), and may obtain operation data corresponding to weight data from the lookup table ( S730 ). Here, the weight data may include a plurality of weight values of a matrix. Specifically, the electronic device 100 may identify n weight values corresponding to n input values in each row of the weight matrix, and may identify binary data corresponding to the n weight values identified from among binary data. Then, the electronic device 100 may obtain operation data corresponding to the identified binary data from the lookup table, and may perform an operation of the neural network model based on the obtained operation data ( S740 ).

Meanwhile, the above-described methods according to various embodiments of the present disclosure may be implemented in the form of software or applications that can be installed in an existing electronic device.

In addition, a non-transitory computer readable medium in which a program for sequentially performing a control method of an electronic device according to the present disclosure is stored may be provided.

The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, etc., and can be read by a device. Specifically, the above-described various applications or programs may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: electronic device
110: memory
120: processor

Claims (14)

In an electronic device for performing computation of a neural network model,
a memory storing weight data including quantized weight values of the neural network model; and
Obtaining operation data based on binary data and input data having at least one bit value different from each other, generating a lookup table in which the operation data is matched to the binary data, and operation data corresponding to the weight data from the lookup table and a processor configured to perform an operation of the neural network model based on the obtained operation data. According to claim 1,
Each of the binary data consists of n bit values,
The input data includes a plurality of input values of a matrix,
The processor is
obtain n input values in each column of the matrix,
and acquiring the operation data for each binary data based on the binary data and the n input values. 3. The method of claim 2,
The weight data includes a plurality of weight values of a matrix,
The processor is
identify n weight values corresponding to the n input values in each row of the matrix, identify binary data corresponding to the identified n weight values among the binary data, and identify the identified binary data from the lookup table Obtaining operation data corresponding to the data, and performing operation of the neural network model based on the obtained operation data. 4. The method of claim 3,
The processor is
From among a plurality of lookup tables generated based on input values of each column of the matrix, a lookup table corresponding to each column of an output matrix for the input data is determined, respectively, and each of the output matrices from the respective lookup tables is determined. An electronic device for obtaining output values of a column. 4. The method of claim 3,
The processor is
The matrix including the plurality of input values is divided into a first matrix and a second matrix based on a predetermined row, and a matrix including the plurality of weight values is divided into a third matrix and a fourth matrix based on a predetermined column is divided into
generating a plurality of lookup tables based on input values of each column of the first matrix, and obtaining operation data corresponding to each row of the third matrix from the plurality of lookup tables;
generating a plurality of lookup tables based on input values of each column of the second matrix, and obtaining operation data corresponding to each row of the fourth matrix from the plurality of lookup tables. 3. The method of claim 2,
The processor is
get 8 input values in each column of the matrix,
and acquiring the operation data for each binary data based on the binary data and the eight input values. 3. The method of claim 2,
The processor is
In a plurality of arithmetic expressions based on the binary data and the n input values, when there are a first arithmetic expression and a second arithmetic expression having the same intermediate arithmetic expression, the operation of the second arithmetic expression is the operation value of the first arithmetic expression based on the electronic device. In the control method of an electronic device for performing calculation of a neural network model,
obtaining operation data based on binary data and input data having at least one bit value different from each other;
generating a lookup table in which the operation data is matched to the binary data;
obtaining computation data corresponding to weight data including quantized weight values of the neural network model from the lookup table; and
and performing an operation of the neural network model based on the obtained operation data. 9. The method of claim 8,
Each of the binary data consists of n bit values,
The input data includes a plurality of input values of a matrix,
The step of obtaining the operation data includes:
obtain n input values in each column of the matrix,
The method for controlling an electronic device, wherein the operation data is obtained for each binary data based on the binary data and the n input values. 10. The method of claim 9,
The weight data includes a plurality of weight values of a matrix,
The operation of the neural network model comprises:
identify n weight values corresponding to the n input values in each row of the matrix, identify binary data corresponding to the identified n weight values among the binary data, and identify the identified binary data from the lookup table A method of controlling an electronic device, obtaining computational data corresponding to the data, and performing computation of the neural network model based on the acquired computational data. 11. The method of claim 10,
The operation of the neural network model comprises:
From among a plurality of lookup tables generated based on input values of each column of the matrix, a lookup table corresponding to each column of an output matrix for the input data is determined, respectively, and each of the output matrices from the respective lookup tables is determined. A method of controlling an electronic device, comprising: obtaining output values of a column. 11. The method of claim 10,
The step of obtaining the operation data includes:
The matrix including the plurality of input values is divided into a first matrix and a second matrix based on a predetermined row, and a matrix including the plurality of weight values is divided into a third matrix and a fourth matrix based on a predetermined column is divided into
generating a plurality of lookup tables based on input values of each column of the first matrix, and obtaining operation data corresponding to each row of the third matrix from the plurality of lookup tables;
generating a plurality of lookup tables based on input values of each column of the second matrix, and obtaining operation data corresponding to each row of the fourth matrix from the plurality of lookup tables. 10. The method of claim 9,
The step of obtaining the operation data includes:
get 8 input values in each column of the matrix,
The method for controlling an electronic device, wherein the operation data is obtained for each binary data based on the binary data and the eight input values. 10. The method of claim 9,
Creating the lookup table comprises:
In a plurality of arithmetic expressions based on the binary data and the n input values, when there are a first arithmetic expression and a second arithmetic expression having the same intermediate arithmetic expression, the operation of the second arithmetic expression is the operation value of the first arithmetic expression A method of controlling an electronic device, including;

Images