KR102447445B1 - Calculation device for efficient parallel processing of matrix operation and memory device including the same - Google Patents

Abstract

The computing device includes one or more operator groups including a plurality of accumulators. Each of the plurality of accumulators includes a first input terminal for receiving input vector components and a second input terminal for receiving matrix elements, and performs a cumulative operation on the input vector components and the matrix elements to obtain an output vector One of the components occurs each. Each of the plurality of accumulators includes a first operator, a second operator, and an accumulation register. The first operator performs a first operation on the input vector components and the elements of the matrix to generate the result of the first operation. The second operator generates a result of the second operation by performing a second operation on the result of the first operation and the accumulated result. The accumulation register generates an accumulation result by accumulating the result of the second operation, and finally generates one of the output vector components. The first input terminals of the plurality of accumulators included in the one operator group are commonly connected.

Description

An arithmetic device for efficient parallel processing of matrix operations and a memory device including the same

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a semiconductor integrated circuit, and more particularly, to an arithmetic device for efficiently parallel processing of matrix operations and a memory device including the arithmetic device.

1A is a diagram illustrating an example of performing a matrix operation according to the prior art.

1A , the elements (a ₁₁ , a ₁₂ , ..., a _1m , a ₂₁ , a ₂₂ , ..., a _2m , ..., a _n1 , a _n2 , ..., a By _{multiplication} of an n*m ( _n , _m is a natural number equal to or greater than ₂ ) dimension matrix containing An n-dimensional output vector comprising the components y ₁ , y ₂ , ..., y _n is obtained. Each component of the output vector is defined as the dot product of the input vector and the corresponding column or row vector of the matrix. In this case, the dot product of two vectors may be obtained by sequentially multiplying the components of each vector and then adding them all together. For example, y ₁ = a ₁₁ *x ₁ +a ₁₂ *x ₂ +...+a _1m *x _m .

FIG. 1B is a block diagram illustrating an arithmetic apparatus according to the related art for performing the matrix operation of FIG. 1A.

Referring to FIG. 1B , the computing device 10 includes a plurality of multiplier-and-accumulators (MACs) 201, 202, ..., 20k (k is a natural number equal to or greater than 2). Each of the plurality of MACs 201 to 20k includes one of the plurality of multipliers 211, 212, ..., 21k and one of the plurality of accumulators 231, 232, ..., 23k.

Each of the plurality of multipliers 211 to 21k is one of the plurality of first inputs IN11, IN12, ..., IN1k and one of the plurality of second inputs IN21, IN22, ..., IN2k. is received, and the two inputs are multiplied and output.

Each of the plurality of accumulators 231 to 23k is connected to one of the plurality of multipliers 211 to 21k, and one of the plurality of adders 251, 252, ..., 25k and a plurality of accumulation registers ( 271, 272, ..., 27k), and after accumulating and adding the output of one of the plurality of multipliers 211 to 21k, among the plurality of outputs OUT1, OUT2, ..., OUTk one occurs

When the dimensions of the matrix and the vector are increased, the dot product may be partially obtained using a plurality of MACs 201 to 20k for parallel processing as shown in FIG. 1B and then the total dot product may be obtained by summing the values. However, when the conventional computing device 10 of FIG. 1B includes k MACs 201 to 20k, each of the matrix elements and k elements of the vector is received as inputs, and for this purpose, Since the components of the vector are repeatedly received, there is a problem in that the use of memory and bus bandwidth increases.

SUMMARY OF THE INVENTION One object of the present invention is to provide an arithmetic device for reducing memory and bus bandwidth usage in performing a matrix operation.

Another object of the present invention is to provide a memory device for improving performance and reducing power consumption in performing a matrix operation by including the computing device.

In order to achieve the above object, a computing device according to embodiments of the present invention includes one or more operator groups including a plurality of accumulators. Each of the plurality of accumulators includes a first input terminal for receiving input vector elements and a second input terminal for receiving matrix elements, and performs an accumulation operation on the input vector elements and the matrix elements. to generate each one of the output vector components. Each of the plurality of accumulators includes a first operator, a second operator, and an accumulation register. The first operator generates a result of the first operation by performing a first operation on the input vector components received from the first input terminal and the elements of the matrix received from the second input terminal. The second operator generates a result of the second operation by performing a second operation on the result of the first operation output from the first operator and the accumulation result provided from the accumulation register. The accumulation register generates the accumulation result by accumulating the results of the second operation output from the second operator, and finally generates one of the output vector components. The first input terminals of the plurality of accumulators included in one operator group are connected in common.

In an embodiment, the plurality of operator groups may receive input vector components in a one-to-one correspondence with a plurality of input vectors and generate output vector components in one-to-one correspondence with a plurality of output vectors. It may include a plurality of crossover operator groups including a plurality of accumulators belonging to the different operator groups. The second input terminals of the plurality of accumulators included in one crossover operator group may be connected in common.

In an embodiment, the first operator is a multiplier, the first operation is a multiplication, the second operator is an adder, the second operation is an addition, and each of the plurality of accumulators is a multiplier-and (MAC) operator. -accumulator).

In an embodiment, each of the plurality of accumulators may further include at least one shadow register that temporarily stores the accumulation result generated by the accumulation register.

In one embodiment, each of the plurality of accumulators is connected to an auxiliary input terminal for receiving an auxiliary input, and the first input terminal for receiving the input vector components, and based on a selection signal, the auxiliary input and the input It may further include a multiplexer that selects one of the vector components and provides it to the first operator.

In an embodiment, a first accumulator of the plurality of accumulators is connected to an auxiliary input terminal for receiving an auxiliary input, and the first input terminal for receiving the input vector components, and based on a selection signal A multiplexer may further include an input and a multiplexer that selects one of the input vector components and provides it to the first operator. At least one accumulator disposed adjacent to the first accumulator among the plurality of accumulators may share the multiplexer with the first accumulator.

In one embodiment, any one of the two inputs of each of the plurality of accumulators is a binary value having a value of 0 or 1, and the first accumulator outputs 0 or the other one according to the binary value. It can be implemented as a gating circuit that bypasses the input.

In one embodiment, one of the two inputs of each of the plurality of accumulators is a base n integer value (n is a natural number greater than or equal to 2) and the other is a value of a power of n having an integer exponent, and the first The accumulator may be implemented as a circuit for shifting the n-base integer input by the integer exponent.

In one embodiment, one of the two inputs of each of the plurality of accumulators is an n-base floating-point value (n is a natural number greater than or equal to 2) and the other is a value to a power of n having an integer exponent, and the The 1 accumulator may be implemented as a circuit that bypasses the mantissa of the n-base floating-point input and adds the exponents of the two inputs.

In one embodiment, the first operator is bypassed by fixing any one of two inputs of each of the plurality of accumulators to 1, and the second operator is the value bypassed by the first operator. It may be implemented as a comparator that compares the value of the accumulation register and outputs the larger of the two values.

In an embodiment, the matrix is a convolution matrix, and the input vector and the output vector may be obtained by changing the input matrix and the output matrix into column vectors, respectively.

In an embodiment, the matrix may be an extended convolution matrix corresponding to a plurality of filter matrices, and the output vector may be an extended output vector corresponding to the plurality of filter matrices.

In one embodiment, the matrix is a convolution matrix or a packed convolution matrix, and the plurality of input vectors and the plurality of output vectors may be obtained by converting a plurality of input matrices and a plurality of output matrices into vectors, respectively. have.

In an embodiment, the matrix is a partitioned convolution matrix, and the plurality of input vectors and the plurality of output vectors may be obtained by converting a plurality of partitioned input matrices and a plurality of partitioned output matrices into vectors, respectively. have.

In order to achieve the above object, a memory device according to embodiments of the present invention includes a memory cell array, a row decoder, a column decoder, a gating circuit, an input/output data driving circuit, an input/output buffer, and an arithmetic circuit. The memory cell array includes a plurality of memory cells arranged to form a plurality of rows and a plurality of columns, and stores elements of a matrix or matrix generation information for generating elements of the matrix. The row decoder generates a row selection signal for selecting a target row from among the plurality of rows based on a row address. The column decoder generates a column selection signal for selecting a target column from among columns included in the target row, based on a column address. The gating circuit selects the target column based on the column selection signal. The input/output data driving circuit writes input data into the target column through the gating circuit or outputs data stored in the target column as output data. The input/output buffer is connected to the input/output data driving circuit and stores input vector components and output vector components. The operation circuit is connected to the gating circuit and the input/output buffer, and includes one or more operator groups including a plurality of accumulators, the input vector components provided from the input/output buffer and the gating circuit provided through the gating circuit. The plurality of accumulators operate on the elements of the matrix generated with reference to the elements of the matrix or the matrix generation information provided through the gating circuit in an input vector reference method to obtain the output vector components, and the output Vector components are provided to the input/output buffer. Each of the plurality of accumulators includes a first input terminal for receiving the input vector components, and a second input terminal for receiving the elements of the matrix, wherein the input vector components and the elements of the matrix An accumulation operation is performed to generate one of the output vector components. Each of the plurality of accumulators includes a first operator, a second operator, and an accumulation register. The first operator generates a result of the first operation by performing a first operation on the input vector components received from the first input terminal and the elements of the matrix received from the second input terminal. The second operator generates a result of the second operation by performing a second operation on the result of the first operation output from the first operator and the accumulation result provided from the accumulation register. The accumulation register generates the accumulation result by accumulating the results of the second operation output from the second operator, and finally generates one of the output vector components. The first input terminals of the plurality of accumulators included in one operator group are connected in common.

In an embodiment, the plurality of operator groups may receive input vector components in a one-to-one correspondence with a plurality of input vectors and generate output vector components in one-to-one correspondence with a plurality of output vectors. It may include a plurality of crossover operator groups including a plurality of accumulators belonging to the different operator groups. The second input terminals of the plurality of accumulators included in one crossover operator group may be connected in common.

In an embodiment, the first operator is a multiplier, the first operation is a multiplication, the second operator is an adder, the second operation is an addition, and each of the plurality of accumulators is a multiplier-and (MAC) operator. -accumulator).

In an embodiment, each of the plurality of accumulators may further include at least one shadow register that temporarily stores the accumulation result generated by the accumulation register.

In one embodiment, each of the plurality of accumulators is connected to an auxiliary input terminal for receiving an auxiliary input, and the first input terminal for receiving the input vector components, and based on a selection signal, the auxiliary input and the input It may further include a multiplexer that selects one of the vector components and provides it to the first operator.

In an embodiment, a first accumulator of the plurality of accumulators is connected to an auxiliary input terminal for receiving an auxiliary input, and the first input terminal for receiving the input vector components, and based on a selection signal A multiplexer may further include an input and a multiplexer that selects one of the input vector components and provides it to the first operator. At least one accumulator disposed adjacent to the first accumulator among the plurality of accumulators may share the multiplexer with the first accumulator.

In one embodiment, any one of the two inputs of each of the plurality of accumulators is a binary value having a value of 0 or 1, and the first accumulator outputs 0 or the other one according to the binary value. It can be implemented as a gating circuit that bypasses the input.

In one embodiment, one of the two inputs of each of the plurality of accumulators is a base n integer value (n is a natural number greater than or equal to 2) and the other is a value of a power of n having an integer exponent, and the first The accumulator may be implemented as a circuit for shifting the n-base integer input by the integer exponent.

In one embodiment, one of the two inputs of each of the plurality of accumulators is an n-base floating-point value (n is a natural number greater than or equal to 2) and the other is a value to a power of n having an integer exponent, and the The 1 accumulator may be implemented as a circuit that bypasses the mantissa of the n-base floating-point input and adds the exponents of the two inputs.

In one embodiment, the first operator is bypassed by fixing any one of two inputs of each of the plurality of accumulators to 1, and the second operator is the value bypassed by the first operator. It may be implemented as a comparator that compares the value of the accumulation register and outputs the larger of the two values.

In an embodiment, the matrix is a convolution matrix, and the input vector and the output vector may be obtained by changing the input matrix and the output matrix into column vectors, respectively.

In an embodiment, the matrix may be an extended convolution matrix corresponding to a plurality of filter matrices, and the output vector may be an extended output vector corresponding to the plurality of filter matrices.

In one embodiment, the matrix is a convolution matrix or a packed convolution matrix, and the plurality of input vectors and the plurality of output vectors may be obtained by converting a plurality of input matrices and a plurality of output matrices into vectors, respectively. have.

In an embodiment, the matrix is a partitioned convolution matrix, and the plurality of input vectors and the plurality of output vectors may be obtained by converting a plurality of partitioned input matrices and a plurality of partitioned output matrices into vectors, respectively. have.

In an embodiment, the memory device is a line selection signal for selecting a target line including the target column from among a plurality of lines included in the target row and each including two or more columns, based on the column address It may further include a line decoder for generating The gating circuit may include a first gating circuit that selects the target line based on the line selection signal and a second gating circuit that selects the target column based on the column selection signal. The input/output data driving circuit may write the input data in the target column or output data stored in the target column as the output data through the first and second gating circuits. The operation circuit may be connected to the first gating circuit.

In one embodiment, the column decoder is a multi-column decoder that generates a multi-column selection signal for selecting a plurality of target columns from among columns included in the target row at a time based on the column address and column selection information, , the gating circuit selects the plurality of target columns at once based on the multi-column selection signal, and the input/output data driving circuit writes the input data to the plurality of target columns at a time through the gating circuit Alternatively, data stored in the plurality of target columns may be output as the output data at a time, and the column addresses corresponding to the plurality of target columns included in the target row may not be consecutive.

In the arithmetic device and the memory device according to the embodiments of the present invention as described above, when receiving input vector components and matrix elements and performing an accumulation operation, a plurality of accumulators ( For example, by connecting the first input terminals of the plurality of MACs in common, it is possible to reduce the number of times that the component of the input vector is applied to the inputs of the plurality of accumulators. Therefore, compared with the conventional computing device, memory and bus bandwidth usage is reduced, and as a result, it is advantageous in terms of performance and power consumption compared to the conventional method.

1A is a diagram illustrating an example of performing a matrix operation according to the prior art.
FIG. 1B is a block diagram illustrating an arithmetic apparatus according to the related art for performing the matrix operation of FIG. 1A.
2A is a diagram illustrating an example of performing a matrix operation according to embodiments of the present invention.
FIG. 2B is a block diagram illustrating an operation device according to embodiments of the present invention that performs the matrix operation of FIG. 2A.
3A is a block diagram illustrating a computing device according to embodiments of the present invention.
3B is a block diagram illustrating another example of an accumulator included in the computing device of FIG. 3A .
FIG. 3C is a block diagram illustrating an example of shortening a calculation execution time by extending the calculation device of FIG. 3A to process a plurality of input vectors in parallel.
4A and 4B are block diagrams illustrating another example of an accumulator included in the computing device of FIG. 3A.
5A, 5B, 5C, 5D, and 5E are diagrams for explaining examples of performing a convolution of two matrices and an operation applying the same by using calculation devices according to embodiments of the present invention.
6A, 6B, and 6C are diagrams for explaining another example of performing a convolution of two matrices using an arithmetic device according to embodiments of the present invention.
7, 8, and 9 are block diagrams illustrating a memory device including an arithmetic unit according to embodiments of the present invention.

With respect to the embodiments of the present invention disclosed in the text, specific structural or functional descriptions are only exemplified for the purpose of describing the embodiments of the present invention, and the embodiments of the present invention may be embodied in various forms. It should not be construed as being limited to the embodiments described in .

Since the present invention can have various changes and can have various forms, specific embodiments are illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to the specific disclosed form, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms may be used for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle. Other expressions describing the relationship between elements, such as "between" and "immediately between" or "neighboring to" and "directly adjacent to", etc., should be interpreted similarly.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, and is intended to indicate that one or more other features or numbers are present. , it is to be understood that it does not preclude the possibility of the presence or addition of steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with the context of the related art, and unless explicitly defined in the present application, they are not to be interpreted in an ideal or excessively formal meaning. .

In the present invention, embodiments related to matrix operations are described by expressing vectors as column vectors for convenience, but the expression of vectors is not limited to column vectors. Considering that a row vector is a transpose matrix of a column vector, even if a vector is expressed as a row vector, an embodiment related to a corresponding matrix operation can be easily constructed and described.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

2A is a diagram illustrating an example of performing a matrix operation according to embodiments of the present invention.

Referring to FIG. 2A , similar to that described above with reference to FIG. 1A , elements a ₁₁ , a ₁₂ , ..., a _1m , a ₂₁ , a ₂₂ , ..., a _2m , ..., n*m (n, m are each a natural number greater than or equal to 2) dimensional matrix containing a _n1 , a _n2 , ..., a _nm ) and components (x ₁ , x ₂ , ..., x _m ) An n-dimensional output vector including the components (y ₁ , y ₂ , ..., y _n ) is obtained by multiplication of the containing m-dimensional input vector. However, unlike the one described above with reference to FIG. 1A, in FIG. 2A, values contributing to each component of the output vector are calculated based on individual components of the input vector, and then accumulated to obtain each component of the output vector. . When the input vector reference method shown in FIG. 2A is applied, one of the input terminals of the plurality of MACs may be commonly bundled as will be described later with reference to FIG. 2B.

FIG. 2B is a block diagram illustrating an operation device according to embodiments of the present invention that performs the matrix operation of FIG. 2A.

Referring to FIG. 2B , the computing device 100 includes a plurality of multiplier-and-accumulators (MACs) 2001, 2002, ..., 200k (k is a natural number equal to or greater than 2). Each of the plurality of MACs 2001 to 200k includes one of a plurality of multipliers 2101, 2102, ..., 210k and one of a plurality of accumulators 2301, 2302, ..., 230k.

Each of the plurality of multipliers 2101 to 210k receives one of the first input IN1 and the plurality of second inputs IN21, IN22, ..., IN2k, and multiplies and outputs the two inputs. For example, the first input IN1 is the components x ₁ , x ₂ , ..., x _m of the input vector of FIG. 2A , and each of the plurality of second inputs IN21 to IN2k is shown in FIG. The elements of the matrix of 2a (a ₁₁ , a ₁₂ , ..., a _1m , a ₂₁ , a ₂₂ , ..., a _2m , ..., a _n1 , a _n2 , ..., a _nm ) may be elements arranged in the same column.

Each of the plurality of accumulators 2301 to 230k is connected to one of the plurality of multipliers 2101 to 210k, and accumulates an output of one of the plurality of multipliers 2101 to 210k to output a plurality of outputs OUT1 and OUT2 , ..., OUTk).

Each of the plurality of accumulators 2301 to 230k includes one of the plurality of adders 2501, 2502, ..., 250k and one of the plurality of accumulation registers 2701, 2702, ..., 270k. . Each of the plurality of adders 2501 to 250k adds and outputs an output of one of the plurality of multipliers 2101 to 210k and an output of one of the plurality of accumulation registers 2701 to 270k. Each of the plurality of accumulation registers 2701 to 270k stores one output of the plurality of adders 2501 to 250k and finally generates one of the plurality of outputs OUT1 to OUTk.

In one embodiment, the first input IN1 and the second input IN21 of the MAC 2001 are the components x ₁ , x ₂ , ..., x _m of the input vector and the elements of the matrix (a ₁₁ , a ₁₂ , ..., a _1m ) is sequentially received, and the output (OUT1) of the MAC 2001 is the component (y ₁ ) of the output vector, that is, a ₁₁ *x ₁ +a ₁₂ * It can correspond to x ₂ +...+a _1m *x _m . For example, during the initial operation, x ₁ and a ₁₁ are respectively input to the first input IN1 and the second input IN21 of the MAC 2001, and multiplication, addition and accumulation are performed thereon to thereby register the accumulation register ( 2701), a ₁₁ *x ₁ may be stored. Thereafter, x ₂ and a ₁₂ are input to the first input IN1 and the second input IN21 of the MAC 2001, respectively, and multiplication, addition, and accumulation are performed thereon, and a ₁₁ is stored in the accumulation register 2701. *x ₁ +a ₁₂ *x ₂ may be stored. In this way, a ₁₁ *x ₁ +a ₁₂ *x ₂ +...+a _1m *x _m may be finally stored and output in the accumulation register 2701 . Similarly, the output OUT2 of the MAC 2002 may correspond to the component y ₂ of the output vector, ie a ₂₁ *x ₁ +a ₂₂ *x ₂ +...+a _2m *x _m . .

In the case of applying the input vector reference method shown in FIG. 2A, as shown in FIG. 2B, when parallel processing is performed using a plurality of MACs 2001 to 200k, the Since the components of the input vectors are all the same, one of the input terminals of the plurality of MACs 2001 to 200k may be commonly bundled. For example, common first input terminals of the plurality of MACs 2001 to 200k receiving the components (x ₁ , x ₂ , ..., x _m ) of the input vector as the first input IN1 It can be used as one input terminal by connecting to

In the arithmetic device 100 according to the embodiments of the present invention, as described above, by connecting the first input terminals of the plurality of MACs 2001 to 200k in common, the component of the input vector is converted into a plurality of MACs ( 2001~200k) can reduce the number of times it is applied. Specifically, when the computing device 100 according to the embodiments of the present invention shown in FIG. 2B includes k MACs 2001 to 200k, (k+1) inputs may be received. Accordingly, as compared with the conventional computing device 10 of FIG. 1B , the use of memory and bus bandwidth is reduced, and as a result, it is advantageous in terms of performance and power consumption compared to the conventional method.

3A is a block diagram illustrating a computing device according to embodiments of the present invention.

Referring to FIG. 3A , the arithmetic unit 300 includes a plurality of accumulators 4001 , 4002 , ..., 400k (k is a natural number equal to or greater than 2). Each of the plurality of accumulators 4001 to 400k is one of the plurality of first operators (4101, 4102, ..., 410k) and one of the plurality of accumulators (4301, 4302, ..., 430k) include

Each of the plurality of first operators 4101 to 410k receives one of a first input IN1 and a plurality of second inputs IN21, IN22, ..., IN2k, and receives a first input for the two inputs. Execute operation and output. For example, the first input IN1 is the components x ₁ , x ₂ , ..., x _m of the input vector of FIG. 2A , and each of the plurality of second inputs IN21 to IN2k is shown in FIG. The elements of the matrix of 2a (a ₁₁ , a ₁₂ , ..., a _1m , a ₂₁ , a ₂₂ , ..., a _2m , ..., a _n1 , a _n2 , ..., a _nm ) may be elements arranged in the same column.

Each of the plurality of accumulators 4301 to 430k is connected to one of the plurality of first operators 4101 to 410k, and accumulates the output of one of the plurality of first operators 4101 to 410k to generate a plurality of outputs. Generates one of (OUT1, OUT2, ..., OUTk).

Each of the plurality of accumulators 4301 to 430k receives one of the plurality of second operators 4501, 4502, ..., 450k and one of the plurality of accumulation registers 4701, 4702, ..., 470k. include Each of the plurality of second operators 4501 to 450k performs a second operation on the output of one of the plurality of first operators 4101 to 410k and the output of one of the plurality of accumulation registers 4701 to 470k. to output Each of the plurality of accumulation registers 4701 to 470k stores one output of the plurality of second operators 4501 to 450k and finally generates one of the plurality of outputs OUT1 to OUTk.

In the case of applying the input vector reference method shown in FIG. 2A, when parallel processing is performed using a plurality of accumulators 4001 to 400k as shown in FIG. 3A, the plurality of accumulators 4001 to 400k Since the components of the applied input vectors are all the same, one of the input terminals of the plurality of accumulators 4001 to 400k may be grouped in common. For example, the first input terminals of the plurality of accumulators 4001 to 400k that receive the components (x ₁ , x ₂ , ..., x _m ) of the input vector as the first input IN1 are It can be used as one input terminal by connecting in common.

In the computing device 300 according to the embodiments of the present invention, as described above, by connecting the first input terminals of the plurality of accumulators 4001 to 400k in common, the component of the input vector is converted into a plurality of accumulators. It is possible to reduce the number of times applied to the inputs of the fields 4001 to 400k. Specifically, when the computing device 300 according to the embodiments of the present invention of FIG. 3A includes k accumulators 4001 to 400k, (k+1) inputs may be received. Therefore, memory and bus bandwidth usage is reduced, which is advantageous in terms of performance and power consumption as a result.

The arithmetic device 300 of FIG. 3A uses the plurality of multipliers 2101 to 210k and the plurality of adders 2501 to 250k of the arithmetic device 100 of FIG. 2B to respectively connect the plurality of first operators 4101 to 410k. and a case of generalization to a plurality of second operators 4501 to 450k.

The plurality of accumulators 4001, 4002, ..., 400k of FIG. 3A may be implemented as a digital circuit or an analog circuit. For example, the plurality of accumulators 4301, 4302, ..., 430k may be implemented as an analog integration circuit.

In one embodiment, when each of the plurality of first operators 4101 to 410k of FIG. 3A is implemented as a multiplier and each of the plurality of second operators 4501 to 450k is implemented as an adder, the arithmetic device of FIG. 3A 300 may perform multiplication of a conventional matrix and an input vector as shown in FIG. 2B .

In one embodiment, when any one of two inputs of each of the plurality of accumulators 4001 to 400k of FIG. 3A is a binary value having a value of 0 or 1, the plurality of first operators of FIG. 3A ( Each of 4101 to 410k) may be implemented as a gating circuit that performs an on-off operation according to the binary value to perform multiplication. For example, when any one input (eg, the first input IN1 ) is 0, each of the plurality of first operators 4101 to 410k may output a value of 0, and any one When the input of (eg, the first input IN1) of is 1, each of the plurality of first operators 4101 to 410k corresponds to the other input (eg, the plurality of second inputs IN21). ~IN2k)) can be output as it is.

In one embodiment, one of the two inputs of each of the plurality of accumulators 4001 to 400k of FIG. 3A is a binary integer value (or an integer value expressed in binary notation) and the other is an integer exponent of 2 In the case of a power value, each of the plurality of first operators 4101 to 410k of FIG. 3A is implemented as a circuit outputting an integer value shifted by the integer exponent by the integer exponent, so that the first operation (for example, multiplication) can be performed. For example, one input (eg, the first input IN1) is 6 (ie, the binary number 1010) and the other input (eg, among the plurality of second inputs IN21 to IN2k) When one) is 2 ² , each of the plurality of first operators 4101 to 410k may output 101000 obtained by shifting 1010 by 2 . In this case, when only the integer exponent is received as an input instead of the power of 2 value having the integer exponent, the data size is reduced, thereby reducing the required memory size and saving the use of memory and bus bandwidth.

The above embodiment can be extended to a case where the input data format is a data format expressed in base n (n is a natural number equal to or greater than 2). That is, one of two inputs of each of the plurality of accumulators 4001 to 400k of FIG. 3A is an n-base integer value (or an integer value expressed in base n) and the other is a power of n having an integer exponent In the case of a square value, each of the plurality of first operators 4101 to 410k of FIG. 3A is implemented as a circuit outputting an integer value shifted by the integer exponent by the n-base integer input, and the first operation (for example, , multiplication) can be performed.

In one embodiment, any one of the two inputs of each of the plurality of accumulators 4001 to 400k of Figure 3A is a binary floating-point value (or a floating-point value with base 2 and exponent and mantissa expressed in binary) and the other one is a power of 2 value having an integer exponent, each of the plurality of first operators 4101 to 410k of FIG. 3A adds exponents to each other, and the mantissa is binary in which the mantissa of the binary floating-point input is maintained as it is. The first operation (eg, multiplication) may be performed by being implemented as a circuit for outputting a floating-point value. In this case, when only the integer exponent is received as an input instead of the power of 2 value having the integer exponent, the data size is reduced, thereby reducing the required memory size and saving the use of memory and bus bandwidth.

The above embodiment can be extended to a case in which the input data format is a data format expressed in base n (n is a natural number equal to or greater than 2). That is, any one of the two inputs of each of the plurality of accumulators 4001 to 400k of FIG. 3A is an n-base floating-point value (or a floating-point value in which the base is n and the exponent and mantissa are expressed in base n) When the other is a power value of n having an integer exponent, each of the plurality of first operators 4101 to 410k of FIG. 3A adds exponents to each other, and the mantissa is n in which the mantissa of the n-base floating-point input is maintained as it is. The first operation (eg, multiplication) may be performed by being implemented as a circuit for outputting a binary floating-point value.

In one embodiment, any one of two inputs of each of the plurality of first operators 4101 to 410k of FIG. 3A (eg, a first input IN1 for receiving a component of the input vector) is 1 fixed to to bypass the plurality of first operators 4101 to 410k, and each of the plurality of second operators 4501 to 450k is bypassed by each of the plurality of first operators 4101 to 410k When the passed value is compared with the value of the accumulation register and implemented as a comparator that outputs the larger of the two values, each of the plurality of accumulators 4001 to 400k is the other one (eg, a plurality of A max pooling operation of outputting a maximum value among the elements of the matrix given as second inputs IN21 to IN2k ) may be performed. Since the comparator is composed of a subtractor and a multiplexer, it can be implemented by adding a little circuit to the adder.

3B is a block diagram illustrating another example of an accumulator included in the computing device of FIG. 3A .

3B, the accumulator 402 includes a first operator 412 and an accumulator 432, and the accumulator 432 includes a second operator 452, an accumulation register 472a, and at least one shadow ( shadow) register 472b.

Except for further including the shadow register 472b, the accumulator 402 of FIG. 3B may be substantially the same as the plurality of accumulators 4001 to 400k of FIG. 3A. The first operator 412, the accumulator 432, the second operator 452, the accumulation register 472a, the first input IN1, the second input IN2, and the output OUT of FIG. 3B are shown in FIG. 3A, respectively. a plurality of first operators 4101 to 410k, a plurality of accumulators 4301 to 430k, a plurality of second operators 4501 to 450k, a plurality of accumulation registers 4701 to 470k, a first input ( IN1), a plurality of second inputs IN21 to IN2k, and a plurality of outputs OUT1 to OUTk.

The shadow register 472b may temporarily store the value of the accumulation register 472a (eg, the accumulation result). Also, the value stored in the shadow register 472b may be written into the accumulation register 472a. For convenience, only one shadow register 472b is illustrated in FIG. 3B , but the number of shadow registers 472b may be variously changed according to an embodiment.

FIG. 3C is a block diagram illustrating an example of shortening the calculation execution time by extending the calculation device 300 of FIG. 3A to process a plurality of input vectors in parallel.

Referring to FIG. 3C , the arithmetic unit 500 includes a plurality of accumulators 5011, 5021. , 50kq (k and q are each a natural number equal to or greater than 2), and each of the accumulators includes a plurality of first inputs ( receiving one of IN11, IN12, ..., IN1q and one of the plurality of second inputs IN21, IN22, ..., IN2k and receiving a plurality of outputs OUT11, OUT21, ..., OUTkq Occurs in one of the operator groups 5001, 5002, , and 500q and belongs to one of the cross operator groups 5010, 5020, , and 50k0. The accumulators belonging to one operator group receive the first input in common because the first input terminals are connected to one another. The accumulators belonging to one cross operator group belong to different operator groups, and the second input terminals are connected to one another to receive the second input in common. Each of the operator groups corresponds to the computing device 300 of FIG. 3A .

When one matrix and a plurality of input vectors are given, the matrix operation on the matrix and each input vector can be sequentially performed by repeatedly using the operation device 300 of FIG. 3A, and the corresponding output vectors can be sequentially obtained. However, as shown in FIG. 3C , by using the arithmetic unit 300 or the arithmetic operator groups 5001 , 5002 , , 500q in one-to-one correspondence to the plurality of input vectors and performing the matrix operation in parallel, the matrix It is possible to shorten the time required to perform calculations. At this time, since the matrix used for the matrix operation is the same, the elements of the matrix applied to each operator group may be the same. The second input terminals of the accumulators belonging to the group may be connected together and the second input may be commonly received.

When an input matrix having each of the plurality of input vectors as a column vector is created, the matrix operation is essentially the same as the matrix operation on the matrix and the input matrix. Accordingly, the matrix and the operation of the matrix may be performed using the calculating device 300 of FIG. 3A or the calculating device 500 of FIG. 3C .

4A and 4B are block diagrams illustrating another example of an accumulator included in the computing device of FIG. 3A.

Referring to FIG. 4A , the accumulator 404 includes a first operator 414 , an accumulator 434 , and a multiplexer 494 , and the accumulator 434 includes a second operator 454 and an accumulation register 474 . include

Except for further including a multiplexer 494, the accumulator 404 of FIG. 4A may be substantially the same as the plurality of accumulators 4001 to 400k of FIG. 3A. The first operator 414, the accumulator 434, the second operator 454, the accumulation register 474, the first input IN1, the second input IN2, and the output OUT of FIG. 4A are shown in FIG. 3A, respectively. a plurality of first operators 4101 to 410k, a plurality of accumulators 4301 to 430k, a plurality of second operators 4501 to 450k, a plurality of accumulation registers 4701 to 470k, a first input ( IN1), a plurality of second inputs IN21 to IN2k, and a plurality of outputs OUT1 to OUTk.

The multiplexer 494 is connected to an auxiliary input terminal receiving the auxiliary input AIN1 and the first input terminal receiving the first input IN1, and based on the selection signal SS1, the auxiliary input AIN1 and One of the first inputs IN1 may be selected and provided to the first operator 414 . Also, the multiplexer 494 may provide a selected one of the auxiliary input AIN1 and the first input IN1 as the auxiliary output AOUT1 . The auxiliary output AOUT1 may be used as an auxiliary input of a multiplexer included in an adjacent accumulator.

In other words, the accumulator 404 of FIG. 4A may select the auxiliary input AIN1 by adding a multiplexer 494 to the first input terminal of the first operator 414, and assist the output of the multiplexer 494. It can be output as output (AOUT1). When parallel processing is performed using a plurality of such accumulators 404, the auxiliary output of an adjacent accumulator can be received as an auxiliary input through the auxiliary input terminal. In addition, the degree of freedom can be increased when allocating an accumulator for multiplication of a matrix and an input vector.

Referring to FIG. 4B , each of the plurality of accumulators 4041, 4042, ..., 404s (s is a natural number greater than or equal to 2) is one of the plurality of first operators 4141, 4142, ..., 414s. and one of the plurality of accumulators 4341, 4342, ..., 434s. Each of the plurality of accumulators 4341 to 434s receives one of the plurality of second operators 4541, 4542, ..., 454s and one of the plurality of accumulation registers 4741, 4742, ..., 474s. include The accumulator 4041 may further include a multiplexer 494 .

Except that the accumulator 4041 further includes a multiplexer 494, the plurality of accumulators 4041 to 404s of FIG. 4B may be substantially the same as the plurality of accumulators 4001 to 400k of FIG. 3A. can 4B, a plurality of first operators 4141 to 414s, a plurality of accumulators 4341 to 434s, a plurality of second operators 4541 to 454s, a plurality of accumulation registers 4741 to 474s, a first The input IN1, the plurality of second inputs IN21, IN22, ..., IN2s, and the plurality of outputs OUT1, OUT2, ..., OUTs are respectively connected to the plurality of first operators ( 4101 to 410k), a plurality of accumulators 4301 to 430k, a plurality of second operators 4501 to 450k, a plurality of accumulation registers 4701 to 470k, a first input IN1, a plurality of second inputs It may correspond to the ones IN21 to IN2k and the plurality of outputs OUT1 to OUTk.

The multiplexer 494 of FIG. 4B may be substantially the same as the multiplexer 494 of FIG. 4A . The multiplexer 494 may select one of the auxiliary input AIN1 and the first input IN1 based on the selection signal SS1 and provide it as the auxiliary output AOUT1 . The accumulators 4041 to 404s may share a multiplexer 494 . The plurality of first operators 4141 to 414s may select and receive the first input IN1 and the auxiliary input AIN1 by receiving the auxiliary output AOUT1 provided from the multiplexer 494 .

In other words, in FIG. 4B, the auxiliary input AIN1 can be selected by adding a multiplexer 494 to the first input terminal shared (that is, commonly connected) by a plurality of accumulators 4041 to 404s. An output of the multiplexer 494 may be output as an auxiliary output AOUT1 . Compared with FIG. 4A , although the degree of freedom of accumulator allocation is lowered, the number of multiplexers is reduced, which may be advantageous in terms of area.

Meanwhile, although not shown, the (s+1)-th accumulator after the s-th accumulator 404s may include a multiplexer similarly to the accumulator 4041, and from the (s+1)-th accumulator to t ( (t-s) accumulators may share the multiplexer included in the (s+1)-th accumulator until t is a natural number greater than s)-th accumulator. Depending on the embodiment, t may be equal to or different from 2*s.

5A, 5B, 5C, 5D, and 5E are diagrams for explaining examples of performing a convolution of two matrices and an operation applying the same by using calculation devices according to embodiments of the present invention.

Referring to FIG. 5A, if a 3*3 filter matrix W is convolved with no padding and a stride is 1 on a 7*8 input matrix X, a 5*6 output matrix Y can be obtained. can During convolution, the components of the output matrix Y are sequentially calculated. In this case, each element of the input matrix X may be referenced multiple times.

In order to express the convolution between two matrices by changing the multiplication between the matrix and the vector, the shapes of the input matrix X and the output matrix Y are changed into column vectors to obtain an input vector x and an output vector y, respectively. In this case, xi and yi are an input subvector and an output subvector having elements of the i-th row of the input matrix X and the output matrix Y as components, respectively.

Since each component of the output vector y to be obtained by convolution is expressed as a linear combination of the components of the input vector x, there is a linear transformation that transforms the input vector x into the output vector y, which is can be expressed together. The output vector y is obtained by multiplying the convolution matrix A by the input vector x, and then the output matrix Y can be obtained by changing the shape to a matrix again. Accordingly, the convolution of the input matrix X and the filter matrix W can be performed using the arithmetic device 300 of FIG. 3A by changing the multiplication of the input vector x and the convolution matrix A. In this case, each component of the input vector x can be referenced only once by applying the input vector reference method.

If the input and output vectors are obtained by changing the shapes of the input and output matrices into row vectors, the convolution matrix may be expressed as the transpose matrix of the matrix A.

Repeated convolution of a plurality of input matrices with the same size can be replaced by repeatedly multiplying a plurality of input vectors with one convolution matrix as described above, and a plurality of output matrices or a plurality of output matrices The output vectors may be obtained in parallel using the arithmetic unit 500 of FIG. 3C .

Referring to FIG. 5D , a plurality of divided output matrices Y ₁ and Y ₂ are generated by dividing the output matrix Y vertically, and a plurality of divided input matrices X ₁ and X ₂ are generated from the input matrix X to form the filter matrix W and Y can be convolved to obtain Y ₁ and Y ₂ respectively. When an input matrix is divided, an overlapping region is generated so that the sum of the number of elements of the divided input matrices X ₁ and X ₂ may be greater than the number of elements of the input matrix X . On the other hand, when processing the input matrix X by dividing the input matrix X into X ₁ and X ₂ as described above, the number of elements of the divided convolution matrix A _d applied may be less than half the number of elements of the convolution matrix A. have.

Similarly, one output matrix can be divided into blocks of the same size to form a plurality of partitioned output matrices, and a plurality of partitioned input matrices can be created. input vectors can be created. Since the plurality of partitioned input matrices have the same size, the partitioned convolution matrix multiplied by the plurality of partitioned input vectors may be the same. Vectors can be found in parallel.

When the input matrix is divided into the plurality of divided input matrices, regions overlapping each other occur, and elements belonging to this region are repeatedly received by the computing device 500 of FIG. 3C . On the other hand, there is an advantage in that the number of elements of the divided convolution matrix received by the computing device 500 of FIG. 3C is reduced, and in addition, in the plurality of accumulators belonging to one intersection operator group of FIG. 3C, The advantage is that the elements of the convolution matrix are received in common.

5E, when a filter matrix V other than the filter matrix W is given and convolution is performed with the input matrix X, a convolution matrix A _v corresponding to the filter matrix V is added to the convolution matrix A to expand the synthesis A product matrix A _e can be created and multiplied by the input vector x to obtain an expanded output vector y _e .

Similarly, when a plurality of filter matrices are convolved to one input matrix, an extended convolution matrix is created from each convolution matrix, and an extended output vector or the plurality of filter matrices is generated using the arithmetic unit 300 of FIG. 3A . A plurality of output vectors corresponding to can be obtained in parallel. In this case, there is an advantage that the elements of the input matrix are received only once in the operation device 300 of FIG. 3A .

6A, 6B, and 6C are diagrams for explaining another example of performing a convolution of two matrices using an arithmetic device according to embodiments of the present invention.

Since the convolution matrix A of FIG. 5C is a sparse matrix with a small number of non-zero elements, time and resource waste may be large when stored in a memory or multiplied by a vector. Looking at the structure of the convolution matrix A, it can be seen that submatrices other than the zero matrix are regularly arranged near the main diagonal, and the submatrices are band matrices. The structural characteristics of the convolution matrix A are maintained even when the size, padding, and stride of the filter matrix are changed. exemplify

Referring to FIG. 6A , a packed convolution matrix A _p can be obtained by resetting the accumulator register after the calculation of one output subvector is finished using the accumulator and allocating the accumulator for the next output subvector calculation. have. In this case, the packed output subvector notation, for example, y ₁ |y ₄ means that the output subvectors y ₁ and y ₄ are obtained by sharing the same accumulators through time division. That is, after obtaining the output subvector y ₁ first, the output subvector y ₄ can be obtained using the same accumulators.

Referring to FIG. 6B , after the calculation of the output vector component for one row of the input matrix is finished using the accumulator, the accumulator value is temporarily stored, and the accumulator is assigned to calculate the next output vector component. A packed convolution matrix A _p can be obtained. Before starting the calculation of the next output vector component, the accumulation register may be reset or a previously temporarily stored value may be written into the accumulation register. In this case, the component notation of the packed output vector, for example, y _i1 |y _i4 , means that the components y _i1 and y _i4 of the output vector are obtained by sharing the same accumulator through time division. That is, after calculation for the output vector component y _i1 is performed with respect to one row of the input matrix, the calculation for the output vector component y _i4 may be performed using the same accumulator.

Referring to FIG. 6C , a packed convolution matrix A _p may be obtained by using both the convolution matrix packing method described above with reference to FIG. 6A and the convolution matrix packing method described above with reference to FIG. 6B .

The convolution matrix and the packed convolution matrices are structurally clear, so that each column can be easily constructed using the elements of the filter matrix. In particular, the previous column or the next column can be obtained by appropriately shifting one column. Accordingly, it is possible to efficiently implement a circuit for generating each column of the convolution matrix or the packed convolution matrix from the elements of the filter matrix.

7, 8, and 9 are block diagrams illustrating a memory device including an arithmetic device according to embodiments of the present invention in order to efficiently perform a matrix operation.

Referring to FIG. 7 , the memory device 1100 includes a memory cell array 1110 , a row decoder 1120 , a column decoder 1130 , a gating circuit 1140 , an input/output data driving circuit 1150 , and an operation circuit 1000 . , and an input/output buffer 1010 .

The memory cell array 1110 includes a plurality of memory cells arranged to form a plurality of rows and a plurality of columns. For example, one row of the memory cell array 1110 constitutes one page. The memory cell array 1110 stores elements of a matrix to be calculated or matrix generation information for generating elements of the matrix. Also, the memory cell array 1110 may store components of an input vector to be calculated and components of an output vector obtained as a result of an operation, and may store temporary values of an output vector component obtained as an intermediate result of an operation. When storing the elements of the matrix, one component of the input vector and the elements of the matrix that are calculated are stored in the same row of the memory cell as much as possible to suit application of the input vector reference method. For this, if necessary, the matrix is transposed to can be saved

The row decoder 1120 generates a row select signal RSEL for selecting a target row including a target memory cell from among the plurality of rows based on the row address RADDR.

The column decoder 1130 generates a column selection signal CSEL for selecting a target column including the target memory cell from among columns included in the target row, based on the column address CADDR.

The gating circuit 1140 is connected to the target row based on the row select signal RSEL. In addition, the gating circuit 1140 selects the target column included in the target row based on the column selection signal CSEL and connects to the input/output data driving circuit 1150 or the operation circuit 1000 . For example, the gating circuit 1140 may further include a sense amplifier for sensing memory cells and stably processing signals.

The input/output data driving circuit 1150 writes input data DIN in the target column or outputs data stored in the target column as output data DOUT. At this time, it is possible to prevent unwanted input data from being written into the target column based on the data mask signal DMS. For example, the input/output data driving circuit 1150 may include an input data driving circuit and an output data driving circuit.

The target column may be connected to the input/output data driving circuit 1150 through the gating circuit 1140 based on the column selection signal CSEL. In the data write operation, the input data DIN received from the input data driving circuit may be provided to the gating circuit 1140 in units of column data CDIN and stored in the target column. In this case, it is possible to prevent unwanted data from being stored in the memory cell array 1110 based on the data mask signal DMS. In the data read operation, page data (or row data) RD stored in the target row is provided to the gating circuit 1140 , and the data of the target column included in the target row is output in units of column data CDOUT. It is provided to the data driving circuit and may be output as output data DOUT.

The input/output buffer 1010 is connected to the input/output data driving circuit 1150 and stores input vectors and components of the output vectors. For example, the input/output buffer 1010 may include an input buffer storing components of the input vector and an output buffer storing components of the output vector. Components of the input vector received from the outside of the memory device may be written into the input/output buffer 1010 through the input/output data driving circuit 1150 , and the components of the input vector stored in the memory cell array 1110 may be converted into the gating circuit 1140 . ) and the input/output data driving circuit 1150 may be written to the input/output buffer 1010 . The output vector components stored in the input/output buffer 1010 may be read out of the memory device through the input/output data driving circuit 1150 , and may be transmitted to the memory cell array 1110 through the input/output data driving circuit 1150 and the gating circuit 1140 . can be entered.

The operation circuit 1000 is connected to the gating circuit 1140 and the input/output buffer 1010, includes a plurality of accumulators, and generates matrix generation information ASINF for generating elements of a matrix through the gating circuit 1140. It may include a matrix element generator that is provided and generates the elements (AS) of the matrix. For example, the matrix element generator may receive elements of a filter matrix used for matrix convolution as the matrix generation information ASINF to generate the elements of a convolution matrix or a packed convolution matrix. The accumulators calculate the elements XS of the input vector provided from the input/output buffer 1010 and the elements AS of the matrix provided through the gating circuit 1140 or the matrix element generator in the input vector reference method. The components YS of the output vector are obtained and provided to the input/output buffer 1010 . When the data format of the components of the input vector and output vector stored in the input/output buffer 1010 or the data format of the elements of the matrix provided through the gating circuit 1140 and the input/output data format of the accumulators are different, the necessary type conversion is performed. can

Each of the plurality of accumulators includes a first input terminal for receiving components of the input vector and a second input terminal for receiving elements of the matrix, wherein components of the input vector and elements of the matrix are An accumulation operation is performed on , to generate one of the components of the output vector. In addition, some or all of the plurality of accumulators may form an operator group in which the first input terminals are connected to one and receive components of the input vector in common, each of the plurality of accumulators and the operator group may be implemented as described above with reference to FIGS. 2A, 2B, 3A, 3B, 4A and 4B. In addition, there may be a plurality of operator groups, each operator group may receive different input vector components, and the accumulators belonging to different operator groups may have second input terminals connected to one another to form an element of the matrix It is possible to form a cross-operator group that commonly receives the values, and can be implemented as described above with reference to FIG. 3C.

Accumulation registers or shadow registers of the plurality of accumulators may serve as an output buffer for storing the component YS of the output vector. In this case, the output buffer of the input/output buffer 1010 may be omitted.

As described above, by connecting the first input terminals or the second input terminals of the plurality of accumulators in common, it is possible to avoid duplicate application of the elements of the input vector or the elements of the matrix, thus Memory and bus bandwidth usage is reduced, which is advantageous in terms of performance and power consumption. In addition, the operation circuit 1000 receives the elements AS of the matrix stored in the memory cell array 1110 through the gating circuit 1140 , and includes the input/output buffer 1010 , the input/output data driving circuit 1150 , and the gating circuit. By receiving the components XS of the input vector stored in the memory cell array 1110 through 1140 and writing the components YS of the output vector to the memory cell array 1110, a path of data related to matrix operation can be confined inside the memory device, thus reducing memory and bus bandwidth usage and advantageous in terms of performance and power consumption.

For implementation reasons, the elements AS or the matrix generation information ASINF provided to the operation circuit 1000 through the gating circuit 1140 are transmitted to the operation circuit 1000 through the input/output data driving circuit 1150 . can be provided.

For implementation reasons, the matrix generation information may be stored in a separate memory device. In this case, the matrix generation information may be provided to the arithmetic circuit 1000 through the input/output data driving circuit 1150 .

For implementation reasons, the arithmetic circuit 1000 and the input/output buffer 1010 may be implemented on separate chips. In this case, an appropriate input/output data driving circuit may be inserted between the gating circuit 1140 and the arithmetic circuit 1000 and between the input/output data driving circuit 1150 and the input/output buffer 1010 to connect the two chips.

In an artificial neural network circuit, the outputs of one layer are obtained by applying an activation function after a matrix operation in which the inputs are multiplied by weights. In order to efficiently perform this, an activation function circuit may be inserted between the operation circuit 1000 and the input/output buffer 1010 .

Meanwhile, although not shown, the row address RADDR, the column address CADDR, and the data mask signal DMS may be provided from an external memory controller. The memory device 1100 and the memory controller may constitute a memory system.

In summary, the memory device 1100 of FIG. 7 can efficiently perform a matrix operation by combining the accumulator according to embodiments of the present invention. The elements of a matrix are stored in the memory cell array 1110 , and these data are transmitted to the operation circuit 1000 through the gating circuit 1140 (and the sense amplifier). The components of the input vector are stored in the input buffer and then transferred to the arithmetic circuit 1000 . The arithmetic circuit 1000 includes a plurality of accumulators, and first input terminals of the accumulators are connected in common. Components of the input vector received from the input buffer are applied to a common terminal of the accumulators, and matrix elements are applied to a second input terminal. For each component of the input vector, a value contributing to each component of the output vector is calculated and accumulated to obtain each component of the output vector and stored in the output buffer.

Referring to FIG. 8 , the memory device 1200 includes a memory cell array 1210 , a row decoder 1220 , a line decoder 1230 , a column decoder 1240 , a first gating circuit 1250 , and a second gating circuit ( 1260 ), an input/output data driving circuit 1270 , an arithmetic circuit 1000 , and an input/output buffer 1010 .

The memory device 1200 of FIG. 8 may have a structure similar to that of the memory device 1100 of FIG. 7 , except that the gating circuit is formed in two stages and thus further includes a line decoder 1230 . When the gating circuit is configured in two stages as shown in FIG. 8 , a plurality of columns selected by the first gating circuit 1250 is defined as one line.

The memory cell array 1210 and the row decoder 1220 may be substantially the same as the memory cell array 1110 and the row decoder 1120 of FIG. 7 , respectively.

The line decoder 1230 generates a line selection signal LSEL for selecting a target line including the target memory cell from among a plurality of lines included in the target row, based on the column address CADDR. Each of the plurality of lines may be defined to be included in one row and include two or more columns.

The column decoder 1240 generates a column selection signal CSEL for selecting a target column included in the target row and included in the target line and including the target memory cell, based on the column address CADDR.

The first gating circuit 1250 is connected to the target row based on the row selection signal RSEL and selects the target line based on the line selection signal LSEL. The second gating circuit 1260 selects the target column included in the target line based on the column selection signal CSEL. For example, the first gating circuit 1250 may further include a sense amplifier for sensing memory cells and stably processing signals.

The input/output data driving circuit 1270 writes input data DIN in the target column or outputs data stored in the target column as output data DOUT. At this time, it is possible to prevent unwanted input data from being written into the target column based on the data mask signal DMS. For example, the input/output data driving circuit 1270 may include an input data driving circuit and an output data driving circuit.

The target column may be connected to the input/output data driving circuit 1270 through the gating circuits 1250 and 1260 based on the line select signal LSEL and the column select signal CSEL. In the data write operation, the input data DIN may be provided to the gating circuits 1250 and 1260 in units of column data CDIN and line data LDIN and stored in the target column. In the data read operation, the page data RD stored in the target row is provided to the first gating circuit 1250 , and the data of the target line and the target column included in the target row are line data LDOUT and column data. It is provided to the second gating circuit 1260 and the output data driving circuit in units of (CDOUT), and may be output as output data DOUT.

The input/output buffer 1010 may be substantially the same as the input/output buffer 1010 of FIG. 7 . The operation circuit 1000 may be substantially the same as the operation circuit 1000 of FIG. 7 , except that it is connected to the first gating circuit 1250 .

In FIG. 8 , by configuring the gating circuit in two stages, the data size supplied to the operation circuit 1000 is larger than the column size, so that more accumulators can be driven at a time.

Referring to FIG. 9 , the memory device 1300 includes a memory cell array 1310 , a row decoder 1320 , a multi-column decoder 1330 , a gating circuit 1340 , an input/output data driving circuit 1350 , a type conversion and operation circuit. 1000 , and an input/output buffer 1010 .

The memory device 1300 of FIG. 9 may have a structure similar to that of the memory device 1100 of FIG. 7 , except that the column decoder 1130 is changed to the multi-column decoder 1330 .

The memory cell array 1310 and the row decoder 1320 may be substantially the same as the memory cell array 1110 and the row decoder 1120 of FIG. 7 , respectively.

The multi-column decoder 1330 includes a plurality of target columns TC (hatched lines) including the target memory cell among columns included in the target row TR, based on the column address CADDR and the column selection information CSINF. portion) to generate a multi-column select signal (MCSEL) for selecting at one time. In this case, there is no restriction that the addresses of the target columns TC are continuous in the column selection information CSINF. In other words, the target columns TC (ie, addresses of the target columns TC) may not be contiguous.

According to an embodiment, the column selection information CSINF may be set based on a predefined column selection table, may be set based on a column selection parameter, or may be set based on a column selection list or a column selection information list. may be

The gating circuit 1340 is connected to the target row TR based on the row select signal RSEL. Also, the gating circuit 1340 selects a plurality of target columns TC included in the target row TR based on the multi-column selection signal MCSEL and connects them to the input/output data driving circuit 150 at once. For example, the gating circuit 1340 may further include a sense amplifier for sensing memory cells and stably processing a signal.

The input/output data driving circuit 1350 writes input data DIN in the plurality of target columns TC at once or outputs data stored in the plurality of target columns TC as output data DOUT at once. In this case, unwanted input data is prevented from being written into the plurality of target columns TC based on the multi-column selection signal MCSEL and the data mask signal DMS. For example, the input/output data driving circuit 1350 may include an input data driving circuit and an output data driving circuit.

The plurality of target columns TC may be connected to the input/output data driving circuit 1350 through the gating circuit 1340 based on the multi-column selection signal MCSEL. In the data write operation, the input data DIN may be provided to the gating circuit 1340 in units of column data CDIN, and may be stored in a plurality of target columns TC. In the data read operation, the page data (or row data) RD stored in the target row TR may be provided to the gating circuit 1340 , and the data of the plurality of target columns TC included in the target row TR may be read. Data may be provided to the output data driving circuit in units of column data CDOUT, and may be output as output data DOUT.

In an embodiment, the size of the input/output data DIN or DOUT may be smaller than or equal to the size of the page data RD. For example, the size of the page data RD may be P (P is a natural number greater than or equal to 2) bits, and the size of the input/output data DIN or DOUT may be K (K is a natural number less than or equal to P) bits.

In an embodiment, the size of the column data CDIN or CDOUT may be smaller than the size of the input/output data DIN or DOUT. For example, the size of the column data CDIN or CDOUT may be C (C is a natural number less than K) bits. In this case, a plurality of columns (eg, K/C columns) may be selected to store one input data DIN or provide one output data DOUT, and accordingly, input/output data DIN or DOUT) is indicated by a single arrow, and column data (CDIN or CDOUT) is indicated by a plurality of arrows.

The input/output buffer 1010 may be substantially the same as the input/output buffer 1010 of FIG. 7 . The operation circuit 1000 may be substantially the same as the operation circuit 1000 of FIG. 7 .

In FIG. 9 , the degree of freedom in selecting data supplied to the operation circuit 1000 may be increased by using the multi-column decoder 1330 . In addition, since non-consecutive memory cells may be accessed at a time, performance and power efficiency of the memory device 1300 may be improved.

The present invention can be applied to various devices and systems including a computing device and a memory device that perform a matrix operation. Accordingly, the present invention is a mobile phone, smart phone, PDA, PMP, digital camera, camcorder, PC, server computer, workstation, notebook, digital TV, set-top box, music player, portable game console, navigation device, wearable device, IoT It may be usefully used in various electronic devices such as devices, VR devices, AR devices, and the like.

Although the above has been described with reference to the preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. you will understand that you can

Claims (30)

each of a first input terminal for receiving input vector components and a second input terminal for receiving elements of a matrix, and performing a cumulative operation on the input vector components and the elements of the matrix to obtain one of the output vector components one or more operator groups including a plurality of accumulators each generating one,
Each of the plurality of accumulators,
a first operator for generating a result of the first operation by performing a first operation on the input vector components received from the first input terminal and elements of the matrix received from the second input terminal;
a second operator for generating a result of the second operation by performing a second operation on the result of the first operation output from the first operator and the accumulation result provided from the accumulation register; and
and the accumulation register generating the accumulation result by accumulating the results of the second operation output from the second operator, and finally generating one of the output vector components,
When one of the two inputs of each of the plurality of accumulators is an n-base floating-point value (n is a natural number greater than or equal to 2) and the other is a power of n having an integer exponent, the first operator is Implemented as a circuit that bypasses the mantissa of a binary floating-point value and adds the exponent of the n-base floating-point value and the exponent of the power of n,
The first input terminal of the plurality of accumulators included in the one operator group is connected in common. The method of claim 1,
A plurality of the operator groups receive input vector components in one-to-one correspondence with a plurality of input vectors and generate output vector components in one-to-one correspondence with a plurality of output vectors,
a plurality of cross operator groups including a plurality of accumulators belonging to the different operator groups;
The second input terminal of the plurality of accumulators included in one crossover operator group is connected in common. The method of claim 1, wherein each of the plurality of accumulators comprises:
and at least one shadow register for temporarily storing the accumulation result generated by the accumulation register. The method of claim 1, wherein each of the plurality of accumulators comprises:
an auxiliary input terminal for receiving an auxiliary input, and the first input terminal for receiving the input vector components, selects one of the auxiliary input and the input vector components based on a selection signal to the first operator Computing device, characterized in that it further comprises a multiplexer to provide. The method of claim 1, wherein a first accumulator among the plurality of accumulators comprises:
an auxiliary input terminal for receiving an auxiliary input, and the first input terminal for receiving the input vector components, selects one of the auxiliary input and the input vector components based on a selection signal to the first operator Further comprising a multiplexer to provide,
and at least one accumulator disposed adjacent to the first accumulator among the plurality of accumulators shares the multiplexer with the first accumulator. The method of claim 1,
When one of the two inputs of each of the plurality of accumulators is an n-base integer value (n is a natural number greater than or equal to 2) and the other is a value of a power of n having an integer exponent,
and the first operator is implemented as a circuit for shifting the n-base integer value by the integer exponent. The method of claim 1,
The first operator is bypassed by fixing any one of two inputs of each of the plurality of accumulators to 1, and the second operator sets the value bypassed by the first operator to the value of the accumulation register. Comparing with and arithmetic device, characterized in that implemented as a comparator that outputs the larger of the two. The method of claim 1,
The matrix is a convolution matrix or a packed convolution matrix, and the input vector and the output vector are obtained by converting an input matrix and an output matrix into vectors, respectively. The method of claim 1,
wherein the matrix is an extended convolution matrix corresponding to a plurality of filter matrices, and the output vector is an extended output vector corresponding to the plurality of filter matrices. 3. The method of claim 2,
The matrix is a convolution matrix or a packed convolution matrix, and the plurality of input vectors and the plurality of output vectors are obtained by converting a plurality of input matrices and a plurality of output matrices into vectors, respectively. 3. The method of claim 2,
The matrix is a divided convolution matrix, and the plurality of input vectors and the plurality of output vectors are obtained by changing a plurality of divided input matrices and a plurality of divided output matrices into vectors, respectively. a memory cell array comprising a plurality of memory cells arranged to form a plurality of rows and a plurality of columns, the memory cell array storing elements of a matrix or matrix generation information for generating elements of the matrix;
a row decoder for generating a row selection signal for selecting a target row from among the plurality of rows based on a row address;
a column decoder for generating a column selection signal for selecting a target column from among columns included in the target row, based on a column address;
a gating circuit for selecting the target column based on the column selection signal;
an input/output data driving circuit for writing input data into the target column or outputting data stored in the target column as output data through the gating circuit;
an input/output buffer connected to the input/output data driving circuit and configured to store input vector components and output vector components; and
connected to the gating circuit and the input/output buffer, and including one or more operator groups including a plurality of accumulators, the input vector components provided from the input/output buffer and the elements of the matrix provided through the gating circuit Alternatively, the plurality of accumulators calculate the elements of the matrix generated with reference to the matrix generation information provided through the gating circuit in an input vector reference method to obtain the output vector components, and use the output vector components as the Comprising an arithmetic circuit providing input and output buffers,
Each of the plurality of accumulators includes a first input terminal for receiving the input vector components, and a second input terminal for receiving the elements of the matrix, wherein the input vector components and the elements of the matrix performing an accumulation operation to generate one of the output vector components,
Each of the plurality of accumulators,
a first operator for generating a result of the first operation by performing a first operation on the input vector components received from the first input terminal and elements of the matrix received from the second input terminal;
a second operator for generating a result of the second operation by performing a second operation on the result of the first operation output from the first operator and the accumulation result provided from the accumulation register; and
and the accumulation register generating the accumulation result by accumulating the results of the second operation output from the second operator, and finally generating one of the output vector components,
When one of the two inputs of each of the plurality of accumulators is an n-base floating-point value (n is a natural number greater than or equal to 2) and the other is a power of n having an integer exponent, the first operator is Implemented as a circuit that bypasses the mantissa of a binary floating-point value and adds the exponent of the n-base floating-point value and the exponent of the power of n,
The first input terminal of the plurality of accumulators included in the one operator group is connected in common. 13. The method of claim 12,
A plurality of the operator groups receive input vector components in one-to-one correspondence with a plurality of input vectors and generate output vector components in one-to-one correspondence with a plurality of output vectors,
a plurality of cross operator groups including a plurality of accumulators belonging to the different operator groups;
and the second input terminals of the plurality of accumulators included in one crossover operator group are connected in common. 13. The method of claim 12, wherein each of the plurality of accumulators comprises:
and at least one shadow register for temporarily storing the accumulation result generated by the accumulation register. 13. The method of claim 12, wherein each of the plurality of accumulators comprises:
an auxiliary input terminal for receiving an auxiliary input, and the first input terminal for receiving the input vector components, selects one of the auxiliary input and the input vector components based on a selection signal to the first operator Memory device, characterized in that it further comprises a multiplexer to provide. 13. The method of claim 12, wherein a first accumulator among the plurality of accumulators comprises:
an auxiliary input terminal for receiving an auxiliary input, and the first input terminal for receiving the input vector components, selects one of the auxiliary input and the input vector components based on a selection signal to the first operator Further comprising a multiplexer to provide,
At least one accumulator disposed adjacent to the first accumulator among the plurality of accumulators shares the multiplexer with the first accumulator. 13. The method of claim 12,
When one of the two inputs of each of the plurality of accumulators is an n-base integer value (n is a natural number greater than or equal to 2) and the other is a value of a power of n having an integer exponent,
and the first operator is implemented as a circuit for shifting the n-base integer value by the integer exponent. 13. The method of claim 12,
The first operator is bypassed by fixing any one of two inputs of each of the plurality of accumulators to 1, and the second operator sets the value bypassed by the first operator to the value of the accumulation register. A memory device, characterized in that it is implemented as a comparator that compares with and outputs the larger of the two values. 13. The method of claim 12,
wherein the matrix is a convolution matrix or a packed convolution matrix, and the input vector and the output vector are obtained by converting an input matrix and an output matrix into vectors, respectively. 13. The method of claim 12,
wherein the matrix is an extended convolution matrix corresponding to a plurality of filter matrices, and the output vector is an extended output vector corresponding to the plurality of filter matrices. 14. The method of claim 13,
wherein the matrix is a convolution matrix or a packed convolution matrix, and the plurality of input vectors and the plurality of output vectors are obtained by converting a plurality of input matrices and a plurality of output matrices into vectors, respectively. 14. The method of claim 13,
wherein the matrix is a partitioned convolution matrix, and the plurality of input vectors and the plurality of output vectors are obtained by converting a plurality of partitioned input matrices and a plurality of partitioned output matrices into vectors, respectively. 13. The method of claim 12,
and a line decoder for generating a line selection signal for selecting a target line including the target column from among a plurality of lines included in the target row and each including two or more columns, based on the column address; ,
The gating circuit includes a first gating circuit that selects the target line based on the line selection signal and a second gating circuit that selects the target column based on the column selection signal,
the input/output data driving circuit writes the input data to the target column or outputs data stored in the target column as the output data through the first and second gating circuits;
and the arithmetic circuit is connected to the first gating circuit. 13. The method of claim 12,
the column decoder is a multi-column decoder that generates a multi-column selection signal for selecting a plurality of target columns from among columns included in the target row at a time based on the column address and column selection information;
the gating circuit selects the plurality of target columns at once based on the multi-column selection signal,
the input/output data driving circuit writes the input data in the plurality of target columns at a time through the gating circuit or outputs data stored in the plurality of target columns as the output data at a time;
and the column addresses corresponding to the plurality of target columns included in the target row are not contiguous.

delete delete delete delete delete delete

Images