KR20230160177A - Apparatus and method for multiplication operation based on outer product - Google Patents

Abstract

A cross product-based multiplication operation apparatus and method are disclosed. An external product-based multiplication operation device according to an embodiment includes first internal operators each performing a MAC (Multiply-Accumulation) operation to generate an intermediate accumulation value, each using the intermediate accumulation value to generate a unit accumulation value (chucking accumulation value). It may include second internal operators that generate , and accumulation data transmission paths that allow the output of any one of the first internal operators to be input to one of the second internal operators.

Description

Outer product-based multiplication operation device and method {APPARATUS AND METHOD FOR MULTIPLICATION OPERATION BASED ON OUTER PRODUCT}

The described embodiment relates to a multiple accumulation operator processing technology for accuracy compensation of low-precision calculations of an artificial intelligence processor.

Quantization techniques are widely used to accelerate the learning and inference of artificial neural networks and reduce memory usage. This is a technique for processing neural network calculations using a low-precision integer or floating point data format of 8 bits or less, rather than calculating in a 32-bit floating point format.

Here, the floating point format consists of a sign (Sign, S), an exponent (E), and a mantissa (M). Among these, the bit width of the mantissa can be used to express the precision of the number. In other words, if the exponent allocation is large and the mantissa part is small, it is a format that sparsely expresses a wide range of numbers. Conversely, if the exponent is small and many mantissas are assigned, it means a format that expresses numbers in a narrow range in detail.

However, when quantization techniques are applied to the accumulation operation of a neural network, information loss due to quantization intensifies. In other words, the lower the bit-width of the mantissa, which expresses the precision of the cumulative calculation data format in floating point format, the larger the calculation error due to information loss can be. This phenomenon is called swamping.

To alleviate this swamping phenomenon, the unit accumulation technique is typically used. A prior literature in which the unit accumulation (Chunking Accumulation) technique is applied to a deep learning accelerator is "Deep learning accelerator architecture with chunking GEMM" (US Public Publication US 2019-0325301).

Looking at previous literature, in the unit accumulation (chunking accumulation) technique, the data subject to the accumulation operation is divided into predetermined units (chunks) and the first step is to accumulate the results for each unit. It proceeds to the second stage.

Through this process, the accumulated value can be distributed evenly, minimizing information loss caused by lack of precision during the floating point addition process.

However, in this unit accumulation (chunking accumulation) technique, the accumulation operation is performed by dividing the data subject to the accumulation operation into predetermined units (chunks), so the accumulation operation of the first stage and the accumulation operation of the second stage can be performed simultaneously through the same operator. does not exist.

Therefore, whenever the accumulation operation is completed for each unit (chunk), the operation result must be stored in external memory. Frequent memory access for this purpose delays the accumulation operation, which can lead to a slowdown in overall learning and inference speed.

Therefore, to solve this problem, a neural network accelerator is being implemented to perform a two-step accumulation operation inside the calculator by embedding hardware dedicated to unit accumulation (chunking accumulation). However, installing a double accumulation operation circuit in all internal operators of a neural network acceleration circuit still causes the problem of expanding the hardware area.

The described embodiment is aimed at compensating the accuracy of a neural network accumulation operation using a low-precision data format in an artificial intelligence processor.

The purpose of the described embodiment is to solve the problem of slowing down learning and inference speed due to frequent memory access when applying the unit accumulation (chunking accumulation) technique in an artificial intelligence processor.

The purpose of the described embodiment is to solve the problem of hardware area expansion for unit accumulation (chunking accumulation) calculation in an artificial intelligence processor.

An external product-based multiplication operation device according to an embodiment includes first internal operators each performing a MAC (Multiply-Accumulation) operation to generate an intermediate accumulation value, each using the intermediate accumulation value to generate a unit accumulation value (chucking accumulation value). It may include second internal operators that generate , and accumulation data transmission paths that allow the output of any one of the first internal operators to be input to one of the second internal operators.

At this time, the second internal operators can each receive the output of any one of the first internal operators.

At this time, the first internal operators may transfer the intermediate accumulated value to one of the second internal operators at every cycle corresponding to a chunk size corresponding to the intermediate accumulated value, and then perform a reset operation.

At this time, the second internal operators may each perform an operation to generate the unit accumulation value at every cycle corresponding to the unit size.

At this time, the operation for generating the unit accumulation value may be an operation of adding a newly input intermediate accumulation value and an intermediate accumulation value stored in the second internal calculator.

At this time, it may further include a control unit that inputs a control signal to adjust the unit size (chunk size) to each of the first internal operators and the second internal operators.

At this time, each of the second internal operators may be switched from an inactive state to an active state every cycle corresponding to a unit size.

The outer product-based multiplication operation method according to the embodiment includes generating an intermediate accumulation value by performing a MAC (Multiply-Accumulation) operation by each of the first internal operators, and generating the intermediate accumulation value through an accumulation data transmission path. It may include transmitting to one of two internal operators, and generating a unit accumulation value (chucking accumulation value) using the intermediate accumulation value by each of the second internal operators.

At this time, the transmitting step may transfer the intermediate accumulated value to one of the second internal operators at each cycle corresponding to a unit size (chunk size) corresponding to the intermediate accumulated value, and then perform a reset operation.

At this time, the step of generating the unit accumulation value may involve performing an operation to generate the unit accumulation value at each cycle corresponding to the unit size.

At this time, the operation for generating the unit accumulation value may be an operation of adding a newly input intermediate accumulation value and an intermediate accumulation value stored in the second internal calculator.

At this time, before the step of generating the unit accumulation value, the second internal operators may further include a step of switching from an inactive state to an active state every cycle corresponding to the unit size.

An external product-based multiplication operation device according to an embodiment includes first internal operators each performing a MAC (Multiply-Accumulation) operation to generate an intermediate accumulation value, each using the intermediate accumulation value to generate a first unit accumulation value (chucking accumulation value). second internal operators that generate a chucking accumulation value, third internal operators that generate a second unit accumulation value (chucking accumulation value) using the first unit accumulation value, respectively, any one of the first internal operators Accumulation data transmission paths for causing an output to be input to one of the second internal operators, and to cause an output of any one of the first internal operators to be input to one of the second internal operators. It may include accumulation data transmission paths.

At this time, the second internal operators may each receive an output from one of the first internal operators, and the third internal operators may each receive an output from one of the second internal operators.

At this time, the first internal operators transfer the intermediate accumulated value to one of the second internal operators at every cycle corresponding to a first unit size (chunk size) corresponding to the intermediate accumulated value, and then perform a reset operation, The second internal operators each calculate the first unit accumulation value to the first unit accumulation value every period corresponding to a size obtained by multiplying the first unit size by a second unit size (chunk size) set for unit accumulation of the first unit accumulation value. 3 After passing it to one of the internal operators, a reset operation can be performed.

At this time, the second internal operators each perform an operation to generate the first unit accumulation value at every cycle corresponding to the first unit size, and the third internal operators respectively perform the operation to generate the first unit accumulation value and the first unit size. An operation for generating the second unit accumulation value may be performed every cycle corresponding to a size multiplied by a second unit size (chunk size) set for unit accumulation of the unit accumulation value.

At this time, the operation for generating the first unit accumulation value is an operation of adding the newly input intermediate accumulation value and the intermediate accumulation value stored in the second internal calculator, and the operation for generating the second unit accumulation value is It may be an operation that adds a newly input first unit accumulation value and a second unit accumulation value stored in the third internal calculator.

At this time, it may further include a control unit that inputs a control signal that adjusts a unit size (chunk size) or a period of performing a reset operation corresponding to each of the first to third internal operators.

At this time, the second internal operators are switched from the inactive state to the active state at every cycle corresponding to the first unit size, and the third internal operators are each switched from the inactive state to the active state at a cycle corresponding to the size multiplied by the first unit size and the second unit size. It can be switched from inactive to active state every time.

The outer product-based multiplication operation device according to the embodiment is for generating an N+2-th unit accumulation value using the N+1-th unit accumulation value (N is an integer that increases by 1 from 1). N+3 internal operators that perform operations, and an accumulation data transmission path that causes the output of one of the N+2 internal operators to be input to one of the N+3 internal operators. ) are repeatedly added as N increases, but the operation to generate the N+2 unit accumulated value is performed using the newly input N+1 unit accumulated value and the N stored in the N+3 internal operator. This may be an operation that adds +2 unit cumulative value.

According to the described embodiment, the accuracy of a neural network accumulation operation using a low-precision data format can be compensated for in an artificial intelligence processor.

According to the described embodiment, when applying the unit accumulation (chunking accumulation) technique in an artificial intelligence processor, the problem of slowing down learning and inference speed due to frequent memory access can be solved.

According to the described embodiment, the problem of hardware area expansion for unit accumulation (Chunking Accumulation) operation in an artificial intelligence processor can be solved. In other words, the structure of the conventional outer-product-based matrix multiplication operator can be reused as is.

According to the described embodiment, accelerator utilization of up to 100% can be achieved by vector-matrix product operation. In the case of vector-matrix product operation, the operation rate of the operator can be improved up to 100% by utilizing the deactivated internal operator (ex. FPU) for multiple accumulation operation, and the unit accumulation has a structure that allows deep accumulation of two or more stages, so quantization It has the effect of reducing errors.

In these described embodiments, low-precision calculations can be widely used in AI semiconductors for fast calculation speed and low power usage. In addition, since unit accumulation is an essential computational element in the process, the hardware structure that accelerates it will be used in various AI semiconductors in the future, so it has high marketability and commercialization potential.

1 is an internal configuration diagram of a general external product-based multiplication operation device.
Figure 2 is a flowchart for explaining the unit accumulation process in a general external product-based multiplication operation device.
Figure 3 is an internal configuration diagram of an external product-based multiplication operation device according to an embodiment.
Figure 4 is a flowchart for explaining a cross product-based multiplication operation method according to an embodiment.
Figure 5 is an example of vector-matrix multiplication performed in a general outer product-based multiplication operation device.
Figure 6 is an internal configuration diagram of a cross product-based multiplication operation device according to another embodiment.
Figure 7 is an internal configuration diagram of a cross product-based multiplication operation device according to another embodiment.
Figure 8 is a diagram showing the configuration of a computer system according to an embodiment.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and will be implemented in various different forms. The present embodiments only serve to ensure that the disclosure of the present invention is complete and that common knowledge in the technical field to which the present invention pertains is not limited. It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

Although terms such as “first” or “second” are used to describe various components, these components are not limited by the above terms. The above terms may be used only to distinguish one component from another component. Accordingly, the first component mentioned below may also be the second component within the technical spirit of the present invention.

The terms used in this specification are for describing embodiments and are not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” or “comprising” implies that the mentioned component or step does not exclude the presence or addition of one or more other components or steps.

Unless otherwise defined, all terms used in this specification can be interpreted as meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, a cross product-based multiplication operation device and method according to an embodiment will be described in detail with reference to FIGS. 1 to 8.

FIG. 1 is an internal configuration diagram of a general external product-based multiplication operation device, and FIG. 2 is a flowchart for explaining a unit accumulation process in a general external product-based multiplication operation device.

Referring to FIG. 1, a general outer product-based matrix operator 100 may be composed of a plurality of internal operators 110 that perform a MAC (Multiply-Accumulation) operation.

A matrix-matrix multiplication operation using a unit accumulation (chucking accumulation) technique can be performed through these plurality of internal operators 110.

Referring to FIG. 2, matrix data for the outer product operation is loaded from the external memory 10 into the outer product-based matrix operator 100 (S210).

Here, the external memory 10 is a memory 1030, a storage 1060, and a network 1080 installed outside the external product-based multiplication operation unit 1090 in the computer system 1000 as shown in FIG. 8, which will be described later. It can include all external memory accessible through .

Then, the outer product-based matrix operator 100 performs a matrix-matrix multiplication operation on matrix data of a predetermined unit size (Chunk Size) from the loaded matrix data (S220).

Then, the outer product-based matrix operator 100 calls the previously stored partial accumulation value from the external memory 10 (S230), and performs element-wise addition with the matrix-matrix product operation result value in S220. ) is performed (S240).

At this time, when the matrix-matrix multiplication operation is performed for the first time, since there is no previously stored partial accumulation value, S230 and S240 may be omitted.

Then, the outer product-based matrix operator 100 performs addition for each element and stores the updated partial accumulation value back in the memory 10 (S250).

However, since the matrix-matrix multiplication operation was performed on matrix data of a predetermined unit size (Chunk Size) rather than the entire matrix data loaded in S220, if there is more matrix data to be calculated (S260), the cross product-based matrix operator ( 100) proceeds to S220 and re-performs S220 to S250.

Alternatively, in FIG. 2, steps S250 and S260 may be performed first before step S230, and the cross product-based matrix operator 100 performs element-by-element addition of all partial accumulation values stored in S230 at S240, and adds the element-by-element addition The result may be passed back to the memory 10.

Alternatively, the outer product-based matrix operator 100 may only perform steps S210, S220, S250, and S260 in FIG. 2, and steps S230 and S240 may be performed by a separate external processor.

As described above, the conventional external product-based matrix operator 100 stores the operation result in the external memory 10 every time the MAC (Multiply-Accumulation) operation on matrix data as large as a unit (chunk) is completed. You have to save it. Frequent memory access in this process delays computation time, which can lead to a slowdown in overall learning and inference speed. Additionally, a separate external processor may be required for element-by-element addition.

Accordingly, the embodiment proposes an external product-based multiplication operation device and method that does not require the addition of additional hardware and does not require the operation result to be stored in an external memory every time the accumulation operation is completed for each unit (chunk).

Figure 3 is an internal configuration diagram of an external product-based multiplication operation device according to an embodiment.

Referring to FIG. 3, the outer product-based multiplication operation apparatus 300 according to an embodiment includes first internal operators 310 and second internal operators 320, and the first internal operators 310 and Accumulation data transmission paths 330 may be formed between the second internal operators 320.

Here, an external product-based multiplication operation device 300 in which internal operators have a 4x4 structure is shown, but this is only an example to aid understanding of the embodiment, and the present invention is not limited thereto.

In addition, the first internal operators 310 are operators that input data from two external input ports in FIG. 3, for example, E0.0, E1.0, ??. It may mean E7.0, and the second internal operators 310 are operators that receive data from the adjacent first internal operators 310 rather than from the external input port in FIG. 3, that is, It can mean E0.1, E1.1,..., E7.1.

For example, the cross product-based multiplication operation device 600 sequentially receives vector data of a first matrix through input ports of A0 to A4, and sequentially inputs vector data of a second matrix through input ports of B0 to B1. You can perform matrix-matrix multiplication operations.

Here, the vector data of the first matrix and the data of the second matrix may include, for example, neural network intermediate calculation data, for example, feature map data, or neural network learning parameters, for example, weights.

At this time, data input through the input ports may be quantized data in which the bit-width of the mantissa, which expresses precision in a floating point format, is adjusted.

The external product-based multiplication operation device 300 according to the embodiment minimizes additional configuration other than the accumulation data transmission paths 330 in hardware configuration by unit accumulation (chunking accumulation) operation. Therefore, it is possible to solve problems caused by unit accumulation (chunking accumulation) operation while utilizing the structure of the external product-based multiplication operation device 300.

Each of the first internal operators 310 may generate an intermediate accumulation value by performing a Multiple-Accumulation (MAC) operation.

Accumulation data transmission paths 330 may allow the output of one of the first internal operators 310 to be input to one of the second internal operators 320.

Accordingly, each of the first internal operators 310 transfers the intermediate accumulated value to one of the adjacent second internal operators 320, rather than storing the intermediate accumulated value in a separate memory.

Then, each of the second internal operators 320 can receive the output of any one of the first internal operators 310.

Each of the second internal operators 320 may perform an operation to generate a unit accumulation value (chucking accumulation value) using the transmitted intermediate accumulation value.

Here, the operation for generating the unit accumulation value may be an operation of adding a newly input intermediate accumulation value and an intermediate accumulation value stored in the second internal calculator 320.

At this time, each of the first internal operators 310 transfers the intermediate accumulated value to one of the second internal operators 320 at every cycle corresponding to a unit size (chunk size) corresponding to the intermediate accumulated value. A reset operation can be performed.

Additionally, the second internal operators 320 may each perform an operation to generate the unit accumulation value at every cycle corresponding to the unit size.

That is, each of the second internal operators 320 may generate a final accumulated value by accumulating only the intermediate accumulated values received from one of the first internal operators 310 at each cycle corresponding to the unit size. .

To this end, the outer product-based multiplication operation device 300 according to the embodiment includes a control unit (not shown) that inputs a control signal for adjusting the unit size (chunk size) to each of the first internal operators and the second internal operators. ) may further be included. According to another embodiment, the control signal may be transmitted from an external control unit.

This unit size (chunk size) is determined by the type of data that is the target of the cross product-based multiplication operation, the bit-width of the mantissa that expresses the precision of the data in floating point format, and the application in which the data is used. It may be adjusted according to at least one of various properties including characteristics.

The first internal operators 310 may each have their intermediate accumulated value transfer timing and reset timing set based on the unit size (chunk size).

Each of the second internal operators 320 may be set at a time to perform an operation for generating the unit accumulation value based on the unit size.

At this time, the second internal operators 320 may be set to switch from the inactive state to the active state at intervals corresponding to each unit size.

Figure 4 is a flowchart for explaining a cross product-based multiplication operation method according to an embodiment.

Referring to FIG. 4, the external product-based multiplication operation device 300 according to the embodiment first loads matrix data from the external memory 10 (S410).

Here, the external memory 10 is a memory 1030, a storage 1060, and a network 1080 installed outside the external product-based multiplication operation unit 1090 in the computer system 1000 as shown in FIG. 8, which will be described later. It can include all external memory accessible through .

Then, each of the first internal operators 310 of the external product-based multiplication operation device 300 performs a MAC (Multiply-Accumulation) operation to generate an intermediate accumulated value (S420).

Then, each of the first internal operators 310 transmits the intermediate accumulated value to one of the second internal operators through the accumulated data transmission path (S430).

Each of the second internal operators 320 generates a unit accumulation value using the intermediate accumulation value (S440).

Then, if calculation target data exists (S450), steps S420 to S440 may be repeatedly performed.

On the other hand, when the calculation target data does not exist (S450), each of the first internal operators 310 transmits an operation completion signal through the accumulated data transmission path (S460), and each of the first internal operators 310 Transfers the unit accumulation value accumulated so far to the memory 10 as the final result value (S470).

In the above-described embodiment, intermediate accumulation values are accumulated by the second internal operators 320, so when comparing the conventional method of FIG. 2 and the method according to the embodiment of FIG. 4, the outer product-based multiplication operation device ( 300) eliminates the need for external memory access.

Additionally, since the element-wise addition process is omitted, the overall result can be derived simply.

However, in the conventional outer product-based matrix operator 100 shown in FIG. 1, the cumulative value of the LxL-dimension matrix can be calculated every cycle. Here, L means the size of the input port. For example, since the cross product-based matrix operator 100 shown in FIG. 1 has four input ports from A0 to A3, the accumulated value of a 4x4 dimensional matrix can be calculated for each cycle.

On the other hand, in the outer product-based matrix operator 300 according to the embodiment shown in FIG. 3, the cumulative value of the Lx(L/2)-dimension matrix can be calculated every cycle. This is because each of the second internal operators 120 does not receive data from the outside, but is used for unit accumulation of intermediate values transmitted from one of the second internal operators 120.

However, as described above, the overall computation speed can be improved because the delay time due to external memory access is reduced.

In addition, as described above, the unit accumulation technique is applied to alleviate the swamping phenomenon caused by the application of the quantization technique to the accumulation operation of the neural network, so accurate operation results with reduced errors due to quantization can be obtained. You can.

Figure 5 is an example of vector-matrix multiplication performed in a general outer product-based multiplication operation device.

The general outer product-based multiplication operator 100 shown in FIG. 1 has an optimized structure to efficiently process matrix-matrix multiplication. Therefore, as shown in FIG. 5, when a vector-matrix multiplication operation is performed in the same hardware structure as the conventional external product-based multiplication operation device 100, the utilization of the internal operators does not achieve 100%. Only a small portion of it is used. For example, only internal operators corresponding to EO to E3 are used, and the remaining internal operators are not utilized.

Therefore, in the embodiment, internal operators that cannot be used in the vector-matrix multiplication operation are used in the multiple accumulation operation. In other words, the unit accumulation (chucking accumulation) technique is applied using unused internal operators.

Figure 6 is an internal configuration diagram of a cross product-based multiplication operation device according to another embodiment.

Referring to FIG. 6, the outer product-based multiplication operation device 500 according to another embodiment includes first internal operators 510 and second internal operators 520, and the first internal operators 510 and accumulation data transmission paths 550-1 may be formed between the second internal calculators 520.

In addition, the outer product-based multiplication operation device 500 according to another embodiment includes third internal operators 530, and transfers accumulated data between the second internal operators 520 and the third internal operators 530. Accumulation data transmission paths 550-2 may be formed.

In addition, the outer product-based multiplication operation device 500 according to another embodiment includes fourth internal operators 540, and transfers accumulated data between the third internal operators 530 and the fourth internal operators 540. Accumulation data transmission paths 550-3 may be formed.

In addition, the first internal operators 510 may refer to operators that input data from two external input ports in FIG. 6, for example, E0.0, E1.0,..., E3.0, , the second internal operators 520 are operators that receive data from the adjacent first internal operators 510 rather than from an external input port in FIG. 6, that is, E0.1, E1.1, ..., may mean E3.1, and the third internal operators 530 are operators that receive data from adjacent second internal operators 520 rather than from the external input port in FIG. , that is, it may mean E0.2, E1.2,..., E3.2, and the fourth internal operators 540 are the adjacent third operators in FIG. 6 where data is not input from the external input port. This may refer to operators that receive data from the internal operators 530, that is, E0.3, E1.3,..., E3.3.

That is, the external product-based multiplication operation device 500 according to the embodiment includes accumulation data transmission paths 550-1, 550-2, and 550 in a hardware configuration based on unit accumulation (chunking accumulation) operation. Minimize additional configuration other than -3). Therefore, it is possible to solve problems caused by unit accumulation (chunking accumulation) operation while utilizing the structure of the external product-based multiplication operation device 300.

Furthermore, in the external product-based multiplication operation device 500 according to another embodiment, the second internal operators 520 to the fourth internal operators 540 are utilized for high-order accumulation, thereby not only increasing the operating efficiency of the internal operators. In addition, quantization error, which is a problem caused by unit accumulation (chunking accumulation) operation, can be further reduced.

For example, when the total accumulated data is 10000, the first internal operators 510 may transfer the accumulated values in units of 10 to the second internal operators 520. The second internal operators 520 accumulate the values accumulated in units of 10 by the first internal operators 510 again in units of 10 and store a total of 100 accumulated values in the third internal operators 530. It can be delivered. The third internal operators 530 accumulate the values accumulated in units of 10 by the second internal operators 520 again in units of 10 and store a total of 1000 accumulated values in the third internal operators 530. It can be delivered. The fourth internal operators 540 may accumulate the values accumulated in units of 10 by the third internal operators 530 again in units of 10 and output a total of 10,000 accumulated final values.

The third internal operators 530 accumulate the values accumulated in units of 10 again in units of 10 and transfer a total of 100 accumulated values to the third internal operators 530, and the first internal operators ( 510) Each can generate an intermediate accumulation value by performing a MAC (Multiply-Accumulation) operation.

Accumulation data transmission paths 550-1 may allow the output of one of the first internal operators 510 to be input to one of the second internal operators 520. .

Accordingly, each of the first internal operators 510 transfers the intermediate accumulated value to one of the adjacent second internal operators 520, rather than storing the intermediate accumulated value in a separate memory.

Then, each of the second internal operators 520 can receive the output of any one of the first internal operators 510.

Each of the second internal operators 520 may perform an operation to generate a first unit accumulation value (chucking accumulation value) using the transmitted intermediate accumulation value.

Here, the operation for generating the unit accumulation value may be an operation of adding a newly input intermediate accumulation value and an intermediate accumulation value stored in the second internal calculator 520.

Accumulation data transmission paths 550-2 may allow the output of one of the second internal operators 520 to be input to one of the third internal operators 530. .

Accordingly, each of the second internal calculators 520 transfers the first unit accumulation value to one of the adjacent third internal calculators 530 rather than storing it in a separate memory.

Then, each of the third internal operators 520 can receive the output of any one of the first internal operators 510.

Each of the third internal operators 530 may perform an operation to generate a second unit accumulation value (chucking accumulation value) using the transmitted first unit accumulation value.

Here, the operation for generating the unit accumulation value may be an operation of adding the newly input first unit accumulation value and the second unit accumulation value stored in the third internal calculator 530.

Accumulation data transmission paths 550-3 may allow the output of one of the third internal operators 530 to be input to one of the fourth internal operators 540. .

Accordingly, each of the third internal calculators 530 transfers the second unit accumulation value to one of the adjacent fourth internal calculators 540 rather than storing it in a separate memory.

Then, each of the third internal operators 540 can receive the output of one of the second internal operators 520.

Each of the fourth internal operators 540 may perform an operation to generate a third unit accumulation value (chucking accumulation value) using the transmitted second unit accumulation value.

Here, the operation for generating the third unit accumulation value may be an operation that adds the newly input second unit accumulation value and the third unit accumulation value stored in the fourth internal calculator 540.

Here, the operation for generating the unit accumulation value is described as being performed in three stages, but this is only an example to aid understanding of the embodiment, and the present invention is not limited thereto. In other words, the operation to generate the unit accumulation value may be performed in three or more stages (stages) depending on the number of internal operators, or may be performed in two stages (stages) without utilizing all of the internal operators shown in FIG. 6. It may be possible.

To this end, the external product-based multiplication operation apparatus 500 according to the embodiment may further include a control unit (not shown) that inputs a control signal that adjusts the number of steps of the operation that generates the unit accumulation value. According to another embodiment, the control signal may be transmitted from an external control unit.

The number of steps of the operation that generates this unit accumulated value depends on the type of data that is the target of the cross product-based multiplication operation, the bit-width of the mantissa expressing the precision of the data in floating point format, and the corresponding data. may be adjusted according to at least one of various properties including characteristics of the application being used.

Meanwhile, the first internal operators 510 transmit the intermediate accumulated value to one of the second internal operators 520 at each cycle corresponding to the first unit size (chunk size) corresponding to the intermediate accumulated value. After that, a reset operation can be performed.

For example, if the total accumulated data is 10,000, the first internal operators 510 may be reset every 10 accumulated time periods, a total of 1,000 times.

Additionally, the second internal operators 320 may each perform an operation to generate the first unit accumulation value at every cycle corresponding to the first unit size.

That is, each of the second internal operators 320 accumulates only the intermediate accumulated values received from one of the first internal operators 310 at every cycle corresponding to the first unit size, and generates the first unit accumulated value. can be created.

In addition, the second internal operators 520 each calculate the first chunk size at every cycle corresponding to a size obtained by multiplying the first unit size by a second unit size (chunk size) set for unit accumulation of the first unit accumulation value. After transferring the unit accumulation value to one of the third internal operators 530, a reset operation can be performed.

For example, if the total accumulated data is 10,000, the second internal operators 520 accumulate the values accumulated in units of 10 by the first internal operators 520 again in units of 10, and a total of 100 are accumulated. It is reset every time period, and can be reset a total of 100 times.

In addition, the third internal operators 330 each calculate the second chunk size at every cycle corresponding to a size obtained by multiplying the first unit size by a second unit size (chunk size) set for unit accumulation of the first unit accumulation value. You can perform operations to generate unit accumulation values.

That is, each of the third internal operators 320 uses only the first unit accumulation values received from one of the second internal operators 310 as a first unit size (chunk size) and a second unit size (chunk size). ) can be accumulated for each period corresponding to the size multiplied by , thereby generating a second unit accumulation value.

In addition, the third internal operators 530 calculate the second unit accumulation value at every cycle corresponding to a size multiplied by the first unit size (chunk size) and the second unit size (chunk size), respectively. After transferring it to one of the operators 530, a reset operation can be performed.

For example, if the total accumulated data is 10000, the third internal operators 530 accumulate the values accumulated in units of 10 by the second internal operators 530 again in units of 10, for a total of 1000. It is reset every time period, and can be reset a total of 10 times.

In addition, the fourth internal operators 540 each correspond to a size obtained by multiplying the first unit size, the second unit size, and the third unit size (chunk size) set for unit accumulation of the second unit accumulation value. An operation to generate a third unit cumulative value may be performed for each cycle.

That is, each of the fourth internal operators 540 only uses the second accumulated values received from one of the third internal operators 530, such as the first unit size, the second unit size, and the second unit accumulated value. The final accumulation value can be generated by accumulating at every cycle corresponding to the size multiplied by the third unit size (chunk size) set for unit accumulation.

For example, if the total accumulated data is 10,000, the fourth internal operators 540 accumulate the values accumulated in units of 10 by the third internal operators 530 again in units of 10, for a total of 10,000. The final accumulated value can be generated.

To this end, the external product-based multiplication operation device 500 according to the embodiment adjusts the unit size (chunk size) or the period for performing the reset operation corresponding to each of the first to third internal operators. It may further include a control unit (not shown) that inputs a control signal. According to another embodiment, the control signal may be transmitted from an external control unit.

These first unit sizes (chunk size) to third unit sizes (chunk size) are the type of data that is the target of the cross product-based multiplication operation, and the bit width (bit-) of the mantissa that expresses the precision of the data in floating point format. width), and the data may be adjusted according to at least one of various properties including characteristics of the application in which the data is used.

Each of the first internal operators 510 may set an intermediate accumulated value transfer time and a reset time based on the first unit size (chunk size).

Each of the second internal operators 520 may be set at a time to perform an operation for generating the unit accumulation value based on the first unit size.

At this time, the second internal operators 320 may be set to switch from the inactive state to the active state at intervals corresponding to the first unit size.

Additionally, the first unit accumulation value transfer time and reset time of the second internal operators 520 may be set based on the first unit size and the second unit size, respectively.

The third internal operators 530 may be set at a time when they perform an operation for generating the second unit accumulation value based on the first unit size and the second unit size, respectively.

At this time, the third internal operators 530 may be set to switch from the inactive state to the active state at every cycle based on the first unit size and the second unit size, respectively.

In addition, the third internal calculators 530 may set the second unit accumulation value transfer time and reset time based on the first unit size to the third unit size, respectively.

The fourth internal operators 540 may each be set at a time to perform an operation for generating the third unit accumulation value based on the first unit size to the third unit size.

At this time, the fourth internal operators 540 may be set to switch from the inactive state to the active state at intervals based on the first to third unit sizes, respectively.

As described above, by utilizing the second internal operators 520 to the fourth operators 540, the unit accumulation (Chunking Accumulation) technique is performed in multi-stage, thereby improving hardware utilization rate. You can do it. In addition, through deep accumulation, it is possible to reduce the information loss rate of accumulated results caused by rapidly large or small values among calculation input values. As a result, calculation results with reduced quantization errors can be derived, and the accuracy of learning and inference using this can be improved.

Figure 7 is an internal configuration diagram of a cross product-based multiplication operation device according to another embodiment.

Referring to FIG. 7 , the cross product-based multiplication operation device 600 according to another embodiment performs matrix-matrix multiplication like the cross product-based multiplication operation device 600 shown in FIG. 3 . As shown, this is an embodiment applying another embodiment in which the operation for generating a unit accumulation value is performed in multi-stages.

For example, the cross product-based multiplication operation device 600 sequentially receives vector data of a first matrix through input ports of A0 to A5, and sequentially inputs vector data of a second matrix through input ports of B0 to B1. It takes a matrix-matrix multiplication operation.

The outer product-based multiplication operation device 600 according to another embodiment includes first internal operators 610 and second internal operators 620, and the first internal operators 610 and the second internal operators 620. Accumulation data transmission paths 640-1 may be formed between the fields 620.

In addition, the outer product-based multiplication operation device 500 according to another embodiment includes third internal operators 630, and accumulates data between the second internal operators 620 and the third internal operators 630. Accumulation data transmission paths 540-2 may be formed.

In addition, the first internal operators 610 may refer to operators in which data is input from two external input ports in FIG. 7, for example, E0.0, E1.0,..., E11.0, , the second internal operators 620 are operators that receive data from the adjacent first internal operators 610 rather than from an external input port in FIG. 6, that is, E0.1, E1.1, ..., can mean E11.1, and the third internal operators 630 are operators that receive data from the adjacent second internal operators 620 rather than from the external input port in FIG. It may mean E0.2, E1.2,..., E11.2.

That is, the external product-based multiplication operation device 600 according to the embodiment includes, in addition to accumulation data transmission paths 640-1 and 640-2, a hardware configuration based on unit accumulation (chunking accumulation) operation. Minimize additional configuration. Therefore, it is possible to solve problems caused by unit accumulation (chunking accumulation) operation while utilizing the structure of the conventional external product-based multiplication operation device.

Furthermore, in the external product-based multiplication operation apparatus 600 according to another embodiment, even when performing a matrix-matrix product operation, the second internal operators 620 and the third internal operators 630 are utilized for high-order accumulation. Thus, not only can the operating efficiency of the internal operators be increased, but also the quantization error, which is a problem caused by unit accumulation (chunking accumulation) calculation, can be further reduced.

Since the operations of the first internal operators 610 to third internal operators 620 are the same as the operations of the first internal operators 510 to third internal operators 520 shown in FIG. 6, detailed description Let's omit it.

However, the third internal operators 520 shown in FIG. 6 do not transmit the second unit accumulated value to one of the adjacent internal operators, but output it to the external memory as the final accumulated value.

Figure 8 is a diagram showing the configuration of a computer system according to an embodiment.

The system to which the external product-based multiplication operation device 1090 is applied in the embodiment may be implemented in a computer system 1000 such as a computer-readable recording medium.

The computer system 1000 may be an artificial intelligence system that trains a neural network or performs inference through a neural network. At this time, the external product-based multiplication operation device 1090 can be used to accelerate the neural network during learning and inference. That is, the outer product-based multiplication operation device 1090 performs a neural network operation through outer product-based multiplication as feature map data and neural network parameters, such as weight data, are input under the control of the processor 1010. It can be done.

At this time, the outer product-based multiplication operation device 1090 may perform the operations described above with reference to FIGS. 3, 4, 6, and 7.

Computer system 1000 may include one or more processors 1010, memory 1030, user interface input device 1040, user interface output device 1050, and storage 1060 that communicate with each other via bus 1020. You can. Additionally, the computer system 1000 may further include a network interface 1070 connected to the network 1080. The processor 1010 may be a central processing unit or a semiconductor device that executes programs or processing instructions stored in the memory 1030 or storage 1060. The memory 1030 and storage 1060 may be storage media including at least one of volatile media, non-volatile media, removable media, non-removable media, communication media, and information transfer media. For example, memory 1030 may include ROM 1031 or RAM 1032.

Depending on the embodiment, the processor 1010 may control the operation of the external product-based multiplication operation device 1090, data input/output, and quantization of input data.

Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention can be implemented in other specific forms without changing the technical idea or essential features. You will understand that it exists. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

Claims (20)

First internal operators each performing a MAC (Multiply-Accumulation) operation to generate an intermediate accumulation value;
second internal operators each generating a unit accumulation value using the intermediate accumulation value; and
Comprising accumulation data transmission paths that allow the output of one of the first internal operators to be input to one of the second internal operators,
Cross product based multiplication operation unit. In claim 1,
The second internal operators each receive the output of one of the first internal operators,
Cross product based multiplication operation unit. In claim 1,
The first internal operators transfer the intermediate accumulated value to one of the second internal operators at every cycle corresponding to a unit size (chunk size) corresponding to the intermediate accumulated value, and then perform a reset operation.
Cross product based multiplication operation unit. In claim 3,
The second internal operators each perform an operation to generate the unit accumulation value at every cycle corresponding to the unit size,
Cross product based multiplication operation unit. In claim 4,
The operation to generate the unit cumulative value is
An operation that adds a newly input intermediate accumulation value and an intermediate accumulation value stored in the second internal operator,
Cross product based multiplication operation unit. In claim 4,
Further comprising a control unit that inputs a control signal for adjusting a unit size (chunk size) to each of the first internal calculator and the second internal calculator,
Cross product based multiplication operation unit. In claim 4,
The second internal operators are each switched from an inactive state to an active state every cycle corresponding to the unit size,
Cross product based multiplication operation unit. generating an intermediate accumulation value by performing a MAC (Multiply-Accumulation) operation by each of the first internal operators;
transmitting the intermediate accumulated value to one of second internal operators through an accumulated data transmission path; and
Generating a unit accumulation value (chucking accumulation value) using the intermediate accumulation value by each of the second internal operators.
Including,
Cross product based multiplication operation method. In claim 8,
The transmitting step is
Transferring the intermediate accumulation value to one of the second internal operators at each cycle corresponding to a unit size (chunk size) corresponding to the intermediate accumulation value, and then performing a reset operation,
Cross product based multiplication operation method. In claim 9,
The steps for generating unit accumulation values are:
Performing an operation to generate the unit accumulation value at each cycle corresponding to the unit size,
Cross product based multiplication operation method. In claim 10,
The operation to generate the unit cumulative value is
An operation that adds a newly input intermediate accumulation value and an intermediate accumulation value stored in the second internal operator,
Cross product based multiplication operation method. In claim 10,
Before generating a unit accumulation value, each of the second internal operators is converted from an inactive state to an active state every cycle corresponding to the unit size.
Containing more,
Cross product based multiplication operation method. First internal operators each performing a MAC (Multiply-Accumulation) operation to generate an intermediate accumulation value;
second internal operators each generating a first unit accumulation value using the intermediate accumulation value;
third internal operators each generating a second unit accumulation value using the first unit accumulation value;
Accumulation data transmission paths that allow the output of one of the first internal operators to be input to one of the second internal operators; and
Accumulation data transmission paths that allow the output of one of the second internal operators to be input to one of the second internal operators.
Including,
Cross product based multiplication operation unit. In claim 13,
The second internal operators each receive an output from one of the first internal operators,
The third internal operators each receive the output of one of the second internal operators,
Cross product based multiplication operation unit. In claim 13,
Each of the first internal operators transfers the intermediate accumulated value to one of the second internal operators at every cycle corresponding to a first unit size (chunk size) corresponding to the intermediate accumulated value, and then performs a reset operation,
The second internal operators each calculate the first unit accumulation value to the first unit accumulation value every period corresponding to a size obtained by multiplying the first unit size by a second unit size (chunk size) set for unit accumulation of the first unit accumulation value. 3, which performs a reset operation after passing it to one of the internal operators,
Cross product based multiplication operation unit. In claim 13,
The second internal operators each perform an operation to generate the first unit accumulation value at every cycle corresponding to the first unit size,
The third internal calculator generates the second unit accumulation value every cycle corresponding to a size obtained by multiplying the first unit size by a second unit size (chunk size) set for unit accumulation of the first unit accumulation value. performing operations for,
Cross product based multiplication operation unit. In claim 16,
The operation for generating the first unit cumulative value is
It is an operation that adds a newly input intermediate accumulation value and an intermediate accumulation value stored in the second internal calculator,
The operation for generating the second unit cumulative value is
An operation that adds the newly input first unit accumulation value and the second unit accumulation value stored in the third internal operator,
Cross product based multiplication operation unit. In claim 16,
Further comprising a control unit that inputs a control signal to adjust a unit size (chunk size) or a period of performing a reset operation corresponding to each of the first to third internal operators,
Cross product based multiplication operation unit. In claim 16,
Each of the second internal operators is switched from an inactive state to an active state every cycle corresponding to the first unit size,
The third internal operators are switched from the inactive state to the active state at each cycle calculated based on the first unit size and the second unit size, respectively.
Cross product based multiplication operation unit. In claim 16,
N+3 internal operators that perform an operation to generate an N+2 unit chucking accumulation value using the N+1 (N is an integer that increases by 1 from 1) unit accumulation value, respectively. ; and
Accumulation data transmission paths that allow the output of one of the N+2 internal operators to be input to one of the N+3 internal operators
Repeat addition as each N increases,
The operation for generating the N+2 unit cumulative value is
An operation that adds the newly input N+1th unit accumulation value and the N+2th unit accumulation value stored in the N+3th internal calculator,
Cross product based multiplication operation unit.

Images