KR20230099190A - Apparatus and method for address generation of multi-dimensional tensor - Google Patents

Abstract

A device for generating an address of a multi-dimensional tensor according to one embodiment comprises: an input part that input-receives information for a multi-dimensional tensor in a memory and a coordinate to access; a determination part that determines an operation required to generate an address for the coordinate, and generates a control signal corresponding to the determined operation; and an operation part that generates an address corresponding to the coordinate according to the control signal. Therefore, the present invention is capable of shortening an overall processing time.

Description

Apparatus and method for generating address of multi-dimensional tensor {APPARATUS AND METHOD FOR ADDRESS GENERATION OF MULTI-DIMENSIONAL TENSOR}

The present invention relates to an apparatus for generating an address of a multi-dimensional tensor and a method for generating an address of a multi-dimensional tensor by the apparatus.

In the field of artificial intelligence, deep learning-based algorithms such as CNN (Convolution Neural Network) and RNN (Recurrent Neural Network) are widely used with excellent performance. However, since there is a disadvantage that the amount of computation required is large, dedicated hardware is required to accelerate it. In particular, since the processing speed of deep learning inference operation is important in real life applications such as self-driving cars, dedicated hardware for deep learning inference operation focuses on increasing the processing speed.

In general, hardware dedicated to deep learning consists of a systolic array for matrix operation and a vector processor for vector operation. On the other hand, as the amount of computation required for deep learning algorithms is large, memory access to data for computation also occurs frequently. To this end, a systolic array has a separate device for accessing and inputting data necessary for operation into memory, but a vector processor needs to calculate a memory address using internal resources, which can cause an increase in processing time.

Republic of Korea Patent Publication No. 10-2019-0113973, published on October 08, 2019.

According to an embodiment, an apparatus and method for generating an address of a multidimensional tensor generating an address corresponding to coordinates of a multidimensional tensor in a memory are provided.

The problem to be solved by the present invention is not limited to those mentioned above, and another problem to be solved that is not mentioned will be clearly understood by those skilled in the art from the description below.

An apparatus for generating an address of a multidimensional tensor according to a first aspect includes an input unit that receives coordinates of a multidimensional tensor in a memory, an operation required to generate an address for the coordinates, and a control signal corresponding to the determined operation. and a calculating unit generating an address corresponding to the coordinates according to the control signal.

A method for generating a multidimensional tensor address performed by an address generating apparatus for a multidimensional tensor according to a second aspect includes receiving coordinates of a multidimensional tensor in a memory, determining an operation necessary for generating an address for the coordinates, and determining the determined and generating an address corresponding to the coordinates according to the operation.

According to the third aspect, a computer program stored on a computer readable recording medium includes instructions for causing the processor to perform the method of generating an address of the multidimensional tensor when the computer program is executed by the processor.

According to an embodiment, by generating and providing an address corresponding to coordinates of a multidimensional tensor in memory, a processor can use the coordinates of a tensor without calculating a memory address. Further, by reusing address operation results or performing only partial operations according to memory access patterns, memory access processing time and energy consumption required for operations may be reduced.

Accordingly, other operations can be performed at the time required for the memory address operation, thereby reducing the overall processing time.

In addition, processor registers required for memory address operation can be used for other operations, so that the scheduling of the compiler can be facilitated.

1 is a block diagram of an apparatus for generating an address of a multi-dimensional tensor according to an embodiment of the present invention.
2 is a flowchart illustrating a method of generating an address of a multi-dimensional tensor according to an embodiment of the present invention.
3 is a structural diagram of a storage device that may be included in a determining unit of a device for generating an address of a multidimensional tensor according to an embodiment of the present invention.
4 is a diagram illustrating an operation of a determination unit of an apparatus for generating an address of a multi-dimensional tensor according to an embodiment of the present invention.
5 is a flowchart illustrating a process of determining an output value of a determination unit of an apparatus for generating an address of a multidimensional tensor according to an embodiment of the present invention.
6 is a diagram illustrating an operation operation of an operation unit of an apparatus for generating an address of a multidimensional tensor according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the embodiments described below in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to completely inform the person who has the scope of the invention, and the present invention is only defined by the scope of the claims.

The terms used in this specification will be briefly described, and the present invention will be described in detail.

The terms used in the present invention have been selected from general terms that are currently widely used as much as possible while considering the functions in the present invention, but these may vary depending on the intention of a person skilled in the art or precedent, the emergence of new technologies, and the like. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, not simply the name of the term.

When it is said that a certain part 'includes' a certain element in the entire specification, it means that other elements may be further included without excluding other elements unless otherwise stated.

In addition, the term 'unit' used in the specification means software or a hardware component such as FPGA or ASIC, and 'unit' performs certain roles. However, 'part' is not limited to software or hardware. A 'unit' may be configured to reside in an addressable storage medium and may be configured to reproduce one or more processors. Thus, as an example, 'unit' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functionality provided within the components and 'parts' may be combined into a smaller number of elements and 'parts' or further separated into additional elements and 'parts'.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted.

1 is a block diagram of an apparatus for generating an address of a multi-dimensional tensor according to an embodiment of the present invention.

Referring to FIG. 1 , an apparatus 100 for generating an address of a multidimensional tensor according to an embodiment includes an input unit 110, a determination unit 120, and an operation unit 130, and may further include an output unit 140. there is. Here, the determining unit 120 and/or the calculating unit 130 may include a computing means such as a microprocessor or a central processing unit (CPU).

The input unit 110 receives coordinates of multi-dimensional tensors in memory. For example, the input unit 110 may receive information about the multidimensional tensor from the processor 10 or a higher-order device driving the processor 10, and may receive information about the multidimensional tensor from the processor 10 requiring access to the multidimensional tensor. Coordinates may be input, and inputted coordinate information may be provided to the determining unit 120 . For example, the input unit 110 may receive information about the multidimensional tensor and provide it to the determination unit 120 at the beginning of the program of the processor 10 or before accessing the multidimensional tensor in the program for the first time. Upon access, the coordinates received from the processor 10 may be provided to the determining unit 120 .

The determination unit 120 determines an operation necessary for generating an address for the coordinates received from the input unit 110, generates a control signal corresponding to the determined operation, and provides the generated control signal to the operation unit 130. For example, the determination unit 120 may store the coordinates received by the input unit 110 and the address generated by the operation unit 130 in a storage device (not shown) as an existing operation result. information can be stored. In addition, when determining an operation necessary for generating an address for coordinates, the determining unit 120 may determine an operation necessary for a dimension having a coordinate change by comparing with an existing operation result. For example, the information on the multi-dimensional tensor may include the size, stride, and address offset of each dimension of the tensor, and the determination unit 120 determines effective coordinates, address offset, and stride as many as the number of dimensions of the tensor and a control signal according to the determined operation. may be provided to the calculation unit 130. In addition, when determining an operation required to generate an address for coordinates, the determination unit 120 may include the previous operation result in the output offset when at least a part of the previous operation result can be reused. In addition, if there is a discontinuous section of the address according to the coordinates in any dimension of the multidimensional tensor, the discrimination unit 120 divides and stores each discontinuous section in several items of the storage device, and then, when the coordinates are input, the coordinates of the corresponding dimension belong The discontinuous section may be retrieved from the storage device by calculating tensor information to which the discontinuous section is allocated. Also, the determining unit 120 may generate a control signal so that the calculating unit 130 generates addresses in a different order from the request order of the processor 10 .

The operation unit 130 generates an address corresponding to the coordinates of the multidimensional tensor according to the control signal generated by the determination unit 120 . The calculation unit 130 converts at least some of the previous calculation results transmitted through the output offset into coordinates of the multidimensional tensor.

can be included in the calculation of the address.

The output unit 140 accesses the memory 20 using the address generated by the arithmetic unit 130 and transmits data of the corresponding address to the processor 10 by the memory 20 .

2 is a flowchart illustrating a method for generating an address of a multidimensional tensor according to an embodiment of the present invention, and FIG. 3 is a determination unit 120 of the apparatus 100 for generating a multidimensional tensor address according to an embodiment 4 is a diagram showing the operation of the determining unit 120 of the apparatus 100 for generating addresses of multi-dimensional tensors according to an embodiment of the present invention, and FIG. A flowchart illustrating a process of determining an output value of the determination unit 120 of the multidimensional tensor address generating apparatus 100 according to an embodiment. FIG. 6 is a multidimensional tensor address generating apparatus 100 according to an embodiment of the present invention. It is a diagram showing the calculation operation of the calculation unit 130 of ).

Hereinafter, with reference to FIGS. 1 to 6 , a method of generating an address of a multidimensional tensor performed by the apparatus 100 for generating an address of a multidimensional tensor according to an embodiment of the present invention will be described in detail.

Image processing is one of the important applications of artificial intelligence algorithms such as convolutional neural networks. At this time, when the input image passes through each layer of the convolutional neural network, it is transmitted to the next layer in the form of a feature map. Since the input image and feature map are composed of width, height, and channel and stored in memory, they can be expressed as a 3D tensor.

A vector processor performs necessary operations by accessing coordinate data consisting of a specific width, height, and channel. At this time, in order to calculate the memory address of data corresponding to each coordinate, tensor information is required. The tensor information required here includes the size of each dimension of the tensor (width, height, channel), the stride indicating the amount by which the memory address changes whenever the coordinates of each dimension change by 1, and the address offset, which is the starting address of the tensor.

To calculate the address of a 3D tensor using this information, 3 multiplications (coordinates and strides in each dimension) and 3 addition operations (results and offsets in each dimension) are required. At this time, when accessing continuous coordinates, the amount of computation is reduced by removing some unnecessary operations from the compiler that generates the code to be executed by the processor. It takes a relatively large amount of time, which may cause an increase in the processing time of the entire neural network.

In order to solve this problem, the multidimensional tensor address generator 100 according to an embodiment of the present invention can obtain the same result even if the processor 10 accesses the memory 20 using only the coordinates of the tensor without performing an address operation. let it be

First, the processor 10 or a higher-order device driving the processor 10 provides information about the multidimensional tensor to the multidimensional tensor address generating device 100, and the input unit 110 of the multidimensional tensor address generating device 100 receives information about the multi-dimensional tensor and provides it to the discrimination unit 120. In this way, receiving information on the multidimensional tensor may be performed at the beginning of the program of the processor 10 or before accessing the multidimensional tensor for the first time in the program. Here, the information on the tensor may be information of a multi-dimensional tensor composed of the size, stride, and address offset of each dimension of the tensor necessary for address calculation (S210).

In addition, the input unit 110 receives coordinates of the multidimensional tensor from the processor 10 that requires access to the multidimensional tensor, and provides the input coordinate information to the determination unit 120 . Here, the coordinate information of the multidimensional tensor transmitted to the determination unit 120 may also include a tensor number for which an address operation is required (S220).

The determination unit 120 may include a storage device 302 for storing existing input coordinates and address calculation results in order to determine necessary operations, and a tensor information storage device 301 for storing tensor information necessary for address calculations. there is. For example, existing input coordinates, address operation results, and tensor information required for address operation may be stored together in one storage device. Here, the number of tensor information that can be stored and the number of previous operation results for each tensor can be set according to the needs of the system.

The determining unit 120 determines an operation necessary for generating an address of the input coordinate (S510), generates a control signal, and transfers it to the calculating unit 130. Here, the operation required for address generation may refer to an additional operation for a dimension having a difference in coordinates (S530 to S550) by comparing the address operation result of the stored existing coordinates (S520), and a control signal for the corresponding operation. may be provided to the calculation unit 130 (S570). If there is no stored result or the coordinates of all dimensions are different, a control signal may be generated (S560) and transmitted to the calculation unit 130 (S570) to perform the entire operation.

The determining unit 120 may pre-calculate the time required for address calculation according to each control signal while generating the control signal. Therefore, a control signal can be generated so that address calculations are performed according to the request order of the processor 10, and a control signal is generated so that address calculations are performed regardless of the request order of the processor 10 in order to reduce processing time according to the request. may come alive

Here, the determination unit 120 may transmit an output offset, a control signal, and effective coordinates and strides equal to the number of tensor dimensions to the operation unit 130 according to the determination result. Here, the effective coordinates and strides that are output are not fixed to a specific dimension of the tensor, but may be variably output along with a control signal only for dimensions that require calculation according to the determination result. At this time, if the existing calculation result stored as the discrimination result can be partially reused, the existing calculation result can be output to the output offset, and the difference between the stored coordinate and the input coordinate can be output to the effective coordinate. If the existing result cannot be reused, the address offset of the tensor can be output in the output offset, and the input coordinate can be output as it is in the valid coordinate. FIG. 4 illustrates data 402 transmitted from the determination unit 120 to the operation unit 130 according to the input tensor coordinates 401 .

In addition, if there is a section in which addresses are discontinuous according to coordinates in a certain dimension of the tensor, the determination unit 120 may divide and store each discontinuous section in several items of the tensor information storage device 301. Then, when the coordinates are input, the number of tensor information to which the discontinuous section to which the coordinates of the corresponding dimension belong is calculated and retrieved from the tensor information storage device 301. At this time, the discontinuous address of each dimension means a case where the address value that changes when the coordinates of the corresponding dimension increase by 1 is not the stride value of the corresponding dimension. Here, the number of discontinuous sections may be pre-determined according to system requirements (S230).

Next, the calculation unit 130 performs an operation on the input data according to the control signal transmitted from the determination unit 120 . For example, the operation unit 130 may perform an operation using a plurality of multipliers 601 , 602 , and 603 , a plurality of adders 604 , 605 , and 606 , and a multiplexer 607 as shown in FIG. 6 . At this time, the number of operations performed is variable according to a control signal, and the control signal includes information enabling a result address to be transmitted to the output unit 140 when necessary operations are completed. For example, if there is a result address stored for the input coordinates, the previous operation result transferred through the output offset may be transferred to the output unit 140 as it is without additional operation. In addition, when a resultant address differing only in height from the input coordinates is stored, the address calculation is completed by calculating only the address corresponding to the input height difference in effective coordinate 1 of FIG. 6 and adding it to the existing result of the output offset. At this time, when some or all of the existing calculation results are reused, the calculation time is shortened and the processing speed is increased (S240).

The output unit 140 accesses the memory 20 using the address generated by the arithmetic unit 130, and transmits data of the corresponding address to the processor 10 by the memory 20 (S250).

When the memory access by the processor 10 ends, the multidimensional tensor address generation process also ends, but if the memory access by the processor 10 remains, steps S220 to S250 may be repeatedly performed (S260).

Meanwhile, each step included in the method for generating an address of a multidimensional tensor performed by the apparatus 100 for generating an address of a multidimensional tensor according to the above-described embodiment records a computer program including instructions for performing these steps. It can be implemented in a computer readable recording medium.

As described so far, by generating and providing an address corresponding to coordinates of a multidimensional tensor in memory according to an embodiment of the present invention, a processor can use the coordinates of a tensor without calculating a memory address. Further, by reusing address operation results or performing only partial operations according to memory access patterns, memory access processing time and energy consumption required for operations may be reduced.

Accordingly, other operations can be performed at the time required for the memory address operation, thereby reducing the overall processing time.

In addition, processor registers required for memory address operation can be used for other operations, facilitating compiler scheduling.

Combinations of each step in each flowchart attached to the present invention may be performed by computer program instructions. Since these computer program instructions may be loaded into a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions executed by the processor of the computer or other programmable data processing equipment function as described in each step of the flowchart. create a means to do them. These computer program instructions can also be stored on a computer usable or computer readable medium that can be directed to a computer or other programmable data processing equipment to implement functions in a particular way, so that the computer usable or computer readable It is also possible that the instructions stored on the recording medium produce an article of manufacture containing instruction means for performing the functions described in each step of the flowchart. The computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operational steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to generate computer or other programmable data processing equipment. Instructions for performing the processing equipment may also provide steps for executing the functions described at each step in the flowchart.

Further, each step may represent a module, segment or portion of code that includes one or more executable instructions for executing the specified logical function(s). It should also be noted that in some alternative embodiments it is possible for the functions mentioned in the steps to occur out of order. For example, two steps shown in succession may in fact be performed substantially concurrently, or the steps may sometimes be performed in reverse order depending on the function in question.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential qualities of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: multi-dimensional tensor address generator
110: input unit
120: determination unit
130: calculation unit
140: output unit

Claims (21)

An input unit for receiving coordinates of a multi-dimensional tensor in memory;
a determination unit determining an operation necessary to generate an address for the coordinates and generating a control signal corresponding to the determined operation;
Comprising a calculation unit for generating an address corresponding to the coordinates according to the control signal
Address generator for multi-dimensional tensors. According to claim 1,
Further comprising an output unit for accessing the memory using the address and transmitting data of the address to the processor by the memory
Address generator for multi-dimensional tensors. According to claim 1,
The input unit receives information about the multidimensional tensor at the beginning of a program of the processor or before accessing the multidimensional tensor for the first time in a program, and provides the information to the discriminating unit.
Address generator for multi-dimensional tensors. According to claim 3,
The determination unit includes a storage device for storing the coordinates received by the input unit and the address generated by the operation unit as an existing operation result, and storing information about the multidimensional tensor
Address generator for multi-dimensional tensors. According to claim 4,
The determination unit, when determining an operation necessary for generating an address for the coordinates, compares the result of the existing operation and determines the operation required for a dimension having a coordinate change.
Address generator for multi-dimensional tensors. According to claim 4,
The information on the multi-dimensional tensor includes the size, stride, and address offset of each dimension of the tensor,
The determination unit provides the control signal and effective coordinates, address offsets, and strides as many as the number of tensor dimensions to the operation unit according to the determined operation.
Address generator for multi-dimensional tensors. According to claim 6,
When determining an operation required to generate an address for the coordinates, the determining unit includes the existing operation result in an output offset when at least a part of the existing operation result can be reused.
Address generator for multi-dimensional tensors. According to claim 4,
If there is a discontinuous section of the address according to the coordinates in any dimension of the multidimensional tensor, the discrimination unit divides and stores each discontinuous section in several items of the storage device, and then, when coordinates are input, the discontinuous section to which the coordinates of the corresponding dimension belong. Calculating the assigned tensor information and bringing the corresponding discontinuous section from the storage device
Address generator for multi-dimensional tensors. According to claim 1,
The determination unit generates the control signal so that the operation unit generates addresses in an order different from the request order of the processor.
Address generator for multi-dimensional tensors. According to claim 7,
The calculation unit includes at least some of the previous calculation results transmitted through the output offset in an address corresponding to the coordinates.
Address generator for multi-dimensional tensors. A method for generating an address of a multidimensional tensor performed by an address generating device of a multidimensional tensor,
Receiving coordinates of a multi-dimensional tensor in memory;
Determining an operation necessary to generate an address for the coordinates, and generating an address corresponding to the coordinates according to the determined operation
How to generate addresses for multidimensional tensors. According to claim 11,
Accessing the memory using the address, and transmitting the data at the address to the processor by the memory
How to generate the addresses of multi-dimensional tensors. According to claim 11,
Receiving information about the multidimensional tensor at the beginning of the program of the processor or before accessing the multidimensional tensor for the first time in the program
How to generate addresses for multidimensional tensors. According to claim 13,
Storing the input coordinates and the generated address as an existing operation result and storing information about the multidimensional tensor
How to generate the addresses of multi-dimensional tensors. 15. The method of claim 14,
When determining the operation necessary to generate an address for the coordinates, comparing the result of the existing operation to determine the operation required for the dimension with coordinate change
How to generate the addresses of multi-dimensional tensors. 15. The method of claim 14,
The information on the multi-dimensional tensor includes the size, stride, and address offset of each dimension of the tensor,
According to the determined operation, effective coordinates, address offsets, and strides as many as the number of tensor dimensions are used for the operation.
How to generate the addresses of multi-dimensional tensors. 17. The method of claim 16,
When determining an operation necessary to generate an address for the coordinates, including an existing operation result in an output offset when at least a part of the existing operation result can be reused
How to generate the addresses of multi-dimensional tensors. 15. The method of claim 14,
If there is a discontinuous section of the address according to the coordinates in any dimension of the multidimensional tensor, each discontinuous section is divided into several items and stored, and then, when coordinates are input, tensor information to which the discontinuous section to which the coordinates of the dimension belong is assigned is calculated. Bringing the corresponding discontinuous section among the stored items
How to generate addresses for multidimensional tensors. According to claim 11,
Generating the addresses in an order different from the request order of the processor.
How to generate addresses for multidimensional tensors. 18. The method of claim 17,
Including at least some of the previous operation results delivered through the output offset to the address corresponding to the coordinates
How to generate the addresses of multi-dimensional tensors. A computer program stored on a computer readable recording medium,
When the computer program is executed by a processor,
Receiving coordinates of a multi-dimensional tensor in memory;
An instruction for causing the processor to perform a method for generating an address of a multidimensional tensor comprising determining an operation necessary to generate an address for the coordinate and generating an address corresponding to the coordinate according to the determined operation.
computer program.

Images