KR20220059409A - Memory apparatus for artificial neural network - Google Patents

Abstract

According to one disclosure of the present specification, proposed is a memory device for an artificial neural network. The memory device may comprise: at least one memory cell array having N number of columns and M number of rows; and a memory control part configured to sequentially operate a data read or write operation of the at least one memory cell array in a burst mode based on the preset sequential access information. Therefore, the present invention is capable of having an effect that substantially eliminates or reduces the data supply delay.

Description

MEMORY APPARATUS FOR ARTIFICIAL NEURAL NETWORK

The present disclosure relates to a memory device, and more particularly, to a memory device for an artificial neural network.

As AI reasoning ability develops, artificial intelligence speaker, smart phone, smart refrigerator, VR device, AR device, AI CCTV, AI robot vacuum cleaner, tablet, laptop computer, autonomous vehicle, biped robot, quadruped Various inference services such as acoustic recognition, voice recognition, image recognition, object detection, driver drowsiness detection, danger moment detection, and gesture detection using artificial intelligence are being installed in various electronic devices such as walking robots and industrial robots.

With the recent development of deep learning technology, the performance of artificial neural network inference services through big data-based learning is developing. This artificial neural network learning and inference service repeatedly learns a huge amount of learning data in the artificial neural network, and infers various and complex data through the learned artificial neural network model. Accordingly, various services are provided to the above-described electronic devices using artificial neural network technology.

However, functions and accuracy required for inference services using artificial neural networks are increasing. Accordingly, the size of the artificial neural network model, the amount of computation, and the size of the training data are increasing exponentially. The performance required for the processor and memory that can handle the reasoning operation of the artificial neural network model is gradually increasing, and the artificial neural network inference service is actively provided in the cloud computing-based server that can easily process big data. there is.

On the other hand, edge computing using artificial neural network model technology is being actively studied. Edge computing means the edge, the perimeter, where computing takes place. Edge computing refers to a terminal that directly produces data or various electronic devices located close to the terminal. Edge computing may be referred to as an edge device. Edge devices can also be utilized when performing necessary tasks immediately and reliably, such as autonomous drones, autonomous robots, or autonomous vehicles that need to process massive amounts of data within 1/100 of a second. Accordingly, fields to which edge devices can be applied are rapidly increasing.

The inventor of the present disclosure recognized the fact that the calculation of the conventional artificial neural network model has problems such as high power consumption, heat generation, a bottleneck of processor operation due to a relatively low memory bandwidth, and latency of the memory. . Therefore, it was recognized that various difficulties exist in improving the computational processing performance of the artificial neural network model, and it was recognized that it was necessary to develop an artificial neural network memory system that can improve these problems.

Accordingly, the inventors of the present disclosure have studied an artificial neural network memory system that can be applied to a server system and/or edge computing. Furthermore, the inventors of the present disclosure have also studied a neural processing unit (NPU), or a neural processing unit, which is a processor of an artificial neural network memory system optimized for artificial neural network model processing.

First, the inventor of the present disclosure recognized that effectively controlling the memory when calculating the artificial neural network model is the key to improving the artificial neural network computation processing speed. The fact that the inventor of the present disclosure may not be able to properly prepare the necessary data in advance to reduce the effective bandwidth of the memory and/or delay the announcement of data in the memory if the memory control is not properly performed when training or inferring the artificial neural network model. was recognized. In addition, the inventors of the present disclosure have recognized the fact that in this case, the processor is in a starvation or idle state that does not receive data to be processed, and thus cannot perform actual calculations, thereby degrading calculation performance.

Second, the inventor of the present disclosure recognized the limitations of the computational processing method of the artificial neural network model in the conventional algorithm level. For example, the conventional prefetch algorithm is a technology in which the processor reads data from the memory in units of layers by interpreting the artificial neural network model in units of conceptual layers. However, the prefetch algorithm cannot recognize the locality of the artificial neural network data in the word unit or memory access request unit of the neural network model existing at the processor-memory level, that is, the hardware level. The inventors of the present disclosure have recognized that it is not possible to optimize the data transmission/reception operation at the processor-memory level using only the prefetch technique.

Third, the inventor of the present disclosure recognized the “artificial neural network data locality”, which is a unique characteristic of the artificial neural network model. The inventor of the present disclosure has artificial neural network data locality in word units or memory access request units at the processor-memory level, and utilizes this to maximize the effective memory bandwidth and minimize the delay in data supply to the processor to learn / It has been recognized that the performance of inference processing can be improved.

Specifically, the term “artificial neural network data locality” of the artificial neural network model recognized by the inventor of the present disclosure means that when the processor processes a specific artificial neural network model, the processor that is performed according to the structure and calculation algorithm of the artificial neural network model creates the artificial neural network. It may mean order information in word units of data required for arithmetic processing. Furthermore, the inventor of the present disclosure has recognized the fact that the computational processing order of such an artificial neural network model has a characteristic in which artificial neural network data locality is maintained for iterative learning and/or inference computation for the artificial neural network model given to the processor. . Therefore, the inventor of the present disclosure recognized the fact that when the locality of the artificial neural network data is maintained, the processing order of data required for the artificial neural network operation processed by the processor is maintained in word units, and the artificial neural network operation is performed by receiving or analyzing this information. recognized that it can be used for In more detail, the word unit of the processor may mean an element unit, which is a basic unit that the processor can process. For example, when the neural processing unit multiplies N-bit input data by an M-bit kernel weight, the word unit of input data of the processor may be N bits and the word unit of weight data may be M bits. In addition, the inventors of the present disclosure also recognized the fact that the word unit of the processor may be set differently depending on the layer, the feature map, the kernel, the activation function, etc. of the artificial neural network model. Accordingly, the inventors of the present disclosure have also recognized the fact that sophisticated memory control technology is required for each word unit operation.

The inventors of the present disclosure have paid attention to the fact that the neural network data locality is configured when the neural network model is compiled to be executed on a specific processor by the compiler. And it was recognized that the locality of artificial neural network data can be configured according to the operating characteristics of the compiler, the algorithms applied to the artificial neural network model, and the processor. To elaborate, the inventor of the present disclosure describes a method in which a processor calculates a corresponding artificial neural network model even in the case of the same artificial neural network model, for example, feature map tiling, a stationary technique of processing elements, etc., processing of the processor The number of elements, cache memory capacity such as feature maps and weights in the processor, the memory hierarchical structure in the processor, and the artificial neural network model to be processed according to the algorithm characteristics of the compiler that determine the order of operation of the processor to process the artificial neural network model. It was recognized that the locality of the artificial neural network data in the neural network model can be configured differently. This is because, even if the same artificial neural network model is processed by the above-described factors, the processor may differently determine the order of data required at each moment in clock units. That is, the inventor of the present disclosure recognized that, conceptually, the order of data required for calculation of the artificial neural network model is the operation order of layers of the artificial neural network, unit convolution, and/or matrix product. Furthermore, the inventors of the present disclosure have recognized the fact that the sequence of data required for physical operation processing constitutes the locality of the artificial neural network data of the corresponding artificial neural network model at the processor-memory level, ie, the hardware level, in units of words. In addition, the inventor of the present disclosure recognized the fact that artificial neural network data locality has a characteristic that is dependent on a processor and a compiler used for the processor.

Fourth, the inventor of the present disclosure recognized the fact that, when an artificial neural network memory system configured to receive and utilize artificial neural network data locality information is provided, the processing performance of the artificial neural network model can be maximized at the processor-memory level.

The inventor of the present disclosure can know even the operation processing order information of the word unit, which is the minimum unit for the processor to process the artificial neural network model, when the artificial neural network memory system can precisely grasp the locality of the artificial neural network data of the artificial neural network model to the word unit. recognized the fact that That is, if an artificial neural network memory system that can utilize artificial neural network data locality is provided, the artificial neural network memory system precisely reads specific data from the memory at a specific timing in word units and provides it to the processor, or whether the processor calculates the specific data. Therefore, we recognized the fact that it is possible to predict in advance whether or not to store in the memory at a specific timing. Accordingly, the inventor of the present disclosure recognized the fact that by providing an artificial neural network memory system, data to be requested by the processor in word units can be prepared in advance.

In other words, the inventor of the present disclosure, if the neural network memory system knows the locality of the artificial neural network data, the kernel moves in a specific direction when the processor calculates the convolution of the specific input data and the specific kernel using a technique such as feature map tiling We recognized the fact that the processing order of the convolution to be processed can also be known in word units.

That is, the artificial neural network memory system utilizes the artificial neural network data locality to predict in advance what kind of data the processor needs, so that the memory read/write operation requested by the processor is predicted in advance, and the data to be processed by the processor is predicted in advance. It has been recognized that it is possible to minimize or eliminate the increase in the effective bandwidth of the memory and/or the delay in data supply of the memory by preparing. In addition, we recognized the fact that if the artificial neural network memory system can supply the data to be processed by the processor at the required timing, the starvation or standby state of the processor can be minimized. Accordingly, the inventors of the present disclosure have recognized that the artificial neural network memory system can provide an effect of improving arithmetic processing performance and reducing power consumption.

Fifth, the inventor of the present disclosure, even if the artificial neural network memory controller is not provided with artificial neural network data locality information, the artificial neural network memory controller is placed in a communication channel between the processor and the memory processing the artificial neural network model, and then the processor By analyzing the data access request requested to the memory when processing the operation of a specific artificial neural network model, it is recognized that the locality of the artificial neural network data of the artificial neural network model being processed by the processor can be inferred from the processor-memory data access request unit. did That is, since each neural network model has its own artificial neural network data locality, the processor at the processor-memory level recognized the fact that the processor generates data access requests in a specific order according to the neural network data locality. In addition, it was recognized that the access order of data stored in the memory for data requests between the processor and memory is also maintained based on the fact that the locality of the artificial neural network data is maintained while the processor repeatedly processes the learning/inference operation of the neural network model.

Accordingly, the inventors of the present disclosure have arranged the artificial neural network memory control unit in a communication channel between the processor and the memory for processing the artificial neural network model. In addition, by observing the data access request between the processor and memory for the first or several times of learning and inference operations, it was recognized that the neural network memory controller can infer the locality of the artificial neural network data in units of data access requests. Therefore, the inventor of the present disclosure recognized the fact that artificial neural network data locality can be inferred by the artificial neural network memory controller even if artificial neural network data locality information is not provided.

Accordingly, the inventor of the present disclosure predicts in advance the memory read/write operation requested by the processor based on the artificial neural network data locality reconstructed in the data access request unit, and prepares the data to be processed by the processor in advance to increase the effective memory bandwidth and/or can minimize or substantially eliminate memory data supply delay. In addition, the inventors of the present disclosure have recognized the fact that if the artificial neural network memory system can supply data to be processed by the processor at a necessary timing, the occurrence rate of starvation or standby state of the processor can be minimized.

Accordingly, an object of the present disclosure is to provide an artificial neural network memory system capable of optimizing the artificial neural network operation of the processor by utilizing the artificial neural network data locality of the artificial neural network model operating at the processor-memory level.

Accordingly, the problem to be solved by the present disclosure is to detect the data access request generated by the processor, generate the data locality pattern of the artificial neural network model being processed by the processor, and prepare the data access request requested by the processor in advance to solve the memory latency problem It is to provide an artificial neural network memory system including an artificial neural network memory control unit that can improve the However, the present disclosure is not limited thereto, and other problems will be clearly understood by those skilled in the art from the following description.

According to one disclosure of the present specification, a memory device for an artificial neural network is provided. The memory device may include: at least one memory cell array having N columns and M rows; and a memory controller configured to sequentially read or write data of the at least one memory cell array in a burst mode based on preset sequential access information.

The at least one memory cell array may further include a plurality of dynamic memory cells having leakage current characteristics.

The at least one memory cell array may include: a column decoder for controlling access to the N columns; a plurality of bit lines connected to the column decoder; a row decoder for controlling access to the M rows; a plurality of word lines connected to the row decoder; and a sense amplifier connected to one end of the plurality of bit lines.

The memory controller may be configured to control data communication between a processor to process an artificial neural network operation and the at least one memory cell array in which data required for the artificial neural network operation is stored based on the sequential access information.

The sequential access information may be generated based on "locality of artificial neural network data" of the artificial neural network that the processor intends to process.

The memory controller is configured to directly control the N column and M row addresses of the at least one memory cell array so that the at least one memory cell array operates in the sequential burst mode based on the sequential access information. can be configured.

The memory controller may be configured to set memory addresses of data for each operation step to be stored in the at least one memory cell array based on the sequential access information.

The memory controller may be configured to sequentially allocate address values corresponding to the N columns and M rows of the at least one memory cell array to store artificial neural network data.

According to one disclosure of the present specification, a memory device for an artificial neural network is provided. The memory device may include at least one memory cell array; and a memory controller configured to directly control a read or write operation of the at least one memory cell array based on artificial neural network data locality.

The artificial neural network data locality may include preset operation order information of the artificial neural network to be processed by the processor.

The artificial neural network data locality may include data size information for each of a preset operation order.

The artificial neural network memory controller may be configured to store a memory map generated in a sequential manner based on preset operation order information and a data size of each corresponding operation order.

The artificial neural network data locality may further include a signal for identifying a weight, an input feature map, and an output feature map. The operation order pattern of the weight data, the input feature map data, and the output feature map data may be determined when compiled based on the characteristics of the processor.

The artificial neural network data locality may be determined based on at least one of a characteristic of an artificial neural network model, a characteristic of a processor, a size of a cache memory, and/or an operation algorithm policy.

According to one disclosure of the present specification, a memory device for an artificial neural network is provided. The memory device includes at least one dynamic memory cell array; and a memory controller configured to store artificial neural network data in the at least one memory cell array in an order based on locality of the artificial neural network data.

The order based on the locality of the artificial neural network data may include at least a repeating pattern of an input feature map, a kernel, and an output feature map order.

The order based on the locality of the artificial neural network data may include at least a repeating pattern of a kernel, an input feature map, and an output feature map order.

The artificial neural network data locality may be configured in a data access request unit that a processor requests from the memory control unit. The artificial neural network data locality may include order information of all data access requests required for one-time inference of the artificial neural network processed by the processor.

The memory controller may be configured to divide the region of the at least one dynamic memory cell array into a kernel region and a feature map region based on identification information configured to distinguish the kernel, the input feature map, and the output feature map.

The at least one dynamic memory cell array may further include a plurality of banks configured to enable an interleaving operation. The memory controller may be configured to divide and store the artificial neural network data in each bank to operate in a burst mode corresponding to the interleaving operation in the plurality of banks.

The memory device may further include a processor configured to provide the artificial neural network data locality information to the memory controller.

The memory device may further include a processor configured to provide at least input feature map, kernel, and output feature map identification information to the memory controller.

According to the embodiments of the present disclosure, there is an effect that can substantially eliminate or reduce the delay in data supply of the memory to the processor by the artificial neural network data locality in the system for processing the artificial neural network.

According to the embodiments of the present disclosure, the artificial neural network memory control unit has an effect of being able to prepare in advance the data of the artificial neural network model processed at the processor-memory level before the processor requests it.

According to the embodiments of the present disclosure, the learning and inference calculation processing time of the artificial neural network model processed by the processor is shortened, so that the calculation processing performance of the corresponding processor is improved, and the power efficiency for the system-level calculation processing can be improved. It works.

Effects according to the present disclosure are not limited by the contents exemplified above, and more various effects are included in the present specification.

1A is a schematic block diagram illustrating a processor and an artificial neural network memory controller of an artificial neural network memory system based on artificial neural network data locality according to an example of the present disclosure.
1B is a schematic diagram illustrating an example of an exemplary neural processing unit for explanation of reconstruction of an artificial neural network data locality pattern that may be applied to various examples of the present disclosure.
2 is a schematic diagram illustrating an artificial neural network data locality pattern according to an example of the present disclosure.
3 is a schematic diagram illustrating an exemplary artificial neural network model for explaining an artificial neural network data locality pattern that can be applied to various examples of the present disclosure.
4 is a schematic diagram illustrating an artificial neural network data locality pattern generated by an artificial neural network memory controller analyzing the artificial neural network model of FIG. 3A according to an example of the present disclosure.
5 is a schematic diagram illustrating a token and identification information corresponding to the artificial neural network data locality pattern of FIG. 4 .
6 is a schematic diagram illustrating a predicted data access request and an actual data access request generated by an artificial neural network memory controller based on an artificial neural network data locality pattern according to an example of the present disclosure.
7 is a flowchart schematically illustrating an operation of an artificial neural network memory controller according to an example of the present disclosure.
8 is a schematic block diagram illustrating an artificial neural network memory system according to another example of the present disclosure.
9 is a schematic diagram illustrating an operation of a memory system according to a comparative example of the present disclosure.
10 is a schematic diagram illustrating a memory system according to another example of the present disclosure.
11 is a schematic block diagram illustrating an artificial neural network memory system according to another example of the present disclosure.
12 is a schematic diagram illustrating exemplary identification information of a data access request.
13 is a schematic diagram illustrating energy consumption per unit operation of an artificial neural network memory system.
14 is a schematic diagram illustrating an artificial neural network memory system according to various examples of the present disclosure.
15 is an exemplary diagram illustrating a substrate and a channel on which a memory is mounted.
16 is an exemplary diagram illustrating a process of reading data from a memory having a multi-bank structure.
17 is an exemplary diagram illustrating latency occurring in a conventional DRAM.
18 is an exemplary diagram illustrating a basic concept of a Sequential Access Memory (SAM) according to the disclosure of the present specification.
19 is a table exemplarily showing the amount of computation and data size for 16 layers.
20 is a table exemplarily showing the amount of computation and data size for 28 layers.
21 is a table showing a first example of accessing a memory according to order information in artificial neural network data locality (ANN DL) information.
22 is an exemplary table showing the table shown in FIG. 21 in a simplified manner.
23 shows an example in which the SAM sets a memory address map according to the table shown in FIG. 22 .
24 is a table showing a second example of accessing a memory according to order information in artificial neural network data locality (ANN DL) information.
25 shows an example in which the SAM sets a memory address map according to the table shown in FIG. 24 .
26 is a table showing a third example of accessing a memory according to order information in artificial neural network data locality (ANN DL) information.
27A and 27B show examples of setting a memory address map according to artificial neural network data locality information.
28 is a conceptual diagram illustrating a control signal of a SAM controller.
29 is an exemplary diagram illustrating an example of setting a memory address map according to a sideband signal shown in FIG. 28 .
30A shows another example of setting a memory address map according to a sideband signal, and FIG. 30B shows an example of a memory address map in which only a kernel is sequentially recorded.
31A is an exemplary diagram illustrating a 'READ_DISCARD' command transmitted through a sideband signal according to an example presented in the present specification, and FIG. 31B illustrates an example of the READ command.
32 shows a part of a circuit diagram of an exemplary SAM implemented in the form of a memory cell of a DRAM according to an example presented herein.
FIG. 33 is an exemplary diagram for explaining a precharge operation in the SAM circuit diagram shown in FIG. 32 .
FIG. 34 is an exemplary diagram for explaining a memory cell access operation in the SAM circuit diagram shown in FIG. 32 .
FIG. 35 is an exemplary diagram for explaining a data detection (DATA SENSE) operation in the SAM circuit diagram shown in FIG. 32 .
FIG. 36 is an exemplary diagram for explaining a READ-DISCARD operation in the SAM circuit diagram shown in FIG. 32 .
37 is an exemplary diagram for explaining a READ operation in the SAM circuit diagram shown in FIG. 32 .
38A is an exemplary waveform diagram of a READ-DISCARD operation, and FIG. 38B is an exemplary waveform diagram of a READ operation.
FIG. 39 is a table showing a part of the table shown in FIG. 21 in order to explain the REFREASH operation.
40 shows an example in which a SAM memory is implemented in various forms according to an example presented herein.
41 is an exemplary diagram illustrating an example of a method of mapping an address of a main memory based on ANN data locality information.
42 is an exemplary diagram illustrating another example of a method of mapping an address of a main memory based on ANN data locality information.
43 is a table showing an example of accessing a memory according to order information in artificial neural network data locality (ANN DL) information.
44 is an exemplary diagram illustrating an example of a memory in which a SAM controller is embedded.
45 is an exemplary diagram illustrating an architecture including a compiler.
46 shows an architecture according to a first example.
47 shows an architecture according to a second example.
48 shows an architecture according to a third example.
49 shows an architecture according to a fourth example.
50 shows an architecture according to a fifth example.
51 shows an architecture according to a sixth example.
52 is an exemplary diagram illustrating an operation according to the sixth example shown in FIG. 51 .
53A and 53B are exemplary views illustrating examples of convolution.
54 illustrates another example of caching data in the main memory in the cache memory and then performing an operation based on a tiling technique.
55 is a schematic diagram illustrating an artificial neural network memory system according to various examples of the present disclosure.
56 shows the detailed operation configuration of the SFU shown in FIG.
57 shows a graph in which the bandwidth of the data bus between the buffer memory (cache) and the main memory is measured.

Advantages and features of the present disclosure, and methods of achieving them, will become apparent with reference to the various examples described below in detail in conjunction with the accompanying drawings. However, the present disclosure is not limited to the examples described below, but may be implemented in various different forms, and only the examples of the present disclosure completely convey the scope of the invention to those of ordinary skill in the art to which the present disclosure belongs. It is provided as a reminder, and the present invention is only defined by the scope of the claims.

The detailed description of the present disclosure may be described with reference to the drawings for convenience of description by way of illustration of specific examples in which the present disclosure may be practiced. Even if components of various examples of the present disclosure are different from each other, manufacturing methods, operating methods, algorithms, shapes, processes, structures, and characteristics described in a specific example may be combined with or included in other examples. In addition, the location or arrangement of individual components within each disclosed example may be changed without departing from the spirit and scope of the present disclosure. Each feature of the various examples of the present disclosure may be partially or wholly combined or combined with each other, and as those skilled in the art will fully understand, technically various interlocking and operation are possible, and each example may be independently implemented with respect to each other, and related relationships can also be performed together.

Since the shape, size, ratio, angle, number, etc. disclosed in the drawings for explaining examples of the present disclosure are exemplary, the present disclosure is not limited thereto with reference to the drawings. Like reference numerals may refer to like elements throughout. In addition, in describing the present disclosure, when it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter of the present disclosure, the detailed description may be omitted. When 'including', 'having', 'consisting', etc. mentioned in this specification are used, other components may be added unless 'only' is used. When a component is expressed in the singular, cases including the plural are included unless otherwise explicitly stated. In interpreting the components, it is interpreted as including an error range even if there is no separate explicit description. In the case of a description of the positional relationship, for example, the positional relationship between the two components is expressed as 'on', 'on', 'on', 'next to', 'adjacent to', etc. When described, one other element may be positioned between two elements unless 'directly' or 'directly' is used. Reference to a device or layer “on” another device or layer includes any intervening layer or other device directly on or in the middle of the other device or layer.

1A is a schematic block diagram illustrating a processor and an artificial neural network memory controller of an artificial neural network memory system based on artificial neural network data locality according to an example of the present disclosure.

Referring to FIG. 1A , the artificial neural network memory system 100 may be configured to include at least one processor 110 and at least one artificial neural network memory controller 120 . That is, there is at least one processor 110 according to examples of the present disclosure, and a plurality of processors may be utilized. That is, there is at least one artificial neural network memory controller 120 according to examples of the present disclosure, and a plurality of artificial neural network memory controllers may be utilized.

For convenience of description below, when the at least one processor 110 is one processor, it may be referred to as a processor 110 .

For convenience of description below, when the at least one artificial neural network memory controller 120 is one artificial neural network memory controller 120 , it may be referred to as an artificial neural network memory controller 120 .

The processor 110 is configured to process an artificial neural network model. For example, the processor 110 may provide an inference result of the artificial neural network model according to input data by processing the inference of the artificial neural network model learned to perform a specific reasoning function. For example, the processor 110 may provide the learned artificial neural network model by processing the learning of the artificial neural network model for performing a specific reasoning function. The specific inference function may include various inference functions that can be inferred by the artificial neural network, such as object recognition, voice recognition, and image processing.

The processor 110 is at least one of a central processing unit (CPU), a graphic processing unit (GPU), an application processor (AP), a digital signal processing unit (DSP), an arithmetic logic unit (ALU), and an artificial neural network processor (NPU) It may be configured to include However, the processor 110 of the present disclosure is not limited to the above-described processors.

The processor 110 may be configured to communicate with the artificial neural network memory controller 120 . The processor 110 may be configured to generate a data access request. The data access request may be transmitted to the artificial neural network memory controller 120 . Here, the data access request may mean a request to access data required when the processor 110 processes inference or learning of the artificial neural network model.

The processor 110 transmits a data access request to the artificial neural network memory controller 120 to receive data necessary for inference or learning of the artificial neural network model from the artificial neural network memory controller 120, or to receive artificial neural network model processing data. The inference or learning result of the neural network may be provided to the artificial neural network memory controller 120 .

The processor 110 may provide an inference result or a learning result obtained by processing a specific artificial neural network model. In this case, the processor 110 may be configured to process operations of the artificial neural network for reasoning or learning in a specific order.

The reason that the processor 110 must process the artificial neural network calculations in a specific order is that each artificial neural network model is configured to have a unique artificial neural network structure. That is, each artificial neural network model is configured to have unique artificial neural network data locality according to a unique artificial neural network structure. Furthermore, the operation order of the artificial neural network model processed by the processor 110 is determined according to the unique locality of the artificial neural network data.

In other words, artificial neural network data locality can be configured when an artificial neural network model is compiled to run on a specific processor by a compiler. Neural network data locality may be configured according to a compiler, algorithms applied to an artificial neural network model, and operating characteristics of a processor.

The artificial neural network model to be processed by the processor 110 may be compiled by the processor 110 and a compiler capable of considering the algorithm characteristics of the artificial neural network model. That is, if the structure and algorithm information of the artificial neural network model are known and the driving characteristics of the processor 110 are known, the compiler may be configured to provide artificial neural network data locality information to the artificial neural network memory controller 120 in word unit order.

For example, a weight value of a specific layer of a specific artificial neural network model of a conventional algorithm level may be calculated for each layer. However, the weight value of the specific layer of the specific artificial neural network model of the processor-memory level according to the examples of the present disclosure may be calculated in units of words scheduled to be processed by the processor 110 .

For example, when the size of the cache memory of the processor 110 is smaller than the data size of the weight values of the specific layer of the artificial neural network model to be processed, the processor 110 compiles so as not to process the weight values of the specific layer at once. can be

That is, when the processor 110 calculates the weight values and node values of a specific layer, since the weight value is too large, a cache memory space to store the result values may be insufficient. In this case, the data access request generated by the processor 110 may be increased to a plurality of data access requests. Accordingly, the processor 110 may be configured to process the increased data access requests in a specific order. In this case, the operation order of the algorithm level and the operation order according to the locality of the artificial neural network data at the processor-memory level may be different from each other.

That is, the neural network operation order at the algorithm level can be reconstructed by the neural network data locality at the processor-memory level in consideration of the hardware characteristics of the processor and memory to process the corresponding artificial neural network model.

The neural network data locality of the artificial neural network model existing at the processor-memory level is the operation sequence of the artificial neural network model processed by the processor 110 at the processor-memory level based on the data access request order requested by the processor 110 to the memory. It can be defined as information that makes predictions.

To elaborate, even in the case of the same artificial neural network model, the number of processing elements of the processor 110, such as the calculation function of the processor 110, for example, a feature map tiling technique, a stationary technique of processing elements, etc. , the processor 110, cache memory capacity such as feature maps and weights, the memory hierarchy in the processor 110, and the algorithm characteristics of the compiler that determine the order of operation of the processor 110 for processing the artificial neural network model The locality of the artificial neural network data of the artificial neural network model may be configured differently according to the like.

For example, feature map tiling is an artificial neural network technique that divides the convolution, and as the convolution region is divided, the feature map is divided and calculated. Therefore, even in the same artificial neural network model by tiling convolution, the locality of the artificial neural network data of the artificial neural network model may be different from each other.

For example, the stationary technique is a technique for controlling a driving method of the processing elements PE in the neural processing unit. According to the stationary technique, a type of processed data, for example, one of an input feature map, a weight, and an output feature map may be fixed to a processing element and reused. Accordingly, the type and order of data requested by the processor 110 from the memory may vary.

That is, even in the case of the same artificial neural network model, artificial neural network data locality may be reconstructed according to various algorithms and/or techniques. Therefore, artificial neural network data locality can be completely or partially reconstructed by various conditions such as processor, compiler, and memory.

1B is a schematic diagram illustrating an example of an exemplary neural processing unit for explanation regarding reconstruction of an artificial neural network data locality pattern that can be applied to various examples of the present disclosure.

Referring to FIG. 1B , exemplary stationary techniques that may be applied when the processor 110 is a neural processing unit (NPU) are illustrated.

The processing elements PE may be configured in an array form, and each processing element may be configured to include a multiplier (x) and an adder (+). The processing elements PE may be connected to a buffer memory or a cache memory, for example, a global buffer. The processing elements PE are configured to fix data of one of an input feature map pixel (I), a filter weight (W), and a subsum (Psum) to a register of the processing elements PE. can In addition, the remaining data may be provided as input data of the processing elements PE. When the accumulation of the subtotal P is completed, it may become an output feature map pixel. However, it is also possible that the plurality of processing elements are implemented not in the form of an array, but in the form of individually driving.

1B (a) illustrates a Weight-Stationary (WS) technique. According to the weight stationary (WS) technique, filter weights W0 to W7 are fixed in the register file of each of the processing elements PE, and an input feature map pixel ( The operation can be executed while moving I) from the 0th input feature map pixel (I0) to the 8th input feature map pixel (I8). The subtotals P0 to P8 may be accumulated in the serially connected processing elements PE. The subtotals P0 to P8 may sequentially move to the next processing element. All multiply and accumulation (MAC) operations using the fixed filter weights W0 to W7 must be mapped to the same processing elements PE for serial processing.

According to the above configuration, there is an effect of minimizing the access energy consumption of the filter weight W by maximizing the reuse of the filter weight W during the convolution operation of the filter weight W in the register file.

It should be noted that, as the weighted stationary (WS) method is applied to the artificial neural network model at the compilation stage, the locality of the artificial neural network data of the artificial neural network model is optimized for the weighted stationary (WS) method at the processor-memory level. is reconstructed for For example, in the weight stationary (WS) technique, the filter weights W0 to W7 may be preferentially stored in the processing elements PE for efficiency of operation. Therefore, the neural network data locality can be reconstructed in the order of the filter weight (W), the input feature map pixel (I), and the subtotal (P), and thus the data access request sequence generated by the processor 110 is also reconstructed. can be determined according to

Figure 1b (b) shows an output stationary (Output-Stationary; OS) technique. According to the output stationary (OS) technique, subtotals P0 to P7 are fixed and accumulated in each register file of the processing elements PE, and the filter weights ( The operation can be executed while moving W) from the 0th input filter weight (W0) to the 7th filter weight (W7). The input feature map pixels I0 to I7 may be moved to serially connected processing elements PE. Each of the subtotals P0 to P7 is fixed to each of the processing elements PE and must be mapped to process a multiply and accumulation (MAC) operation.

According to the above configuration, the subtotal P is fixed to the register file of the processing elements PE during the convolution operation of the filter weights W in the processing elements PE to maximize the reuse of the subtotal P and to maximize the subsum There is an effect that can minimize the energy consumption according to the movement of (P). When the accumulation of the fixed subtotal P is completed, it may become an output feature map.

It should be noted that, as the processor 110 applies the output stationary (OS) technique, the neural network data locality of the neural network model is reconfigured to be optimized for the output stationary (OS) technique at the processor-memory level. do. For example, the output stationary (OS) technique may be configured to preferentially store the subtotals P0 to P7 in the processing elements PE for efficiency of operation. Therefore, the neural network data locality can be reconstructed in the order of the subtotal (P), the filter weight (W), and the input feature map pixel (I). The neural network model compiler may receive hardware characteristic information of the processor 110 and the memory and convert the artificial neural network model into code that can operate at the processor-memory level. At this time, since the artificial neural network model is converted into code executed by a processor, it can be converted into a low-level code.

That is, according to each of the factors described above, the processor 110 may change the order of data required at every moment in clock units even when the same artificial neural network model is processed. Therefore, the locality of the artificial neural network data of the artificial neural network model may be configured differently at the hardware level.

However, when the configuration of the artificial neural network data locality is completed, the operation order of the processor 110 and the data processing order necessary for the operation may be accurately repeated for each learning operation or inference operation of the artificial neural network model.

Hereinafter, the neural network memory system 100 according to an example of the present disclosure described above predicts the next data to be requested by the processor 110 in advance based on the precise operation sequence provided by the artificial neural network data locality, thereby reducing the memory delay problem and memory bandwidth. By improving the problem, it can be configured to improve the processing performance of the artificial neural network and reduce power consumption.

The artificial neural network memory controller 120 according to an example of the present disclosure is configured to receive locality information of the artificial neural network data of the artificial neural network model to be processed by the processor 110, or the artificial neural network of the artificial neural network model being processed by the processor 110 It is characterized in that it is configured to analyze data locality.

The artificial neural network memory controller 120 may be configured to receive a data access request generated by the processor 110 .

The artificial neural network memory control unit 120 may be configured to monitor or record the data access request received from the processor 110 . The artificial neural network memory control unit 120 has the effect of accurately predicting the data access sequence to be requested later by observing the data access requests output by the processor 110 processing the artificial neural network model. One data access request may be configured to include data of at least one word unit.

The artificial neural network memory control unit 120 may be configured to sequentially record or monitor data access requests received from the processor 110 .

The data access requests recorded by the artificial neural network memory controller 120 may be stored in various formats, such as a log file, a table, and a list. However, the artificial neural network memory controller 120 according to an example of the present disclosure is not limited to the recorded form or form of the data access request.

The data access requests monitored by the artificial neural network memory controller 120 may be stored in any memory in the artificial neural network memory controller 120 . However, the artificial neural network memory controller 120 according to an example of the present disclosure is not limited to the monitoring method of the data access request.

The artificial neural network memory control unit 120 may be configured to further include an arbitrary memory for recording or monitoring a data access request. However, the artificial neural network memory controller 120 according to an example of the present disclosure is not limited thereto, and may be configured to communicate with an external memory.

The artificial neural network memory controller 120 may be configured to analyze the data access requests by monitoring or recording the data access requests received from the processor 110 .

That is, the artificial neural network memory controller 120 may be configured to analyze the received data access requests to analyze the artificial neural network data locality of the artificial neural network model being processed by the processor 110 .

That is, the artificial neural network memory controller 120 may be configured to analyze the locality of the artificial neural network data of the artificial neural network model compiled to operate at the processor-memory level.

That is, the artificial neural network memory controller 120 analyzes the computational processing sequence of the artificial neural network in units of memory access requests generated by the processor, based on the processor-memory level data locality of the artificial neural network, and analyzes the artificial neural network data locality of the artificial neural network model. can be configured to analyze.

According to the above configuration, the artificial neural network memory controller 120 has an effect of analyzing the locality of the artificial neural network data reconstructed at the processor-memory level.

In some examples, the compiler may be configured to analyze the neural network data locality of the artificial neural network model to a word unit.

In some examples, the at least one artificial neural network memory controller may be configured to receive the artificial neural network data locality analyzed by the compiler in units of words. Here, the word unit may be 8-bit, 16-bit, 32-bit, 64-bit, or the like, depending on the word unit of the processor 110 . Here, the word unit may be set in different word units, such as 2 bit, 3 bit, or 5 bit, depending on the quantization algorithm such as the kernel and feature map of the compiled artificial neural network model.

The artificial neural network memory control unit 120 may be configured to include a special function register. The special function register may be configured to store artificial neural network data locality information.

The artificial neural network memory controller 120 may be configured to operate in different modes depending on whether artificial neural network data locality information is stored.

If the artificial neural network memory control unit 120 stores artificial neural network data locality information, the artificial neural network memory controller 120 may predict the data processing order of the artificial neural network model to be processed by the processor 110 in advance in word unit order. Therefore, it may be configured not to record a separate data access request. However, the present invention is not limited thereto, and the artificial neural network memory controller 120 may be configured to verify whether an error exists in the stored artificial neural network data locality while comparing the stored artificial neural network data locality information with the data access request generated by the processor.

If the artificial neural network memory controller 120 is not provided with the artificial neural network data locality information, the artificial neural network memory controller 120 observes the data access request generated by the processor 110 and processes the artificial neural network data by the processor 110 . It may be configured to operate in a mode for predicting artificial neural network data locality of a neural network model.

In some examples, the neural network memory system may include a processor, a memory, and a cache memory, and may be configured to generate a predicted data access request including data to be requested by the processor based on the neural network data locality information. The predicted data access request may be referred to as a data access request predicted based on the ANN DL or a data access request to be requested. For convenience of description below, the predicted data access request may be referred to as a prior data access request. The artificial neural network memory system may be configured to store data corresponding to the data access request predicted from the memory in the cache memory before the processor requests it. At this time, the artificial neural network memory system is one of a first mode configured to operate by receiving artificial neural network data locality information or a second mode configured to predict and operate artificial neural network data locality information by observing data access requests generated by the processor It can be configured to operate as According to the above configuration, when the artificial neural network data locality information is provided, the artificial neural network memory system has the effect of predicting and preparing the data requested by the processor in word units in advance, even if the artificial neural network data locality information is not provided. By monitoring the data access requests generated by the processor for a certain period of time, it has the effect of predicting the locality of the artificial neural network data being processed by the processor in units of data access requests. Furthermore, even if the artificial neural network data locality information is provided, the artificial neural network memory system can reconstruct the artificial neural network data locality by monitoring data access requests by itself and use it for the purpose of verifying the provided artificial neural network data locality. Accordingly, an effect of detecting a change in the artificial neural network model or occurrence of an error may be provided.

In some examples, the at least one artificial neural network memory controller and the at least one processor may be configured to communicate directly. According to the above configuration, since the artificial neural network memory control unit can directly receive a data access request from the processor, there is an effect of eliminating delay time that may be caused by the system bus between the processor and the artificial neural network memory control unit. . In more detail, for direct communication between the processor and the artificial neural network memory controller, it may be configured to further include a dedicated bus or to further include a dedicated communication channel. However, the present invention is not limited thereto.

In some examples, the artificial neural network data locality information may be configured to be selectively stored in the processor 110 and/or the artificial neural network memory controller 120 . The artificial neural network data locality information may be configured to be stored in a special function register included in the processor 110 and/or the artificial neural network memory controller 120 . However, the present invention is not limited thereto, and the artificial neural network data locality information may be stored in any memory, register, or the like that can communicate with the artificial neural network memory system.

2 is a schematic diagram illustrating an artificial neural network data locality pattern according to an example of the present disclosure. Hereinafter, the artificial neural network data locality and the artificial neural network data locality pattern of the artificial neural network model will be described with reference to FIG. 2 .

The artificial neural network memory control unit 120 is configured to sequentially record or monitor the data access requests received from the processor 110 .

The artificial neural network memory controller 120 is configured to generate an artificial neural network data locality pattern including the data locality of the artificial neural network model being processed by the processor 110 . That is, the artificial neural network memory control unit 120 may be configured to generate a repeating specific pattern by analyzing data access requests related to the artificial neural network model generated by the processor 110 . That is, when observing a data access request, artificial neural network data locality information may be stored as an artificial neural network data locality pattern.

Referring to FIG. 2 , for example, 18 data access requests are sequentially recorded in the artificial neural network memory controller 120 . Each data access request is configured to include identification information.

The identification information included in the data access request may be configured to include various information.

For example, the identification information is configured to include at least a memory address value and an operating mode value.

For example, the memory address value may be configured to include memory address values corresponding to the requested data. However, the present disclosure is not limited thereto.

For example, the memory address value may be configured to include a start value and an end value of the memory address corresponding to the requested data. According to the above configuration, it is considered that data is sequentially stored between the start value and the end value of the memory address. Accordingly, there is an effect that the capacity for storing memory address values can be reduced. That is, when the trigger value is activated, the memory can also operate in burst mode.

For example, the memory address value may be configured to include a start value of the memory address corresponding to the requested data and a data continuous read trigger value. According to the above configuration, data can be read continuously from the start value of the memory address until the continuous read trigger value is changed. According to the above-described configuration, since data can be read continuously, there is an effect that an effective memory bandwidth can be increased.

For example, the memory address value may be configured to include a start value of a memory address corresponding to the requested data and information on the number of data. The unit of the number of data may be determined based on the unit of the capacity of the memory. The unit may be, for example, one of 8 bits of 1 byte, 4 bytes of 1 word, or 1024 bytes of 1 block. However, the present disclosure is not limited thereto. According to the above-described configuration, data can be read continuously from the start value of the memory address as much as the number of data of the set unit size. According to the above configuration, since data can be read continuously, there is an effect that an effective memory bandwidth can be increased.

For example, when the memory is a non-volatile memory, the memory address value may further include a physical-logical address mapping table or flash translation layer information. However, the present disclosure is not limited thereto.

For example, the operation mode may be configured to include a read mode and a write mode. Reading and writing may further include a burst mode.

For example, the mode of operation may be configured to further include overwrite. However, the present disclosure is not limited thereto.

The artificial neural network memory controller 120 may be configured to determine whether the identification information of each of the data access requests is the same.

For example, the artificial neural network memory controller 120 may be configured to determine whether the memory address and operation mode of each of the data access requests are the same. In other words, the artificial neural network memory controller 120 may be configured to detect a data access request value having the same memory address value and the same operation mode.

For example, when the memory address value and operation mode of the first data access request and the memory address value and operation mode of the tenth data access request are the same, the artificial neural network memory controller 120 controls the memory address value and operation mode It is configured to generate an artificial neural network data locality pattern corresponding to .

The artificial neural network data locality pattern may be configured to include data in which addresses of data access requests are sequentially recorded.

That is, the neural network memory control unit 120 detects a repetition period of data access requests having the same memory address value and operation mode, and generates an artificial neural network data locality pattern composed of data access requests having a repeated memory address value and operation mode. can be configured to create

That is, the artificial neural network memory control unit 120 may be configured to generate an artificial neural network data locality pattern by detecting a repeating pattern of a memory address included in the data access request.

Referring to FIG. 2 , when the neural network memory controller 120 confirms that the memory address values and operation modes of the first data access request and the tenth data access request are the same, the neural network memory controller 120 is It can be configured to generate one artificial neural network data locality pattern from a data access request initiated among the same data access requests to a previous data access request of a repeated data access request. In this case, the neural network memory control unit 120 may be configured to generate the neural network data locality pattern including the first data access request to the ninth data access request.

That is, the artificial neural network data locality pattern described in the example of FIG. 2 is a first data access request, a second data access request, a third data access request, a fourth data access request, a fifth data access request, and a sixth data access request. , the seventh data access request, the eighth data access request, and the ninth data access request sequence may be configured to include memory address values and operation mode values.

The artificial neural network data locality pattern generated by the artificial neural network memory controller 120 may be stored in various forms such as a log file, a table, and a list, and the artificial neural network according to an example of the present disclosure The memory control unit 120 is not limited to the recorded form or form of the artificial neural network data locality pattern.

The artificial neural network data locality pattern generated by the artificial neural network memory controller 120 may be stored in an arbitrary memory of the artificial neural network memory controller 120, and the artificial neural network memory controller 120 according to an example of the present disclosure It is not limited to the structure or method of the memory for storing the data locality pattern.

The artificial neural network memory control unit 120 may be configured to further include an arbitrary memory for storing artificial neural network data locality patterns. However, the artificial neural network memory controller 120 according to an example of the present disclosure is not limited thereto, and may be configured to communicate with an external memory.

That is, the artificial neural network memory system 100 according to an example of the present disclosure sequentially records the data access request and at least one processor 110 configured to generate a data access request corresponding to the artificial neural network operation, so that the artificial neural network data locality It may be configured to include an artificial neural network memory controller 120 configured to generate a pattern.

When the artificial neural network memory controller 120 generates the artificial neural network data locality pattern, the artificial neural network memory controller 120 determines that the memory address value and the operation mode value of each data access request received from the processor 110 are previously generated. It may be configured to determine whether any one of memory address values and operation mode values included in the artificial neural network data locality pattern matches.

Referring to FIG. 2 , when the artificial neural network memory controller 120 receives the tenth data access request from the processor 110 , the artificial neural network memory controller 120 determines that the received data access request is an artificial neural network data locality pattern. and determine whether it has the same memory address value as the memory address value contained in the .

2, when the artificial neural network memory controller 120 receives the tenth data access request, the neural network memory controller 120 controls the memory address value of the tenth data access request, starting value [0]. and detecting that the end value [0x1000000] and the start value [0] and the end value [0x1000000] that are the memory address values of the first data access request are the same, and the read mode value of the operation mode of the tenth data access request and the first Detect that the read mode values of the operation modes of the data access request are identical to each other, and determine that the tenth data access request is the same as the first data access request, and that the tenth data access request is an artificial neural network operation. .

When the artificial neural network memory control unit 120 receives the eleventh data access request, the start value [0x1100000] that is the memory address value of the eleventh data access request, the end value [0x1110000], and the start value that is the memory address value of the second data access request [ 0x1100000] detecting that the end value [0x1110000] is the same, detecting that the write mode value of the operation mode of the eleventh data access request and the write mode value of the operation mode of the second data access request are the same It may be configured to determine that the access request is identical to the second data access request, and the eleventh data access request is an artificial neural network operation.

That is, the artificial neural network memory controller 120 may distinguish the start and the end of the artificial neural network data locality pattern. In addition, the artificial neural network memory controller 120 may prepare in advance for the start of the artificial neural network data locality pattern if there is no special command after the end of the artificial neural network data locality pattern. Therefore, when the same operation is repeated, there is an effect that data can be prepared before the start of the next speculation by predicting the start of the next speculation based on the end of the speculation. Therefore, when the same artificial neural network data locality pattern is repeated, it is possible to prevent or reduce the delay time at the beginning and the end.

Referring back to FIG. 2 , the neural network memory controller 120 exemplifies a case in which the neural network data locality pattern is not generated from the first data access request to the ninth data access request. In this case, the artificial neural network memory controller 120 may be initialized or the processor 110 may not perform the artificial neural network operation. Therefore, the artificial neural network memory controller 120 does not detect a case in which the pattern matches until the ninth data access request. When the tenth data access request is made, the artificial neural network memory controller 120 may determine the same as the first data access request, generate an artificial neural network data locality pattern, and record whether the pattern matches. Since the tenth data access request to the eighteenth data access request are the same as the first data access request to the ninth data access request, the artificial neural network memory control unit 120 determines that the pattern of the tenth data access request to the eighteenth data access request is artificial. It can be determined that it matches the neural network data locality pattern.

That is, the artificial neural network memory controller 120 according to an example of the present disclosure may be configured to determine whether the operation being processed by the processor 110 is an artificial neural network operation by utilizing the artificial neural network data locality pattern. According to the above configuration, the artificial neural network memory control unit 120 determines that the processor 110 is processing the artificial neural network operation even if only the data access request including the memory address value and the operation mode value generated by the processor 110 is received. possible effects can be provided. Accordingly, the artificial neural network memory controller 120 may provide an effect of determining whether the processor 110 currently performs an artificial neural network operation based on the artificial neural network data locality pattern even without additional identification information.

2 , each data access request may be configured to be stored as a token. For example, each data access request of the artificial neural network may be stored by tokenizing the data access request. For example, each data access request of an artificial neural network can be tokenized based on identification information. For example, each data access request of an artificial neural network can be tokenized based on a memory address value. However, examples of the present disclosure are not limited thereto, and the token may be referred to as a code or an ID. For example, the token may be a word unit, a data access request unit, or an ANN DL unit.

For example, the first data access request may be stored as a token [1]. The fourth data access request may be stored as a token [4]. The seventh data access request may be stored as a token [7]. For example, the neural network data locality pattern may be stored as a token [1-2-3-4-5-6-7-8-9]. For example, the tenth data access request may be stored as token [1] because it has the same memory address value and the same operation mode value as token [1]. Since the thirteenth data access request has the same memory address value and operation mode value as the token [4], it may be stored as the token [4]. Accordingly, the neural network memory controller 120 may be configured to determine that the corresponding data access request is an artificial neural network operation when detecting the same token as the token of the artificial neural network data locality pattern.

According to the above configuration, the artificial neural network memory control unit 120 has the effect of easily and quickly recognizing and classifying a data access request by using the tokenized artificial neural network data locality pattern, and furthermore, additional identification information and / Or even when more data is added, the same token can be used to provide the effect of quickly and easily recognizing and distinguishing a data access request by using the token even when the additional information of the data access request increases.

In some examples, the artificial neural network data locality pattern stored in the artificial neural network memory controller may be deleted or initialized. For example, when the artificial neural network data locality pattern is not utilized for more than a preset time, for example, when a data access request matching the artificial neural network data locality pattern is not generated for a specific time, the artificial neural network memory controller determines that the frequency of use of the artificial neural network data locality pattern is low, and may delete or initialize the artificial neural network data locality pattern.

According to the above configuration, there is an effect that can improve the utilization of the storage space of the memory for storing the artificial neural network data locality pattern.

In some examples, the artificial neural network memory controller may be configured to store the updated pattern and the previous pattern of the artificial neural network data locality pattern to determine whether to change the artificial neural network model. That is, when the number of artificial neural network models is plural, the artificial neural network memory controller may be configured to further generate artificial neural network data locality patterns corresponding to the number of artificial neural network models.

For example, the first neural network data locality pattern is token [1-2-3-4-5-6-7-8-9] and the second neural network data locality pattern is token [11-12-13-14 -15-16], when the processor generates a data access request corresponding to the token [1], the neural network memory control unit may be configured to select the first artificial neural network data locality pattern. Alternatively, when the processor generates a data access request corresponding to the token [11], the artificial neural network memory control unit may be configured to select the second artificial neural network data locality pattern.

According to the above configuration, the artificial neural network memory controller can store a plurality of artificial neural network data locality patterns, and when the artificial neural network model processed by the processor is changed to another artificial neural network model, the previously stored artificial neural network data locality pattern can be quickly applied. can have an effect.

In some examples, the neural network memory controller may be configured to determine whether the data access requests are requests of one artificial neural network model or a mixture of requests of a plurality of neural network models. Also, the artificial neural network memory controller may be configured to predict a data access request corresponding to the artificial neural network data locality of each of the plurality of artificial neural network models.

For example, the processor may simultaneously process a plurality of artificial neural network models, and in this case, the data access request generated by the processor may be mixed with data access requests corresponding to the plurality of artificial neural network models.

For example, the first neural network data locality pattern is token [1-2-3-4-5-6-7-8-9] and the second neural network data locality pattern is token [11-12-13-14 -15-16], the processor 110 requests data access in the order of [1-11-2-3-12-13-14-4-5-6-15-16-7-8-9] You can create a token corresponding to .

Since the neural network memory controller knows each neural network data locality pattern, even if token [1] is generated and then token [11] is generated, the neural network memory controller can predict that token [2] will be generated next. . Therefore, the neural network memory control unit can generate a dictionary data access corresponding to the token [2]. Also, even after token [11] is generated and then token [2] is generated, the artificial neural network memory controller can predict that token [12] will be generated next. Therefore, the neural network memory control unit can generate a dictionary data access corresponding to the token [12].

According to the above configuration, the artificial neural network memory control unit 120 predicts the data access request to be generated by the processor 110 for processing a plurality of artificial neural network models for each artificial neural network model, and pre-predicts the data to be requested by the processor 110 . It has the effect of being able to predict and prepare for it.

In some examples, the neural network memory controller may be configured to store a plurality of neural network data locality patterns.

For example, when the processor processes two artificial neural network models, the artificial neural network memory controller may be configured to store artificial neural network data locality patterns of each artificial neural network model.

According to the above configuration, when the operation of each artificial neural network model is processed, the actual data access request corresponding to each model can be predicted, so the example of the present invention can improve the processing speed of the artificial neural network operation. there is an effect

In some examples, the artificial neural network memory control unit may be configured to further include an artificial neural network model configured to machine-learning artificial neural network data locality patterns.

According to the above configuration, the artificial neural network model of the artificial neural network memory controller may be configured to reinforce-learning data access requests generated by the processor in real time. In addition, the artificial neural network model of the artificial neural network memory controller may be a model learned by using artificial neural network data locality patterns of conventionally well-known artificial neural network models as learning materials. Therefore, the artificial neural network memory control unit has the effect of extracting artificial neural network data locality patterns from various artificial neural network models. In particular, this method can be effective when processing various artificial neural network models according to requests from multiple users, such as a server.

2 , the artificial neural network memory controller 120 may be configured to dynamically or in real time monitor the artificial neural network model processed by the processor 110 and determine whether to change the artificial neural network model.

For example, the artificial neural network memory controller 120 may be configured to statistically utilize the pattern matching frequency of the artificial neural network data locality pattern to determine the reliability of the artificial neural network data locality pattern. The reliability of the neural network data locality pattern may be configured to increase as the pattern matching frequency of the data locality pattern increases, and the reliability of the neural network data locality pattern may be configured to decrease as the pattern matching frequency of the data locality pattern decreases.

According to the above configuration, when the processor 110 repeatedly processes a specific artificial neural network model, the artificial neural network memory control unit 120 has an effect that the locality prediction reliability of the artificial neural network data of the specific artificial neural network model can be improved.

3 is a schematic diagram illustrating an exemplary artificial neural network model for explaining an artificial neural network data locality pattern that can be applied to various examples of the present disclosure.

The exemplary artificial neural network model 1300 being processed by the processor 110 shown in FIG. 3 may be any artificial neural network model trained to perform a specific reasoning function. For convenience of explanation only, although an artificial neural network model in which all nodes are fully-connected is illustrated, the present disclosure is not limited thereto.

Although not shown in FIG. 3 , the artificial neural network model applicable to the present disclosure may be a convolutional neural network (CNN), which is a type of a deep neural network (DNN). Exemplary artificial neural network models include VGG, VGG16, DenseNet and FCN (Fully Convolutional Network) with encoder-decoder structure, SegNet, DeconvNet, DeepLAB V3+, Deep neural network (DNN) such as U-net, SqueezeNet, Alexnet, ResNet18 , MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, Resnet101, Inception-v3, etc., or an ensemble model based on at least two different models. However, the artificial neural network model of the present disclosure is not limited thereto.

The above-described exemplary artificial neural network models may be configured to have artificial neural network data locality.

Referring again to FIG. 3 , the locality of the artificial neural network data of the artificial neural network model processed by the processor 110 will be described in detail.

An exemplary artificial neural network model 1300 includes an input layer 1310 , a first connection network 1320 , a first hidden layer 1330 , a second connection network 1340 , a second hidden layer 1350 , and a third connection network 1360 . ), and an output layer 1370 .

The artificial neural network has a corresponding weight value. The weight value of the network is multiplied by the input node value, and the accumulated value of the multiplied values is stored in the node of the corresponding output layer.

To elaborate, the connection network of the artificial neural network model 1300 is shown as a line and the weight is shown as ⓧ.

To elaborate, it may be configured to additionally provide various activation functions for imparting nonlinearity to the accumulated value. The activation function is, for example, a sigmoid function, a hyperbolic tangent function, an ELU function, a Hard-Sigmoid function, a Swish function, a Hard-Swish function, a SELU function, a CELU function, a GELU function, a TANHSHRINK function, a SOFTPLUS function, a MISH function. , it may be a Piecewise Interpolation Approximation for Non-linear function or a ReLU function. However, the present disclosure is not limited thereto.

The input layer 1310 of the exemplary artificial neural network model 1300 includes x1 and x2 input nodes.

The first connection network 1320 of the exemplary artificial neural network model 1300 includes connections having six weight values that connect each node of the input layer 1310 and the nodes of the first hidden layer 1330 .

The first hidden layer 1330 of the exemplary artificial neural network model 1300 includes nodes a1, a2, and a3. The weight values of the first connection network 1320 are multiplied by the node values of the corresponding input layer 1310 , and the accumulated value of the multiplied values is stored in the first hidden layer 1330 .

The second connection network 1340 of the exemplary artificial neural network model 1300 includes connections having nine weight values connecting nodes of the first hidden layer 1330 and nodes of the second hidden layer 1350 .

The second hidden layer 1350 of the exemplary artificial neural network model 1300 includes nodes b1, b2, and b3. A weight value of the second connection network 1340 is multiplied by a corresponding node value of the first hidden layer 1330 , and an accumulated value of the multiplied values is stored in the second hidden layer 1350 .

The third connection network 1360 of the exemplary artificial neural network model 1300 includes connections having six weight values connecting each node of the second hidden layer 1350 and each node of the output layer 1370. .

The output layer 1370 of the exemplary artificial neural network model 1300 includes nodes y1 and y2. The weight value of the third connection network 1360 is multiplied by the input node value of the corresponding second hidden layer 1350 , and the accumulated value of the multiplied values is stored in the output layer 1370 .

According to the structure of the artificial neural network model 1300 described above, it can be recognized that the operation for each layer must be sequentially performed. That is, when the structure of the artificial neural network model is determined, the order of operation for each layer must be determined, and if the order is calculated differently, a problem that the inference result may be inaccurate may occur. The order of operations or data flow according to the structure of the artificial neural network model can be defined as artificial neural network data locality.

In more detail, although the description has been made on a layer-by-layer basis in FIG. 2 only for convenience of description, examples of the present disclosure are not limited to a layer-by-layer basis. Since the processor 110 according to examples of the present disclosure processes data based on artificial neural network data locality, it may be operated in units of words or data access requests, not in units of layers. Here, the data size of the data access request may be less than or equal to the data size of the corresponding layer.

Referring again to FIG. 3 , for example, for multiplication of weight values of the first connection network 1320 and node values of the input layer 1310 , the processor 110 may generate a data access request on a layer-by-layer basis.

However, the feature map division convolution of the processor 110 , the stationary technique of processing elements, the number of processing elements of the processor, the cache memory capacity of the processor 110 , the memory hierarchy of the processor 110 , and/or the processor 110 . can

When the data access request requested by the processor 110 is divided into a plurality of pieces, the order of requesting the divided data access requests may be determined by the artificial neural network data locality. At this time, the artificial neural network memory control unit 120 is provided with the artificial neural network data locality, and the processor 110 is configured to prepare to provide data corresponding to the actual data access request requested next, that is, the next data access request. It is possible. Hereinafter, the actual data access request may be referred to as a next data access request. Alternatively, the artificial neural network memory control unit 120 may be configured to predict the artificial neural network data locality and prepare to provide data corresponding to the actual data access request requested by the processor 110 .

The data access requests and artificial neural network data locality generated by the processor 110 during the artificial neural network operation of the artificial neural network model 1300 shown in FIG. 3 will be described.

The processor 110 generates a first data access request for reading the input node values of the input layer 1310 of the artificial neural network model 1300 . The first data access request includes a first memory address value and a read mode value. The first data access request may be stored as a token [1].

Next, the processor 110 generates a second data access request for reading the weight values of the first connection network 1320 of the artificial neural network model 1300 . The second data access request includes a second memory address value and a read mode value. The second data access request may be stored as a token [2].

Next, the processor 110 multiplies the weight values of the first connection network 1320 of the artificial neural network model 1300 and the node values of the input layer 1310 and stores the accumulated node values of the first hidden layer 1330. Create a third data access request for The third data access request includes a third memory address value and a write mode value. The third data access request may be stored as a token [3].

Next, the processor 110 generates a fourth data access request for reading node values stored in the first hidden layer 1330 of the artificial neural network model 1300 . The fourth data access request includes a third memory address value and a read mode value. The fourth data access request may be stored as a token [4].

Next, the processor 110 generates a fifth data access request for reading the weight values of the second connection network 1340 of the artificial neural network model 1300 . The fifth data access request includes a fifth memory address value and a write mode value. The fifth data access request may be stored as a token [5].

Next, the processor 110 multiplies the weight values of the second connection network 1340 of the artificial neural network model 1300 and the node values of the first hidden layer 1330 and accumulates the node values of the second hidden layer 1350. Generate a sixth data access request for storage. The sixth data access request includes a sixth memory address value and a write mode value. The sixth data access request may be stored as a token [6].

Next, the processor 110 generates a seventh data access request for reading node values stored in the second hidden layer 1350 of the artificial neural network model 1300 . The seventh data access request includes a sixth memory address value and a read mode value. The seventh data access request may be stored as a token [7].

Next, the processor 110 generates an eighth data access request for reading the weight values of the third connection network 1360 of the artificial neural network model 1300 . The eighth data access request includes an eighth memory address value and a read mode value. The eighth data access request may be stored as a token [8].

Next, the processor 110 multiplies the weight values of the third connection network 1360 of the artificial neural network model 1300 and the node values of the second hidden layer 1350 and stores the accumulated node values of the output layer 1370. Create a ninth data access request for The ninth data access request includes a ninth memory address value and a write mode value. The ninth data access request may be stored as a token [9]. The node values may be a feature map, an activation map, or the like. However, the present invention is not limited thereto. The weight values may be a kernel window. However, the present invention is not limited thereto.

That is, the processor 110 must generate the first to ninth data access requests for inference of the exemplary artificial neural network model 1300 . If the order of the data access requests generated by the processor 110 is mixed, the locality of the artificial neural network data of the artificial neural network model 1300 is damaged, so that an error occurs in the inference result of the artificial neural network model 1300 or the accuracy is impaired. can For example, when the processor 110 calculates the second layer first and then calculates the first layer, and the like. Accordingly, the processor 110 may be configured to sequentially generate data access requests based on artificial neural network data locality. Therefore, the artificial neural network memory controller 120 may assume that the processor 110 sequentially generates data access requests based on the artificial neural network data locality during computation of the artificial neural network.

However, as described above, each data access request may be reinterpreted at the processor-memory level according to the hardware characteristics of the processor. The above-described example has been described as an example in which the available capacity of the cache memory of the processor is sufficient, and the data size of the node value and the data size of the weight value are smaller than the available capacity of the cache memory. Accordingly, each layer may be described as being processed in one data access request unit. If the data size of the artificial neural network model weight value, feature map, kernel, activation map, etc. is larger than the available capacity of the cache memory of the processor, the corresponding data access request may be divided into a plurality of pieces. In this case, the artificial neural network The neural network data locality of the model can be reconstructed.

Since the artificial neural network memory control unit 120 according to an example of the present disclosure can generate an artificial neural network data locality pattern, the effect of being able to operate in response to the artificial neural network data locality of the artificial neural network model actively processed by the processor is effective. there is.

That is, the artificial neural network memory control unit 120 analyzes the recorded data access request even if the processor 110 does not know the actual artificial neural network data locality of the artificial neural network model being processed. there is

That is, the artificial neural network memory control unit 120 has the effect of substantially analyzing the artificial neural network data locality by analyzing the recorded data access request, even if the processor 110 does not provide structural information of the artificial neural network model being processed. .

In some examples, the neural network memory control unit may be configured to receive the neural network data locality pattern previously generated at the processor-memory level.

4 is a schematic diagram illustrating an artificial neural network data locality pattern generated by an artificial neural network memory controller analyzing the artificial neural network model of FIG. 3 according to an example of the present disclosure. 5 is a schematic diagram illustrating a token and identification information corresponding to the artificial neural network data locality pattern of FIG. 4 .

The artificial neural network data locality pattern 1400 shown in FIG. 4 is only shown as a token for convenience of description. 1A to 4, the artificial neural network data locality pattern 1400 of the artificial neural network model 1300 is stored as a token [1-2-3-4-5-6-7-8-9]. there is. A token corresponding to the artificial neural network data locality pattern 1400 shown in FIG. 5 and identification information corresponding to the token are shown.

Each data access request may be configured to include identification information. Each data access request can be expressed as a token. However, this is only for convenience of description, and the present disclosure is not limited to tokens.

According to the artificial neural network data locality pattern 1400 , the artificial neural network memory controller 120 has an effect of sequentially predicting the order of tokens to be generated after the current token.

For example, the artificial neural network data locality pattern 1400 may be configured to have a pattern in the form of a loop in which the sequence is connected from the last token to the start token. However, the present disclosure is not limited thereto.

For example, the artificial neural network data locality pattern 1400 may be composed of memory addresses having a repeated loop characteristic. However, the present disclosure is not limited thereto.

For example, the artificial neural network data locality pattern 1400 may be configured to further include identification information that can identify the start and end of the computation of the artificial neural network model. However, the present disclosure is not limited thereto.

For example, the start and end of the artificial neural network data locality pattern 1400 may be configured to be divided into a start token and a last token of the pattern. However, the present disclosure is not limited thereto.

According to the above configuration, when the processor 110 repeatedly infers a specific artificial neural network model, since the artificial neural network data locality pattern 1400 is a loop-type pattern, even if the current inference of the specific artificial neural network model ends, the next inference has the effect of predicting the onset of

For example, in the case of an artificial neural network model that recognizes an object in the image of a front camera mounted on an autonomous vehicle at a speed of 30 IPS (inference per second) per second, the same reasoning is continuously repeated at a specific cycle. Therefore, if the aforementioned loop-type artificial neural network data locality pattern is utilized, there is an effect of predicting repeated data access requests.

To elaborate on the identification information for example, it can be confirmed that token [3] and token [4] of the artificial neural network data locality pattern 1400 have the same memory address value but have different operation modes. Accordingly, the artificial neural network memory controller 120 may be configured to classify the third data access request and the fourth data access request into different tokens because the operation modes are different even if the memory address values are the same. However, identification information of examples of the present disclosure is not limited to an operation mode, and may be configured to predict an artificial neural network data locality pattern only with a memory address value.

The neural network memory controller 120 may be configured to generate a corresponding predicted data access request (ie, a prior data access request) based on the neural network data locality pattern 1400 .

The artificial neural network memory controller 120 may be configured to sequentially further generate a predicted data access request based on the artificial neural network data locality pattern 1400 .

According to the above configuration, when the processor 110 generates a specific data access request included in the artificial neural network data locality pattern 1400, the neural network memory controller 120 transmits at least one or more data access requests after the specific data access request. There is an effect that can be predicted sequentially. For example, when the processor 110 generates the token [1], the artificial neural network memory control unit 120 has the effect of predicting that a data access request corresponding to the token [2] will be generated next. For example, when the processor 110 generates the token [3], the artificial neural network memory control unit 120 has the effect of predicting that the data access request corresponding to the token [4] will be generated next. For example, when the processor 110 generates the token [1], the artificial neural network memory control unit 120 requests data access corresponding to the token [2-3-4-5-6-7-8-9] in the order It has the effect of predicting what will be created.

In other words, when the processor 110 processes a plurality of artificial neural network models, an unexpected data locality pattern may be interposed between tokens of the artificial neural network data locality pattern 1400 . For example, a new token [41] can invade after token [2]. However, even in this case, the artificial neural network memory controller 120 has the effect of predicting that the processor 110 will generate the token [3] after the token [2] and preparing it.

For example, when the processor 110 generates the token [9], the artificial neural network memory control unit 120 may predict that the processor 110 will generate the token [1].

6 is a schematic diagram illustrating a predicted data access request and an actual data access request generated by an artificial neural network memory controller based on an artificial neural network data locality pattern according to an example of the present disclosure.

The neural network memory control unit 120 according to an example of the present disclosure may be configured to generate a predicted data access request by predicting the actual data access request that the processor 110 will request next by using the neural network data locality pattern. .

Referring to FIG. 6 , the data access request token means a token corresponding to the data access request received by the artificial neural network memory controller 120 from the processor 110 . The predicted data access request token is a token corresponding to the data access request that the neural network memory controller 120 predicts in advance based on the neural network data locality pattern for the data access request to be requested next by the processor 110 . The actual data access request token is a data access request token actually generated by the processor 110 after the predicted data access request token is generated. However, the token of the present disclosure is merely an example for convenience of description, and the present disclosure is not limited to the token.

The data access request and the dictionary data access may correspond to the data access request token. In this case, the data access request matching the specific data access request token and the predicted data access request may be configured to have the same memory address. That is, the data access request and the dictionary data access may be configured to include the same memory address.

For example, when the data access request token is [3] and the predicted data access request token is [3], the memory address values of the respective tokens may be the same. That is, the data access request and the dictionary data access may be configured to include the same operation mode value. For example, when the data access request token is [3] and the predicted data access request token is [3], the operation mode values of each token may be the same.

Referring to FIG. 6 , when the processor 110 generates a data access request corresponding to the token [1], the artificial neural network memory controller 120 generates a predicted data access request corresponding to the token [2]. The processor 110 generates an actual data access request corresponding to the token [2] after generating the predicted data access request. And the artificial neural network memory control unit 120 is configured to determine whether the predicted data access request accurately predicted the actual data access request. The artificial neural network memory controller 120 may determine that the patterns match because the tokens corresponding to the predicted data access request and the actual data access request are the same.

Next, for example, when the processor 110 generates a data access request corresponding to the token [2], the artificial neural network memory control unit 120 generates a predicted data access request corresponding to the token [3]. The processor 110 generates an actual data access request corresponding to the token [3] after generating the predicted data access request. And the artificial neural network memory control unit 120 is configured to determine whether the predicted data access request accurately predicted the actual data access request. The artificial neural network memory controller 120 may determine that the patterns match because the tokens corresponding to the predicted data access request and the actual data access request are the same.

Again, for example, when the processor 110 generates a data access request corresponding to the token [9], the artificial neural network memory control unit 120 generates a predicted data access request corresponding to the token [1]. The processor 110 generates an actual data access request corresponding to the token [9] after generating the predicted data access request. And the artificial neural network memory control unit 120 is configured to check whether the predicted data access request accurately predicted the actual data access request to be generated later. The artificial neural network memory controller 120 may determine that the patterns match because the tokens corresponding to the predicted data access request and the actual data access request are the same.

After the artificial neural network memory controller 120 generates the predicted data access request, when the processor 110 generates an actual data access request, the artificial neural network memory controller 120 performs the predicted data access request and the actual data access request. may be configured to determine whether these requests are the same as each other.

According to the above configuration, the artificial neural network memory system 100 has an effect of detecting a change in the locality of the artificial neural network data of the artificial neural network model processed by the processor 110 . Accordingly, the artificial neural network memory controller 120 has an effect of analyzing the locality of the changed artificial neural network data even if the artificial neural network model is changed.

When the neural network memory controller 120 determines that the predicted data access request and the actual data access request are the same, the neural network memory controller 120 may be configured to maintain the neural network data locality pattern.

According to the above-described configuration, the artificial neural network memory system 100 detects that the artificial neural network model processed by the processor 110 is repeatedly used, and can prepare or provide data requested by the processor 110 more quickly. there is an effect

When the neural network memory controller 120 determines that the predicted data access request and the actual data access request are different, the neural network memory controller 120 updates the neural network data locality pattern or further generates a new artificial neural network data locality pattern can be configured to

According to the above configuration, the artificial neural network memory system 100 detects that the artificial neural network model processed by the processor 110 is changed, and has the effect of generating a predicted data access request corresponding to the changed artificial neural network model. .

In some examples, the neural network memory controller may be configured to generate a sequence of predicted data access requests.

For example, when the data access request token is [2], the predicted data access request generated by the artificial neural network memory controller may be a data access request corresponding to the token [3]. However, the present invention is not limited thereto, and for example, the predicted data access request generated by the artificial neural network memory controller may be a plurality of data access requests corresponding to the token [3-4]. However, the present invention is not limited thereto, and for example, the predicted data access request generated by the artificial neural network memory controller may be a plurality of data access requests corresponding to the token [3-4-5-6].

According to the above-described configuration, the neural network memory controller has an effect of generating a predicted data access request that predicts all the sequences of continuously repeated data access requests, based on the neural network data locality pattern.

According to the above configuration, the artificial neural network memory controller has an effect of generating a predicted data access request that predicts the order of at least some data access requests in advance, based on the artificial neural network data locality pattern.

7 is a flowchart schematically illustrating an operation of an artificial neural network memory controller according to an example of the present disclosure.

Referring to FIG. 7 , for artificial neural network operation processing, the processor 110 may be configured to generate a data access request corresponding to an artificial neural network model based on artificial neural network data locality.

The artificial neural network memory controller 120 sequentially records the data access requests generated by the processor 110 to generate an artificial neural network data locality pattern (S710).

The artificial neural network memory controller 120 compares the generated artificial neural network data locality pattern with the data access request generated by the processor 110 to generate a predicted data access request that predicts the actual data access request to be generated by the processor 110 can be configured to

The artificial neural network memory system 100 according to an example of the present disclosure sequentially records the data access request and at least one processor 110 configured to generate a data access request corresponding to the artificial neural network operation, and the artificial neural network of the artificial neural network operation A data locality pattern is generated (S720). At least one memory neural network memory system 100 is configured to generate a predicted data access request that predicts the actual data access request of the data access request generated by the at least one processor 110 based on the neural network data locality pattern It may be configured to include the artificial neural network memory control unit 120 of the.

That is, the at least one artificial neural network memory controller 120 generates a predicted data access request before generating the actual data access request (S730).

That is, the at least one processor 110 is configured to transmit a data access request to the at least one artificial neural network memory controller 120 , and the at least one artificial neural network memory controller 120 transmits data predicted in response to the data access request. It may be configured to output an access request.

The artificial neural network memory system 100 according to an example of the present disclosure receives the data access request generated by the at least one processor 110 and the at least one processor 110 configured to generate a data access request corresponding to the artificial neural network operation. It is configured to sequentially record and generate an artificial neural network data locality pattern of an artificial neural network operation, and based on the artificial neural network data locality pattern to generate a data access request that predicts the actual data access request generated by the at least one processor 110 It may be configured to include at least one configured artificial neural network memory controller 120 .

According to the above configuration, since the artificial neural network memory controller 120 can predict in advance the actual data access request to be generated by the artificial neural network model being processed by the processor 110 based on the artificial neural network data locality pattern, the processor 110 ) has the advantage of being able to prepare to provide the data in advance before requesting it.

The artificial neural network memory control unit 120 compares the generated predicted data access request with the actual data access request generated by the processor 110 after generating the predicted data access request to determine whether the artificial neural network data locality pattern matches. It can be (S740).

According to the above configuration, the artificial neural network memory controller 120 may prepare to provide data in advance by generating a predicted data access request before generating the actual data access request. Accordingly, the artificial neural network memory controller 120 has an effect of substantially eliminating or reducing a delay time that may occur when providing data to the processor 110 .

8 is a schematic block diagram illustrating an artificial neural network memory system according to another example of the present disclosure.

Referring to FIG. 8 , the artificial neural network memory system 200 may be configured to include a processor 210 , an artificial neural network memory controller 220 , and a memory 230 .

When the artificial neural network memory system 200 according to another example of the present disclosure is compared with the artificial neural network memory system 100 according to an example of the present disclosure, the artificial neural network memory system 200 further includes a memory 230 . Since they are substantially the same except for, redundant descriptions may be omitted for convenience of description only.

The artificial neural network memory system 200 according to another example of the present disclosure includes a memory 230 configured to communicate with the artificial neural network memory controller 220 , and the memory 230 is a memory output from the artificial neural network memory controller 220 . It may be configured to act in response to an access request.

The processor 210 may be configured to communicate with the artificial neural network memory controller 220 . The processor 210 may be configured to generate a data access request to be transmitted to the artificial neural network memory controller 220 . The data access request may be generated based on the neural network data locality of the neural network model being processed. The processor 210 is configured to receive data corresponding to the data access request from the artificial neural network memory controller 220 .

The artificial neural network memory controller 220 may be configured to receive the data access request generated by the processor 210 . The artificial neural network memory controller 220 may be configured to generate an artificial neural network data locality pattern by analyzing the artificial neural network data locality of the artificial neural network model being processed by the processor 210 .

The artificial neural network memory controller 220 may be configured to control the memory 230 by generating a memory access request. The artificial neural network memory controller 220 may be configured to generate a memory access request corresponding to the data access request. That is, the artificial neural network memory controller 220 may be configured to generate a memory access request corresponding to the data access request generated by the processor 210 . For example, when the artificial neural network memory controller 220 does not generate the artificial neural network data locality pattern, the artificial neural network memory controller 220 generates a memory access request based on the data access request generated by the processor 210. can be configured. In this case, the memory access request may be configured to include a memory address value and an operation mode value among identification information included in the data access request.

The artificial neural network memory controller 220 may be configured to generate a memory access request corresponding to the predicted data access request. That is, the neural network memory controller 220 may be configured to generate a memory access request based on the predicted data access request generated based on the neural network data locality pattern. For example, when the neural network memory controller 220 generates the neural network data locality pattern, the neural network memory controller 220 may be configured to generate a memory access request based on the predicted data access request.

According to the above configuration, the artificial neural network memory controller 220 can send and receive data to and from the memory 220 through a memory access request, and when the corresponding memory access request is generated based on the predicted data access request, the artificial neural network The memory system 200 has the effect of more rapidly providing data to the processor 210 .

The neural network memory controller 220 may be configured to generate a memory access request based on one of a data access request generated by the processor 210 and a predicted data access request generated by the neural network memory controller 220 . That is, the memory access request generated by the artificial neural network memory controller 220 may be selectively generated based on the data access request or the predicted data access request.

The artificial neural network memory control unit 220 may be configured to generate a memory access request including at least some of the identification information included in the data access request and the predicted data access request. For example, the data access request generated by the processor 210 may include a memory address value and an operation mode value. In this case, the memory access request generated by the artificial neural network memory controller 220 may be configured to include a memory address value and an operation mode value of the corresponding data access request.

That is, each of the data access request, the predicted data access request, and the memory access request may be configured to include a corresponding memory address value and an operation mode value, respectively. The operation mode may be configured to include a read mode and a write mode. For example, the memory access request generated by the artificial neural network memory controller 220 may be configured in the form of data having the same structure as the data access request or the predicted data access request. Therefore, the memory 230 can perform the memory access request task according to the instruction of the artificial neural network memory controller 220 without distinguishing between the data access request and the predicted data access request.

According to the above configuration, the memory 230 has an effect that can operate regardless of whether the memory access request generated by the artificial neural network memory controller 220 is based on a data access request or a predicted data access request. there is. Therefore, even if the artificial neural network memory control unit 220 operates based on the artificial neural network data locality, there is an effect that it can operate compatible with various types of memories.

The artificial neural network memory controller 220 transmits a memory access request to the memory 230 , and the memory 230 is configured to perform a memory operation corresponding to the memory access request.

Memory according to examples of the present disclosure may be implemented in various forms. The memory may be implemented as a volatile memory and a non-volatile memory.

The volatile memory may include a dynamic RAM (DRAM) and a static RAM (SRAM). Non-volatile memory includes Programmable ROM (PROM), Erasable PROM (EPROM), Electrically EPROM (EEPROM), Flash Memory, ferroelectric RAM (FRAM), Magnetic RAM (MRAM), and and a phase change memory device (RAM), and the like. However, the present disclosure is not limited thereto.

The memory 230 may be configured to store at least one of inference data, weight data, and feature map data of the artificial neural network model being processed by the processor 210 . The inference data may be an input signal of an artificial neural network model.

The memory 230 may be configured to receive a memory access request from the artificial neural network memory controller 220 . The memory 230 may be configured to perform a memory operation corresponding to the received memory access request. The operation mode for controlling the memory operation may include a read mode or a write mode.

For example, when the operation mode of the received memory access request is the write mode, the memory 230 may store data received from the artificial neural network memory controller 220 in a corresponding memory address value.

For example, when the operation mode of the received memory access request is the read mode, the memory 230 may transmit data stored in a corresponding memory address value to the artificial neural network memory controller 220 . The artificial neural network memory controller 220 may be configured to transmit the received data back to the processor 210 .

The memory 230 may have latency. The delay time of the memory 230 may mean a delay time when the artificial neural network memory controller 220 processes a memory access request. That is, when the memory 230 receives a memory access request from the neural network memory controller 220 , the data actually requested is output from the memory 230 after a delay time of a specific clock cycle.

In order for the memory 230 to process the memory access request, the memory 230 may access a memory address value included in the memory access request. Therefore, time is required to access the memory address value, and this time can be defined as the memory delay time. For example, the CAS latency of DDR4 SDRAM memory is about 10ns. If data is not supplied to the processor 210 while the delay time occurs, the processor 210 may be in an IDLE state and may not be able to perform an actual operation.

More specifically, in the case of DRAM, which is a type of memory 230 , several clocks for activating word lines and bit lines, several clocks for activating column lines, and data 230) It takes several clocks to pass through the external transmission path, and in the case of NAND Flash, since the unit activated at one time is large, several clocks may be additionally required to search for data of a required address among them.

The memory 230 may have a bandwidth. A data transfer rate of the memory 230 may be defined as a memory bandwidth. For example, the bandwidth of DDR4 SDRAM memory is about 4GBytes/sec. The higher the memory bandwidth, the faster the memory 230 can transmit data to the processor 210 .

That is, the processing speed of the artificial neural network memory system 200 is higher than the processing performance of the processor 210 , the delay time generated when the processor 210 supplies data to be processed and the bandwidth performance of the memory 230 are relatively higher. affects

In other words, although the bandwidth of the memory is gradually increased, the improvement rate of the delay time of the memory is relatively slow compared to the improvement rate of the bandwidth. In particular, whenever a memory access request is generated, since a delay time of the memory 230 is generated, frequent memory access requests may be an important cause of a decrease in the processing speed of the artificial neural network.

That is, even if the processing speed of the processor 210 is fast, if there is a delay in fetching data required for the operation, the processor 210 may be in a standby state in which no operation is performed. In this case, the operation of the processor 210 is Processing speed may be reduced.

Accordingly, the artificial neural network memory system according to examples of the present disclosure may be configured to improve the bandwidth and/or delay time of the memory 230 .

9 is a schematic diagram illustrating an operation of a memory system according to a comparative example of the present disclosure.

Referring to FIG. 9 , a processor may generate a data access request, and the conventional memory system may transmit a memory access request corresponding to the data access request to the memory. In this case, since the memory has a delay time, the processor can receive the requested data from the memory after waiting for the delay time.

For example, the conventional memory system receives the data access request [1] generated by the processor, and transmits the memory access request [1'] corresponding to the data access request [1] to the memory. The memory can deliver data[1''] to the memory system after the delay time. Accordingly, the processor may delay the processing time by the delay time of the memory for each data access request. Accordingly, the time of the artificial neural network reasoning operation may be as slow as the memory delay time. In particular, as the processor generates more data access requests, the artificial neural network reasoning operation time of the conventional memory system may be further delayed.

10 is a schematic diagram illustrating a memory system according to another example of the present disclosure.

Referring to FIG. 10 , the processor 210 generates a data access request [1], and the neural network memory control unit 220 requests a memory access corresponding to the predicted data access request generated based on the artificial neural network data locality pattern. may be transferred to the memory 230 . At this time, even if the memory 230 has a delay time, since the processor 210 generates a memory access request corresponding to the predicted data access request, when the processor 210 generates the actual data access request, the artificial neural network memory controller The 220 may directly provide the data requested by the processor 210 to the processor 210 .

For example, the artificial neural network memory controller 220 receives the data access request [1] generated by the processor 210, generates a predicted data access request [2], and responds to the predicted data access request [2] It transfers the memory access request [2'] to the memory 230 . The memory 230 may transmit the data [2''] to the artificial neural network memory controller 220 after the delay time. However, the data [2''] provided by the memory 230 is data corresponding to the memory access request [2'] based on the predicted data access request [2]. Therefore, when the processor 210 generates an actual data access request [2], the artificial neural network memory controller 220 may immediately provide the data [2''] to the processor 210 .

If the time between the memory access request based on the predicted data access request and the actual data access request is equal to or greater than the delay time of the memory 230 , the artificial neural network memory controller 220 receives the actual data access request from the processor 210 . Data may be provided to the processor 210 as soon as it is done. In this case, the artificial neural network memory controller 220 has the effect of substantially eliminating the delay time of the memory 230 .

In other words, when the memory access request based on the predicted data access request is transmitted to the memory 230, the delay time of the memory 230 may be less than or equal to the time from the predicted data access request generation to the actual data access request generation. . In this case, the artificial neural network memory control unit 220 has the effect of being able to provide data immediately without delay as soon as the processor 210 generates an actual data access request.

If the time between the memory access request and the actual data access request based on the predicted data access request is less than the delay time of the memory 230 , the delay time of the memory 230 by the time between the memory access request and the actual data access request has the effect of substantially reducing

According to the above configuration, the artificial neural network memory controller 220 has an effect of substantially eliminating or reducing the delay time of data to be provided to the processor 210 .

In some examples, the artificial neural network memory control unit of the artificial neural network memory system may be configured to measure the delay time of the memory or to receive the delay time value of the memory from the memory.

According to the above-described configuration, the artificial neural network memory controller may be configured to determine the generation time of the memory access request based on the predicted data access request based on the delay time of the memory. Therefore, there is an effect that the artificial neural network memory controller can generate a memory access request based on the predicted data access request that substantially minimizes the delay time of the memory.

In some examples, the memory of the artificial neural network memory system may be a memory configured to include a refresh function capable of updating a voltage of a memory cell (ie, a memory cell array). The artificial neural network memory controller may be configured to selectively control the refresh of a memory address region of a memory corresponding to a memory access request corresponding to the predicted data access request. For example, the memory may be a SAM or DRAM with a refresh function.

In the DRAM, if the voltage of the memory cell is not refreshed, the memory cell is gradually discharged, and stored data may be lost. Therefore, the voltage of the memory cell must be refreshed every specific period. If the refresh timing overlaps with the memory access request of the artificial neural network memory controller, the artificial neural network memory system may be configured to advance or delay the timing of refreshing the voltage of the memory cell.

The artificial neural network memory system may predict or calculate the generation timing of the memory access request based on the artificial neural network data locality pattern. Accordingly, the artificial neural network memory system may be configured to limit the voltage refresh of the memory cell during a memory access request operation.

In other words, since the reasoning operation of the artificial neural network operation operates with the concept of accuracy, even if the voltage refresh of the memory cell is delayed and some loss occurs in the stored data, the degradation of the reasoning accuracy may be substantially negligible.

According to the above-described configuration, the artificial neural network memory system has an effect of being able to receive data according to a memory access request from the memory by adjusting the voltage refresh period of the memory cell. Therefore, the artificial neural network memory system has an effect of improving the artificial neural network operation speed decrease due to the voltage refresh of the memory cell without substantially lowering the inference accuracy.

In some examples, the memory of the artificial neural network memory system may be configured to further include a precharge function capable of charging the global bit line of the memory to a specific voltage. In this case, the artificial neural network memory controller may be configured to selectively provide a precharge to a memory address area of a memory corresponding to a memory access request corresponding to the predicted data access request.

In some examples, the neural network memory controller may be configured to precharge or delay a bit line of a memory to perform a memory operation corresponding to a data access request predicted based on the neural network data locality pattern.

In general, a memory receives a memory access request and performs a read operation or a write operation, and a precharge operation is performed. When one memory operation is completed, signals remain on the bit line and each data input/output line on which the data read/write operation has been performed. These lines must be precharged to a preset level to smoothly perform the next memory operation there is. However, since the time required for the precharge is quite long, when the timing of generating the memory access request and the timing of the precharge overlap, the memory operation may be delayed by the precharge time. Therefore, the processing time of the data access request requested by the processor may be delayed.

The artificial neural network memory controller may predict that a memory operation is performed on a bit line of a particular memory in a particular order based on the artificial neural network data locality pattern. Accordingly, the artificial neural network memory control unit may advance or delay the precharge timing so that the precharge timing does not overlap with when a memory operation is performed on a specific bit line.

In other words, since the reasoning operation of the artificial neural network model operates based on the concept of accuracy, even if some loss occurs in the stored data due to the delay in precharging, the degradation of the inference accuracy may be substantially negligible.

To elaborate, the artificial neural network is a mathematical model modeled by mimicking the brain neural network of biology. Human nerve cells called neurons exchange information through the junctions of nerve cells called synapses. pay This structure has the advantage of being very strong against small errors because even if some nerve cells transmit erroneous information, the overall information is not greatly affected. That is, due to the above-described characteristics, even if the precharge and refresh functions of the memory for storing the data of the artificial neural network model are selectively restricted, the accuracy of the artificial neural network model may not be substantially problematic, and the memory delay due to the precharge or refresh It has the effect of saving time.

According to the above-described configuration, the artificial neural network memory system has an effect of improving the artificial neural network computation speed decrease due to precharge while not substantially lowering inference accuracy.

In some examples, the artificial neural network memory controller may be configured to respectively control a refresh function and a precharge function of the memory based on the neural network data locality pattern.

11 is a schematic block diagram illustrating an artificial neural network memory system according to another example of the present disclosure.

Referring to FIG. 11 , the artificial neural network memory system 300 may be configured to include a processor 310 , an artificial neural network memory controller 320 including a cache memory 322 , and a memory 330 .

The memory 330 that may be included in various examples of the present disclosure may be a memory specialized for artificial neural network operation, and may be referred to as SEQUENTIAL ACCESS MEMORY (SAM). However, the present disclosure is not limited thereto, and the memory of various examples of the present disclosure may refer to a memory specialized for an artificial neural network that can be controlled based on artificial neural network data locality.

When the artificial neural network memory system 300 according to another example of the present disclosure is compared with the artificial neural network memory system 200 according to another example of the present disclosure, the artificial neural network memory system 300 further includes a cache memory 322 . Since they are substantially the same except for the following, duplicate description may be omitted for convenience of description only.

The artificial neural network memory system 300 according to another example of the present disclosure is artificial including a cache memory 322 configured to store data transmitted by the memory 330 in response to a memory access request based on the predicted data access request. It may be configured to include a neural network memory controller 320 .

According to the above configuration, the artificial neural network memory controller 320 may read data in response to the memory access request based on the predicted data access request from the memory 330 and store it in the cache memory 322 . Therefore, when the processor 310 generates an actual data access request, the artificial neural network memory control unit 320 has an effect of directly providing the data stored in the cache memory 322 to the processor 310 .

The delay time of the cache memory 322 is relatively much shorter than the delay time of the memory 330 . The bandwidth of the cache memory 322 is relatively higher than the bandwidth of the memory 330 .

The artificial neural network model processing performance of the artificial neural network memory system 300 including the cache memory 322 according to another example of the present disclosure may be relatively better than the artificial neural network memory system 200 according to another example of the present disclosure. there is an effect

Again, an artificial neural network memory system 300 according to another example of the present disclosure will be described with reference to the artificial neural network model 1300 of FIG. 3 .

The artificial neural network model 1300 may be compiled by a specific compiler and calculated by the processor 310 . The compiler may be configured to provide the neural network data locality pattern to the neural network memory controller 320 .

In order to infer the neural network model 1300, the processor 310 is configured to generate data access requests in an order based on the neural network data locality. Accordingly, the neural network memory controller 320 may generate the neural network data locality pattern 1400 by monitoring data access requests. Alternatively, the artificial neural network memory controller 320 may store the previously generated artificial neural network data locality pattern 1400 .

Hereinafter, a case in which the artificial neural network data locality pattern 1400 is not generated will be described.

First, the processor 310 may generate a data access request of the token [1] corresponding to reading the node value of the input layer 1310 . Accordingly, the artificial neural network memory control unit 320 may generate a memory access request of the token [1] and transmit the node value of the input layer 1310 received from the memory 330 to the processor 310 .

Subsequently, the processor 310 may generate a data access request of the token [2] corresponding to the reading of the weight value of the first connection network 1320 . Accordingly, the artificial neural network memory controller 320 may generate a memory access request of the token [2] and transmit the weight value of the first connection network 1320 received from the memory 330 to the processor 310 .

Subsequently, the processor 310 may receive the node value of the input layer 1310 and the weight value of the first connection network 1320 to calculate the node value of the first hidden layer 1330 . That is, the processor 310 may generate a data access request of the token [3] corresponding to the writing of the node value of the first hidden layer 1330 . Accordingly, the artificial neural network memory control unit 320 may generate a memory access request of the token [3] and store the node value of the first hidden layer 1330 in the memory 330 .

Subsequently, the processor 310 may generate a data access request of the token [4] corresponding to the reading of the node value of the first hidden layer 1330 . Accordingly, the artificial neural network memory control unit 320 may generate a memory access request of the token [4] and transmit the node value of the first hidden layer 1330 received from the memory 330 to the processor 310 .

Subsequently, the processor 310 may generate a data access request of the token [5] corresponding to the reading of the weight value of the second connection network 1340 . Accordingly, the artificial neural network memory control unit 320 may generate a memory access request of the token [5] and transmit the weight value of the second connection network 1340 received from the memory 330 to the processor 310 .

Subsequently, the processor 310 may receive the node value of the first hidden layer 1330 and the weight value of the second connection network 1340 to calculate the node value of the second hidden layer 1350 . That is, the processor 310 may generate a data access request of the token [6] corresponding to the writing of the node value of the second hidden layer 1350. Accordingly, the artificial neural network memory controller 320 may generate a memory access request of the token [6] and store the node value of the second hidden layer 1350 in the memory 330 .

Subsequently, the processor 310 may generate a data access request of the token [7] corresponding to the reading of the node value of the second hidden layer 1350 . Accordingly, the artificial neural network memory control unit 320 may generate a memory access request of the token [7] and transmit the node value of the second hidden layer 1350 received from the memory 330 to the processor 310 .

Subsequently, the processor 310 may generate a data access request of the token [8] corresponding to the reading of the weight value of the third connection network 1360 . Accordingly, the artificial neural network memory control unit 320 may generate a memory access request of the token [8] and transmit the weight value of the third connection network 1360 received from the memory 330 to the processor 310 .

Subsequently, the processor 310 may calculate the node value of the output layer 1370 by receiving the node value of the second hidden layer 1350 and the weight value of the third connection network 1360 . That is, the processor 310 may generate a data access request of the token [9] corresponding to the writing of the node value of the output layer 1370 . Accordingly, the artificial neural network memory control unit 320 may generate a memory access request of the token [9] and store the node value of the output layer 1370 in the memory 330 .

Accordingly, the artificial neural network memory system 300 may store the inference result of the artificial neural network model 1300 in the output layer 1370 .

The above-described example is a case in which the artificial neural network data locality pattern 1400 is not generated in the artificial neural network memory controller 320 . Thus, the above example cannot generate a predicted data access request. Therefore, since the artificial neural network memory controller 320 fails to provide data in advance, a delay time of the memory 330 may occur for each memory access request.

However, since the artificial neural network memory control unit 320 records data access requests, when the processor 310 again generates a data access request of the token [1] corresponding to reading the node value of the input layer 1310, the artificial neural network A data locality pattern 1400 may be generated.

Hereinafter, a case in which the artificial neural network data locality pattern 1400 is generated will be described with reference to FIG. 4 again.

The following example may be a case in which the artificial neural network data locality pattern 1400 is generated and the processor 310 is repeatedly inferring the artificial neural network model 1300 . However, the present invention is not limited thereto.

The processor 310 may generate the artificial neural network data locality pattern 1400 by detecting the data access request of the repeated token [1]. To elaborate, since the artificial neural network memory controller 320 sequentially stores the token [1] to the token [9], when the artificial neural network memory controller 320 detects the token [1] again, the neural network data locality is can decide

However, as described above, the artificial neural network memory control unit according to the examples of the present disclosure is not limited to the token, and the token is only for convenience of description, and is an example of the present disclosure by the identification information included in the data access request and the memory access request. can be implemented.

For example, when the processor 310 generates a data access request of the token [9], the artificial neural network memory control unit 320 generates a predicted data access request of the token [1]. Accordingly, the artificial neural network memory controller 320 may generate a memory access request of the token [1] and store the node value of the input layer 1310 in the cache memory 322 in advance.

That is, if the data access request of the token [9] is the last step of the artificial neural network model 1300, the artificial neural network memory control unit 320 generates the data access request of the token [1], which is the starting stage of the artificial neural network model 1300. can be predicted to be

Subsequently, when the processor 310 generates a data access request of the token [1], the artificial neural network memory control unit 320 determines whether the predicted data access request of the token [1] and the data access request of the token [1] are the same do. When it is determined that they are the same, the node value of the input layer 1310 stored in the cache memory 322 may be directly provided to the processor 310 .

At this time, the artificial neural network memory control unit 320 generates a predicted data access request of the token [2].

Accordingly, the artificial neural network memory control unit 320 may generate a memory access request of the token [2] and store the weight value of the first connection network 1320 in the cache memory 322 in advance.

Subsequently, when the processor 310 generates a data access request of the token [2], the artificial neural network memory control unit 320 determines whether the predicted data access request of the token [2] and the data access request of the token [2] are the same do. When it is determined that they are the same, the node value of the first connection network 1320 stored in the cache memory 322 may be directly provided to the processor 310 .

At this time, the artificial neural network memory control unit 320 generates the predicted data access request of the token [3].

Subsequently, the processor 310 may receive the node value of the input layer 1310 and the weight value of the first connection network 1320 to calculate the node value of the first hidden layer 1330 . When the processor 310 generates a data access request of the token [3], the artificial neural network memory control unit 320 determines whether the predicted data access request of the token [3] and the data access request of the token [3] are the same. When it is determined that they are the same, the calculated node value of the first hidden layer 1330 may be stored in the memory 330 and/or the cache memory 322 .

To elaborate on the cache memory 322, the same data without the cache memory 322 is stored in the memory 330 as a memory access request of the token [3], and again in the memory 330 with the memory access request of the token [4]. ), the delay time of the memory 330 may be doubled.

In this case, the neural network memory control unit 320 determines the node value of the layer calculated on the basis that the memory address values of consecutive tokens are the same, the operation mode of the preceding token is the write mode, and the operation mode of the next token is the read mode. and to determine to use the corresponding node value as an input value for the next layer.

That is, when data of the token [3] is stored in the cache memory 322 , a data access request corresponding to the token [3] and the token [4] may be processed in the cache memory 322 . Therefore, the artificial neural network memory control unit 320 may be configured not to generate memory access requests corresponding to the data access request of the token [3] and the data access request of the token [4]. According to the above configuration, there is an effect that the delay time of the memory 330 due to the memory 330 can be eliminated by the memory access request of the token [3] and the memory access request of the token [4]. In particular, the cache memory 322 operation policy may be executed based on the artificial neural network data locality pattern 1400 .

At this time, the artificial neural network memory control unit 320 generates the predicted data access request of the token [4].

Subsequently, when the processor 310 generates a data access request of the token [4], the artificial neural network memory control unit 320 determines whether the predicted data access request of the token [4] and the data access request of the token [4] are the same do. When it is determined that they are the same, the node value of the first hidden layer 1330 stored in the cache memory 322 may be directly provided to the processor 310 .

At this time, the artificial neural network memory control unit 320 generates the predicted data access request of the token [5].

Accordingly, the artificial neural network memory controller 320 may generate a memory access request of the token [5] and store the weight value of the second connection network 1340 in the cache memory 322 in advance.

Subsequently, when the processor 310 generates a data access request of the token [5], the artificial neural network memory control unit 320 determines whether the predicted data access request of the token [5] and the data access request of the token [5] are the same do. When it is determined that they are the same, the weight value of the second connection network 1340 stored in the cache memory 322 may be directly provided to the processor 310 .

At this time, the artificial neural network memory control unit 320 generates the predicted data access request of the token [6].

Subsequently, the processor 310 may receive the node value of the first hidden layer 1330 and the weight value of the second connection network 1340 to calculate the node value of the second hidden layer 1350 . When the processor 310 generates a data access request of the token [6], the artificial neural network memory control unit 320 determines whether the predicted data access request of the token [6] and the data access request of the token [6] are the same. When it is determined that they are the same, the calculated node value of the second hidden layer 1350 may be stored in the memory 330 and/or the cache memory 322 .

At this time, the artificial neural network memory control unit 320 generates the predicted data access request of the token [7].

Subsequently, when the processor 310 generates a data access request of the token [7], the artificial neural network memory control unit 320 determines whether the predicted data access request of the token [7] and the data access request of the token [7] are the same do. When it is determined that they are the same, the node value of the second hidden layer 1350 stored in the cache memory 322 may be directly provided to the processor 310 .

At this time, the artificial neural network memory control unit 320 generates the predicted data access request of the token [8].

Accordingly, the artificial neural network memory controller 320 may generate a memory access request of the token [8] and store the weight value of the third connection network 1360 in the cache memory 322 in advance.

Subsequently, when the processor 310 generates a data access request of the token [8], the artificial neural network memory control unit 320 determines whether the predicted data access request of the token [8] and the data access request of the token [8] are the same do. When it is determined that they are the same, the weight value of the third connection network 1360 stored in the cache memory 322 may be directly provided to the processor 310 .

At this time, the artificial neural network memory control unit 320 generates the predicted data access request of the token [9].

Subsequently, the processor 310 may calculate the node value of the output layer 1370 by receiving the node value of the second hidden layer 1350 and the weight value of the third connection network 1360 . When the processor 310 generates a data access request of the token [9], the artificial neural network memory control unit 320 determines whether the predicted data access request of the token [9] and the data access request of the token [9] are the same. When it is determined that they are the same, the calculated node value of the output layer 1370 may be stored in the memory 330 and/or the cache memory 322 .

Accordingly, the artificial neural network memory system 300 may store the inference result of the artificial neural network model 1300 in the output layer 1370 .

The artificial neural network memory system 300 has the effect of preparing to immediately start the next inference even after the inference of the artificial neural network model 1300 is finished by the artificial neural network data locality pattern 1400 .

That is, the neural network memory system 300 according to another example of the present disclosure generates a predicted data access request based on the artificial neural network data locality, and determines whether the predicted data access request and the actual data access request are the same, If equal, it may be configured to further generate the next sequence of predicted data access requests. According to the above configuration, the artificial neural network memory control unit 320 has an effect of removing or reducing the delay time of the memory 320 when processing each data access request.

In some examples, the artificial neural network memory controller may be configured to generate at least one predicted data access request to operate to minimize the free space of the cache memory.

That is, the artificial neural network memory controller compares the free memory space of the cache memory with the size of the data value to be stored. can be configured to be minimized.

That is, the artificial neural network memory control unit may be configured to generate a plurality of predicted data access requests according to the capacity of the cache memory.

That is, the artificial neural network memory control unit may be configured such that the remaining capacity of the cache memory is minimized by sequentially generating at least one or more memory access requests based on the remaining capacity of the cache memory.

An example will be described with reference to FIGS. 2 to 6 . When the processor generates a data access request of the token [1], the artificial neural network memory control unit generates a predicted data access request of the token [2] and stores the weight value of the first connection network 1320 in the cache memory in advance. . Subsequently, the artificial neural network memory controller may store the node value operation processing result of the first hidden layer 1330 corresponding to the token [3] and the token [4] and allocate a read space to the cache memory in advance. Subsequently, the artificial neural network memory controller may store the weight value of the second connection network 1340 corresponding to the token [5] in the cache memory in advance. Here, the artificial neural network memory control unit may be configured to sequentially further generate a predicted data access request based on the artificial neural network data locality pattern when there is room in the cache memory. That is, when the cache memory has a sufficient capacity, the artificial neural network memory control unit may be configured to pre-store weight values in the cache memory based on the artificial neural network data locality pattern or to secure a region to store the artificial neural network calculation results in advance. .

If the capacity of the cache memory is sufficient, it may be configured to store the weight values of all the connected networks of the artificial neural network model 1300 in the cache memory. In particular, in the case of an artificial neural network model that has been trained, the weight values are fixed. Therefore, when the weight values reside in the cache memory, there is an effect that memory delay time due to a memory access request for reading the weight values can be eliminated.

According to the above configuration, there is an effect of optimizing the operation efficiency of the cache memory and improving the processing speed of the artificial neural network memory system 300 by storing necessary data in the cache memory based on the locality of the artificial neural network data.

According to the above configuration, since the cache memory sequentially generates predicted data access requests in consideration of both the neural network data locality pattern and the capacity of the cache memory, there is an effect that the processing speed of the artificial neural network memory system can be improved.

According to the above configuration, when the processor generates a specific data access request included in the artificial neural network data locality pattern 1400, the neural network memory controller can sequentially predict at least one or more data access requests after the specific data access request. there is For example, if the processor generates a data access request for token [1], the artificial neural network memory controller generates data access requests corresponding to token [2-3-4-5-6-7-8-9] It has the ability to predict what will happen.

According to the above-described configuration, the artificial neural network memory control unit 320 may allow specific weight values to reside in the cache memory for a specific period. For example, when the processor makes inference using an artificial neural network model at a speed of 30 times per second, the weight value of a specific layer can be resident in the cache memory. In this case, the artificial neural network memory control unit has the effect of reusing the weight value stored in the cache memory for each inference. Accordingly, there is an effect of selectively deleting the corresponding memory access request. Therefore, there is an effect that can remove the delay time due to the memory access request.

In some examples, the cache memory may be configured as a plurality of layered cache memories. For example, it may include a cache memory configured to store a weight value or a cache memory configured to store a feature map, and the like.

In some examples, when the artificial neural network data locality pattern 1400 is generated, the artificial neural network memory controller may be configured to predict a weight value and a node value based on identification information included in the data access request. Accordingly, the artificial neural network memory controller may be configured to identify a data access request corresponding to a weight value. Specifically, assuming that the learning is completed and the weight value of the connection network is fixed, the weight value in the artificial neural network data locality pattern 1400 may be configured to operate only in the read mode. Therefore, the artificial neural network memory controller can determine the token [2], token [5], and token [8] as weight values. To elaborate, it can be determined that the token [1] is an input node value because it is the starting stage of inference. To elaborate, it can be determined that the token [9] is the output node value because it is the last stage of inference. In more detail, since tokens [3] [4] have the order of the write mode and the read mode of the same memory address value, it can be determined as a node value of the hidden layer. However, this may vary depending on the locality of the artificial neural network data of the artificial neural network model.

The artificial neural network memory controller may be configured to analyze the artificial neural network data locality pattern to determine whether each data access request is a weight value of the neural network model, a kernel window value, a node value, an activation map value, or the like.

In some examples, the neural network memory system is configured to store a processor configured to generate a data access request corresponding to an artificial neural network operation, a compiler generated neural network data locality pattern, and the processor generated based on the artificial neural network data locality pattern and a neural network memory controller, configured to generate a predicted data access request that predicts the actual data access request of the one data access request, and a memory configured to communicate with the neural network memory controller. The memory may be configured to operate in response to a memory access request output from the artificial neural network memory controller.

According to the above configuration, the artificial neural network memory controller may be configured to receive the artificial neural network data locality pattern generated by the compiler. In this case, the artificial neural network memory controller has the effect of preparing in advance the data access requests of the artificial neural network model being processed by the processor in the cache memory based on the artificial neural network data locality pattern generated by the compiler. In particular, the artificial neural network data locality pattern generated by the compiler has the effect of being more accurate than the artificial neural network data locality pattern generated by monitoring the artificial neural network data locality.

In other words, the artificial neural network memory controller may be configured to store the artificial neural network data locality pattern generated by the compiler and the artificial neural network data locality pattern generated by monitoring data access requests by itself.

12 is a schematic diagram illustrating exemplary identification information of a data access request.

The data access request generated by the processor according to examples of the present disclosure may be configured to further include at least one piece of additional identification information. The additional identifying information may also be referred to as a side band signal or information.

The data access request generated by the processor may be an interface signal having a specific structure. That is, the data access request may be an interface signal for communication between the processor and the artificial neural network memory controller. The data access request may be configured to further include additional bits in the interface signal to additionally provide identification information necessary for artificial artificial intelligence network operation. However, the present disclosure is not limited thereto, and may be configured to provide additional identification information in various ways.

In some examples, the data access request of the artificial neural network memory system may be configured to further include identification information capable of identifying whether the artificial neural network operation is performed. However, examples of the present disclosure are not limited thereto.

For example, the artificial neural network memory system may be configured to add a 1-bit identification code to the data access request to identify whether the data access request received by the artificial neural network memory controller is a data access request related to an artificial neural network operation. However, the number of bits of the identification code according to examples of the present disclosure is not limited, and may be adjusted according to the number of cases to be identified.

For example, when the identification code is [0], the artificial neural network memory control unit may be configured to determine that the corresponding data access request is related to the artificial neural network operation.

For example, when the identification code is [1], the artificial neural network memory control unit may be configured to determine that the corresponding data access request is not related to the artificial neural network operation.

In this case, the artificial neural network memory controller may be configured to generate an artificial neural network data locality pattern by recording only a data access request related to an artificial neural network operation based on identification information included in the data access request. According to the above configuration, the artificial neural network memory controller may not record data access requests that are not related to artificial neural network operation. Therefore, it is possible to improve the accuracy of the artificial neural network data locality pattern generated by recording data access requests. However, examples of the present disclosure are not limited thereto.

In some examples, the data access request of the artificial neural network memory system may be configured to further include identification information capable of identifying whether the artificial neural network operation is an operation for learning or an operation for inference. However, examples of the present disclosure are not limited thereto.

For example, the neural network memory system may be configured to add a 1-bit identification code to the data access request so that the data access request received by the artificial neural network memory controller identifies whether the operation type of the neural network model is learning or inference. there is. However, the number of bits of the identification code according to examples of the present disclosure is not limited, and may be adjusted according to the number of cases to be identified.

For example, when the identification code is [0], the artificial neural network memory controller may be configured to determine the corresponding data access request as a learning operation.

For example, when the identification code is [1], the artificial neural network memory controller may be configured to determine the reasoning operation of the corresponding data access request.

In this case, the artificial neural network memory controller may be configured to separately record the data access request of the learning operation and the data access request of the inference operation to generate the artificial neural network data locality pattern. For example, in the learning mode, weight values of each layer and/or kernel window of the artificial neural network model may be updated, and an evaluation step of determining the inference accuracy of the learned artificial neural network model may be further included. Therefore, even if the structure of the artificial neural network model is the same, the locality of the artificial neural network data processed by the processor may be different from each other during a learning operation or an inference operation.

According to the above-described configuration, the artificial neural network memory controller may be configured to separately generate an artificial neural network data locality pattern of a learning mode and an artificial neural network data locality pattern of an inference mode of a specific artificial neural network model. Accordingly, there is an effect of improving the accuracy of the artificial neural network data locality pattern generated by the neural network memory controller recording data access requests. However, examples of the present disclosure are not limited thereto.

In some examples, the data access request of the artificial neural network memory system may be configured in an operation mode including identification information capable of identifying a memory read operation and a memory write operation. However, the present invention is not limited thereto, and the data access request of the artificial neural network memory system may be configured in an operation mode that further includes identification information for identifying an overwrite operation and/or a protection operation. However, examples of the present disclosure are not limited thereto.

For example, it may be configured to include a read operation and a write operation by adding a 1-bit identification code to the data access request of the artificial neural network memory system. Alternatively, it may be configured to identify a read operation, a write operation, an overwrite operation, and a protection operation by adding a 2-bit identification code to the data access request of the artificial neural network memory system. However, the number of bits of the identification code according to examples of the present disclosure is not limited, and may be adjusted according to the number of cases to be identified.

In other words, for the operation of the artificial neural network memory system, the data access request must include at least a memory address value and identification information for identifying read and write operations. The artificial neural network memory controller may be configured to receive a data access request and generate a corresponding memory access request to perform a memory operation.

For example, when the identification code is [000], the artificial neural network memory controller may be configured to determine the corresponding data access request as a read operation.

For example, when the identification code is [001], the artificial neural network memory control unit may be configured to determine the corresponding data access request as a write operation.

For example, when the identification code is [010], the artificial neural network memory controller may be configured to determine the corresponding data access request as an overwrite operation.

For example, when the identification code is [011], the artificial neural network memory control unit may be configured to determine the corresponding data access request as a protection operation.

For example, when the identification code is [100], the artificial neural network memory controller may be configured to determine the corresponding data access request as a read-burst operation.

For example, when the identification code is [001], the artificial neural network memory controller may be configured to determine the corresponding data access request as a write-burst operation.

However, examples of the present disclosure are not limited thereto.

According to the above configuration, the artificial neural network memory controller may receive various data of the artificial neural network model from the memory or store it in the memory by controlling the memory according to the read mode or the write mode.

According to the above configuration, the artificial neural network memory controller may update the weight value of a specific layer by the overwrite mode during the learning operation of the artificial neural network. In particular, since the updated weight value is stored in the same memory address value, a new memory address may not be allocated. Therefore, the overwrite mode may be more efficient in the learning operation than the write mode.

According to the above configuration, the artificial neural network memory controller can protect data stored in a specific memory address by the protection mode. In particular, there is an effect that can prevent the data of the artificial neural network model from being deleted arbitrarily in an environment accessed by multiple users, such as a server. It is also possible to protect the weight values of the artificial neural network model that has been trained in the protection mode.

In some examples, the data access request of the artificial neural network memory system may be configured to further include identification information capable of identifying whether inference data, weights, feature maps, training data sets, evaluation data sets, and others. However, examples of the present disclosure are not limited thereto.

For example, the artificial neural network memory system may be configured to add a 3-bit identification code to the data access request so that the artificial neural network memory controller identifies a domain of data to be accessed. However, the number of bits of the identification code according to examples of the present disclosure is not limited, and may be adjusted according to the number of cases to be identified.

For example, when the identification code is [000], the artificial neural network memory controller may be configured to determine that the corresponding data is data unrelated to the artificial neural network model.

For example, when the identification code is [001], the artificial neural network memory controller may be configured to determine the corresponding data as inferred data of the artificial neural network model.

For example, when the identification code is [010], the artificial neural network memory controller may be configured to determine the corresponding data as a feature map of the artificial neural network model.

For example, when the identification code is [011], the artificial neural network memory control unit may be configured to determine the corresponding data as a weight of the artificial neural network model.

For example, when the identification code is [100], the artificial neural network memory control unit may be configured to determine the corresponding data as a training data set of the artificial neural network model.

For example, when the identification code is [101], the artificial neural network memory control unit may be configured to determine the corresponding data as an inference data set of the artificial neural network model.

According to the above-described configuration, the artificial neural network memory controller may be configured to identify domains of data of the artificial neural network model, and allocate addresses of memories in which data corresponding to each domain is stored. For example, the artificial neural network memory controller may set the start number and end address of the memory area allocated to each domain. According to the above-described configuration, data allocated to each domain may be stored to correspond to the sequence of the artificial neural network data locality pattern.

For example, data of each domain of the artificial neural network model may be sequentially stored in a memory area allocated to each domain. In this case, the corresponding memory may be a memory capable of supporting a read-burst function. According to the above configuration, when the artificial neural network memory controller reads data of a specific domain from the memory, the specific data is stored according to the artificial neural network data locality pattern, so it may be configured to be optimized for the read-burst function. That is, the artificial neural network memory controller may be configured to set the storage area of the memory in consideration of the read-burst function.

In some examples, the memory may further include a read-burst function, and the at least one artificial neural network memory controller may be configured to write the storage area of the at least one memory in consideration of the read-burst function.

In some examples, the data access request of the artificial neural network memory system may be configured to further include identification information capable of identifying the quantization of the artificial neural network model. However, examples of the present disclosure are not limited thereto.

For example, when the data access request includes at least a memory address value, a domain, and quantization identification information, the artificial neural network memory system may be configured to identify quantization information of data of a corresponding domain.

For example, when the identification code is [00001], the artificial neural network memory control unit may be configured to determine the corresponding data as data quantized to 1 bit.

For example, when the identification code is [11111], the artificial neural network memory controller may be configured to determine the corresponding data as 32-bit quantized data.

In some examples, various identification information may be optionally included in the data access request.

According to the above-described configuration, the artificial neural network memory controller has an effect of analyzing the identification code of the data access request to generate a more accurate artificial neural network data locality pattern. In addition, it has the effect of selectively controlling the storage policy of the memory by identifying each identification information.

For example, if learning and reasoning can be identified, each artificial neural network data locality pattern can be generated.

For example, if the domain of data can be identified, there is an effect of improving the efficiency of memory operation by establishing a policy for storing the data of the artificial neural network data locality pattern in a specific memory area.

In some examples, when the artificial neural network memory system is configured to process a plurality of artificial neural network models, the artificial neural network memory controller includes identification information of the artificial neural network model, for example, the first artificial neural network model, the second artificial neural network model, etc. It may be configured to further generate additional identifying information. In this case, the artificial neural network memory controller may be configured to classify the artificial neural network models based on the artificial neural network data locality of each artificial neural network model. However, the present invention is not limited thereto.

The sideband signal and ANN (artificial neural network) data locality information shown in FIG. 12 may be selectively integrated or separated.

Artificial Neural Network Calculation: It is possible to determine whether ANN operation of the data is performed in the SAM MEMORY CONTROLLER.

Operation type: In the SAM MEMORY CONTROLLER, it is possible to determine whether the data is learning or inference. (Schedule for updating weight values during inference)

Operation mode: RAM operation can be controlled by SAM MEMORY CONTROLLER (Kernel can view and refresh domain, and feature map can read-discard)

DOMAIN: This may be information required for MEMORY MAP setting in SAM MEMORY CONTROLLER. (DOMAIN may allocate the same data to a specific area according to ANN data locality information)

Quantization: SAM MEMORY CONTROLLER can provide quantization information of the corresponding data.

ANN MODEL #: SAM MEMORY CONTROLLER can allocate each model to MEMORY MAP according to ANN data locality information. The minimum ANN's total DATA size can be secured.

MULTI-THREAD: The kernel shares the SAM MEMORY CONTROLLER according to the number of THREADs of each ANN MODEL, and each feature map can be allocated.

ANN data locality (DATA LOCALITY): Information indicating the current processing stage of data locality information of ANN.

Meanwhile, all sideband signals may be implemented as PACKET.

13 is a schematic diagram illustrating energy consumption per unit operation of an artificial neural network memory system.

Referring to FIG. 13 , it is a table schematically explaining energy consumed per unit operation of the artificial neural network memory system 300 . Energy consumption can be divided into memory access, addition operation, and multiplication operation.

“8b Add” refers to the 8-bit integer addition operation of the adder. An 8-bit integer addition operation can consume 0.03 pj of energy.

“16b Add” refers to the 16-bit integer addition operation of the adder. A 16-bit integer addition operation can consume 0.05pj of energy.

“32b Add” refers to the 32-bit integer addition operation of the adder. A 32-bit integer addition operation can consume 0.1pj of energy.

“16b FP Add” refers to the 16-bit floating-point addition operation of the adder. A 16-bit floating-point addition operation can consume 0.4pj of energy.

“32b FP Add” refers to the 32-bit floating-point addition operation of the adder. A 32-bit floating-point addition operation can consume 0.9pj of energy.

“8b Mult” refers to the multiplier's 8-bit integer multiplication operation. An 8-bit integer multiplication operation can consume 0.2pj of energy.

“32b Mult” refers to the multiplier's 32-bit integer multiplication operation. A 32-bit integer multiplication operation can consume 3.1pj of energy.

“16b FP Mult” refers to the multiplier's 16-bit floating-point multiplication operation. A 16-bit floating-point multiplication operation can consume 1.1pj of energy.

“32b FP Mult” refers to the multiplier's 32-bit floating-point multiplication operation. A 32-bit floating-point multiplication operation can consume 3.7 pj of energy.

“32b SRAM Read” refers to 32-bit data read access when the cache memory 322 of the artificial neural network memory system 300 is a static random access memory (SRAM). 5pj of energy may be consumed to read 32-bit data from the cache memory 322 to the processor 310 .

“32b DRAM Read” refers to 32-bit data read access when the memory 330 of the artificial neural network memory system 300 is DRAM. Reading 32-bit data from the memory 330 to the processor 310 may consume 640pj of energy. Energy unit means pico-joule (pj).

When the artificial neural network memory system 300 performs 32-bit floating-point multiplication and 8-bit integer multiplication, energy consumption per unit operation is approximately 18.5 times different. When 32-bit data is read from the memory 330 composed of DRAM and when 32-bit data is read from the cache memory 322 composed of SRAM, energy consumption per unit operation is approximately 128 times different.

That is, in terms of power consumption, as the bit size of data increases, power consumption increases. In addition, using floating-point arithmetic increases power consumption compared to integer arithmetic. Also, when data is read from DRAM, power consumption increases rapidly.

Accordingly, the neural network memory system 300 according to another example of the present disclosure may be configured such that the capacity of the cache memory 322 is sufficient to store all the data values of the neural network model 1300 .

The cache memory according to examples of the present disclosure is not limited to SRAM. Static memories capable of high-speed driving such as SRAM include SRAM, MRAM, STT-MRAM, eMRAM, and OST-MRAM. Furthermore, MRAM, STT-MRAM, eMRAM, and OST-MRAM are static memories and have non-volatile characteristics. Therefore, there is an effect that the artificial neural network model 1300 does not need to be provided again from the memory 330 when the artificial neural network memory system 300 is rebooted after the power is cut off. However, examples according to the present disclosure are not limited thereto.

According to the above-described configuration, the artificial neural network memory system 300 can significantly reduce power consumption by the read operation of the memory 330 during inference calculation of the artificial neural network model 1300 based on the artificial neural network data locality pattern 1400. can have an effect.

14 is a schematic diagram illustrating an artificial neural network memory system according to various examples of the present disclosure.

Hereinafter, various examples according to the present disclosure will be described with reference to FIG. 14 . 14 may explain the number of various instances in which various examples according to the present disclosure may be practiced.

According to various examples of the present disclosure, the artificial neural network memory system 400 includes at least one processor, at least one memory, and at least one processor, and receives at least one data access request from at least one processor. It may be configured to include at least one artificial neural network memory controller (ANN Memory Controller: AMC) configured to provide a memory access request to the memory. At least one artificial neural network memory controller (AMC) may be configured substantially the same as the exemplary artificial neural network memory controllers 120 , 220 , 320 . However, the present invention is not limited thereto, and one artificial neural network memory controller of the artificial neural network memory system 400 may be configured differently from other artificial neural network memory controllers. Hereinafter, overlapping descriptions of the neural network memory controllers 411 , 412 , 413 , 414 , 415 , 416 , and 417 and the aforementioned artificial neural network memory controllers 120 , 220 , 320 may be omitted for convenience of description.

The at least one artificial neural network memory controller is configured to connect at least one processor and at least one memory. In this case, the data movement path between the at least one processor and the at least one memory may exist in a corresponding artificial neural network data locality. Accordingly, the artificial neural network memory controller located in the corresponding data movement path may be configured to extract the corresponding artificial neural network data locality pattern.

Each artificial neural network memory control unit (AMC) may be configured to monitor each data access request to generate each artificial neural network data locality pattern. The artificial neural network memory system 400 may be configured to include at least one processor. there is. At least one processor may be configured to process an artificial neural network operation alone or in cooperation with another processor.

The artificial neural network memory system 400 may be configured to include at least one internal memory. The artificial neural network memory system 400 may be configured to be connected to at least one external memory. Internal or external memory includes DRAM (Dynamic RAM), HBM (High bandwidth memory), SRAM (Static RAM), PROM (Programmable ROM), EPROM (Erasable PROM), EEPROM (Electrically EPROM), Flash Memory, It may include a ferroelectric RAM (FRAM), a flash memory, a magnetic RAM (MRAM), a hard disk, a phase change memory device (phase change RAM), and the like. However, the present disclosure is not limited thereto.

The artificial neural network memory system 400 may include an external memory interface connected to an external memory (External MEM). The external memory interface may transmit a memory access request to at least one external memory of the artificial neural network memory system 400 and receive data in response to the memory access request from the at least one external memory. The configuration and functions disclosed in the exemplary neural network memory controllers 120, 220, 320 are distributed to a plurality of artificial neural network memory controllers 411, 412, 413, 414, 415, 416, 417, and the artificial neural network memory system 400 ) can be placed in a specific position. In some examples, the processor may be configured to include an artificial neural network memory controller.

In some examples, the memory may be DRAM, and in this case, the artificial neural network memory controller may be configured to be included in the DRAM.

For example, at least one of the artificial neural network memory controllers 411 , 412 , 413 , 414 , 415 , 416 , and 417 may be configured to have a cache memory embedded therein. In addition, the cache memory may be configured to be included in the processor, internal memory, and/or external memory.

For example, at least one of the artificial neural network memory controllers 411 , 412 , 413 , 414 , 415 , 416 , and 417 may be configured to be distributed and disposed in a data transmission path between the memory and the processor.

For example, the artificial neural network memory controller that can be implemented in the artificial neural network memory system 400 is an artificial neural network memory controller 411 configured in an independent form, an artificial neural network memory controller 412 included in the system bus, and an interface of the processor. The configured artificial neural network memory controller 413, the artificial neural network memory controller 414 included in the Wrapper Block between the memory interface of the internal memory and the system bus, the artificial neural network memory controller included in the memory interface of the internal memory, and included in the internal memory The artificial neural network memory controller 415, the artificial neural network memory controller included in the memory interface corresponding to the external memory, the artificial neural network memory controller 416 included in the wrapper block between the memory interface of the external memory and the system bus, and/or external It may be configured as one of the artificial neural network memory controllers 417 included in the memory. However, the artificial neural network memory controller according to examples of the present disclosure is not limited thereto.

For example, the artificial neural network data locality patterns generated by the first artificial neural network memory controller 411 and the second artificial neural network memory controller 412 may be the same or different from each other.

In more detail, the first artificial neural network memory controller 411 may be configured to connect the first processor 1 and the first internal memory 1 through a system bus. In this case, a corresponding first artificial neural network data locality may exist in a data movement path between the first processor 1 and the first internal memory 1 .

At this time, although the third artificial neural network memory controller 413 is shown in the corresponding path, this is only for illustration, and the third artificial neural network memory controller 413 may be deleted. That is, when at least one artificial neural network memory controller is disposed between the processor and the memory, the artificial neural network data locality pattern of the artificial neural network model processed by the processor may be generated.

In more detail, the second artificial neural network memory controller 412 may be configured to connect the second processor 2 and the first external memory 1 external MEM 1 . In this case, a corresponding second artificial neural network data locality may exist in a data movement path between the second processor 2 and the first external memory 1 .

For example, the first artificial neural network model processed by the first processor 1 may be an object recognition model, and the second artificial neural network model processed by the second processor 2 may be a voice recognition model. Accordingly, each artificial neural network model may be different from each other, and corresponding artificial neural network data locality patterns may also be different from each other.

That is, the artificial neural network data locality pattern generated by each of the neural network memory controllers 411, 412, 413, 414, 415, 416, and 417 may be determined according to the pattern characteristic of the data access request generated by the corresponding processor.

That is, even if the artificial neural network memory control unit of the artificial neural network memory system 400 is disposed between an arbitrary processor and an arbitrary memory, there is an effect that can provide the adaptability to generate the artificial neural network data locality pattern of the corresponding location.

In other words, when two processors cooperate to process one artificial neural network model in parallel, the neural network data locality pattern of the corresponding artificial neural network model can be divided and assigned to each processor. For example, the convolution operation of the first layer may be processed by the first processor and the convolution operation of the second layer may be processed by the second processor to distribute the computation of the artificial neural network model. In this case, even if the artificial neural network model is the same, the locality of the artificial neural network data of the artificial neural network model processed by each processor can be reconfigured in units of data access requests. In this case, each artificial neural network memory control unit has the effect of providing adaptability that can be configured to respectively generate artificial neural network data locality patterns corresponding to the data access request of the processor processed by each artificial neural network memory controller.

The data access request unit may be composed of at least one word unit. The artificial neural network data locality (ANN DL) unit may consist of at least one data access request unit.

According to the above configuration, even if a plurality of artificial neural network memory controllers are distributed among a plurality of processors and a plurality of memories, the performance of the artificial neural network memory system 400 is performed by artificial neural network data locality patterns generated according to each situation. This has the effect that it can be optimized. That is, since each artificial neural network memory controller can analyze locality of artificial neural network data at each location, there is an effect that can be optimized for artificial neural network calculations that are variably processed in real time.

In some examples, at least one of the neural network memory controllers 411, 412, 413, 414, 415, 416, 417 may include at least one of the number of memories, memory types, effective bandwidth of memory, delay time of memory, and memory size. may be configured to verify information.

In some examples, at least one of the neural network memory controllers 411 , 412 , 413 , 414 , 415 , 416 , and 417 may be configured to measure an effective bandwidth of a memory in response to a memory access request. Here, there may be at least one memory, and each artificial neural network memory controller may measure an effective bandwidth of a channel communicating with each memory. The effective bandwidth may be calculated by the artificial neural network memory controller generating a memory access request, measuring the time at which the memory access request is completed, and the data transmission bit rate.

In some examples, at least one of the neural network memory controllers 411 , 412 , 413 , 414 , 415 , 416 , and 417 may be configured to receive information on a required bandwidth of at least one memory in response to a memory access request.

In some examples, the neural network memory system 400 may include a plurality of memories, and at least one artificial neural network memory controller may be configured to measure an effective bandwidth of the plurality of memories, respectively.

In some examples, the artificial neural network memory system 400 may include a plurality of memories, and at least one artificial neural network memory controller may be configured to measure delay times of the plurality of memories, respectively.

That is, the at least one artificial neural network memory controller may be configured to auto-calibrate each memory connected thereto. Auto-calibration can be configured to run when the neural network memory system starts up or at specific intervals. The at least one artificial neural network memory control unit may be configured to collect information such as the number of memories connected thereto, the type of memory, the effective bandwidth of the memory, the delayed renewal of the memory, the size of the memory, and the like through auto-calibration.

According to the above configuration, the artificial neural network memory system 400 can know the delay time and effective bandwidth of the memory corresponding to the artificial neural network memory controller.

According to the above configuration, even when an independent type of artificial neural network memory control unit is connected to the system bus, there is an effect that the processor can control the memory by generating the artificial neural network data locality of the artificial neural network model being processed.

In some examples, the at least one artificial neural network memory control unit of the artificial neural network memory system 400 calculates the effective bandwidth required for the artificial neural network operation by calculating the time and data size required for one repetition of the artificial neural network data locality pattern. can be configured to Specifically, when all data access requests included in the artificial neural network data locality pattern are processed, the processor may determine that the inference of the artificial neural network model has been completed. The artificial neural network memory system 400 may be configured to calculate the number of inferences per second (IPS) by measuring the time taken for one inference based on the artificial neural network data locality pattern. In addition, the artificial neural network memory system 400 may receive information on the number of inferences per second in Mokpo from the processor. For example, a specific application may require the inference speed of a specific artificial neural network model to be 30 IPS. If the measured IPS is lower than the target IPS, the artificial neural network memory control unit 400 may be configured to operate to improve the artificial neural network model processing speed of the processor.

In some examples, the neural network memory system 400 may be configured to include an artificial neural network memory controller, a processor, and a system bus configured to control communication of the memory. In addition, at least one artificial neural network memory control unit may be configured to have a master authority of the system bus.

In other words, the artificial neural network memory system 400 may not be a dedicated device for artificial neural network computation. In this case, various peripheral devices such as Wi-Fi, a display, a camera, and a microphone may be connected to the system bus of the artificial neural network memory system 400 . In this case, the artificial neural network memory system 400 may be configured to control the bandwidth of the system bus for stable artificial neural network operation.

In some examples, the at least one artificial neural network memory control unit may be configured to operate to preferentially process an artificial neural network operation during a processing time of a memory access request, and to process operations other than the artificial neural network operation during other times.

In some examples, the at least one artificial neural network memory controller may be configured to secure an effective bandwidth of the system bus until the at least one memory completes the memory access request.

In some examples, the at least one artificial neural network memory controller may be disposed in the system bus, and the system bus may be configured to dynamically vary the bandwidth of the system bus based on the artificial neural network data locality pattern generated in the system bus.

In some examples, the at least one artificial neural network memory controller is disposed in the system bus, and the at least one artificial neural network memory controller stores the control authority of the system bus until the at least one memory completes a response to the memory access request. It can be configured to increase relatively higher than when there is no access request.

In some examples, the at least one artificial neural network memory control unit may be configured to set the priority of a data access request of a processor processing an artificial neural network operation among a plurality of processors higher than a processor processing an operation other than the artificial neural network operation. there is.

In some examples, the artificial neural network memory controller may be configured to directly control the memory.

In some examples, an artificial neural network memory controller may be included in the memory, and the neural network memory controller may be configured to generate at least one access que. The artificial neural network memory control unit may be configured to separately generate an access sequence dedicated to artificial neural network computation.

In some examples, at least one of the plurality of memories may be a SAM or a DRAM. In this case, the at least one artificial neural network memory control unit may be configured to readjust the access order of the memory access request. Such access reordering may be an access queue re-order.

In some examples, the neural network memory control unit may be configured to include an access order of a plurality of memory access requests. In this case, the first access order may be an access order dedicated to artificial neural network operation, and the second access order may be an access order other than artificial neural network operation. The artificial neural network memory control unit may be configured to provide data by selecting each access order according to a priority setting.

In some examples, the at least one artificial neural network memory controller is configured to calculate a specific bandwidth required for the system bus to process a specific memory access request based on the neural network data locality pattern, and the at least one artificial neural network memory controller includes a specific It may be configured to control the effective bandwidth of the system bus based on the bandwidth.

According to the above-described configurations, the neural network memory system 400 is configured to lower the priority of memory access requests of various peripheral devices or improve the priority of the predicted data access requests based on the neural network data locality pattern. can

According to the above-described configurations, the artificial neural network memory controller readjusts the processing order of the data access request of the system bus to maximize the bandwidth of the system bus while the artificial neural network operation is being processed, and when there is no artificial neural network operation, other peripheral devices Bandwidth can be yielded for processing of data.

According to the above-described configurations, the artificial neural network memory controller may readjust the priority of the data access request based on the artificial neural network data locality pattern. In addition, the priority can be re-adjusted based on the identification information included in the data access request. That is, the effective bandwidth of the system bus can be dynamically changed from the viewpoint of artificial neural network operation, so that the effective bandwidth can be improved. Accordingly, there is an effect that the operating efficiency of the system bus can be improved. Therefore, there is an effect that the effective bandwidth of the system bus can be improved from the point of view of the artificial neural network memory controller.

In some examples, the at least one artificial neural network memory control unit may be configured to machine-learning a data access request. That is, the at least one artificial neural network memory controller may further include an artificial neural network model configured to machine-learning artificial neural network data locality patterns. That is, since the artificial neural network data locality pattern is machine learned, it can be configured to learn and predict specific patterns interrupted by other data access requests in the middle of data access request processing according to the actual artificial neural network data locality.

The artificial neural network model built into the artificial neural network memory control unit can be machine-learned to increase the control authority of the system bus relatively higher when predicted data access requests are generated than when predicted data access requests are not generated. .

In some examples, the at least one artificial neural network memory control unit further includes a plurality of layered cache memories, and the at least one artificial neural network memory control unit performs machine learning of inter-layer data access requests of the plurality of layered cache memories. can be configured.

In some examples, the at least one artificial neural network memory control unit may be configured to further receive at least one of effective bandwidth, power consumption, and delay time information of each layer of a plurality of layered cache memories.

According to the above configuration, the artificial neural network memory control unit may be configured to generate an artificial neural network data locality pattern through machine learning, and the machine-learned artificial neural network data locality pattern is a specific pattern of various data access requests independent of the artificial neural network operation. When generated with In addition, the efficiency of artificial neural network computation can be improved by predicting the characteristics of various artificial neural network models and other operations processed by the processor by reinforcement learning.

In some examples, the at least one artificial neural network memory controller may be configured to divide and store data stored in the plurality of memories based on the effective bandwidth and delay time of each of the plurality of memories.

For example, the data is composed of a bit group of L bits, the plurality of memories further comprising a first memory and a second memory, wherein the first memory includes the L bits based on a first effective bandwidth or a first delay time. is configured to divide and store M-bit data among the bit group of The sum of M bits and N bits may be configured to be less than or equal to L bits. In addition, the plurality of memories further include a third memory, wherein the third memory is configured to store data of O bits of the bit group of L bits based on the third effective bandwidth or the third delay time, M bits, N bits The sum of bits and O bits may be configured to be equal to L bits.

For example, the data is composed of P data bundles, the plurality of memories includes a first memory and a second memory, and the first memory is one of the P data bundles based on the first effective bandwidth or the first delay time. configured to store R data bundles, and the second memory is configured to store S data bundles among the P data bundles based on a second effective bandwidth or a second delay time, wherein the R and S sums are equal to the P It may be configured to be the same or smaller. In addition, the plurality of memories further include a third memory, wherein the third memory is configured to store T data bundles among the P data bundles based on the third effective bandwidth or the third delay time, R, S and The sum of T may be configured to be equal to P.

According to the above-described configuration, when the bandwidth of one memory is low, the artificial neural network memory controller can store or read data by distributing data in a plurality of memories, thereby improving the effective bandwidth of the memory. For example, the artificial neural network memory controller may be configured to store or read an 8-bit quantized weight value divided by 4 bits in a first memory and 4 bits in a second memory. Therefore, there is an effect that the effective bandwidth of the memory can be improved from the point of view of the artificial neural network memory controller.

The artificial neural network memory control unit may be configured to further include a cache memory configured to merge and store data divided and stored in a plurality of memories. That is, the at least one artificial neural network memory controller may further include a cache memory, and the at least one artificial neural network memory controller may be configured to merge data stored in a plurality of memories and store the merged data in the cache memory. Accordingly, the processor may be provided with the merged data.

In order to merge the divided data, the at least one artificial neural network memory control unit may be configured to store division information of the divided and stored data in a plurality of memories. Various examples of the present disclosure may be described as follows.

According to the examples of the present disclosure, the artificial neural network memory system sequentially records at least one processor configured to generate a data access request corresponding to the artificial neural network operation and the data access request to obtain the neural network data locality pattern of the artificial neural network operation. at least one neural network memory controller configured to generate a predicted data access request that predicts the actual data access request of the data access request generated by the at least one processor based on the neural network data locality pattern can be configured. Here, the artificial neural network data locality may be the artificial neural network data locality reconstructed at the processor-memory level.

According to examples of the present disclosure, an artificial neural network memory system is configured to store at least one processor configured to process an artificial neural network model and artificial neural network data locality information of the artificial neural network model, and based on the artificial neural network data locality information, at least one The processor may be configured to include at least one artificial neural network memory controller configured to predict data to be requested and generate a predicted data access request.

The artificial neural network memory system may be configured to further include at least one memory and a neural network memory controller, at least one processor, and a system bus configured to control communication of the at least one memory. According to examples of the present disclosure, The neural network memory system includes a processor, a memory, and a cache memory, and is configured to generate a predicted data access request including data to be requested by the processor based on the neural network data locality information, and to respond to the predicted data access request from the memory. It may be configured to store the corresponding data in the cache memory before the processor requests it.

According to the examples of the present disclosure, the artificial neural network memory system is configured to predict and operate the artificial neural network data locality information by observing the data access requests generated by the first mode or the processor configured to operate by receiving the artificial neural network data locality information. It may be configured to operate in one of two modes.

The at least one artificial neural network memory controller may be configured to sequentially further generate a predicted data access request based on the neural network data locality pattern.

The at least one artificial neural network memory controller may be configured to generate a predicted data access request before generating an actual data access request.

The at least one processor may be configured to transmit a data access request to the at least one artificial neural network memory controller.

The at least one artificial neural network memory control unit may be configured to output a predicted data access request in response to the data access request.

The data access request may be configured to further include a memory address.

The data access request may be configured to further include a start address and an end address of the memory.

The at least one artificial neural network memory controller may be configured to generate a memory access request based on one of a data access request generated by the at least one processor and a predicted data access request generated by the artificial neural network memory controller.

The data access request may be configured to further include a start address of memory and a continuous data read trigger (trigger).

The data access request may be configured to further include information on the number of consecutive data and a start address of the memory.

The data access request and dictionary data access may be configured to further include a matching data access request token at the same memory address.

The data access request may be configured to further include identification information capable of identifying whether a memory read or write command is issued.

The data access request may be configured to further include identification information capable of identifying whether or not an overwrite command is performed.

The data access request may be configured to further include identification information capable of identifying whether inference data, weight data, and feature map data are present.

The data access request may be configured to further include identification information that can identify whether the learning data and the evaluation data.

The data access request may be configured to further include identification information capable of identifying whether the artificial neural network operation is an operation for learning or an operation for inference.

When the at least one processor generates the actual data access request, the at least one artificial neural network memory controller may be configured to determine whether the predicted data access request and the actual data access request are the same request.

The at least one artificial neural network memory controller may be configured to maintain the neural network data locality pattern when the predicted data access request and the actual data access request are the same.

The at least one artificial neural network memory controller may be configured to update the neural network data locality pattern when the predicted data access request and the actual data access request are different.

The artificial neural network data locality pattern may be configured to further include data that sequentially records addresses of memory of data access requests.

The at least one artificial neural network memory controller may be configured to generate an artificial neural network data locality pattern by detecting a repeating pattern of a memory address included in the data access request.

The artificial neural network data locality pattern may be composed of memory addresses having a repeating loop characteristic.

The artificial neural network data locality pattern may be configured to further include identification information capable of identifying the start and end of the computation of the artificial neural network model.

At least one processor may be configured to receive data corresponding to the data access request from the artificial neural network memory controller.

The at least one artificial neural network memory control unit may be configured to further include an artificial neural network model configured to machine-learning artificial neural network data locality patterns.

The at least one artificial neural network memory control unit may be configured to store an updated pattern and a previous pattern of the artificial neural network data locality pattern to determine whether to change the artificial neural network model.

The at least one artificial neural network memory controller may be configured to determine whether the data access requests are requests of one artificial neural network model or a mixture of requests of a plurality of artificial neural network models.

The at least one artificial neural network memory controller may be configured to further generate artificial neural network data locality patterns corresponding to the number of artificial neural network models when the number of artificial neural network models is plural.

The at least one neural network memory control unit may be configured to respectively generate corresponding predicted data access requests based on the neural network data locality patterns.

The at least one artificial neural network memory controller may be configured to further generate a memory access request corresponding to the data access request.

The at least one artificial neural network memory controller may be configured to further generate a memory access request corresponding to the predicted data access request.

Each of the data access request, the predicted data access request, and the memory access request may be configured to include a corresponding memory address value and an operation mode, respectively.

The at least one artificial neural network memory control unit may be configured to further generate a memory access request configured to include at least a portion of information included in the data access request and the predicted data access request.

It may further include at least one memory configured to communicate with the at least one artificial neural network memory controller, and the at least one memory may be configured to operate in response to a memory access request output from the at least one artificial neural network memory controller.

The at least one memory may be configured to store at least one of inference data, weight data, and feature map data.

The at least one artificial neural network memory control unit may be configured to further include a cache memory configured to store data transmitted by the at least one memory in response to a memory access request.

When the at least one processor outputs the actual data access request, the at least one artificial neural network memory control unit determines whether the predicted data access request and the actual data access request are the same request, and if they are the same, the at least one artificial neural network memory control unit The control unit is configured to provide the data stored in the cache memory to the at least one processor, and if not identical, the at least one artificial neural network memory control unit may be configured to generate a new memory access request based on the actual data access request.

The at least one artificial neural network memory control unit may be configured to sequentially generate at least one or more memory access requests based on the remaining capacity of the cache memory to minimize the remaining capacity of the cache memory.

The at least one artificial neural network memory controller may be configured to measure an effective bandwidth of at least one memory in response to a memory access request.

The at least one artificial neural network memory controller may be configured to receive information on a required bandwidth of at least one memory in response to a memory access request.

The at least one artificial neural network memory control unit may be configured to measure the number of inferences per second (IPS) of the artificial neural network operation by calculating the number of repetitions for a specific time of the artificial neural network data locality pattern.

The at least one artificial neural network memory controller may be configured to calculate an effective bandwidth required for an artificial neural network operation by calculating a time and data size required for one repetition of the neural network data locality pattern.

The at least one memory further includes a DRAM including a refresh function capable of updating a voltage of a cell of the memory, and the at least one artificial neural network memory control unit responds to a memory access request corresponding to the predicted data access request. It may be configured to selectively control refresh of a memory address region of the corresponding at least one memory.

The at least one memory further includes a precharge function capable of charging the global bit line of the memory to a specific voltage, and the at least one artificial neural network memory control unit includes at least one memory access request corresponding to the predicted data access request. may be configured to selectively provide precharge to a memory address region of a memory of

The at least one memory may further include a plurality of memories, and the at least one artificial neural network memory controller may be configured to measure an effective bandwidth of the plurality of memories, respectively.

The at least one memory may further include a plurality of memories, and the at least one artificial neural network memory controller may be configured to measure latencies of the plurality of memories, respectively.

The at least one memory further includes a plurality of memories, and the at least one artificial neural network memory control unit may be configured to divide and store data stored in the plurality of memories based on the effective bandwidth and delay time of each of the plurality of memories. .

The data is composed of a bit group of L bits, and the plurality of memories further include a first memory and a second memory, wherein the first memory includes M among the bit group of L bits based on the first effective bandwidth or the first delay time. The second memory is configured to divide and store data of bits, and the second memory is configured to divide and store N bits of data among the L bits bit group based on the second effective bandwidth or the second delay time, and the M bits and the N bits The sum of can be configured to be less than or equal to L bits.

The plurality of memories further includes a third memory, wherein the third memory is configured to store data of O bits of the bit group of L bits based on the third effective bandwidth or the third delay time, the M bits, N bits and The sum of O bits may be configured to be equal to L bits.

The at least one artificial neural network memory controller may be configured to further include a cache memory configured to merge and store data divided and stored in a plurality of memories.

The data is composed of P data bundles, and the plurality of memories further include a first memory and a second memory, wherein the first memory includes R data of the P data bundles based on the first effective bandwidth or the first delay time. and the second memory is configured to store S data bundles among the P data bundles based on a second effective bandwidth or a second delay time, wherein R and the S sum are equal to the P or may be configured to be small.

The plurality of memories further includes a third memory, wherein the third memory is configured to store T data bundles of the P data bundles based on a third effective bandwidth or a third delay time, R, the S and The sum of the T numbers may be configured to be equal to the P numbers.

The at least one memory further includes a plurality of memories, the at least one artificial neural network memory controller further includes a cache memory, and the at least one artificial neural network memory controller merges data stored in the plurality of memories distributed to the cache memory. It can be configured to store in .

The at least one memory may further include a plurality of memories, and the at least one artificial neural network memory controller may be configured to store division information of data divided and stored in the plurality of memories.

The at least one artificial neural network memory controller may be configured to store a portion of data as much as a latency in the cache memory based on the predicted data access request and the latency value of the at least one memory.

The at least one artificial neural network memory control unit may be configured to store a portion of the data in the cache memory based on the predicted data access request and the data bandwidth requirement of the at least one memory.

When the at least one processor generates an actual data access request, the at least one artificial neural network memory control unit provides the data stored in the cache memory first, and controls the remainder of the data from the at least one memory in a read-burst mode, It may be configured to reduce latency of the memory.

When the at least one artificial neural network memory control unit generates an actual data access request in at least one processor based on the predicted data access request and the latency value of the at least one memory, the read-burst mode of at least one memory in advance by the latency value Starting with , it may be configured to reduce latency of at least one memory.

It may be configured to further include a system bus configured to control communication of an artificial neural network memory controller, the at least one processor, and the at least one memory.

At least one artificial neural network memory control unit may be configured to have a master authority of the system bus.

The at least one artificial neural network memory control unit further includes an artificial neural network model, wherein, when a predicted data access request is generated, the control right of the system bus is relatively greater than when the predicted data access requests are not generated. It can be machine learned to increase high.

The at least one artificial neural network memory control unit may be configured to secure an effective bandwidth of the system bus until the at least one memory completes the memory access request.

The at least one artificial neural network memory control unit calculates a specific bandwidth required for the system bus to process a specific memory access request based on the artificial neural network data locality pattern, and the at least one artificial neural network memory control unit calculates a specific bandwidth required for the system bus based on the specific bandwidth. may be configured to control the effective bandwidth of

The at least one artificial neural network memory controller may be disposed inside the system bus, and the system bus may be configured to dynamically vary the bandwidth of the system bus based on the artificial neural network data locality pattern generated in the system bus.

The at least one artificial neural network memory control unit may be configured to operate to preferentially process an artificial neural network operation during a processing time of a memory access request, and to process an operation other than an artificial neural network operation for other times.

The at least one artificial neural network memory controller and the at least one processor may be configured to communicate directly.

The artificial neural network memory control unit further includes a first access order that is an access order dedicated to artificial neural network operation and a second access order that is an access order other than artificial neural network operation, and the artificial neural network memory control unit selects each access order according to a priority setting to provide data.

The at least one artificial neural network memory control unit further includes a plurality of layered cache memories, and the at least one artificial neural network memory control unit further includes an artificial neural network model configured to machine learning a data access request between layers of a plurality of layered cache memories. can be configured to

The at least one artificial neural network memory controller may be configured to further receive at least one of effective bandwidth, power consumption, and latency information of each layer of a plurality of layered cache memories.

At least one processor configured to generate a data access request corresponding to the artificial neural network operation and configured to store an artificial neural network data locality pattern of an artificial neural network operation generated by a compiler, wherein the at least one processor is configured to store an artificial neural network data locality pattern based on the artificial neural network data locality pattern at least one neural network memory controller configured to generate a predicted data access request that predicts the actual data access request of the generated data access request, and at least one memory configured to communicate with the at least one neural network memory controller, at least One memory may be configured to operate in response to a memory access request output from at least one artificial neural network memory controller.

The at least one neural network memory system may be configured to further include at least one memory and a system bus configured to control communication of the neural network memory controller, the at least one processor, and the at least one memory.

The at least one artificial neural network memory control unit is disposed in the system bus, and the at least one artificial neural network memory control unit grants control authority of the system bus until the at least one memory completes a response to the memory access request. It can be configured to increase relatively higher than in the absence of it.

At least a portion of the at least one artificial neural network memory controller may be configured to be included in the DRAM.

At least a portion of the at least one artificial neural network memory controller may be configured to be included in the at least one processor.

The DRAM may further include or at least one memory is DRAM, and the at least one artificial neural network memory controller may be configured to readjust an access que of a memory access request. That is, it may be configured to control a reorder cue of the memory controller of the DRAM.

The artificial neural network memory controller may be configured to further include priority information that the memory controller of the memory can interpret in the request for accessing the artificial neural network operation-related memory provided to the memory controller of the memory.

According to the above configuration, the memory controller of the memory is configured to store the memory inside the memory controller based on the priority information included in the memory access request generated by the artificial neural network memory controller regardless of whether the memory access request is related to the artificial neural network operation. It may be configured to re-order the access order. Therefore, the access order of memory access requests for artificial neural network operation processing can be processed before the access order of other types of memory access requests. Accordingly, the artificial neural network memory controller has an effect of increasing the effective bandwidth of the corresponding memory.

It may be configured to readjust the memory access request processing order determined by the memory controller of the DRAM according to the priority information provided by the artificial neural network memory controller.

For example, if the priority of the memory access request generated by the artificial neural network memory controller is set to urgent, the memory controller of the DRAM may change the processing order of the memory access request to the first priority.

The artificial neural network memory controller may be configured to generate at least one access sequence.

The at least one memory may include an artificial neural network memory controller, and the artificial neural network memory controller may be configured to separately generate an access sequence dedicated to artificial neural network computation.

The at least one artificial neural network memory control unit may be configured to readjust the access order of the memory access request.

The at least one memory may further include a read-burst function, and the at least one artificial neural network memory controller may be configured to set a storage area of the at least one memory in consideration of the read-burst function.

The at least one memory may further include a read-burst function, and the at least one artificial neural network memory controller may be configured to process a write operation in the storage area of the at least one memory in consideration of the read-burst function.

The at least one processor further includes a plurality of processors, and the at least one artificial neural network memory control unit is configured to prioritize a data access request of a processor that processes an artificial neural network operation among the plurality of processors. A processor that processes operations other than the artificial neural network operation It can be configured to set higher than that.

Each of the at least one AMC may be configured to operate independently based on each ANN DL stored therein. Each ANN DL may be the same as or different from each other. Each ANN DL may be configured to have a different ANN DL according to the arrangement position of each AMC. To elaborate, the ANN DL is configured to analyze the ANN DL of the ANN model processed on the communication bus where the AMC is located, and prepare the predicted data in advance. Accordingly, the first ANN DL recognized by the first AMC at the first arrangement position may be different from the second ANN DL recognized by the second AMC at the second arrangement position. However, each AMC has the advantage of being able to operate independently at a corresponding location based on each ANN DL.

15 is an exemplary diagram illustrating a substrate and a channel on which a memory is mounted.

As shown, a plurality of pins for communication with a memory bus are formed on a board on which the memory is mounted, that is, the board.

The memory bus of the memory may include an address bus (eg, 17 bits), a command and control bus (eg, 6 bits), and a data bus (eg, 64 bits). To elaborate, it is also possible that the memory bus optionally further includes the side band signal illustrated in FIG. 12 .

That is, according to the added sideband signal signal, the SAM controller according to various examples of the present disclosure may be configured to selectively classify and control the memory cell region of the memory. However, the present invention is not limited thereto, and may be implemented in an IP header packet method replacing the sideband signal.

The present specification provides a memory configured to operate according to an artificial neural network (ANN) data locality (DL), for example, a sequential access memory (SAM) and a SAM controller. SAM may mean an artificial neural network dedicated memory. The SAM control unit may mean a memory control unit that controls the SAM.

That is, the SAM according to an example of the present disclosure excludes the random access (RANDOM ACCESS) characteristic of the DRAM memory, and is set to sequentially operate according to the artificial neural network data locality (ANN DL) information. It means a memory specialized for artificial neural network processing. can do. However, the memory cell structure of the SAM is not limited to DRAM, and it may be applied to a memory having a memory cell having a structure similar to that of DRAM. That is, based on the artificial neural network data locality information, sequential access information capable of sequentially accessing an address of a memory can be derived.

SAM can be configured to process read/write commands in BURST MODE basically. In this case, the read/write command may be set to operate in units of artificial neural network data locality (ANN DL) information. That is, the SAM control unit may be configured to request a memory operation from the SAM in units of ANN DLs. In this case, the memory map may be set so that the SAM operates the memory operation of the ANN DL unit in the actual BURST MODE without a separate BURST MODE command. Here, the ANN DL unit may mean a data access request of the minimum unit requested by the processor to the memory or AMC based on the artificial neural network data locality information.

As the ANN DL is provided, the SAM can substantially eliminate the random access nature of the memory. Since SAM operates in the actual Burst mode based on artificial neural network data locality (ANN DL) information, the frequency of occurrence of CAS latency & RAS latency can be minimized.

In other words, the conventional random access operation of the memory is effective only in a situation in which the memory operation order of the processor cannot be predicted.

Conversely, the SAM may know in advance the order of memory operation requests to be requested by the processor based on the ANN DL. Therefore, the SAM can provide a memory operation with minimized power consumption and latency based on locality information of artificial neural network data.

The memory bus between the SAM and the SAM controller may further include at least one side band signal.

The system bus between the SAM controller and the processor may further include at least one side band signal. The number of side band signals of each of the memory bus and the system bus may be the same or different from each other.

However, the present disclosure is not limited thereto, and may be implemented in the form of a packet including information corresponding to a sideband signal.

Since the SAM removes the random access characteristic of the memory and operates based on artificial neural network data locality (ANN DL) information, it may be configured to enable precise refresh timing control of the memory cells of the SAM. A dynamic memory cell needs periodic refresh, and a dynamic memory implemented with SAM may be configured such that refresh is controlled based on ANN DL.

Since the SAM removes the random access characteristic of the memory and operates based on artificial neural network data locality (ANN DL) information, it can be configured to enable precise pre-charge timing control of the memory cells of the SAM. The dynamic memory cell requires precharge for the sense amplifier operation, and the dynamic memory implemented as SAM may be configured to control the precharge based on the ANN DL.

The SAM may be configured to determine an area of memory allocated for each artificial neural network data locality (ANN DL) information or for each domain.

16 is an exemplary diagram illustrating a process of reading data from a SAM having a multi-bank structure.

The SAM shown in FIG. 16 may utilize a part of the memory cell operation of a conventional DRAM. The SAM may include at least one bank.

Referring to FIG. 16 , the SAM may be configured such that memory cells in a matrix form have row and column addresses. One bank may be configured by bundling a plurality of memory cells.

In order to improve the bandwidth (BANDWIDTH) of the SAM, it may be configured to interlace memory cells in each bank of the SAM.

In order to improve the bandwidth of the SAM, it may be configured to perform interleaving in units of banks of the SAM.

A row address strobe (RAS) signal and/or a column address strobe (CAS) signal of a memory cell of the SAM may be directly controlled according to the artificial neural network data locality (ANN DL) information. Accordingly, the SAM control unit may control the SAM to sequentially read data or write data according to the ANN DL.

12, 15 or 16 , the SAM according to an example of the present disclosure may include a plurality of banks. In this case, the SAM may be configured to allocate a specific bank and/or a specific memory cell region for a specific purpose based on at least one side band signal.

For example, depending on the domain, the first bank of the SAM may be allocated exclusively for the feature map.

For example, depending on the domain, the second bank of the SAM may be allocated exclusively for the kernel.

A SAM according to an example of the present disclosure may include at least one bank. In this case, the SAM may be configured to allocate specific rows of at least one bank for a specific purpose based on at least one side band signal.

For example, the SAM may allocate the rows of the first area of the first bank exclusively to the feature map according to the domain.

For example, the SAM may allocate the rows of the second area of the first bank exclusively for weights according to domains.

12 or 15 again, the SAM according to an example of the present disclosure may be configured to allocate specific rows of a specific bank for a specific purpose based on the ANN MODEL# signal. SAM according to an example of the present disclosure may be configured to allocate specific rows of a specific bank to a specific purpose based on at least ANN DATA LOCALITY. That is, the SAM may be configured to allocate memory cells of a specific bank or specific rows to a specific purpose based on at least one side band signal.

However, the present disclosure is not limited thereto, and even if there is no separate sideband signal, the SAM controller may be implemented by directly controlling the memory address of the SAM based on the artificial neural network data locality (ANN DL) information.

17 is an exemplary diagram illustrating latency occurring in a conventional DRAM.

Referring to FIG. 17 , the latency between a CPU, a conventional memory controller, and a conventional DRAM is schematically illustrated.

A conventional CPU uses a virtual memory utilizing a TLB (Translation LOOKASIDE BUFFER) to process various operations. Therefore, artificial neural network data stored in the conventional DRAM is fragmented and stored in the DRAM.

The operation of the CPU reading data from the DRAM may include steps A through F. Each process may incur latency.

In step A, the CPU generates a transaction request. In this process, the transaction request may be temporarily queued in a queue of the CPU, and thus latency may occur. In step B, the CPU may transmit the transaction request to the memory controller. In step C, the memory controller may convert the transaction request into command sequences. In step D, the memory controller may transfer the command sequences to the DRAM. In step E, the DRAM may use one CAS signal, a combination of a RAS signal and a CAS signal, or a combination of a pre-charge (PRE) signal, a RAS signal, and a CAS signal to process the command sequences. In process F, data according to the transaction is transferred to the CPU.

The latency from process A to process F may include A+B+C+D+E+F.

Process E1 may occur when all data corresponding to a data operation requested by the conventional DRAM is latched in the sense amplifier shown in FIG. 31A .

Process E2 may occur when a portion of data corresponding to a data operation requested by the conventional DRAM is fragmented in memory cells of a plurality of rows.

Process E3 may occur when a portion of data corresponding to a data operation requested by the conventional DRAM is fragmented into a plurality of rows of memory cells, and the memory cells are precharged for various reasons.

Here, RAS means a Row Address Strobe (RAS) signal, CAS means a Column Address Strobe (CAS) signal, and PRE means a pre-charge signal. Each signal includes a corresponding respective delay time.

When conventional DRAM and conventional memory controllers process artificial neural network data, ANN DL information is not considered. Accordingly, the artificial neural network data is fragmented and processed as a virtual memory. Therefore, in the conventional case, the cases of E2 and E3 rather than the case of E1 occur frequently. Therefore, it is possible to cause a bottleneck of artificial neural network processing by the conventional DRAM.

In contrast, in the case of the SAM according to an example of the present disclosure, since it is based on the ANN DL, the frequency of occurrence of E1 may be maximized by removing or minimizing the frequency of occurrence of E2 and E3. The delay time according to the CAS signal, the RAS signal, and the PRE signal may be reduced. Therefore, it is possible to improve the processing speed of the artificial neural network calculation.

18 is an exemplary diagram illustrating a basic concept of a Sequential Access Memory (SAM) according to the disclosure of the present specification.

18 shows the main memory, the SAM, the SAM controller, and the processor. The SAM controller is disposed between the processor and the SAM to control the SAM.

The SAM controller may be integrated with the SAM, which is the main memory, or may be implemented physically separated from the SAM. Alternatively, the SAM controller may be embedded in the processor. In addition, the SAM controller may be implemented in various forms.

The SAM controller may receive artificial neural network data locality (ANN DL) information from a processor to process an artificial neural network (ANN), for example, an NPU or a compiler (COMPILER).

The artificial neural network data locality (ANN DL) information may be included in a register map for NPU control or may be provided as a separate register map or table.

The artificial neural network data locality (ANN DL) information may be included in a register map for controlling the SAM controller or may be provided as a separate register map or table.

The artificial neural network data locality information may be provided to the processor (ie, NPU) and the SAM controller, respectively. In addition, the artificial neural network data locality information provided to the NPU and the SAM control unit may be identical to each other, or may be at least partially identical to each other.

The SAM controller may transmit a read/write command to the SAM, which is the main memory, according to order information in the artificial neural network data locality (ANN DL) information, and may serve to provide data requested by the processor.

The SAM, which is the main memory, may determine a data size to request according to the order information in the artificial neural network data locality (ANN DL) information. Artificial neural network data locality (ANN DL) information includes the number of PEs in the processor (ie NPU), the cache memory size of the processor (ie NPU), the kernel to be used for the layer (KERNEL), the feature map (FEATURE MAP) size, etc. Therefore, it may vary.

For example, when the size of data is larger than the size of the cache memory (CACHE MEMORY), the processor may use the TILING algorithm. In addition, the SAM controller may be configured to operate according to the processing method of the processor.

For example, for latency hiding of the main memory, artificial neural network data locality (ANN DL) may be determined. In other words, for latency hiding, it is also possible to set the locality of the artificial neural network data so that data of a size corresponding to the minimum number of clocks is first cached.

When the processing method of the processor is changed, for example, the artificial neural network data locality (ANN DL) information may be changed according to the WEIGHT STATIONARY, INPUT STATIONARY, and OUTPUT STATIONARY methods.

Unless there is a special circumstance, the SAM according to examples of the present disclosure may be configured to minimize the frequency of occurrence of the E2 or E3 latency described above with reference to FIG. 17 . That is, the memory operation of the SAM may operate to sequentially access rows of memory cells of the bank unless there is a special circumstance. Therefore, the frequency of occurrence of latency E2 or E3 of FIG. 17 described above can be minimized.

That is, the SAM may be configured to operate by sequentially addressing memory cells for each memory operation of the ANN DL unit. Memory cells of all columns corresponding to one row of one memory cell of the SAM may be latched by the corresponding sense amplifier. All data latched in the sense amplifier can be read without additional RAS latency. Accordingly, memory cells of columns corresponding to one row may be sequentially read.

However, the present disclosure is not limited thereto, and the order of reading data latched in the sense amplifier may be changed, and even in this case, separate RAS latency may not occur.

In other words, the sequential addressing technique of the SAM may mean that the address of the row and column of a memory cell is gradually changed when processing a memory operation in the ANN DL unit.

The SAM controller may be configured to directly control the storage address of the SAM, which is the main memory, in ANN DL units. Accordingly, the SAM controller may be configured to directly control the RAS signal and the CAS signal for accessing the memory cell of the SAM.

19 is a table exemplarily showing the amount of computation and data size for 16 layers.

The example of FIG. 19 describes the input feature map, the output feature map, and the structure information of the kernel for each of 16 layers when the artificial neural network model is VGG16. Various examples of the present disclosure may be configured to generate at least one artificial neural network data locality (ANN DL) information based on at least one artificial neural network model.

In the table shown in FIG. 19 , layers 1 to 13 are layers for convolutional, and layers 14 to 16 include fully-connected layers.

In general, the artificial neural network model should be calculated according to the order of the layers, but for various reasons, the operation order of the artificial neural network model processed by the processor may increase or decrease.

Theoretically, one layer of an artificial neural network model can be structurally processed with one convolution operation. However, convolution can be performed several times under various conditions. That is, the number of convolution operations may increase as much as the number of tilings.

For example, the artificial neural network data locality (ANN DL) information may be changed according to the layer structure of the artificial neural network, the PE array structure of the processor (ie, NPU), and the size of the internal memory of the processor.

For example, if the size of the internal memory for the kernel is 256Kbyte and the size of the kernel of layer 1 is 3.2MByte, the number of tiling suitable for the internal memory for the kernel may be 13 times.

Also, the order of the 13 tiling processing to be processed by the processor may be determined.

That is, the number of ANN DLs may change according to the size of the processor's internal memory. Therefore, it is also possible to increase the number of ANN DLs. On the other hand, if the size of the internal memory for the input feature map is 256 Kbyte and the size of the input feature map of layer 1 is 1.7 KByte, tiling may not be necessary. Even for the output feature map of layer 1, tiling may be unnecessary.

That is, when the operation order of the artificial neural network model processed by the processor is changed, the artificial neural network data locality (ANN DL) information of the artificial neural network model is also changed.

Accordingly, the artificial neural network data locality (ANN DL) information of the artificial neural network model may be configured to include order information changed according to tiling.

20 is a table exemplarily showing the amount of computation and data size for 28 layers.

The example of FIG. 20 describes the input feature map, the output feature map, and the structure information of the kernel for each 28 layers when the artificial neural network model is Mobilenet V1.0. Various examples of the present disclosure may be configured to generate at least one artificial neural network data locality (ANN DL) information based on at least one artificial neural network model.

In the table shown in FIG. 20 , layers 1 to 28 include a layer for convolutional, a layer for depth-wise convolution, and a layer for point-wise convolution.

In general, the artificial neural network model should be calculated according to the order of the layers, but the order of operation may be changed for various reasons. If the operation order is changed, the artificial neural network data locality (ANN DL) information of the artificial neural network model is also changed.

For example, when one processor processes two artificial neural network models, the artificial neural network data locality (ANN DL) information processed by the one processor is the artificial neural network model shown in FIGS. 19 and 20. Neural network data locality information may be integrated with each other in a specific order.

For example, when two processors process one artificial neural network model, the artificial neural network data locality (ANN DL) information processed by the two processors is the artificial neural network data locality information of the artificial neural network model shown in FIG. It may be separate to be processed by each processor.

21 is a table showing a first example of accessing a memory according to order information in artificial neural network data locality (ANN DL) information.

In the first example shown in FIG. 21, the SAM control unit performs an operation on 28 layers when the artificial neural network model is Mobilenet V1.0. Artificial neural network data locality (ANN DL) including 84 steps It can be configured to have information. That is, sequential access information may be determined from the viewpoint of the row and column addresses of the SAM based on the order of each step.

The SAM, which is the main memory, may be configured to operate based on artificial neural network data locality (ANN DL) information included in the SAM controller.

Here, the artificial neural network data locality (ANN DL) information refers to the data processing order of the artificial neural network model of the processor generated by the compiler or the SAM controller in consideration of at least some of the following conditions.

a. ANN's model structure (VGG16 or Mobilenet V1.0, etc.)

b, the architecture of the processor (eg, depending on the architecture of the CPU, GPU, NPU)

For example, in the case of NPU, the number of PEs, stationary structure (input, output, weight), structure, etc.

c. Size of cache memory (Tiling algorithm needs to be applied when the size of cache memory is smaller than the size of data, etc.)

d. Data size for each domain and each layer

For example, the domain includes an input feature map (IFMAP), an output feature map (OFMAP), and a kernel (Kernel).

e. PROCESSING POLICY

f. Data Reuse Rate

For example, a data request order of a specific domain may be determined, such as an input feature map (IFMAP) read first request or a kernel (kernel) read first request.

The policy may vary according to the structure of the processor or the compiler algorithm.

The SAM controller according to examples of the present disclosure may set the row and column addresses of memory cells of the SAM to be sequential based on the ANN DL. For example, it may be set so that the row and column addresses of the memory cells of the SAM are sequential in units of ANN DL.

22 is an exemplary table showing the table shown in FIG. 21 in a simplified manner.

In FIG. 22, data sizes and memory addresses are denoted by symbols for convenience of explanation.

As can be seen with reference to FIG. 22 , the SAM control unit may establish a policy regarding address allocation of the SAM according to order information in the artificial neural network data locality (ANN DL) information. In more detail, the SAM controller may be configured to directly control the row and column addresses of memory cells of the SAM.

According to order information in the artificial neural network data locality (ANN DL) information, at least some or all data may be stored in the memory in units of ANN DLs. In this case, all of the data may be optimized and stored in the burst mode.

The ANN DL information may include, for example, pattern information in the order of i) reading the input feature map, ii) reading the corresponding kernel, and iii) writing the output feature map. However, the present disclosure is not limited to the above pattern, and various patterns are disclosed. In addition, the pattern may be set differently for each layer.

At this time, the SAM, which is the main memory, may control the CAS signal or the RAS signal according to the ANN DL information to operate in the burst mode. 41 or 42 , an example of directly controlling the row address decoder and the column multiplexer/demultiplexer by controlling the CAS signal, the RAS signal, and the address signal is shown.

In this case, the SAM control unit may predict that the processor requests data in a specific order based on order information in the ANN DL information.

The SAM controller analyzes the ANN DL information of the compiled artificial neural network model and directly controls the CAS signal and/or the RAS signal of the SAM, so that the data requested by the processor can be sequentially arranged in the memory cells of the SAM. Alternatively, fragmented data can be continuously rearranged. Accordingly, the SAM can sequentially provide data to the SAM control unit.

That is, the SAM, which is the main memory, may be configured to operate in burst mode up to the start address and the end address in units of ANN DLs.

Alternatively, the SAM, which is the main memory, may analyze the compiled ANN DL information, arrange the data requested by the NPU into consecutive addresses, and then sequentially provide the data.

Accordingly, the SAM control unit according to examples of the present disclosure may set the row and column addresses of the memory cells of the SAM to be sequential based on the ANN DL.

Each ANN DL unit may have a corresponding data size. For example, the first ANN DL unit may have data of size A, and may have a start address and an end address corresponding to size A. Accordingly, the operation mode of the SAM may be configured to basically operate in the burst mode of the DRAM, and the SAM may basically operate in the burst mode when the SAM controller performs a read command.

Also, even when the processor's command is in the read mode instead of the read burst mode, the SAM may operate in the actual burst mode based on the ANN DL.

In addition, all data may operate in the burst mode based on the ANN DL, but the present disclosure is not limited thereto, and most data may be set in the burst mode. That is, at least some may not be in burst mode.

23 shows an example in which the SAM sets a memory address map according to the table shown in FIG. 22 .

The SAM control unit may control the CAS signal and/or the RAS signal of the SAM based on the compiled ANN DL information to arrange data to be requested by the processor so as to be contiguous in the memory map.

Since the SAM already knows in what order the processor will perform a read or write command for data of a specific size at a specific address in the memory, the data can be arranged in the above order.

According to the example shown in FIG. 23 , each of data A to data K is stored using sequential addresses. As described above, since data is continuously stored based on the ANN DL, it may be possible to operate in a burst mode at least for each ANN DL unit. In addition, according to examples of the present disclosure, since adjacent ANN DLs may also have sequential addresses, a burst mode operation in units of a plurality of ANN DLs is also possible. can be saved. Therefore, the SAM can be operated in burst mode. The sequential address may mean that addresses of rows and columns of the memory cell array are sequentially increased.

Accordingly, each data may be read in the burst mode, and data stored with consecutive addresses may also be read in the burst mode.

Preferably, all of the ANN DLs #1 to #15 may operate in the burst mode, but the present disclosure is not limited thereto, and data of at least some ANN DL units may be configured to operate in the burst mode.

Hereinafter, a procedure after the memory address map is set based on the ANN DL information will be described.

ANN DL #1: Processor or SAM controller requests SAM to read A data in read-burst mode.

In the case of ANN DL #1, since data A is sequentially stored, the SAM can be operated in read-burst mode while all data A is read.

ANN DL #2: The processor or SAM controller requests the SAM to read B data in read-burst mode.

In the case of ANN DL #2, since data is sequentially stored, it may be operated in a read-burst mode while all data B is read.

Since A and B data are sequentially stored in the memory map, A and B data, that is, data of continuous ANN DL units may be operated in read-burst mode.

ANN DL #3: The processor or SAM controller requests the SAM to write the output feature map (OFMAP) C data in write-burst mode.

Since C data has a memory address that follows B data, it can be written into the SAM according to the write-burst mode.

ANN DL #4: The processor or SAM controller requests the SAM to read the C data, which is the input feature map (IFMAP), again in read-burst mode.

ANN DL #3 and ANN DL #4 will be further described. C data, which is an output feature map of the first layer, is reused as an input feature map in the second layer.

As described above, since the artificial neural network model can request to read again immediately after writing the same data, the refresh operation of the memory cell that may occur during write and read operations can be omitted based on the ANN DL. Accordingly, the time required for C data refresh and power consumption can be reduced.

In addition, the feature map of a specific artificial neural network model may have a feature that is no longer reused when the ANN DL sequence passes. Therefore, since such a feature map does not need to maintain data any longer, an error may not occur in artificial neural network operation even if data is damaged by not refreshing.

ANN DL #5: The processor and/or SAM controller requests the SAM to read D data in read-burst mode.

Since D data has an address following that of C data, it can be read continuously in read-burst mode.

ANN DL #6: The processor and/or SAM controller requests the SAM to write E data in write-burst mode.

Since E data has an address following that of D data, it can be successively written into the SAM according to the write-burst mode.

ANN DL #7: Processor and/or SAM controller requests SAM to read E data back in read-burst mode.

ANN DL #6 and ANN DL #7 will be further described. Data E, which is an output feature map (OFMAP) of the second layer, is reused as an input feature map (IFMAP) in the third layer. In this way, since the artificial neural network model can request to read again immediately after writing the same data, the refresh operation performed on the memory cell that may occur between writing and reading can be omitted based on the ANN DL. there is. Accordingly, the time required to refresh the E data and power consumption can be reduced.

In addition, the feature map of a specific artificial neural network model may not be reused any more when the ANN DL sequence passes due to the nature of the artificial neural network model. Therefore, since the feature map does not need to maintain data any longer, an error may not occur in the artificial neural network operation even if the memory cell is not refreshed.

24 is a table showing a second example of accessing a memory according to order information in artificial neural network data locality (ANN DL) information.

In the second example shown in FIG. 24 , the ANN DL may be configured to read the kernel before the input feature map when the artificial neural network model is Mobilenet V1.0. To this end, the order information in the ANN DL information may include an order of reading the kernel before the input feature map.

This second example may be more effective as the data size of the feature map increases.

After the kernel is read from SAM, CONVOLUTION can be done in the order in which the input feature maps (IFMAPs) are received.

Specifically, the ANN DL information according to the second example may include information on an order pattern i) first read the kernel, ii) read the corresponding input feature map, and 3) write the output feature map to the memory.

The SAM, which is the main memory, may control the CAS signal and/or the RAS signal according to the ANN DL information to operate in a burst mode.

The data operation request order of the PROCESSOR is based on the preset ANN DATA LOCALITY order.

The SAM control unit may control the CAS signal and/or the RAS signal of the SAM based on the compiled ANN DL information to sequentially align data requested by the processor or NPU, and then sequentially operate in burst mode.

Based on the compiled ANN DL information, the SAM control unit may perform optimization such that the SAM operates in the burst mode by successively arranging data requested by the processor or NPU.

Compared with the first example of FIG. 23 , the memory address map of the second example may be different from the memory address map of the first example even if the same artificial neural network model is processed.

25 shows an example in which the SAM sets a memory address map according to the table shown in FIG. 24 .

The SAM, which is the main memory, may control the CAS signal and/or the RAS signal based on the compiled ANN DL information, so that data requested by the processor are arranged so that they are contiguous in the memory address map.

The SAM control unit may control the CAS signal and/or the RAS signal of the SAM based on the compiled ANN DL information to continuously align data requested by the processor in the memory address map.

Since the SAM control unit knows whether the processor will transmit a read command or a write command of data of a specific size to a specific address in the memory, it can know in what order the data will be processed.

Referring to FIG. 25 , each of data A to K is stored according to sequential memory addresses. Since data is continuous in this way, the SAM can be operated in burst mode at least in units of ANN DLs. In addition, according to examples of the present disclosure, since adjacent ANN DLs may also have sequential addresses, a burst mode operation in units of a plurality of ANN DLs is also possible.

Since the data bits in the A data to K data are also stored sequentially, the memory can be operated in burst mode.

That is, bits constituting each data may be read or written in the burst mode, and since each data is continuous with each other, it may be read or written in the burst mode.

Hereinafter, some explanation of the inference steps that follow after setting the memory map based on the ANN DL,

i) The processor and/or the SAM controller requests the SAM to read data according to the read-burst mode.

Since data is sequentially stored, the read-burst mode may be performed while data A is read.

ii) The processor and/or the SAM controller requests the SAM to read B data according to the read-burst mode.

Since the data are sequentially stored, the read-burst mode may be performed while the B data is read.

Since the A and B data are stored sequentially, the A and B data can be operated in read-burst mode. That is, continuous data of ANN DL may be operated in continuous read-burst mode.

iii) The processor and/or the SAM controller requests the SAM to write the C data, which is the output feature map, according to the write-burst mode.

Since C data has a memory address following that of B data, it can be written into memory according to the write-burst mode.

iv) The processor and/or the SAM controller requests the SAM to read D data according to the read-burst mode.

v) The processor and/or the SAM controller may request the SAM to read the C data again according to the read-burst mode. That is, the output feature map (OFMAP) of the previous layer may be used as the input feature map (IFMAP) in the next layer.

Since the processor and/or the SAM controller know in advance that it will request the SAM to read the C data after the D data, it selectively schedules operations such as pre-charge and/or refresh for the corresponding memory cell in advance. can do.

vi) The processor and/or the SAM controller may request the SAM to write E data according to the write-burst mode.

Since the processor and/or the SAM controller know in advance that it will request the SAM to write the E data after the C data, it selectively schedules operations such as pre-charge and/or refresh for the corresponding memory cell in advance. can do.

vii) The processor and/or the SAM controller may request the SAM to read the F data according to the read-burst mode.

Since F data has an address following that of E data, it is possible to continuously operate in read-burst mode.

The processor and/or the SAM controller may predict to request the SAM to read back from ANN DL #8 the same data written to ANN DL #6. Therefore, based on the ANN DL, it is possible to predict or calculate when the data access request corresponding to ANN DL #6 and ANN DL #8 will be performed. For the prediction or calculation, it is also possible to utilize information such as the clock speed of the processor, the size of E data corresponding to the ANN DL # 6 & ANN DL # 8, and the bandwidth of the memory bus. Accordingly, the SAM control unit or the SAM may omit a task such as pre-charge and/or refresh of the corresponding memory cell or schedule it with an optimal timing.

26 is a table showing a third example of accessing a memory according to order information in artificial neural network data locality (ANN DL) information.

In the third example illustrated in FIG. 26 , a specific region in the memory may be set as a common region for the input feature map and the output feature map. That is, the SAM and/or the SAM control unit may be configured to classify a region of the SAM based on a specific domain.

Since the input feature map and/or the output feature map may be data that cannot be reused once used, they may be alternately recorded in the same area.

M_FMAP in the illustrated table indicates the size of the largest feature map among the plurality of input feature maps and the plurality of output feature maps. Since the size of the feature maps for each layer is different, if the maximum value of the feature maps of the artificial neural network model is set, problems such as overflow can be prevented.

All of the start addresses for reading or writing the feature map may be the same, and the end address may be changed according to the actual size of the corresponding feature map.

In the third example, since a specific area of the memory is used in common, the following condition must be satisfied.

M_FMAP >= C, E, G, I, K (partially omitted from the figure, but the example ANN DL information for Mobilenet V1.0 includes 84 steps, the maximum value among all feature maps)

In memory, kernels may be stored sequentially.

27A and 27B show examples of setting a memory address map according to artificial neural network data locality information.

The SAM control unit may control the CAS signal and/or the RAS signal of the SAM based on the compiled ANN DL information to sequentially align data requested by the processor (eg, NPU) in the memory address map.

27A and 27B, since the SAM knows that the processor (eg, NPU) will instruct the processor (eg, NPU) to read or write a specific size of data to or from a specific address in memory, it knows in what order the data will be processed. Able to know.

The feature map may be recorded in an overwrite format in a common area in the memory, and kernels are stored using sequentially ordered memory addresses. Accordingly, continuous data can be read or written according to the burst mode.

The first to third examples described so far will be briefly described as follows.

Referring to the first to third examples, a memory address map may be set according to ANN DL information. The memory address map is set according to the above-described various conditions, performance algorithms, structures of the ANN model, and the like. Furthermore, the SAM may set a memory address map based on the ANN DL information so that data can be read or written in burst-mode according to the ANN DL information.

According to the first to third examples, the kernel can achieve performance improvement due to the sequential nature of the SAM.

On the other hand, the feature map is repeated in order from a write operation to a read operation.

When the main memory adopts the memory cell structure of DRAM, due to the inherent characteristics of DRAM, once data is read from within the memory cell, the charges charged in the capacitor in the memory cell are discharged, resulting in data loss. Therefore, a restore operation for recharging the electric charge must be performed.

The first example is for the most sequential pattern, but since the input feature map IFMAP is first read from the memory, the convolution operation can be performed after the kernel KERNEL is read.

In the second example, since the kernel is first read from the memory, the convolution operation can be started after the input feature map (IFMAP) is read. From the viewpoint of performing convolution, the second example can be said to be excellent.

In the third example, it may be effective when the capacity of the main memory is small. Alternatively, it may be effective when the feature map and the kernel are separated, such as when the main memory is a memory having two channels.

Since the DRAM bus is generally a single channel, data can be sent and received in the manner according to the first to third examples. It is also possible to implement SAM separately.

However, the examples of the present disclosure are not limited thereto, and the examples described in FIGS. 22 to 27 may be set differently for each layer of the artificial neural network model according to the size of the feature map and the kernel of each layer.

28 is a conceptual diagram illustrating a control signal of a SAM controller.

28 shows the memory, the SAM controller and the processor. Signals transmitted from the processor may not be transmitted through individual physical wires, but may be logical signals transmitted through one or more wires. However, the present disclosure is not limited thereto.

The SAM controller may include an internal memory that stores ANN DL information.

The ANN DL information may include information compiled to suit a processor (eg, an NPU).

Read/write command: Refers to a read command signal or a write command signal delivered according to the sequence information in the ANN DL information. A memory address corresponding to each command signal may be transmitted together with a start address and an end address of the memory or count information.

Sideband signal (SIDEBAND SIGNAL): Various control signals for increasing processing efficiency according to ANN DL information may be selectively included as needed.

RESET signal: When the ANN model is changed, it can be used to initialize (RESET) the memory address map.

ENABLE signal: When the ENABLE signal is ON, data can be transferred to the processor.

ANN DL information and SIDEBAND SIGNAL may have some overlapping signals. However, the ANN DL information may be static information according to the artificial neural network structure, and the sideband signal may be a dynamic control signal for ANN operation.

29 is an exemplary diagram illustrating an example of setting a memory address map according to a sideband signal shown in FIG. 28 .

As shown in FIG. 29 , a memory address map may be configured to process a plurality of ANN models.

When the processor (eg, NPU) performs time division operations in the # order of the ANN MODEL, the memory address map may be sequentially set in the # order of the ANN MODEL. Therefore, when the ANN model is changed, it can be operated in burst mode. A memory address map for each ANN model may be set according to the first to third examples described above.

30A shows another example of setting a memory address map according to a sideband signal, and FIG. 30B shows an example of a memory address map in which only a kernel is sequentially recorded.

As shown in FIG. 30A , multiple threads THREAD may be set for a specific ANN model (eg, ANN MODEL #1). That is, the memory address map can be set to process the ANN model in multi-threads when accessing multiple users.

By using multiple threads, multiple users can jointly use the kernel of one ANN model. Each thread may be allocated a memory address for storing an input feature map and/or an output feature map.

Alternatively, when using multiple threads, only the kernel may be sequentially mapped into the memory address map as shown in FIG. 30B . M_FMAP may be additionally created as many as the number of threads.

That is, the kernel can be used jointly regardless of the number of users, and the feature map may be configured to increase in proportion to the number of users.

31A is an exemplary diagram illustrating a 'READ_DISCARD' command transmitted through a sideband signal according to an example presented in this specification, and FIG. 31B illustrates an example of the READ command.

t _RAS shown in FIG. 31b means Data sense(t _RCD ) + Data restored to DRAM cells time.

Data sense(t _RCD ) time means the time for latching data to the sense amplifier. For the latching operation, precharge, access, and sense operations may be required. For the above operations, reference may be made to FIGS. 32, 33 and 34 .

Also, reference may be made to FIG. 17 for the description of this example.

According to an example presented herein, the 'READ_DISCARD' command may be executed only during t _RCD , and data restored to DRAM cells may not be executed.

Therefore, the latency and power required for data restored to DRAM cells can be reduced. For example, after writing C data to the memory according to ANN DL #3 in the ANN DL information of FIG. 25, and after reading and using the C data from the memory in ANN DL #5, the C data is ANN DL There is no need to perform the data restored to DRAM cells operation because it will never be used again as a basis. That is, order information and/or domain information may be used to determine the operation.

For example, the READ_DISCARD command can be set to a specific ANN DL #.

For example, the output feature map (OFMAP) of ANN DL #3 can be reused as the input feature map (IFMAP) of the next layer, which is ANN DL #5. That is, the input feature map (IFMAP) can utilize the fact that it is not reused after convolution with the kernel (KERNEL).

That is, when reading the input feature map (IFMAP), the READ_DISCARD command can be set to the corresponding ANN DL #.

For example, if the output feature map OFMAP of the first layer is written to the memory, the corresponding data is used as the input feature map IFMAP of the second layer and read from the memory. However, since the input feature map (IFMAP) is not used again, even if the data is lost because data restored to DRAM cells is not performed, the ANN operation is not affected. Accordingly, according to an example presented in the present disclosure, the SAM controller may be configured to instruct the READ-DISCARD command to the memory.

This principle is shown in Figure 31a. Data sense (t _RCD ) shown in FIG. 31A is a time when sense AMP reads a value stored in memory cells of a specific row.

In more detail, the 'READ_DISCARD' command may be executed in units of rows of memory.

In the 'Data restored to DRAM cells', since the data stored in the memory cell was lost by performing a read operation using the 'sense AMP (amplifier)', the latched data using the 'sense AMP' is again transferred to the memory cell. It means the operation to restore again.

32 shows a part of a circuit diagram of an exemplary SAM implemented in the form of a memory cell of a DRAM according to an example presented herein.

The circuit diagram of the SAM shown in FIG. 32 includes the Sense AMP. The SENSE AMP of SAM amplifies the difference between the reference voltage (Vref) supplied to the bit line and the voltage on the bit line to generate a digital signal of 0 or 1.

The SAM's SENSE AMP can selectively store charge in the discharged memory cell through the bit line. When the READ command is executed, RESTORE is also executed. In the case of a READ-DISCARD command, RESTORE may not be performed.

SAM's SENSE AMP provides a buffer memory function that latches the sensed voltage.

Here, the capacitor of the memory cell may have a leakage current characteristic.

FIG. 33 is an exemplary diagram for explaining a precharge operation in the SAM circuit diagram shown in FIG. 32 .

에 인가된다. 따라서 Bitline 및

이 Vref 전압을 가진다. 또한 VOLTAGE EQ. CIRCUIT이 Bitline 및

를 단락(short)시켜서 양쪽 라인의 전압이 동일하게 된다. In the pre-charge section, the EQ (equalizing) signal is supplied, and the VOLTAGE EQ. CIRCUIT is activated. When the EQ signal is supplied, the voltage Vref = Vcc/2 passes through each Tr to Bitline and

is authorized to Therefore, Bitline and

It has this Vref voltage. Also VOLTAGE EQ. CIRCUIT is Bitline and

is shorted so that the voltage on both lines is the same.

FIG. 34 is an exemplary diagram for explaining a memory cell access operation in the SAM circuit diagram shown in FIG. 32 .

In the memory cell access period, the bit line may be charged in the following order.

i) Over-drive a wordline corresponding to a precharged bitline to be accessed with a voltage of Vcc + Vt as shown in the illustrated thick wordline.

A voltage of Vcc + Vt is supplied to the word line to turn on the transistor Tr of the memory cell to be accessed. CSL Tr is turned off and data output is blocked.

ii) As shown in the illustrated thick bit line, when the value stored in Cap of Tr of the memory cell is 1, the capacitor Cap is discharged and the voltage of the bit line rises from Vref to Vref+. At this time, the voltage at Cap decreases and the stored data is lost

FIG. 35 is an exemplary diagram for explaining a data detection (DATA SENSE) operation in the SAM circuit diagram shown in FIG. 32 .

In the detection (SENSE) section, the sensing circuit charges the voltage of the bit line in the following order.

i) As shown by the thick solid line shown, when the bit line becomes Vref+ voltage, the lower left transistor tr of the detection circuit is turned on.

에 인가된다. Therefore, the SAN, which is the ground (GND) voltage, is

is authorized to

은 GND 전압이 된다. 따라서 우측 상단의 Tr이 on 된다. thus

is the GND voltage. Therefore, Tr in the upper right corner is on.

=0V 이 인가된다. ii) As shown by the thick dotted line shown, when the upper right Tr of the detection circuit is turned on, the SAP (PFet sense amplifier), which is the Vcc voltage, is applied to the bitline. In addition, the voltage Vcc from the voltage Vref+ is applied to Tr at the lower left. So Bitline=3V,

=0V is applied.

iii) Now we are ready to read data from DRAM. When the CSL signal is supplied, the output of the SENSE AMP can be generated.

FIG. 36 is an exemplary diagram for explaining a READ-DISCARD operation in the SAM circuit diagram shown in FIG. 32 .

In the READ-DISCARD section, the CSL (Column Select Line) signal is supplied, and the SENSE AMP outputs data accordingly.

According to the example presented herein, after the data stored in the memory cell is read, the power consumption and the door-to-door time can be reduced by not performing a restore for recharging the charge in the corresponding memory cell. there is. For example, it is applicable when storing the output feature map and then using the output feature map as the input feature map of the next layer.

Referring to FIG. 31A and FIG. 31B by comparison, memory latency can be reduced by t _RAS time.

Meanwhile, in order to prevent charge from being charged in the capacitor Cap, the overdrive (VCC + Vt) word line may be turned off. Therefore, power can be reduced.

37 is an exemplary diagram for explaining a READ operation in the SAM circuit diagram shown in FIG. 32 .

In the READ section, the CSL signal is supplied and the SENSE AMP outputs data.

According to the example presented in this specification, after data stored in the memory cell is read, restoration of recharging the electric charge in the corresponding memory cell may be performed.

Referring to Figure 31b, it takes t _RAS time for restoration.

Keeping the wordline active allows the fully driven bitline voltage to restore the charge on the capacitor through the access transistor. Therefore, power consumption according to the restoration is required.

38A is an exemplary waveform diagram of a READ-DISCARD operation, and FIG. 38B is an exemplary waveform diagram of a READ operation.

In contrast to FIG. 38B , referring to FIG. 38A , since there is no RESTORE process, the t _RAS time may be shortened. 31A and 31B may be referred to in order to understand the corresponding operation.

FIG. 39 is a table showing a part of the table shown in FIG. 21 in order to explain the REFREASH operation.

The table shown in FIG. 39 is for conceptually explaining the time it takes to perform one-time inference using the ANN model.

The time required for each ANN DL # unit can be measured, calculated or predicted based on the processing speed of the processor, the bandwidth of the data bus, and the operating speed of the memory.

The SAM controller may control whether to REFRESH data in a specific area (DOMAIN) of the memory based on the ANN DL information. The SAM controller may measure inference time (IT) based on the ANN DL information. For example, the repetition time of the same ANN DL # in the ANN DL information may be measured. That is, after performing the operation for ANN DL #1, it is possible to measure the time at which operation #1 returns again. As another example, it is possible to measure the time it takes to perform until the start # and the end # in the ANN DL information. That is, from #1 to #84 in the ANN DL information, the time taken to perform the operation may be measured. As another example, the processing time may be measured by setting a specific period in the ANN DL information.

If the SAM controller determines that one inference has been completed within the threshold time, memory refresh may be disabled. For example, if the inference is completed once within the threshold time according to the ANN DL information, the refresh of the memory area in which the kernel is stored may be disabled.

The reason for doing this is as follows. When one inference is completed according to the ANN DL information, “read” or “write” of all kernels of the ANN model is completed once in memory. In the case of the memory cell structure of DRAM, “read” is substantially the same as REFRESH, so data can be preserved even if REFRESH is not repeatedly performed. However, when speculation is interrupted in the middle or a threshold time is exceeded, the SAM controller may refresh only the memory cell (eg, Row) in which the kernel is stored. For another example, if one inference is completed within a threshold time according to the ANN DL information, the SAM controller may disable REFRESH for only the memory area in which the feature map is stored. The reason for doing this is as follows. Since the feature map cannot be reused, it is insensitive to data loss, so it is okay to DISALBE REFRESH. As another example, when a READ-DISCARD operation is performed, it may be effective to disable REFRESH because data has already been lost.

In more detail, since KERNEL may be a fixed value, it may be periodically REFRESHed. However, as described above, KERNEL can also disable REFERESH if it is possible to predict that reading will be repeated within a critical time based on the ANN DL.

However, the examples of the present disclosure are not limited thereto, and the READ, WRITE, and READ-DISCARD commands of the SAM may be appropriately selected based on the ANN DL, considering the characteristics of data, processing time, reuse or the like.

Meanwhile, the aforementioned threshold time may be exemplarily set to REFRESH THRESHOLD TIME, RT _th = 32ms to 64ms. The recommended time to prevent data loss may vary depending on the capacity of the memory capacitor and the amount of leakage current.

Also, when IT < RT _th is satisfied, the memory cells in which the feature map is stored may not receive a refresh command.

Data REHRESH policy can be set differently in ANN DL information.

For example, in the case of the kernel, a REHRESH policy may be set to increase the data protection level, and in the case of a feature map, the REHRESH policy may be set to decrease the data protection level.

By disabling REFRESH based on the ANN DL information, it is possible to reduce memory operation delay and reduce power consumption.

When the artificial neural network model is distributed and stored in a plurality of banks in the memory, based on the ANN DL information, the precharge timing of the banks can be controlled respectively.

40 shows an example in which a SAM memory is implemented in various forms according to an example presented herein.

The SAM memory may be implemented in various forms according to application fields.

The memory bus, which is a data transfer path between caches in the processor, may be implemented as a single channel or a dual channel. When the number of channels increases, power consumption increases, but there is an advantage in that the bandwidth can be improved by managing the kernel and/or the feature map, respectively. When two channels are used, the bandwidth is doubled compared to when one channel is used, so more data can be delivered to the cache in the processor. The operations may be controlled based on the ANN DL.

A plurality of SAM memories may be grouped and driven in a “rank”.

Each SAM memory includes a "bank", which is a collection of independently operating memory arrays. For example, one bank may include eight memory arrays. Interleaving multiple memory banks can use low bandwidth devices to implement high bandwidth memory buses. Each memory array may include a row decoder, a column decoder, a sense amplifier, and an input/output buffer. “Row” refers to a row of a memory array. "Column" means a column of a memory array.

41 is an exemplary diagram illustrating an example of a method of mapping an address of a main memory based on ANN data locality information.

Referring to FIG. 41, the basic structure of the SAM is shown. The SAM includes a plurality of memory cells in a matrix structure having addresses of rows and columns. The SAM may be implemented as, for example, DRAM. However, examples of the present disclosure are not limited thereto.

A sense amplifier is disposed at lower ends of the plurality of memory cells of the matrix structure. The row address decoder selects a specific row. RAS latency is required to perform the corresponding operation. Data of the memory cells of the selected row are latched in the sense amplifier. The column address decoder selects necessary data from the data latched in the sense amplifier and transmits it to the data buffer. CAS Latency is required to perform the corresponding operation. The structure may be referred to as a bank of DRAM. A DRAM may include a plurality of banks.

At this time, when the DRAM operates in the burst mode, data is read or written while the addresses of the memory cells are sequentially increased. Therefore, compared to the case of reading fragmented address data, RAS latency and CAS latency are minimized.

To elaborate, even if the AMC or NPU instructs the burst mode to the main memory, if the data stored in the DRAM is actually fragmented, the RAS latency and CAS latency corresponding to the fragmentation occur. Therefore, it is difficult to actually reduce RAS latency and CAS latency by simply executing the burst mode command.

Conversely, in the case of SRAM, data fragmentation does not actually cause latency. Therefore, in the buffer memory or internal memory composed of SRAM, latency generation due to data fragmentation may not be fatal.

Referring to FIG. 41 , based on the ANN data locality information (ANN DL), the memory map may be set in consideration of the order and size of data requested by the NPU to the memory cells of the DRAM. The memory map may be set based on a start address and an end address based on each data size. Therefore, if memory operations are performed in the order of ANN data locality information (ANN DL) in SAM, all memory operations may be operated in burst mode.

Accordingly, the main memory shown in FIG. 41 can be controlled based on the memory address and operation mode shown in Table 1.

Layer start address end address operation mode domain ANN DL Size (Byte) One 0 A=A' Read-Burst IFMAP One A One A'+1 A+1+B=B' Read-Burst Kernel 2 B One B'+1 B'+1+C=C' Write-Burst OFMAP 3 C 2 B'+1 B'+1+C=C' Read-Burst IFMAP 4 C 2 C'+1 C'+1+D=D' Read-Burst Kernel 5 D 2 D'+1 D'+1+E=E' Write-Burst OFMAP 6 E 3 D'+1 D'+1+E=E' Read-Burst IFMAP 7 E 3 E'+1 E'+1+F=F' Read-Burst Kernel 8 F 3 F'+1 F'+1+G=G' Write-Burst OFMAP 9 G 4 F'+1 F'+1+G=G' Read-Burst IFMAP 10 G 4 G'+1 G'+1+H=H' Read-Burst Kernel 11 H 4 H'+1 H'+1+I=I' Write-Burst OFMAP 12 I 5 H'+1 H'+1+I=I' Read-Burst IFMAP 13 I 5 I'+1 I'+1+J=J' Read-Burst Kernel 14 J 5 J'+1 J'+1+K=K' Write-Burst OFMAP 15 K

In more detail, for the domain of Table 1, it is also possible to utilize the domain information described with reference to FIG. 12 . In more detail, the operation mode of Table 1 may utilize the operation mode information described with reference to FIG. 12 .

Since the data is mapped to sequential addresses according to the ANN data locality information (ANN DL), the data can be processed with a burst mode command.

That is, the AMC can cache the necessary data before the NPU makes a request based on the ANN data locality information (ANN DL), and can determine the order of all requests. Therefore, the cache hit probability of the buffer memory of the AMC can theoretically be 100%.

Also, since the memory map of the main memory is set based on the ANN data locality information (ANN DL), it is also possible for all memory operations to operate in the burst mode.

Although a single memory bank is exemplarily shown in FIG. 29 , address mapping may be performed in a bank interleaving method according to the configuration of a bank, a rank, and a channel of the memory.

If there is no ANN data locality information (ANN DL), it is practically impossible to sequentially store data requested by the NPU in DRAM. That is, even if there is general artificial neural network model information, if there is no ANN data locality information (ANN DL) described in various examples, it is impossible to know all the orders of data operations that the NPU will request from the main memory.

If the AMC does not have the ANN data locality information (ANN DL), it is difficult for the AMC to know whether the NPU will first request the kernel of the first layer of the artificial neural network model or the input feature map first. Accordingly, it becomes substantially difficult to set the memory map considering the burst mode in the main memory.

42 is an exemplary diagram illustrating another example of a method of mapping an address of a main memory based on ANN data locality information.

Since the structure of the main memory shown in FIG. 42 is substantially the same as that of the main memory shown in FIG. 41 , a redundant description will be omitted.

Referring to FIG. 42 , based on the ANN data locality information (ANN DL), the memory map may be set in consideration of the order and size of data requested by the NPU to the memory cells of the DRAM. The memory map may be set based on a start address and an end address based on each data size. Accordingly, if memory operations are performed in the order of the ANN data locality information (ANN DL) in the DRAM, all memory operations may be operable in the burst mode.

Accordingly, the main memory shown in FIG. 42 can be controlled based on the memory address and operation mode shown in Table 2.

The ANN data locality information (ANN DL) corresponding to FIG. 42 and Table 2 is an example of a case in which the NPU is set to use the input feature map and the output feature map in common.

layer name start address end address operation mode domain ANN DL size
(Byte) One 0 M_FMAP=A' Read-Burst IFMAP One M_FMAP One A'+1 A'+1+B=B' Read-Burst Kernel 2 B One 0 C Write-Burst OFMAP 3 C 2 0 C Read-Burst IFMAP 4 C 2 B'+1 B'+1+D=D' Read-Burst Kernel 5 D 2 0 E Write-Burst OFMAP 6 E 3 0 E Read-Burst IFMAP 7 E 3 D'+1 D'+1+F=F' Read-Burst Kernel 8 F 3 0 G Write-Burst OFMAP 9 G 4 0 G Read-Burst IFMAP 10 G 4 F'+1 F'+1+H=H' Read-Burst Kernel 11 H 4 0 I Write-Burst OFMAP 12 I 5 0 I Read-Burst IFMAP 13 I 5 H'+1 H'+1+J=J' Read-Burst Kernel 14 J 5 0 K Write-Burst OFMAP 15 K

The value of the kernel is fixed when training of the artificial neural network model is completed. Therefore, the value of the kernel has a fixed characteristic. On the other hand, since the input feature map and the output feature map are inputs of image data, camera, microphone, radar, lidar, etc., once used, they may not be reused any more.

Referring to FIG. 20 as an example, the sizes of the input feature map and the output feature map of the artificial neural network model are defined. Therefore, it is possible to select the largest data size (M_FMAP) among the input feature map and the output feature map of the artificial neural network model. In the case of the artificial neural network model of FIG. 20, the feature map (M_FMAP) of the maximum size is 802,816 bytes. Therefore, the input feature maps and output feature maps of each layer of the artificial neural network model in Table 2 are set to have the same start address. That is, the input feature map and the output feature map may operate in the form of overwriting the same memory address. As described above, due to the characteristics of the artificial neural network model, when the input feature map and the kernel are convolutional, an output feature map is generated, and the output feature map becomes the input feature map of the next layer. Therefore, the feature map of the previous layer is not reused and may be deleted.

According to the above configuration, the size of the memory map of the main memory can be reduced by setting the memory area set based on the maximum feature map as the shared area of the input feature map and the output feature map.

43 is a table showing an example of accessing a memory according to order information in artificial neural network data locality (ANN DL) information.

The SAM controller may control the burst length (BURST LENGTH) based on the ANN DL information. The SAM controller can efficiently control the BURST-MODE according to the ANN DL information by using the BURST-TERMINATE command. The burst length can be selected according to the AXI interface.

1) In case of ANN DL #1, it has a start 0 end A' address. Therefore, the READ-BURST end command indicated by the SAM controller may correspond to the data size A.

2) In case of ANN DL #2, it has a start A'+1 an end B' address. Therefore, the READ-BURST end command indicated by the SAM controller may correspond to the data size B.

3) In case of ANN DL #3, it has a start B'+1 end C' address. Therefore, the WRITE-BURST end command indicated by the SAM controller may correspond to the data size C.

4) In case of ANN DL #4, it has a start B'+1 end C' address. Therefore, the READ-BURST end command indicated by the SAM controller may correspond to the data size C.

5) In case of ANN DL #5, it has start C'+1 end D' address. Therefore, the READ-BURST end command indicated by the SAM controller may correspond to the data size D.

Examples of the present disclosure are not limited to the contents described above, and the burst length is programmable in the following ways.

a. Use a short fixed burst length.

b. Explicitly identifies the burst length of a read or write command.

c. The burst length is programmed using the DRAM's fuse. (laser programmable fuses, electrically programmable fuses)

d. Use a long, fixed burst length with the burst end command

e. It uses a BEDO style protocol where each CAS/pulse toggles one column of data. (Burst-mode extended data out; BEDO DRAM)

44 is an exemplary diagram illustrating an example of a memory in which a SAM controller is embedded.

The illustrated memory is an improved dedicated memory for an artificial neural network, and the memory may have a SAM controller embedded therein. That is, DSAM may refer to a SAM implemented based on DRAM.

45 is an exemplary diagram illustrating an architecture including a compiler.

The compiler converts the artificial neural network model into machine code that can be run on the NPU.

The compiler may include a front end and a backend. An intermediate representation (IR) may exist between the front end and the rear end. These IRs are abstract concepts of programs and are used for program optimization. The artificial neural network model can be converted to various levels of IR.

A high-level IR may exist on the front side of the compiler. The front end of the compiler receives information about the artificial neural network model. For example, the information on the artificial neural network model may be the information exemplified in FIG. 23 . The front end of the compiler may perform hardware-independent conversion and optimization.

The high-level IR is at the graph level and can optimize computation and control flow. The low-level IR may be located at the end of the compiler.

The back end of the compiler can convert the high-level IR to the low-level IR. The rear end of the compiler performs NPU optimization, CODE generation, and compilation tasks.

The compiler rear end may perform optimization tasks such as hardware intrinsic mapping and memory-allocation.

ANN data locality information may be generated or defined in the lower-level IR.

The ANN data locality information may include all memory operation order information requested by the NPU from the main memory. Therefore, the AMC can know the order of all memory operations that the NPU will request. As described above, the ANN data locality information may be generated by the compiler, or the AMC may be generated by analyzing the repetition pattern of the memory operation requested by the NPU from the main memory.

ANN data locality information may be generated in the form of a register map or a lookup table.

After the ANN data locality information (ANN DL) is analyzed or provided, the compiler may generate a caching schedule of the AMC and/or NPU based on the ANN DL. The caching schedule may include a caching schedule of an on-chip memory of the NPU and/or a caching schedule of a buffer memory of the AMC.

Meanwhile, the compiler may compile an artificial neural network model reflecting optimization algorithms (eg, Quantization, Pruning, Retraining, Layer fusion, Model Compression, Transfer Learning, AI Based Model Optimization, and Other Model Optimization).

In addition, the compiler may generate ANN data locality information of an artificial neural network model optimized for NPU. The ANN data locality information may be separately provided to the AMC, and it is also possible for the NPU and the AMC to receive the same ANN data locality information, respectively. Also, as described above with reference to FIG. 14 , there may be at least one AMC.

The ANN data locality information may include an operation sequence configured as a memory operation request unit of the NPU, a data domain, a data size, and a memory map configured for sequential addressing.

The illustrated scheduler in the NPU may perform an artificial neural network operation by receiving machine code in a binary form from the compiler.

The compiler provides sequentially aligned memory address map information of the main memory to the DMA, which is an artificial neural network memory controller (ANN Memory Controller, AMC), and the AMC is based on the sequential memory address map. The artificial neural network model data in the main memory can be arranged or rearranged. The AMC may perform an operation of rearranging data in the main memory during initialization of the NPU or runtime.

In this case, the AMC may optimize the read-burst operation in performing the arrangement or realignment. The arrangement or rearrangement may be performed when the NPU operation is initialized. In addition, the arrangement or rearrangement may be performed upon detection of a change in the ANN DL. These functions may be independently performed in the AMC during the operation of the NPU regardless of the compiler.

The AMC and the NPU may receive or provide ANN data locality information to each other. That is, the compiler may provide the ANN data locality information to the AMC and the NPU. The AMC may receive real-time information on the operation step of the ANN data locality information being processed by the NPU. In addition, the AMC may synchronize the ANN data locality information with the NPU.

If the current NPU is processing the data corresponding to the ANN data locality information token #N, the AMC predicts that the data corresponding to the data locality information token #(N+1) will be requested from the NPU, In consideration of the delay, data corresponding to the ANN data locality information token #(N+1) is requested from the main memory. The corresponding operation may be independently performed by the AMC before the memory operation request of the NPU.

The compiler may generate a caching policy to store data necessary for a prediction operation according to ANN data locality in a buffer memory in the AMC. The compiler caches as much data as possible before the NPU requests it according to the buffer size of the DMA.

For example, the compiler provides AMC with a caching policy to cache as many ANN data locality information tokens as #(N+M). Here, M may be an integer value that satisfies the case where the data size of the ANN data locality information tokens #(N+1) to #(N+M) is equal to or smaller than the cache capacity of the AMC.

When the remaining capacity of the cache memory of the AMC is larger than the data size of the ANN data locality information token #(N+M+1), the compiler stores the data corresponding to the ANN data locality information token #(N) in the ANN data locality Can store information token #(N+M+1) data.

In more detail, the caching may be independently performed by the AMC without an instruction from the NPU based on the ANN DL stored in the ANN data locality information management unit of the AMC.

The compiler may provide a model lightweight function. The compiler can further optimize and lighten the deep learning model to fit the corresponding NPU architecture.

46 shows an architecture according to a first example.

Referring to FIG. 46 , an NPU, an artificial neural network memory controller (AMC), and a main memory that is an external memory are shown. In some cases, the main memory may be referred to as an external memory.

For convenience of description below, the artificial neural network memory controller of various examples of the present disclosure may be referred to as an AMC.

The NPU may include an NPU scheduler, an internal memory, and a PE array.

The PE array of various examples of this disclosure includes a plurality of processing elements. The plurality of processing elements may be individually drivable independently or drivable as a group. A PE array may be referred to as a plurality of processing elements.

The NPU may further include a Special Function Unit (SFU).

The PE array can perform operations for artificial neural networks. For example, when input data is input, the PE array may perform an operation of deriving an inference result through an artificial neural network.

The NPU scheduler is configured to control the operation of the PE array for the reasoning operation of the NPU and the read and write order of the NPU internal memory. To elaborate, the NPU scheduler may be configured to control the PE array and the NPU internal memory based on ANN (Artificial Neural Network) data locality information.

The NPU scheduler may analyze the structure of the artificial neural network model to be operated in the PE array or may be provided with the analyzed information. For example, the compiler of the NPU may be configured to analyze artificial neural network data locality. The data that the artificial neural network model may include includes at least an input feature map of each layer according to the locality of the artificial neural network data, kernel data, and an output feature map. Each layer may be selectively tiled according to the size of the layer and the size of the internal memory.

ANN data locality information may be stored in a memory provided inside the NPU scheduler or in the NPU internal memory. The NPU scheduler can access the main memory to read or write necessary data. In addition, the NPU scheduler may utilize ANN data locality information or information on the structure based on data such as a feature map and kernel data for each layer of the artificial neural network model. A kernel may also be referred to as a weight. The feature map may also be referred to as node data. For example, ANN data locality may be generated when designing an artificial neural network model, completing training, or compiling. The NPU scheduler may store ANN data locality information in the form of a register map. However, the present invention is not limited thereto.

The NPU scheduler may schedule the operation order of the artificial neural network model based on the locality information of the ANN data.

The NPU scheduler may acquire a memory address value in which the feature map and kernel data of each layer of the artificial neural network model are stored based on the locality information of the ANN data. For example, the NPU scheduler may obtain a memory address value in which a feature map and kernel data of a layer of an artificial neural network model stored in a memory are stored. Therefore, the NPU scheduler may prefetch at least a part of the feature map and kernel data of the layer of the artificial neural network model to be driven from the main memory, and then provide it to the NPU internal memory in a timely manner. The feature map of each layer may have a corresponding memory address value. Each kernel data may have a corresponding respective memory address value.

The NPU scheduler may schedule the operation order of the PE array based on ANN data locality information, for example, information about arrangement data or structure of layers of an artificial neural network of an artificial neural network model.

Since the NPU scheduler schedules operations based on ANN data locality information, it may operate differently from the general CPU scheduling concept. Scheduling of a general CPU operates to achieve the best efficiency by considering fairness, efficiency, stability, and response time. That is, it is scheduled to perform the most processing within the same time in consideration of priority and operation time.

Conventional CPUs use an algorithm for scheduling tasks in consideration of data such as priority order of each processing and operation processing time.

That is, general CPU scheduling is random and difficult to predict, so it is determined based on statistics, probability, and priorities. Conversely, since the artificial neural network operation order is predictable rather than random, more efficient scheduling is possible. In particular, since artificial neural network computation has a huge amount of data, the computational processing speed of artificial neural network can be significantly improved according to efficient scheduling.

The NPU scheduler may determine the operation order based on the ANN data locality information.

Further, the NPU scheduler may determine the operation order based on the ANN data locality information and/or the data locality information of the NPU to be used or information about the structure.

According to the structure of the artificial neural network model, calculations for each layer are sequentially performed. That is, when the structure of the artificial neural network model is determined, the operation order for each layer may be determined. The order of operations or data flow according to the structure of the artificial neural network model can be defined as the data locality of the artificial neural network model at the algorithm level.

The PE array (ie, a plurality of processing elements) refers to a configuration in which a plurality of PEs configured to calculate a feature map and kernel data of an artificial neural network are disposed. Each PE may include a multiply and accumulate (MAC) operator and/or an Arithmetic Logic Unit (ALU) operator. However, examples according to the present disclosure are not limited thereto.

On the other hand, the internal memory in the NPU may be a static memory. For example, the internal memory may be SRAM or a register. The internal memory may simultaneously process a read operation and a write operation. To this end, the AMC and the NPU may be connected through a dual-port communication interface. Alternatively, when the AMC and the NPU are connected through a one-port communication interface, a read operation and a write operation may be sequentially performed in a TDM manner.

The AMC may include an ANN data locality information management unit and a buffer memory.

The AMC may monitor the operation order information of the NPU through the ANN data locality information management unit.

The ANN data locality information management unit may order and manage the data to be provided to the PEs according to the operation order of the NPU. The buffer memory may temporarily store the data read from the main memory before providing the data to the NPU. Also, the buffer memory may temporarily store the output feature map provided from the NPU before transferring it to the main memory.

The AMC reads data to be requested by the NPU from the main memory before the NPU requests it, based on the ANN data locality information, and stores it in the buffer memory. The AMC immediately provides the corresponding data stored in the buffer memory when the NPU actually requests the corresponding data. Therefore, as the AMC is provided, the RAS latency and CAS latency that may be generated by the main memory can be substantially removed by monitoring the operation sequence of the artificial neural network model processed by the NPU.

The main memory may be a dynamic memory. For example, the main memory may be a SAM or DRAM. The main memory, which is the DRAM, and the AMC may be connected through a system bus (eg, an AXI interface). The system bus may be implemented as a one-port. In this case, the DRAM may not be able to simultaneously process a read operation and a write operation.

Meanwhile, the AMC may rearrange data in the main memory so that a read operation becomes a burst operation based on the ANN data locality information.

Accordingly, when the DRAM, which is the main memory, supplies data to the buffer memory in a burst operation, the buffer memory may stream the data to the NPU.

The buffer memory may be implemented in the form of a first input first output (FIFO). The AMC switches to a standby state when the buffer memory is full. When the buffer memory transfers data to the NPU, the AMC reads data from the main memory based on the ANN data locality information and stores the data in the buffer memory.

If the size of the buffer memory is small (eg, 1 KB), the buffer memory may perform only a caching role to reduce latency between the main memory and the NPU. In this case, a large amount of data may be transferred at once between the main memory and the NPU according to a burst operation. If the burst operation is performed well in this way, the bandwidth of the main memory may be substantially maximized.

As a modified example of FIG. 46 , the AMC may be embedded in the NPU, embedded in the main memory, or embedded in a system bus.

47 shows an architecture according to a second example.

Referring to FIG. 47 , an NPU, an AMC, and a main memory are shown. In the second example, duplicate descriptions described in other examples may be omitted for convenience of description. Configurations of other examples are selectively applicable to this example.

The NPU may include an NPU scheduler, a plurality of internal memories, and a PE array.

Unlike FIG. 46, the plurality of internal memories in the NPU shown in FIG. 47 include a first internal memory for kernel data, a second internal memory for an input feature map, and a third internal memory for an output feature map. can do. The first to third internal memories may be a plurality of regions allocated in one physical memory. Each internal memory may be provided with a port capable of communicating with the PE array, respectively. If each port is provided for each internal memory, the bandwidth of each internal memory can be guaranteed.

The size of each internal memory may be variably adjusted. For example, the total of each internal memory is 1 MByte, and the size of each internal memory may be divided in a ratio of A:B:C. For example, the size of each of the internal memories may be divided in a ratio of 1:2:3. The ratio of each internal memory may be adjusted according to the size of the input feature map, the size of the output feature map, and the size of the kernel data for each operation order of the artificial neural network model.

Unlike FIG. 46 , the AMC shown in FIG. 47 may include a direct memory access (DMA) controller.

The external main memory may be SMA or DRAM.

While the PE array in the NPU performs an operation for inference, the DMA controller independently reads data from the main memory based on the ANN data locality information and stores it in the buffer memory, even if there is no command from the NPU. can

The DMA controller reads data to be requested by the NPU based on the ANN data locality information from the main memory before the NPU requests it, and stores it in the buffer memory. The DMA controller directly provides the corresponding data stored in the buffer memory when the NPU actually requests the corresponding data. Accordingly, as the DMA controller is provided, RAS latency and CAS latency that may be generated by the main memory can be substantially eliminated.

48 shows an architecture according to a third example.

Referring to FIG. 48 , an NPU, an AMC, and a main memory are shown. In the third example, duplicate descriptions described in other examples may be omitted for convenience of description. Configurations of other examples are selectively applicable to this example.

The NPU may include an NPU scheduler, a plurality of internal memories, and a PE array.

Unlike FIG. 46, the plurality of internal memories in the NPU shown in FIG. 48 include a first internal memory for kernel data, a second internal memory for an input feature map, and a third internal memory for an output feature map. can do. The first to third internal memories may be a plurality of regions allocated in one physical memory.

Unlike FIG. 46 , the AMC shown in FIG. 48 may include an ANN data locality information management unit, a swap memory, and a buffer memory.

The external main memory may be a SAM or DRAM.

A swap memory in the AMC may be used to reorder data in the main memory.

In the main memory, data may be fragmented and stored at random addresses. However, when data is randomly stored in this way, in order to read data from the main memory, a non-sequential memory address must be used. In this case, column address strobe (CAS) latency and row address strobe (RAS) latency may occur frequently.

To solve this problem, the AMC may rearrange the data in the main memory based on the ANN data locality information. Specifically, the AMC temporarily stores at least a portion of the fragmented data in the main memory in the swap memory. Subsequently, the data stored in the main memory may be rearranged to enable a burst operation based on the ANN data locality information.

The data rearrangement operation may be performed only once during the initial operation. However, the present invention is not limited thereto. If the ANN data locality information is changed, the reordering operation may be performed again based on the changed ANN data locality information.

Meanwhile, as a modification, the AMC may allocate a swap area in the main memory without using the swap memory and then perform the data rearrangement.

49 shows an architecture according to a fourth example.

Referring to FIG. 49 , an NPU, an AMC, and a main memory are shown. In the fourth example, duplicate descriptions described in other examples may be omitted for convenience of description. Configurations of other examples are selectively applicable to this example.

The NPU may include an NPU scheduler, a plurality of internal memories, and a PE array.

Unlike FIG. 46, the plurality of internal memories in the NPU shown in FIG. 49 include a first internal memory for kernel data, a second internal memory for an input feature map, and a third internal memory for an output feature map. can do.

The AMC may include an ANN data locality information management unit and a plurality of buffer memories.

Unlike FIG. 46 , the plurality of buffer memories shown in FIG. 49 may include a first buffer memory for kernel data, a second buffer memory for an input feature map, and a third buffer memory for an output feature map. can The first to third buffer memories may be a plurality of regions allocated in one physical memory.

Each internal memory in the NPU may be connected to each buffer memory in the AMC. For example, the first internal memory may be directly connected to the first buffer memory, the second internal memory may be directly connected to the second buffer memory, and the third internal memory may be connected to the third buffer memory.

Each buffer memory may be provided with a port capable of communicating with each internal memory of the NPU, respectively.

The size of each buffer memory may be variably adjusted. For example, the total of each buffer memory is 1 MByte, and the size of each buffer memory may be divided in a ratio of A:B:C. For example, the size of each buffer memory may be divided in a ratio of 1:2:3. The ratio of each buffer memory may be adjusted according to the size of the input feature map, the size of the output feature map, and the size of the kernel data for each operation order of the artificial neural network model.

The AMC may individually store data for an operation operation of the NPU in each of the buffer memories based on the ANN data locality information.

On the other hand, as can be seen with reference to FIG. 23, when the artificial neural network model is based on Mobilenet V1.0, the size deviation of the kernel (ie, weight) for depth-wise convolution and/or point-wise convolution is quite large. can

Accordingly, the size of each internal memory may be adjusted based on the ANN data locality information. Similarly, the size of each buffer memory may be adjusted.

50 shows an architecture according to a fifth example.

Referring to FIG. 50 , an NPU, an AMC, and a main memory are shown. In the fifth example, duplicate descriptions described in other examples may be omitted for convenience of description. Configurations of other examples are selectively applicable to this example.

The NPU may include an NPU scheduler, a plurality of internal memories, and a PE array.

Unlike FIG. 46, the plurality of internal memories in the NPU shown in FIG. 50 include a first internal memory for kernel data, a second internal memory for an input feature map, and a third internal memory for an output feature map. can do.

The AMC may include an ANN data locality information management unit and a buffer memory.

As mentioned in other examples, data may be randomly fragmented in the main memory. However, when data is randomly stored in this way, to read data from the main memory, non-sequential memory addresses must be used, so CAS (Column Address Strobe) Latency and RAS (Row Address Strobe) Latency may occur. There is a possibility.

To solve this problem, the AMC may rearrange the data in the main memory based on the ANN data locality information. Specifically, the AMC temporarily stores at least a portion of the fragmented data in the main memory in the buffer memory. Subsequently, the data stored in the main memory may be rearranged to enable a burst operation based on the ANN data locality information.

Meanwhile, when data is rearranged, a memory address may be changed. Accordingly, the ANN data locality information management unit in the AMC and the NPU scheduler may communicate with each other. Specifically, the ANN data locality information management unit stores the updated memory address after the data rearrangement. Then, the ANN data locality information management unit may update the existing memory address stored in the NPU scheduler.

51 shows an architecture according to a sixth example.

Referring to FIG. 51 , an NPU, an AMC, and a main memory are shown. In the sixth example, duplicate descriptions described in other examples may be omitted for convenience of description. Configurations of other examples are selectively applicable to this example.

The NPU may include an NPU scheduler, a plurality of internal memories, and a PE array.

46, the plurality of internal memories in the NPU shown in FIG. 51 may include a first internal memory for weights, a second internal memory for input feature maps, and a third internal memory for output feature maps. can The first to third internal memories may be a plurality of regions allocated in one physical memory.

The AMC may include an ANN data locality information management unit, a translation lookaside buffer (TLB), and a buffer memory.

Data may be randomly stored in the main memory. However, when data is randomly stored in this way, to read data from the main memory, non-sequential memory addresses must be used, so CAS (Column Address Strobe) Latency and RAS (Row Address Strobe) Latency may occur. There is a possibility.

To solve this problem, the AMC may rearrange the data in the main memory based on the ANN data locality information. Specifically, after temporarily storing the data stored in the main memory in the buffer memory, the AMC may rearrange the data stored in the main memory to enable a burst operation based on the ANN data locality information.

Meanwhile, when data is rearranged, a memory address may be changed. Accordingly, the TLB in the AMC may store the old memory address before the rearrangement and the new memory address after the rearrangement in the form of a table.

When the scheduler in the NPU requests data using the old memory address, the TLB in the AMC converts the old memory address to the new memory address, reads data from the main memory, and stores the data in the buffer memory. can Therefore, unlike FIG. 21, the main memory can operate in burst mode without needing to update the memory address stored in the NPU scheduler through the TLB.

In the various examples described above, the AMC and the NPU are illustrated as separate configurations, but the AMC may be configured to be included in the NPU.

52 is an exemplary diagram illustrating an operation according to the sixth example shown in FIG. 51 .

As can be seen with reference to FIG. 52 , the memory address map may be set based on a table according to the ANN DL. In the buffer memory of the AMC, data are sequentially cached in advance according to the order in the ANN DL information. In order not to fill the buffer memory, data in the oldest order may be deleted based on the size of the buffer memory.

53A and 53B are exemplary views illustrating examples of convolution.

Referring to FIG. 53A , a first layer for performing a convolution operation is shown. The size of the input feature map may be 9x9x1, the size of the kernel including weights may be 3x3x1, the stride is 1, and the size of the output feature map is 7x7x1.

If the first input feature map is read from the main memory in the direction of the arrow, the convolution operation can be started faster. The reading direction of the first input feature map may be reading in a horizontal direction after reading in a vertical direction by the height of the kernel.

Referring to FIG. 53B , a second layer for performing a convolution operation is exemplarily shown. The size of the input feature map may be 7x7x1, the size of the kernel including weights may be 3x3x1, the stride is 1, and the size of the output feature map is 5x5x1.

If the second input feature map is read from the main memory in the direction of the arrow, the convolution operation can be started faster. The reading direction of the second input feature map may be reading in a horizontal direction after reading in a vertical direction by the height of the kernel.

54 illustrates another example of caching data in the main memory in the buffer memory and then performing an operation based on a tiling technique.

Referring to FIG. 54 , the main memory and the buffer memory (cache memory) in the AMC are shown. The main memory and the buffer memory may be connected to each other through a system bus. 54 is an example to which the tiling concept is applied. Hereinafter, an example of tiling will be described.

At least one of a kernel, an input feature map, and an output feature map stored in the main memory may be tiled. The memory map of the main memory may be tiled.

At least one of a kernel, an input feature map, and an output feature map stored in the buffer memory may be tiled. The memory map of the buffer memory may be tiled.

As shown, it is assumed that the input feature map for the first layer Conv1 has a size of 18x18x1 for convenience of description only. The input feature map may be tiled into four input feature maps having a size of 9x9x1.

That is, the first input feature map for the first layer Conv1 is a first input feature map tile (IFMAP_1-1), a second input feature map tile (IFMAP_1-2), and a third input feature map tile (IFMAP_1-3). ), and a fourth input feature map tile (IFMAP_1-4). The four input feature map tiles may be combined to form a first input feature map.

In this case, the first kernel Kernel_1 of the first layer Conv1 may be reused. Therefore, the same kernel may be used for the convolution of each tile. In this case, the first kernel Kernel_1 may be reused in the NPU internal memory until the four tiling is completed.

That is, when the first kernel Kernel_1 and the first input feature map tile IFMAP_1-1 are convolved, a first output feature map tile OFMAP_1-1 is generated. A second output feature map tile OFMAP_1-2 is generated by convolution of the first kernel Kernel_1 and the second input feature map tile IFMAP_1-2. When the first kernel Kernel_1 and the third input feature map tile IFMAP_1-3 are convolved, a third output feature map tile OFMAP_1-3 is generated. A fourth output feature map tile OFMAP_1-4 is generated by convolution of the first kernel Kernel_1 and the fourth input feature map tile IFMAP_1-4. The four output feature map tiles may be combined to form a first output feature map.

In this case, the memory map of the main memory may be set to operate in a burst mode based on the tiled artificial neural network data locality information. That is, the artificial neural network data locality information may be changed according to the tiling method. The tiling rule may be variously modified.

That is, the ANN data locality information includes the order of data requested by the NPU to the main memory, and also includes the order according to tiling.

For example, the ANN data locality information includes a first input feature map tile (IFMAP_1-1), a second input feature map tile (IFMAP_1-2), a third input feature map tile (IFMAP_1-3), and a fourth input It may include the order of the feature map tiles (IFMAP_1-4).

For example, the ANN data locality information includes a fourth input feature map tile (IFMAP_1-4), a third input feature map tile (IFMAP_1-3), a second input feature map tile (IFMAP_1-2), and a first input The order of the feature map tiles (IFMAP_1-1) may be included.

That is, the buffer memory of the AMC may receive or generate ANN data locality information, predict the order requested by the NPU, and sequentially cache data corresponding to the order.

55 is a schematic diagram illustrating an artificial neural network memory system according to various examples of the present disclosure.

Referring to FIG. 55 , the NPU and one or more internal memories are implemented in the form of a System on Chip (SoC). The internal memory may be SRAM. Accordingly, the NPU and the internal memory may be connected through an SRAM interface.

An AMC may be disposed between the SoC and the main memory. The AMC may be configured to predict the data to be requested by the NPU between the main memory and the internal memory based on the ANN DL information, and to cache the data in the main memory in advance until the NPU requests it.

The internal memory may include a first internal memory for storing weights, a second internal memory for storing an input feature map, and a third internal memory for storing an output feature map. The three internal memories may be a plurality of logical regions allocated in one physical memory. For example, the size of the second internal memory may be 128 KB, and the size of the third internal memory may be 196 KB.

The NPU may include a PE array including PEs and a special function unit (SFU). The NPU reads the weights from the first internal memory, reads the input feature map from the second internal memory, performs a convolution operation on the input feature map and the weights, and generates an output feature map print out And the SFU of the NPU stores the output feature map in the third internal memory.

In addition, one or more external main memories exist outside the SoC, and are electrically connected to the SoC. The external main memory may be a SAM or DRAM. Accordingly, the one or more external main memories and the SoC may be connected through a DRAM interface.

The external main memory may include a first external main memory for storing weights and a second external main memory for storing a feature map. The two external main memories may be a plurality of areas allocated within one physical memory.

The SoC reads the weight in the first external main memory and the feature map in the second external main memory through a read command, and stores the reads in the first internal memory and the second internal memory, respectively. Also, the SoC may store the output feature map in the third internal memory in the second external main memory through a write command.

Figure 56 shows the detailed operation configuration of the SFU shown in Figure 57.

Each operation configuration of the SFU shown in Figure 56 may be organized in the following table.

explanation movement Zero point add Offset addition by filter or tensor (Dequantize offset operation) Int add Int2float type casting Scale Scale multiplication by filter or tensor (dequantize offset operation) float mul Bias add Add bias value with filter paddy Float add Batch Complex point values per filter and multiplication/addition Float mul, Float add Skip add Pre-block output and element-wise addition
(skip connection add) Float add Activation activation function SE mul Channel wise multiplication of SE block output with previous output (multiplication with SE module output) float mul Avgpool Divide Feature Dimensions After Stacking Float add, Float mul Quantize Zero point addition and scale multiplication Float add, Float mul float2int type casting

57 is a graph showing a measurement of the bandwidth of the data bus between the buffer memory (cache) and the main memory.

The graph shown in FIG. 57 shows the result of measuring the bandwidth when the buffer memory (cache) and the main memory are connected through the AXI4 interface.

The bandwidth measurement was performed in a situation where 2 Mbyte of data was read from DRAM, which is the main memory, to SRAM, which is the buffer memory, and was performed 10 times for each AXI burst length (1 to 16).

The graph shown in FIG. 59 is summarized in a table as follows.

burst length 1 burst 2 bursts 4 bursts 8 bursts 16 bursts Linear
Address Time (ns) 2,310,440 1,198,699 654,484 378,766 242,023 Bandwidth (Gb/sec) 6.93 13.35 24.45 42.24 66.11 Random
Address Time (ns) 6,108,015 1,738,665 983,017 617,457 363,018 Bandwidth (Gb/sec) 2.62 9.20 16.28 25.91 44.07

Regardless of the burst length, when the address ADDRESS is linear, the transmission bandwidth, that is, the transmission speed is improved.

If the burst lengths are the same, using a linear address may result in a faster transfer rate. It may be advantageous to efficiently allocate the address of the DRAM, which is the main memory, to be read-burst.

The burst length means the length of reading at a time in bursts. In the linear case, even if the burst length is short, since the DRAM addresses are contiguous, RAS delay and/or CAS delay can be reduced.

That is, if the memory map of the main memory is set linearly based on the ANN data locality information, the bandwidth increases compared to the random case. Accordingly, the effective bandwidth between the main memory and the buffer memory can be increased.

<Simplified summary of the disclosures of the present specification>

According to one disclosure of the present specification, a memory device for an artificial neural network is provided. The memory device may include: at least one memory cell array having N columns and M rows; and a memory controller configured to sequentially operate the data read or write operation of the at least one memory cell array in a burst mode based on preset sequential access information.

The at least one memory cell array may further include a plurality of dynamic memory cells having leakage current characteristics.

The at least one memory cell array may include: a column decoder for controlling access to the N columns; a plurality of bit lines connected to the column decoder; a row decoder for controlling access to the M rows; a plurality of word lines connected to the row decoder; and a sense amplifier connected to one end of the plurality of bit lines.

The memory controller may be configured to control data communication between a processor to process an artificial neural network operation and the at least one memory cell array in which data required for the artificial neural network operation is stored based on the sequential access information.

The sequential access information may be generated based on "locality of artificial neural network data" of the artificial neural network that the processor intends to process.

The memory controller is configured to directly control the N column and M row addresses of the at least one memory cell array so that the at least one memory cell array operates in the sequential burst mode based on the sequential access information. can be configured.

The memory controller may be configured to set memory addresses of data for each operation step to be stored in the at least one memory cell array based on the sequential access information.

The memory controller may be configured to sequentially allocate address values corresponding to the N columns and M rows of the at least one memory cell array to store artificial neural network data.

According to one disclosure of the present specification, a memory device for an artificial neural network is provided. The memory device may include at least one memory cell array; and a memory controller configured to directly control a read or write operation of the at least one memory cell array based on artificial neural network data locality.

The artificial neural network data locality may include preset operation order information of the artificial neural network to be processed by the processor.

The artificial neural network data locality may include data size information for each of a preset operation order.

The artificial neural network memory control unit may be configured to store a memory map generated in a sequential manner based on preset operation order information and a data size of each corresponding operation order.

The artificial neural network data locality may further include a signal for identifying a weight, an input feature map, and an output feature map. An operation order pattern of the weight data, the input feature map data, and the output feature map data may be determined when compiled based on the characteristics of the processor.

The artificial neural network data locality may be determined based on at least one of a characteristic of an artificial neural network model, a characteristic of a processor, a size of a cache memory, and/or an operation algorithm policy.

According to one disclosure of the present specification, a memory device for an artificial neural network is provided. The memory device includes at least one dynamic memory cell array; and a memory controller configured to store artificial neural network data in the at least one memory cell array in an order based on locality of the artificial neural network data.

The order based on the locality of the artificial neural network data may include at least a repeating pattern of an input feature map, a kernel, and an output feature map order.

The order based on the locality of the artificial neural network data may include at least a repeating pattern of a kernel, an input feature map, and an output feature map order.

The artificial neural network data locality may be configured in a data access request unit that a processor requests from the memory control unit. The artificial neural network data locality may include order information of all data access requests required for one-time inference of the artificial neural network processed by the processor.

The memory controller may be configured to divide the region of the at least one dynamic memory cell array into a kernel region and a feature map region based on identification information configured to distinguish the kernel, the input feature map, and the output feature map.

The at least one dynamic memory cell array may further include a plurality of banks configured to enable an interleaving operation. The memory controller may be configured to divide and store the artificial neural network data in each bank to operate in a burst mode corresponding to the interleaving operation in the plurality of banks.

The memory device may further include a processor configured to provide the artificial neural network data locality information to the memory controller.

The memory device may further include a processor configured to provide at least input feature map, kernel, and output feature map identification information to the memory controller.

Features, structures, effects, etc. described in the above examples are included in one example of the present disclosure, and are not necessarily limited to one example. Furthermore, the features, structures, effects, etc. illustrated in each example can be combined or modified for other examples by those of ordinary skill in the art to which the examples belong. Accordingly, the contents related to such combinations and variations should be interpreted as being included in the scope of the present disclosure.

In addition, although it has been described mainly by way of example in the above, it is only an example and does not limit the present invention, and those of ordinary skill in the art to which the present disclosure pertains are not exemplified above in a range that does not depart from the essential characteristics of this example. It can be seen that various modifications and applications are possible. For example, each component specifically shown in the example may be implemented by modification. And differences related to such modifications and applications should be construed as being included in the scope of the present invention defined in the appended claims.

Claims (22)

at least one memory cell array having N columns and M rows; and
A memory device comprising a; a memory controller configured to operate the data read or write operation of the at least one memory cell array in a sequential burst mode based on preset sequential access information. According to claim 1,
wherein the at least one memory cell array further comprises a plurality of dynamic memory cells having a leakage current characteristic. The method of claim 1 , wherein the at least one memory cell array comprises:
a column decoder for controlling access to the N columns;
a plurality of bit lines connected to the column decoder;
a row decoder for controlling access to the M rows;
a plurality of word lines connected to the row decoder; and
and a sense amplifier connected to one end of the plurality of bit lines. According to claim 1,
The memory controller is configured to control data communication between a processor to process an artificial neural network operation and the at least one memory cell array in which data required for the artificial neural network operation is stored based on the sequential access information. According to claim 1,
The memory device, characterized in that the sequential access information is generated based on the "artificial neural network data locality" of the artificial neural network to be processed by the processor. According to claim 1,
The memory controller is configured to directly control the N column and M row addresses of the at least one memory cell array so that the at least one memory cell array operates in the sequential burst mode based on the sequential access information. configured, memory device. According to claim 1,
The memory controller is configured to set memory addresses of data for each operation step to be stored in the at least one memory cell array based on the sequential access information. According to claim 1,
The memory controller is configured to store artificial neural network data by sequentially allocating address values corresponding to the N columns and M rows of the at least one memory cell array. at least one memory cell array; and
A memory device comprising: a memory controller configured to directly control a read or write operation of the at least one memory cell array based on artificial neural network data locality. 9. The method of claim 8,
The artificial neural network data locality, the memory device, characterized in that it includes predetermined operation order information of the artificial neural network to be processed by the processor. 9. The method of claim 8,
The artificial neural network data locality, the memory device, characterized in that it includes data size information of each of the predetermined operation order. 9. The method of claim 8,
The artificial neural network memory control unit is configured to store a memory map generated in a sequential manner based on preset arithmetic order information and a data size of each corresponding arithmetic order. 9. The method of claim 8,
The artificial neural network data locality further includes a signal for identifying a weight, an input feature map, and an output feature map,
and an operation order pattern of the weight data, the input feature map data, and the output feature map data is determined when compiled based on a characteristic of a processor. 9. The method of claim 8,
The artificial neural network data locality, characterized in that determined based on at least one of a characteristic of an artificial neural network model, a characteristic of a processor, a size of a cache memory, and/or an arithmetic algorithm policy, the memory device. at least one dynamic memory cell array; and
and a memory controller configured to store artificial neural network data in an order based on artificial neural network data locality in the at least one memory cell array. 16. The method of claim 15,
The memory device, wherein the order based on the artificial neural network data locality includes at least a repeating pattern of an input feature map, a kernel, and an output feature map order. 16. The method of claim 15,
The memory device, wherein the order based on the artificial neural network data locality includes at least a repeating pattern of a kernel, an input feature map, and an output feature map order. 16. The method of claim 15,
The artificial neural network data locality consists of a data access request unit requested by the processor to the memory control unit,
The artificial neural network data locality includes order information of all data access requests required for one-time inference of the artificial neural network processed by the processor. 16. The method of claim 15,
The memory control unit is configured to divide the region of the at least one dynamic memory cell array into a kernel region and a feature map region based on identification information configured to distinguish the kernel, the input feature map, and the output feature map, memory device. 16. The method of claim 15,
The at least one dynamic memory cell array further includes a plurality of banks configured to enable an interleaving operation,
The memory control unit is characterized in that configured to divide and store the artificial neural network data in each bank to operate in a burst mode corresponding to the interleaving operation in the plurality of banks. 16. The method of claim 15,
The memory device, further comprising a processor, configured to provide the artificial neural network data locality information to the memory controller. 16. The method of claim 15,
and a processor configured to provide at least input feature map, kernel, and output feature map identification information to the memory controller.

Images