KR20210133784A - Method and apparatus for controlling memory - Google Patents

Abstract

Provided are a memory control device and a method thereof. The method, which is the method of controlling a plurality of external memories in an artificial intelligence neural network chip, includes the following steps of: synchronizing, by a memory control device, clock synchronization between the external memories with respect to the plurality of external memories; performing, by the memory control device, a delay time prediction setting test for writing data to a setting address and reading data from the corresponding address with respect to the external memory according to a set parameter; and determining an optimal read/write timing for the external memory based on the parameter used in the test, if read/write to the external memory is successful according to the delay time prediction setting test. Accordingly, it is possible to solve a problem of uncertain latency time without redesigning and implementing the memory control device.

Description

METHOD AND APPARATUS FOR CONTROLLING MEMORY

The present invention relates to a memory control apparatus and method.

A convolutional neural network (CNN), a visual cognitive neural network that mimics the human optic nerve structure to simulate visual information processing, has been applied to various applications (eg, object recognition and detection, speech recognition, etc.). can be implemented as a hardware circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

In order to perform the processing of recognizing input images in real time in a visual cognitive neural network, it is necessary to design a high-speed memory controller that can control an external high-speed memory while maximizing the bus bandwidth of the neural network accelerator. Physical delay caused by the memory controller's printed circuit board (PCB) line of the external memory, the chip of the neural network accelerator, and the package pad delay when implementing a system according to the application of general-purpose object recognition It should support the ability to predict time.

In the past, there is a problem that neural network accelerators have an error in delay time depending on the application, so that the controller needs to be redesigned for each external memory, or additional time or cost occurs due to the redesign of the system board, etc.

An object of the present invention is to provide a memory control apparatus and method capable of fine delay time control for a memory connected to an artificial intelligence neural network chip (hardware accelerator).

In addition, the problem to be solved by the present invention is to provide an apparatus and method for controlling the memory so as to compensate the physical delay time that occurs when an artificial intelligence neural network chip or hardware accelerator is implemented on an external memory and a PCB board. will be.

According to an embodiment of the present invention, there is provided a method for controlling a plurality of external memories in an artificial intelligence neural network chip. The method includes the steps of: synchronizing, by a memory control device, clock synchronization between external memories for a plurality of external memories; performing, by the memory control device, a delay time prediction setting test for writing data to a setting address and reading data from the corresponding address with respect to the external memory according to a set parameter; and when the read/write to the external memory is successful according to the delay time prediction setting test, determining an optimal read/write timing for the external memory based on parameters used for the test.

In one implementation, the performing the delay time prediction setting test includes, in the delay time prediction setting test, when reading and writing to the corresponding external memory fails, performing coarse delay adjustment on the clock signal input to the memory controller to read and write to the external memory.

In one implementation, the step of reading and writing to the external memory by performing the coarse delay adjustment may include performing the coarse delay adjustment with a resolution of 1/2 of a memory clock cycle.

In one implementation, the performing of the delay time prediction setting test may include, after performing the coarse delay adjustment to read and write to the external memory, when reading and writing to the external memory fails, the memory The method may further include reading and writing to the external memory by performing fine delay adjustment on the clock signal input to the controller.

In one embodiment, the step of performing the reading and writing to the external memory by performing the fine delay adjustment includes performing fine time control with a resolution of ps unit, and performing the fine delay adjustment within a range of twice the memory clock period. can be done

In one implementation, the set parameter may include a memory transfer clock, a time parameter of a connected external memory, a memory read clock, and an initial memory delay time.

In one implementation, the determining of the optimal read/write timing includes determining a time for performing a read to the corresponding external memory and a time for performing a write to the external memory as an optimal memory read/write time may include

In one implementation, the step of synchronizing may include delaying a clock signal input by a finely variable delay controller according to a set fine delay value and outputting it to the external memories, thereby causing a skew between clock signals input to the external memories. ) can be adjusted.

According to another embodiment of the present invention, an apparatus for controlling a plurality of external memories in an artificial intelligence neural network chip is provided. The apparatus includes: a memory controller configured to control a plurality of external memories; a first fine variable delay controller configured to control an input clock signal and provide it to the external memory; and a second fine variable delay controller configured to control the input clock signal and provide it to the memory controller, wherein the memory controller writes data to a set address for the external memory according to a set parameter and writes data at the corresponding address When the delay time prediction setting test for reading data is performed, and the read/write to the external memory is successful according to the delay time prediction setting test, the optimal read/write to the external memory is performed based on the parameters used for the test configured to determine timing.

In one implementation, in the delay time prediction setting test, when reading and writing to the corresponding external memory fails, the second finely variable delay controller may be configured to perform coarse delay adjustment on the clock signal input to the memory controller and the memory controller may be configured to read and write to the external memory according to a clock signal adjusted and input from the second fine variable delay controller.

In one implementation, the coarse delay adjustment may be adjusted to a resolution of 1/2 of a memory clock period.

In one embodiment, when reading and writing to the external memory fails as a result of reading and writing to the external memory according to the coarse delay adjustment, the second fine variable delay controller responds to the clock signal input to the memory controller. It may be configured to perform fine delay adjustment, and the memory controller may be configured to read and write to the external memory according to a clock signal adjusted and input from the second fine variable delay controller.

In one implementation, the fine delay adjustment may be performed within a range of twice the memory clock period with a resolution of ps.

In one implementation, the set parameter may include a memory transfer clock, a time parameter of a connected external memory, a memory read clock, and an initial memory delay time.

In one implementation, the second fine variable delay controller may delay the input clock signal according to a set fine delay value and output it to the external memories to adjust a skew between the clock signals input to the external memories. .

In one implementation, the first finely variable delay controller and the second finely variable delay controller may include a fine delay register for setting a delay time, respectively, and a plurality of buffers including a buffer and a delay element.

In one implementation, the artificial intelligence neural network chip may be implemented as hardware and mounted on a printed circuit board (PCB) board, and the memory control device may be configured to control the plurality of external memories on the PCB board.

According to an embodiment, an uncertain delay time that occurs when a neural network accelerator that requires a memory controller for an external memory is implemented as actual hardware can be simply predicted through a time delay test. Thus, the uncertain latency problem can be solved without memory controller redesign and implementation.

In addition, the apparatus and method according to an embodiment of the present invention can be easily applied even if the memory access time, memory type, and time setting are changed for each company for the neural network accelerator, and the system board such as the system PCB is not redesigned. You can apply it right away without having to.

1 is a diagram illustrating an example to which a memory control apparatus according to an embodiment of the present invention is applied.
2 is a diagram illustrating a structure of a memory control apparatus according to an embodiment of the present invention.
3 is an exemplary implementation diagram of a finely variable delay controller according to an embodiment of the present invention.
4 is an exemplary diagram illustrating a clock waveform of a finely variable delay controller according to an embodiment of the present invention.
5 is an exemplary diagram illustrating memory data read timing through a read clock and coarse delay time control after fine delay control according to an embodiment of the present invention.
6 is a flowchart of a memory control method according to an embodiment of the present invention.
7 is a structural diagram of a memory control apparatus according to another embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

In the present specification, expressions described in the singular may be construed in the singular or plural unless an explicit expression such as “a” or “single” is used.

Also, terms including an ordinal number such as first, second, etc. used in an embodiment of the present invention may be used to describe the constituent elements, but the constituent elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

Hereinafter, an apparatus and method for controlling a memory according to an embodiment of the present invention will be described with reference to the drawings.

1 is a diagram illustrating an example to which a memory control apparatus according to an embodiment of the present invention is applied.

1 , in a convolutional neural network (CNN) neural network, the memory control device according to an embodiment of the present invention, that is, the memory controller 1 is connected to the neural network MAC (multiply accumulate) operator 2 through a bus interface. connected and may be connected with the external memory 3 . Here, the external memory 3 is, for example, a volatile storage device such as SRAM (Static RAM), SSRAM (synchronous static RAM), DRAM (Dynamic RAM), SDRAM (Synchronous DRAM), DDR SDRAM (Double Data Rate SDRAM), etc. can be

The CNN neural network requires an external memory to store the layer filter coefficients and the input/output layer feature map, which is the result of the calculation. Accordingly, the neural network MAC operator 2 communicates with the external memory 3 through the memory controller 1 before and after the operation is performed, as shown in FIG. 1 . The memory controller 1 may control the external memory 3 and read data requested by the neural network MAC operator 2 from the external memory 3 and store it in a buffer (not shown). The neural network MAC operator 2 reads data stored in the buffer and performs an operation, and the calculated data (eg, updated weights) may be stored in the external memory 3 through the buffer.

2 is a diagram illustrating a structure of a memory control apparatus according to an embodiment of the present invention.

When the artificial intelligence neural network chip is implemented as hardware and mounted on a printed circuit board (PCB) system, the memory control device 10 may be connected to the external memory 3 on the PCB board as shown in FIG. 2 .

The memory control device 10 includes a memory controller 11 connected to the external memory 3 , and may include a plurality of finely variable delay controllers. The plurality of finely variable delay controllers includes a finely variable delay controller (also called a first finely variable delay controller) 12 connected to the external memory 3 and a finely variable delay controller (second finely variable delay controller) connected to the memory controller 11 . Also called variable delay controller) 13 . The number of first fine variable delay controllers 12 may be plural according to the number of external memories 3 . The finely variable delay controllers 12 and 13 may be connected to a phase lock loop (PLL) 14 that processes a signal output from an oscillator (not shown) and provides it as a clock signal.

When the memory controller 11 is connected to the external memory 3 on the PCB board, a physical time delay occurs. Such time delay may be uncertain delay time due to differences in memory input/output (I/O) load, package routing delay time, and uncertain delay time due to board trace. The uncertain time delay can vary with capacitance in the PCB line itself. And the signal transmission time in the artificial intelligence neural chip may vary depending on Best/Worst/Typical conditions. Moreover, errors occur in the theoretical delay value predicted during system design and the measurement delay value when the system is actually implemented on the PCB board. In order to compensate for almost all of these unpredictable physical delay times, fine delay control and coarse delay control are performed. The fine delay control can perform time control with a resolution of several tens of ps (picosecond), and can be implemented to be controlled in a range more than twice the memory system clock period. Meanwhile, the resolution of the coarse delay control may vary depending on the clock implementation of the artificial intelligence neural network chip, and may have various resolutions of approximately 1/8 to 1/2 of the control clock cycle.

According to an embodiment of the present invention, it is possible to accurately read and write memory data by predicting an uncertain physical delay time as described above. In addition, when the neural network accelerator is implemented in hardware and communicates with the high-speed memory on the PCB board, it adaptively compensates for the read/write time delay according to the length and arrangement of the lines between the chips quickly and accurately.

3 is an exemplary implementation diagram of a finely variable delay controller according to an embodiment of the present invention.

The finely variable delay controllers 12 and 13 of the memory control apparatus according to an embodiment of the present invention, as shown in FIG. 3, include a plurality of buffer units 121 for delaying an input clock signal and a fine delay time setting possible. It includes a fine delay register 122 .

Each buffer unit 121 includes a signal output unit 1211 configured to output a signal input to any one input terminal according to a control signal input from the fine delay register 122 , and a signal output unit 1211 outputted from the signal output unit 1211 . and a plurality of delay elements 1212 for delaying the signal. The signal output unit 1211 functions as a buffer configured to output a signal input through one of the first input terminal and the second input terminal according to the input control signal, for example, a flip-flop (flip-flop). flop). The delay element 1212 delays the signal output from the signal output unit 1211 of the previous stage and provides it to the signal output unit connected to the next stage.

Accordingly, the clock signal input through the buffer unit 121 is delayed and output.

The finely variable delay controllers 12 and 13 having such a structure can be implemented as a combination of buffer cells or equivalent delay cells of the corresponding foundry when implemented with an ASIC chip, and can be configured with a corresponding buffer library when implemented with an FPGA.

The finely variable delay controllers 12 and 13 having such a structure may set a fine delay time using the fine delay register 122 . Depending on the implementation, it is possible to set the fine delay time in conjunction with the latency prediction test in the software.

As shown in FIG. 2 , the finely variable delay controller 12 may be implemented in front of the input clock of the external memory 3 to reduce clock skew of the external memory 3 .

4 is an exemplary diagram illustrating a clock waveform of a finely variable delay controller according to an embodiment of the present invention.

As illustrated in FIG. 4 , the finely variable delay controller 12 delays the input external memory clock signal (the clock signal output from the PLL 14 ), and outputs the delayed external memory clock. Here, external memory clock #1 to external memory clock #4 provided to each of the four external memories are taken as an example.

External memory clock #1 to external memory clock #4 are output through the fine variable delay controller 12 according to an embodiment of the present invention, and as shown in FIG. 4 , the output is compared to the external memory clock input to the fine variable delay controller. It can be seen that the skew of the clock of each external memory to be used is reduced. Accordingly, fine delay control is performed, and clock synchronization between external memory chips can be achieved.

5 is an exemplary diagram illustrating memory data read timing through a read clock and coarse delay control after fine delay control according to an embodiment of the present invention.

For example, in the implementation environment shown in FIG. 2 , the memory output data is input to the neural network chip by adding a physical delay time after the memory access time, as illustrated in FIG. 5 .

After changing the fine delay adjustment values for internal memory and external memory, accurate memory data reading is possible by finely adjusting the time adjustment value centering on the time set in the coarse delay adjustment. In this example, the coarse delay time is adjusted with a resolution of 1/2 of the memory system clock period.

Next, a memory control method according to an embodiment of the present invention will be described.

6 is a flowchart of a memory control method according to an embodiment of the present invention.

Through initialization of the neural network accelerator, memory control initial environment registers such as a memory bandwidth connected to the memory controller 11, a transmission clock, a read clock, and a read delay time are set. That is, an initial memory parameter is set (S100).

Next, fine delay adjustment of the external memory clock is performed (S110). Adjusts the skew between clocks input to external memories by performing fine delay adjustment for external memory. Specifically, as described above, the fine delay processing is performed on the external memory clock input using the fine variable delay controller 12 to adjust the skew between the clocks input to the external memories. Accordingly, clock synchronization between the external memories can be matched.

And the memory controller 11 performs a delay time prediction setting test (S120). Specifically, the memory controller 11 performs a delay time prediction setting test for predicting a possible delay time for each memory by initially writing data to a specific address for each memory and reading data from the corresponding address. In this case, the delay time prediction setting test can be implemented using the firmware of the artificial intelligence neural network chip, and the memory R (read)/W (write) data can be automated (Macro) by changing the specific value of the setting register within the memory controller. A delay time prediction setting test can be implemented in a way that compares , but is not limited thereto.

In this delay time prediction setup test, in the process of writing data to a specific address for each memory and reading data from that address, the memory transfer clock, time parameters of the connected memory (such as access time), memory read clock and initial memory latency (latency) and the like are combined to perform a delay time prediction setting test using the timing to read memory data as a reference point.

In this delay time prediction setting test, when reading and writing to the memory fails (S130), coarse delay adjustment is performed (S140). For example, the resolution of the coarse delay adjustment is variable according to the clock implementation of the artificial intelligence neural network chip. For example, the coarse delay adjustment is performed with a resolution of 1/2 of the clock period of the memory system. For example, in the structure of FIG. 2 , the clock input to the memory controller 11 through the finely variable delay controller 13 is controlled to 1/2 of the system clock cycle, and read into the memory of the memory controller 11 Allows write operations to be coarsely delayed.

A coarse delay adjustment is performed, a delay time prediction setting test is performed, and as a result, when reading and writing to the memory fails, fine delay adjustment is performed (S150, S160). In fine delay adjustment, for example, the fine delay adjustment may be performed in a range of two times or more of a memory system clock period. For example, in the structure of FIG. 2 , the clock input to the memory controller 11 through the finely variable delay controller 13 is controlled within a range of 2 times or more of the system clock period, for example, 1 of the control clock period. By controlling /8 to 1/2, the read/write operation to the memory of the memory controller 11 is finely delayed.

Fine delay adjustment is performed and delay time prediction setting test is performed. As a result, when reading and writing to the memory fails (S170), the above steps (S140 to S170) are repeatedly performed.

On the other hand, read/write to the memory is successful in step S130, or read/write to the memory is successful after coarse delay adjustment in step S150, or read/write to the memory after fine delay adjustment in step S170 If is successful, the time at which the read to the corresponding memory is performed and the time at which the write to the memory is performed are set as the optimal memory read/write time. Accordingly, an optimal memory read/write time parameter for the corresponding memory is set ( S180 ).

The process as described above may be individually performed for each memory connected to the neural network chip, or may be performed simultaneously for all connected memories.

According to this embodiment of the present invention, an uncertain delay time that may occur when a neural network accelerator that requires a memory controller for controlling an external memory is implemented as actual hardware can be simply predicted through an initial time delay test. Accordingly, the uncertain latency problem can be easily solved without redesigning and implementing the memory controller, such as redesigning or implementing the memory controller.

7 is a structural diagram of a memory control apparatus according to another embodiment of the present invention.

As illustrated in FIG. 7 , the memory control apparatus 200 according to another embodiment of the present invention may be implemented as a computer system.

The memory control device 200 includes a processor 210 , a memory 220 , an input interface device 230 , an output interface device 240 , and a storage device 250 . Each of the components may be connected by a bus 260 to communicate with each other. In addition, each of the components may not be connected to the common bus 260 but to the processor 210 through an individual interface or a separate bus.

The processor 210 may execute a program command stored in at least one of the memory 220 and the storage device 260 . The processor 210 may refer to a central processing unit (CPU) or a dedicated processor on which methods according to embodiments of the present invention are performed. Such a processor 210 may be configured to implement a function corresponding to the method described based on FIGS. 1 to 6 above.

The memory 220 is connected to the processor 210 and stores various information related to the operation of the processor 210 . The memory 220 may store instructions to be executed by the processor 210 or may temporarily store instructions by loading the instructions from the storage device 250 . The processor 210 may execute instructions stored in or loaded from the memory 220 . The memory may include a ROM 221 and a RAM 222 .

In an embodiment of the present invention, the memory 220/storage device 250 may be located inside or outside the processor 210 , and may be connected to the processor 210 through various known means.

The embodiment of the present invention is not implemented only through the apparatus and/or method described above, but may be implemented through a program for realizing a function corresponding to the configuration of the embodiment of the present invention, a recording medium in which the program is recorded, etc. Also, such an implementation can be easily implemented by those skilled in the art to which the present invention pertains from the description of the above-described embodiments.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improved forms of the present invention are also provided by those skilled in the art using the basic concept of the present invention as defined in the following claims. is within the scope of the right.

Claims (17)

A method of controlling a plurality of external memories in an artificial intelligence neural network chip, comprising:
synchronizing, by the memory control device, clock synchronization between the external memories with respect to the plurality of external memories;
performing, by the memory control device, a delay time prediction setting test for writing data to a setting address and reading data from the corresponding address with respect to the external memory according to a set parameter; and
If the read/write to the external memory is successful according to the delay time prediction setting test, determining the optimal read/write timing for the external memory based on the parameters used for the test
A control method comprising a. According to claim 1,
The step of performing the delay time prediction setting test is,
In the delay time prediction setting test, when reading and writing to the corresponding external memory fails, performing read/write to the external memory by performing coarse delay adjustment on the clock signal input to the memory controller;
Including, a control method. 3. The method of claim 2,
The step of performing reading and writing to the external memory by performing the coarse delay adjustment includes performing the coarse delay adjustment with a resolution of 1/2 of a memory clock cycle. 3. The method of claim 2,
The step of performing the delay time prediction setting test is,
After reading and writing to the external memory by performing the coarse delay adjustment, when reading and writing to the external memory fails, fine delay adjustment is performed on the clock signal input to the memory controller to perform the fine delay adjustment to the external memory. Steps to read and write to memory
Further comprising, a control method. 5. The method of claim 4,
The step of performing the read/write to the external memory by performing the fine delay adjustment includes performing fine time control with a resolution of ps unit, and performing the fine delay adjustment within a range of twice the memory clock period. Way. According to claim 1,
The set parameter includes a memory transfer clock, a time parameter of a connected external memory, a memory read clock and an initial memory delay time. According to claim 1,
The step of determining the optimal read/write timing includes:
A control method comprising the step of determining a time for performing a read to the corresponding external memory and a time for performing a write to the external memory as an optimal memory read/write time. According to claim 1,
The step of synchronizing
A control method, comprising: a fine variable delay controller delaying an input clock signal according to a set fine delay value and outputting it to the external memories to adjust a skew between clock signals input to the external memories. A device for controlling a plurality of external memories in an artificial intelligence neural network chip, comprising:
a memory controller configured to control the plurality of external memories;
a first fine variable delay controller configured to control an input clock signal and provide it to the external memory; and
a second fine variable delay controller configured to control the input clock signal and provide it to the memory controller
including,
The memory controller performs a delay time prediction setting test for writing data to a setting address and reading data from the corresponding address with respect to the external memory according to a set parameter, and reading into the external memory according to the delay time prediction setting test If the /write is successful, the control device is configured to determine an optimal read/write timing for the external memory based on the parameters used for the test. 10. The method of claim 9,
In the delay time prediction setting test, when reading and writing to the corresponding external memory fails, the second fine variable delay controller is configured to perform coarse delay adjustment on the clock signal input to the memory controller,
and the memory controller is configured to read and write to the external memory according to a clock signal adjusted and input from the second fine variable delay controller. 11. The method of claim 10,
The control device of claim 1, wherein the coarse delay adjustment is adjusted to 1/2 resolution of the memory clock period. 11. The method of claim 10,
When reading and writing to the external memory fails as a result of reading and writing to the external memory according to the coarse delay adjustment, the second fine variable delay controller performs fine delay adjustment of the clock signal input to the memory controller. is configured to perform
and the memory controller is configured to read and write to the external memory according to a clock signal adjusted and input from the second fine variable delay controller. 13. The method of claim 12,
The control device, wherein the fine delay adjustment is made within a range of twice the memory clock period with a resolution of ps. 10. The method of claim 9,
The set parameter includes a memory transfer clock, a time parameter of a connected external memory, a memory read clock and an initial memory delay time, the control device. 10. The method of claim 9,
and the second fine variable delay controller delays the input clock signal according to a set fine delay value and outputs it to the external memories to adjust a skew between the clock signals input to the external memories. 10. The method of claim 9,
The first finely variable delay controller and the second finely variable delay controller each include a fine delay register for setting a delay time, and a plurality of buffers including a buffer and a delay element. 10. The method of claim 9,
The artificial intelligence neural network chip is implemented in hardware and mounted on a printed circuit board (PCB) board, and the memory control device is configured to control the plurality of external memories on the PCB board.

Images