JP7179237B1 - neural network device - Google Patents

Abstract

The neural network analysis unit (102) of the neural network device (100) includes a network structure analysis unit (201) for analyzing the operation structure of the neural network, and each operation obtained by dividing the neural network into a circuit. and a neural network dividing unit (202) that determines whether to perform software processing. A neural network division unit (202) includes a convolution layer circuitization unit (501) that groups layers having the same or similar parameters among the layers that perform convolution operations of the neural network; For each operation, a circuit scale calculation unit (503) for calculating a circuit scale when the convolution operation is circuitized, and an operation to be circuitized is determined based on the circuit scale calculated by the circuit scale calculation unit (503). and a circuitization location determination unit (504).

Description

TECHNICAL FIELD The present disclosure relates to artificial intelligence technology, and more particularly to an apparatus and method for creating a program for processing neural networks.

For example, in fields such as image processing, neural networks are widely used these days because they can perform processing with extremely high accuracy. In addition, neural networks include a large number of operations and are known to have a high processing load. In order to complete processing within a desired time, a neural network is realized by hardware such as a dedicated processor such as GPGPU (General Purpose Graphics Processing Unit), FPGA (Field Programmable Gate Array), and ASIC (Application Specific Integrated Circuit). There are many things.

A neural network contains a large number of operations, and is composed of a combination of operations such as convolution, activation function, and total connection, and its structure is relatively simple. For example, Japanese Unexamined Patent Application Publication No. 2002-200002 proposes a technique for solving the problem of a large amount of computation by using the characteristics of this neural network. In the technique of Patent Document 1, different network structures are compiled into operations of the same arithmetic unit by controlling the same arithmetic unit based on a single instruction corresponding to each layer operation of multi-layer operations of the neural network, thereby performing the same operation. The device is designed to implement logic operations of all layers (eg, Patent Document 1 below).

JP 2019-139747 A

In the technique of Patent Document 1, hardware resources are efficiently used by connecting and using a single instruction prepared in advance on hardware according to the network structure, and neural network processing is realized at high speed. can be done. However, since processing is performed using a combination of single instructions prepared in advance on hardware, there is concern that processing may not be completed within a desired time.

The present disclosure has been made to solve the above problems, and aims to enable the design of neural network processing that can achieve desired performance (execution time and accuracy) on hardware with fewer resources. aim.

The neural network device according to the present disclosure includes a neural network analysis unit that determines an operation method for each operation that constitutes a neural network, whether to circuitize the operation or to perform software processing, and a circuit that performs the operation that has been determined to be circuitized. and a neural network operation method output unit that creates and outputs a program for software processing of operations determined to be processed by software and circuit information for processing, wherein the neural network analysis unit is , a network structure analysis unit that analyzes the operation structure of the neural network; and a neural network division unit that determines whether each operation obtained by dividing the neural network is circuitized or software-processed; and the network structure analysis unit classifies each layer of the neural network into an operation structure classification unit that classifies each layer according to the type of operation that constitutes the layer, and a layer that performs a convolution operation by the operation structure classification unit. a convolutional layer analyzer that identifies parameters of the convolutional operation for each layer, wherein the neural network partitioner has the same or similar parameters based on the parameters identified by the convolutional layer analyzer. It has a convolutional layer circuitizer that groups the layers.

The present disclosure enables the design of neural network processing that achieves desired performance (in-time processing and accuracy) on hardware with fewer resources.

Objects, features, aspects and advantages of the present disclosure will become more apparent with the following detailed description and accompanying drawings.

1 is a block diagram showing the configuration of a neural network device according to Embodiment 1; FIG. 2 is a block diagram showing the configuration of a neural network analysis unit according to Embodiment 1; FIG. 2 is a block diagram showing the configuration of a neural network arithmetic method output unit according to Embodiment 1; FIG. 3 is a block diagram showing the configuration of a network structure analysis unit according to Embodiment 1; FIG. 2 is a block diagram showing the configuration of a neural network dividing unit according to Embodiment 1; FIG. 4 is a flowchart showing processing of a convolutional layer circuitization unit according to Embodiment 1; 7 is a flow chart showing processing of an activation layer circuitization unit according to the first embodiment; 7 is a flow chart showing processing of a circuit scale calculator according to the first embodiment; FIG. 3 is a diagram showing a hardware configuration example of a neural network construction unit; FIG. 3 is a diagram showing a hardware configuration example of a neural network construction unit; FIG. 9 is a block diagram showing the configuration of a neural network device according to Embodiment 2;

<Embodiment 1>
FIG. 1 is a block diagram showing the configuration of a neural network device 100 according to Embodiment 1. As shown in FIG. As shown in FIG. 1 , the neural network device 100 includes a neural network constructing section 101 having a neural network analysis section 102 , a neural network arithmetic method output section 103 , and a storage section 104 .

The neural network analysis unit 102 reads the network structure data of the neural network stored in the storage unit 104, analyzes the network structure, determines the operation method including the program and circuit for operating the neural network, and determines the operation Output the scheme. In other words, the neural network analysis unit 102 determines the calculation method of circuitization or software processing for each calculation that constitutes the neural network.

Based on the calculation method received from the neural network analysis unit 102, the neural network calculation method output unit 103 outputs data of a program that operates on a processor such as a CPU (Central Processing Unit), and data for constructing an calculation circuit on an FPGA. Create and output circuit information. That is, the neural network operation method output unit 103 performs software processing of circuit information for circuitizing operations determined to be circuitized by the neural network analysis unit 102 and operations determined to be software-processed by the neural network analysis unit 102. Create a program to do and output.

FIG. 2 is a block diagram showing the configuration of the neural network analysis unit 102. As shown in FIG. As shown in FIG. 2 , the neural network analysis section 102 has a network structure analysis section 201 and a neural network division section 202 .

The network structure analysis unit 201 analyzes the operation structure in the network based on the network structure data of the neural network read from the storage unit 104, and outputs the analysis result.

The neural network dividing unit 202 receives the analysis result of the operation structure in the network input from the network structure analysis unit 201, divides the operations forming the network structure into operation units of a predetermined scale, and It determines whether the processing of each operation is to be performed by the CPU or the FPGA (that is, whether to be processed by software or by circuitization), and the result of the determination is output in association with each operation after division.

FIG. 3 is a block diagram showing the configuration of the neural network arithmetic method output unit 103. As shown in FIG. As shown in FIG. 3 , the neural network arithmetic method output unit 103 has a control program creation unit 301 , an arithmetic circuit creation unit 302 , and a control data generation unit 303 for data acquisition circuit.

The control program creation unit 301 receives the calculation method of each process of the neural network determined by the neural network analysis unit 102, creates and outputs a program for the CPU for calculations to be software-processed by the CPU. The program also includes a control program that manages input and output of arithmetic circuits running on the FPGA and controls the FPGA to enable processing of the entire neural network. The control program performs processing such as inputting data to the arithmetic circuit A, receiving an arithmetic result from the arithmetic circuit A, and inputting it to the arithmetic circuit B, for example. Also, part of the arithmetic processing may be included in the control program so that part of the arithmetic processing is performed on the CPU instead of on the FPGA. In that case, the control program performs processing such as calculating the product of the outputs of the arithmetic circuit A and the arithmetic circuit B, for example. The program output by the control program creation unit 301 is, for example, binary data of an execution program obtained by compiling a code written in C language or the like with a compiler for a specific CPU.

The arithmetic circuit creation unit 302 receives the arithmetic method including the program and circuit for operating the neural network determined by the neural network analysis unit 102, and creates and outputs circuit information for constructing an arithmetic circuit that operates on the FPGA. .

The data acquisition circuit control data generator 303 calculates, based on the circuit information, parameters to be provided to a dedicated circuit that receives data from a shared memory or the like, which is used when the FPGA receives data from the CPU. For example, an arithmetic circuit A on an FPGA receives data of a certain size (data width) and calculates the average of a certain number of data contained therein, and the data of a certain size is stored in the shared memory. Then, when arithmetic circuit A starts an operation without receiving an operation execution command from the CPU, that is, when arithmetic circuit A autonomously acquires data, the above parameters enable the operation. It is for incorporating in advance a value of "fixed size" to be used in the arithmetic circuit on the FPGA.

FIG. 4 is a block diagram showing components of the network structure analysis unit 201. As shown in FIG. As shown in FIG. 4 , the network structure analysis unit 201 has an arithmetic structure classification unit 401 , a convolutional layer analysis unit 402 and an activation layer analysis unit 403 .

Based on the network structure data of the neural network read from the storage unit 104, the operation structure classification unit 401 analyzes what kind of operation each layer in the neural network is composed of, and converts the analysis result into the operation information of each layer. , which is associated with each layer and output. The operation associated with each layer is, for example, a convolution operation or an operation using an activation function. Note that the operations associated with each layer may include operations other than convolution operations and activation functions. In that case, other operations may be circuitized or processed by the CPU. For example, the operations forming the fully connected layer may be circuitized as they are, or may be regarded as convolutional operations with large parameters and circuitized as a convolutional layer.

The convolutional layer analysis unit 402 receives the operation information of each layer in the neural network from the operation structure classification unit 401, and analyzes the layer that performs the convolution operation among those layers. Specifically, the parameters of the convolution operation performed in each layer in which the convolution operation is performed are specified. The parameters are, for example, input size, output size, number of kernels, kernel size, BorderMode, and the like.

The activation layer analysis unit 403 receives the operation information of each layer in the neural network from the operation structure classification unit 401, and analyzes the layer that performs processing based on the activation function among those layers. Specifically, it identifies what activation function is used in each layer that performs processing based on the activation function.

FIG. 5 is a diagram showing the configuration of the neural network dividing unit 202. As shown in FIG. As shown in FIG. 5, the neural network dividing unit 202 has a convolutional layer circuitization unit 501, an activation layer circuitization unit 502, a circuit scale calculation unit 503, and a circuitization location determination unit 504. FIG.

The convolutional layer circuitization unit 501 receives the analysis result by the network structure analysis unit 201, and outputs information related to circuitization of the convolution arithmetic processing.

The activation layer circuitization unit 502 receives the analysis result from the network structure analysis unit 201 and outputs information on circuitization of the activation function.

The circuit scale calculation unit 503 receives information related to circuitization output from the convolutional layer circuitization unit 501 and the activation layer circuitization unit 502, and calculates the circuit scale when each operation is circuitized.

A circuitization location determination unit 504 receives information on the circuit scale of each operation calculated by the circuit scale calculation unit 503, and separates calculations to be circuitized from calculations to be processed by a processor such as a CPU without circuitization.

FIG. 6 is a flow chart showing the processing of the convolutional layer circuitization unit 501. As shown in FIG. The operation of the convolutional layer circuitization unit 501 will be described below with reference to FIG.

The convolutional layer circuitization unit 501 first acquires information of one convolutional layer from the operation information of each layer in the neural network received from the network structure analysis unit 201 (step S601). Then, the convolutional layer circuitizing unit 501 confirms the parameter included in the acquired information, associates the information of the parameter with the convolutional layer, and saves it as convolutional layer information (step S602).

Subsequently, the convolutional layer circuitization unit 501 checks whether there is a convolutional layer whose information has not yet been acquired among the operation information of each layer in the neural network (step S603). If there remains a convolutional layer for which information has not been acquired, the process returns to step S601, and the convolutional layer circuitization unit 501 performs the processes of steps S601 and S602 on the next convolutional layer.

If the information of all the convolutional layers has already been acquired, the convolutional layer circuitization unit 501 extracts layers having the same parameters from among the convolutional layers corresponding to the stored convolutional layer information, and extracts the layers having the same parameters. The layers that have are grouped (step S604). In other words, the convolutional layer circuitization unit 501 groups convolutional layers according to their parameters. Here, the grouping unit is a layer unit, but the sum-of-products operation that constitutes the convolution operation in the layer may be a smaller grouping unit.

When the grouping in step S604 is completed, the convolutional layer circuitization unit 501 confirms whether or not the number of created groups is equal to or greater than a predetermined number (step S605). If the number of groups is equal to or greater than the predetermined number, the convolutional layer circuitization unit 501 further groups groups having similar parameters (step S606). "Similar" here may be defined as a relationship in which parameters are close to each other within a certain range, or may be defined as an inclusion relationship in which parameters in one group include parameters in the other group. . The reason for grouping groups in which parameters have an inclusion relationship is that if the operation on the including side is circuitized, the operation on the included side can be executed using a part of the circuit of the operation on the including side. . When storing the grouping result, information indicating how many times the operation belonging to each group is used in the entire neural network is stored in association with the grouping result.

FIG. 7 is a flow chart showing the processing of the activation layer circuitization unit 502. As shown in FIG. The operation of the activation layer circuitization unit 502 will be described below with reference to FIG.

Activation layer circuitization unit 502 first acquires the activation function of each layer included in the operation information of each layer in the neural network received from network structure analysis unit 201, and stores it as activation function information (step S701). Then, the activation layer circuitization unit 502 extracts the same activation functions from the saved activation functions and groups the same activation functions (step S702). In other words, the activation layer circuitization unit 502 groups the layers to be processed based on the activation function for each activation function. When storing the grouping result, information indicating how many times the operation belonging to each group is used in the entire neural network is stored in association with the grouping result.

Next, the activation layer circuitization unit 502 acquires one of the grouped activation functions (step S703), and checks whether the activation function can be linearly approximated (step S704). If the activation function can be linearly approximated, the linearly approximated function is stored in association with the activation function (step S705).

Subsequently, the activation layer circuitization unit 502 checks whether there is an activation function that has not yet been acquired among the grouped activation functions (step S706). If there remains an activation function that has not yet been acquired, the process returns to step S703, and the convolutional layer circuitization unit 501 performs the processes of steps S703 to S706 for the next activation function. That is, the convolutional layer circuitization unit 501 performs the processing of steps S703 to S706 for all grouped activation functions.

The circuit scale calculation unit 503 performs circuitization for each group of the convolutional layer information grouped by the convolutional layer circuitization unit 501 and the activation function information grouped by the activation layer circuitization unit 502. Calculate the scale of The scale of the circuit can be calculated based on the number of calculators such as adders and multipliers included in the circuit, for example.

FIG. 8 is a flow chart showing the processing of the circuit scale calculator 503. As shown in FIG. The operation of the circuit scale calculator 503 will be described below with reference to FIG.

The circuit scale calculation unit 503 first acquires one convolutional layer group included in the grouped convolutional layer information (step S801). Then, the circuit scale calculation unit 503 determines arithmetic units necessary for circuitization of the calculation of the convolutional layer based on the parameters stored in association with the acquired convolutional layer (step S802). Then, the circuit scale calculation unit 503 calculates the circuit scale when the calculation of the convolutional layer is circuitized from the determined type and number of arithmetic units (step S803).

It should be noted that the arithmetic unit selected in the circuitization of the arithmetic does not have to be the smallest arithmetic unit capable of executing the arithmetic operation, and an arithmetic unit with a large circuit scale may be selected. If the circuit scale is a power of 2, data access is efficient. by increasing

After that, the circuit scale calculation unit 503 checks whether there is a convolutional layer group that has not yet been acquired in the grouped convolutional layer information (step S804). If there are groups of convolutional layers that have not yet been acquired, the process returns to step S801, and the circuit scale calculation unit 503 performs the processes of steps S801 to S803 on the next group. That is, the circuit scale calculation unit 503 performs the processing of steps S801 to S803 for all convolutional layer groups.

If the groups of all convolutional layers have already been acquired, the circuit scale calculation unit 503 acquires one group of activation functions corresponding to the activation layer included in the grouped activation function information (step S805). ). Next, the circuit scale calculation unit 503 determines arithmetic units required to circuitize the activation function (step S806). Then, the circuit scale calculation unit 503 calculates the circuit scale when the activation function is circuitized from the types and the number of arithmetic units (step S807).

Subsequently, the circuit scale calculator 503 determines whether the activation function can be linearly approximated (step S808). If the activation function can be linearly approximated, the approximation function is specified (step S809), the arithmetic units required for circuitization of the approximation function are determined (step S810), and the circuit is constructed from the type and number of the determined arithmetic units. A scale is calculated (step S811), and the calculated circuit scale is stored in association with the activation function.

After that, the circuit scale calculation unit 503 checks whether or not a group of activation functions that have not yet been acquired remains in the grouped activation function information (step S812). If there remains a group of activation functions that have not yet been acquired, the process returns to step S805, and the circuit scale calculation unit 503 performs the processes of steps S805 to S812 on the next group. That is, the circuit scale calculation unit 503 performs the processing of steps S805 to S812 for all activation function groups.

The circuit location determination unit 504 receives a list of circuits necessary for processing the neural network and information on the circuit scale from the circuit scale calculation unit 503, and considers the capacity of the FPGA to be implemented, and determines the neural Among the network processes, the processes to be circuitized are determined. At this time, the processing determined not to be circuitized is processed by software on a processor such as a CPU. The selection criteria for the processing to be circuitized (that is, the criteria for determining whether to circuitize each processing) are whether the execution time of the neural network processing is reduced (shortened) and whether the accuracy of the neural network is increased. Two factors are taken into consideration.

Usually, the execution time of neural network processing is shorter when each processing is circuitized than when each processing is processed by software. Therefore, when selecting processing to be circuitized based on the execution time of neural network processing as a selection criterion, among the list of circuits necessary for processing the neural network received from the circuit scale calculation unit 503, It is determined that the larger (longer) processing time should be preferentially circuitized. At this time, the number of times each circuit is used in the processing of the neural network is also considered. For example, even if the execution time for software processing of process A is 5 milliseconds and the execution time for process B is 30 milliseconds for software processing, the number of times processing A is used in the neural network is 20 times. If B is used twice, the processing time of process A (5 milliseconds × 20) is longer than the processing time of process B (30 milliseconds × 2) in terms of the neural network as a whole. It is determined that A should be circuitized with priority over process B.

On the other hand, the accuracy of the neural network is basically the same regardless of whether each process is circuitized or software processed. However, if the activation function is processed using a linear approximation function, the accuracy may decrease. Therefore, it is necessary to consider both the degree of shortening of the execution time and the degree of deterioration of accuracy due to circuitization in determining whether or not to circuitize the processing of the activation function. Regarding the decrease in accuracy due to linear approximation, the larger the range error (difference between linear approximation and non-linear approximation) in the domain of possible function inputs in the neural network caused by linear approximation of the activation function, , it can be said that the degree of decrease in accuracy is large. Also, it can be said that the more frequently an activation function is used in a neural network, the greater the decrease in precision when the activation function is linearly approximated. Furthermore, the location where the activation function is used in the neural network also affects the degree of accuracy degradation when the activation function is linearly approximated. For example, the activation function near the input layer, which is the first layer of the neural network, directly affects the input data to the neural network. Since the activation function is used for processing after feature quantity extraction, it is considered that a decrease in precision caused by linear approximation is small. However, it is not always necessary to evaluate that the lower the accuracy due to linear approximation, the greater the decrease in the accuracy of the activation function closer to the input layer. The circuitization location determining unit 504 evaluates the degree of deterioration of the accuracy of the neural network by integrating these. For example, the circuitization location determination unit 504 calculates the product of the error caused by linearly approximating the activation function and the number of times the activation function is used in the neural network, and calculates the accuracy reduction due to the linear approximation of the activation function. may be calculated as the degree of

In addition, since circuitization beyond the capacity of the FPGA cannot be realized, an upper limit is set for processing that can be circuitized due to the limitation of the capacity of the FPGA. On the other hand, the execution time of neural network processing is reduced more as more processing is circuitized. Furthermore, since the circuit scale of the activation function processing including nonlinear processing is larger than the circuit scale of activation function processing including nonlinear processing, If the processing of the transformation function is circuitized, the number of processing that can be circuitized is reduced due to the limitation of the capacity of the FPGA. This problem can be avoided by linearly approximating the activation function to form a circuit, but the linear approximation causes a loss of precision. In other words, there is a trade-off between the execution time of neural network processing (that is, the number of processes to be circuitized) and the accuracy of the neural network. Therefore, the circuitization location determining unit 504 determines the circuitization location by considering the balance between conflicting elements, namely, the execution time of the neural network processing and the accuracy of the neural network. For example, a method is conceivable in which a weighted linear sum of the execution time of neural network processing and the accuracy of the neural network is used as an evaluation function, and a portion to be circuitized is determined so that the value of the evaluation function is minimized. At this time, the upper limit of the execution time of the neural network processing, the lower limit of the accuracy of the neural network, the upper limit of the shared memory capacity required between the CPU and the FPGA, the bandwidth of the data transfer bus, etc. are added as constraint conditions. may Moreover, since an extreme increase in execution time and a decrease in accuracy are not preferable, a non-linear function that penalizes an increase in execution time and a decrease in accuracy may be used as the evaluation function.

As described above, according to the neural network device 100 according to the first embodiment, each processing of the neural network is performed in consideration of the degree of reduction in the execution time of neural network processing and the degree of deterioration in accuracy due to circuitization of processing. Neural network processing is designed by determining whether to circuitize on an FPGA or to perform software processing on a processor such as a CPU. As a result, it is possible to design a neural network in which the processor and the circuit on the FPGA cooperate to perform processing, and to realize a neural network capable of performing high-speed processing on hardware with small resources.

9 and 10 are diagrams showing examples of hardware configurations of the neural network constructing unit 101, respectively. Each function of the components of the neural network constructing unit 101 shown in FIG. 1 is implemented by, for example, the processing circuit 10 shown in FIG. That is, the neural network constructing unit 101 determines, for each operation that constitutes the neural network, an operation method, that is, whether the operation is to be circuitized or software processed, and a circuit for circuitizing the operation determined to be circuitized. A processing circuit 10 is provided for creating and outputting a program for software processing of information and operations determined to be processed by software. The processing circuit 10 may be dedicated hardware, or a processor (central processing unit (CPU: Central Processing Unit), processing device, arithmetic device, microprocessor, microcomputer, etc.) that executes a program stored in a memory. Also called a DSP (Digital Signal Processor)).

When the processing circuit 10 is dedicated hardware, the processing circuit 10 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these. Each function of the components of the neural network constructing unit 101 may be realized by individual processing circuits, or these functions may be collectively realized by one processing circuit.

FIG. 10 shows an example of the hardware configuration of the neural network construction unit 101 when the processing circuit 10 is configured using a processor 11 that executes programs. In this case, the functions of the components of the neural network construction unit 101 are implemented by software or the like (software, firmware, or a combination of software and firmware). Software or the like is written as a program and stored in the memory 12 . The processor 11 reads out and executes programs stored in the memory 12 to achieve the functions of each unit. That is, when executed by the processor 11, the neural network constructing unit 101 performs a process of determining an operation method for each operation constituting the neural network, that is, whether the operation is to be circuitized or software-processed. Circuit information for circuitizing the determined operation, processing to create and output a program for software processing of the determined operation, and a program to be executed as a result are stored. A memory 12 is provided for In other words, it can be said that this program causes a computer to execute the procedures and methods of operation of the components of the neural network constructing unit 101 .

Here, the memory 12 is, for example, a non-volatile or Volatile semiconductor memories, HDDs (Hard Disk Drives), magnetic disks, flexible disks, optical disks, compact disks, mini disks, DVDs (Digital Versatile Disks) and their drive devices, as well as all storage media that will be used in the future. good too.

The configuration in which the functions of the components of the neural network constructing unit 101 are realized by either hardware or software has been described above. However, the configuration is not limited to this, and a configuration in which some components of the neural network construction unit 101 are realized by dedicated hardware and another part of the components is realized by software or the like may be used. For example, the functions of some of the components are implemented by the processing circuit 10 as dedicated hardware, and the processing circuit 10 as the processor 11 executes the programs stored in the memory 12 for some of the other components. Its function can be realized by reading and executing it.

As described above, the neural network constructing unit 101 can implement each of the functions described above using hardware, software, etc., or a combination thereof.

<Embodiment 2>
FIG. 11 is a block diagram showing the configuration of neural network device 100 according to the second embodiment. In FIG. 11, elements identical or equivalent to those shown in Embodiment 1 (FIG. 1) are denoted by the same reference numerals, and descriptions thereof are omitted here.

As shown in FIG. 11 , the neural network device 100 according to the second embodiment includes a neural network construction section 101 and a neural network execution section 901 . The neural network execution unit 901 has a storage unit 905 , a CPU 902 , an FPGA 903 , a memory 904 and a data acquisition circuit 906 .

The neural network execution unit 901 executes neural network arithmetic processing in which the CPU 902 and the FPGA 903 work together based on the program and circuit information created by the neural network construction unit 101 .

A storage unit 905 stores the program and circuit information created by the neural network construction unit 101 . The CPU 902 reads a program stored in the storage unit 905 and performs arithmetic processing of the neural network assigned to the CPU 902 and control of the FPGA 903 based on the program. The FPGA 903 reads circuit information stored in the storage unit 905 , configures an arithmetic circuit based on the circuit information, and performs arithmetic processing of the neural network assigned to the FPGA 903 .

The memory 904 is for relaying data exchanged between the CPU 902 and the FPGA 903 . More specifically, the CPU 902 stores input data for computation using a circuit constructed on the FPGA 903 in the memory 904, and the FPGA 903 reads out this input data and uses it for computation on the circuit. The FPGA 903 stores the calculation result in the memory 904, and the CPU 902 reads the calculation result from the memory 904 and uses it for software processing.

A data acquisition circuit 906 is a circuit used when the FPGA 903 reads data from the memory 904 . In this embodiment, the data acquisition circuit 906 is built on the FPGA 903 as one of the arithmetic circuits.

In general, when an FPGA reads out data from an external memory and performs an operation, it receives a notification that necessary data has been stored in a predetermined location in the memory and starts acquiring data. The data acquisition circuit 906 is for omitting this notification process. Specifically, the data acquisition circuit 906 predetermines the size of the input data for each circuit on the FPGA 903, and automatically transfers the data to the FPGA 903 when the data of that size is available on the memory 904. . Since the size of the input data is determined when the neural network construction unit 101 determines the circuit configuration, it can be calculated by the data acquisition circuit control data generation unit 303 of the neural network construction unit 101 . The neural network construction unit 101 includes the size of the input data of each circuit calculated by the data acquisition circuit control data generation unit 303 in the circuit information and stores the circuit information in the storage unit 905 .

In FIG. 11, the CPU 902, FPGA 903, and memory 904 are shown as separate blocks. good.

In addition, it is possible to freely combine each embodiment, and to modify or omit each embodiment as appropriate.

It is to be understood that the above description is illustrative in all aspects and that countless variations not illustrated can be envisaged.

100 neural network device, 101 neural network construction unit, 102 neural network analysis unit, 103 neural network calculation method output unit, 104 storage unit, 201 network structure analysis unit, 202 neural network division unit, 301 control program creation unit, 302 arithmetic circuit Creation unit 303 Control data generation unit for data acquisition circuit 401 Operation structure classification unit 402 Convolution layer analysis unit 403 Activation layer analysis unit 501 Convolution layer circuitization unit 502 Activation layer circuitization unit 503 Circuit scale Calculation unit 504 Circuit location determination unit 901 Neural network execution unit 902 CPU 903 FPGA 904 Memory 905 Storage unit 906 Data acquisition circuit 10 Processing circuit 11 Processor 12 Memory.

Claims (7)

a neural network analysis unit that determines an operation method for each operation that constitutes the neural network, whether the operation is circuitized or software-processed;
a neural network operation method output unit that creates and outputs circuit information for circuitizing the operation determined to be circuitized and a program for software processing the operation determined to be processed by software;
A neural network device comprising:
The neural network analysis unit is
a network structure analysis unit that analyzes the operation structure of the neural network;
a neural network dividing unit that determines whether each of the operations obtained by dividing the neural network is to be circuitized or software-processed;
The network structure analysis unit
an operation structure classification unit that classifies each layer of the neural network according to the type of operation that constitutes the layer;
a convolution layer analysis unit that identifies parameters for the convolution operation for layers classified by the operation structure classification unit as layers that perform convolution operations;
The neural network dividing unit
a convolutional layer circuitization unit that groups layers having the same or similar parameters based on the parameters identified by the convolutional layer analysis unit;
Neural network device. The neural network dividing unit further
a circuit scale calculation unit for calculating a circuit scale when the convolution operation is circuitized for each convolution operation of the layers grouped by the convolution layer circuitization unit;
a circuitization location determination unit that determines an operation to be circuitized based on the circuit scale calculated by the circuit scale calculation unit;
The neural network device according to claim 1. The neural network operation method output unit,
an arithmetic circuit creation unit that creates the circuit information;
a control program creation unit that creates the program,
The program created by the control program creation unit includes a control program for managing input/output of an arithmetic circuit constructed based on the circuit information.
3. The neural network device according to claim 1 or 2 . The network structure analysis unit
further comprising an activation layer analysis unit that specifies an activation function used in each layer classified by the arithmetic structure classification unit as a layer that performs processing based on the activation function;
The neural network dividing unit
Grouping the same activation functions among the activation functions used in each layer classified into layers that perform processing based on the activation functions by the arithmetic structure classification unit, and linearly approximable among the grouped activation functions further comprising an activation layer circuitization unit that stores a linear approximation function obtained by linearly approximating an object in association with the activation function;
The circuit scale calculation unit further calculates a circuit scale when each of the grouped activation function and the linear approximation function is circuitized.
3. The neural network device according to claim 2 . The neural network device is
Further comprising a neural network execution unit that executes neural network processing based on the circuit information and the program output by the neural network operation method output unit,
The neural network execution unit is
a storage unit that stores the circuit information and the program;
a CPU that executes the program;
an FPGA that constructs an arithmetic circuit based on the circuit information and executes an arithmetic operation by the arithmetic circuit;
a memory for relaying data between the CPU and the FPGA;
comprising a
The neural network device according to any one of claims 1 to 4 . The neural network execution unit is
further comprising a data acquisition circuit that automatically acquires data passed from the CPU to the FPGA via the memory from the memory and reads the data into the FPGA;
The neural network operation method output unit,
further comprising a data acquisition circuit control data generation unit that creates data for controlling the data acquisition circuit based on the circuit information;
The neural network device according to claim 5 . The data for controlling the data acquisition circuit is data indicating the size of the input data of the arithmetic circuit.
The neural network device according to claim 6 .

Images