KR20200138414A - Network-on-chip data processing method and device - Google Patents

Abstract

The present application relates to a network-on-chip data processing method applied to a network-on-chip processing system, wherein the network-on-chip processing system performs machine learning calculations, and includes a storage device and a calculation device. The method includes the steps of: accessing a storage device of the network-on-chip processing system through a first computing device of the network-on-chip processing system to obtain first operation data; Obtaining a first operation result by performing an operation on the first operation data through the first calculation device; And transmitting the first calculation result to a second calculation device of the network on-chip processing system. This method can reduce computational overhead and improve data read/write efficiency.

Description

Network-on-chip data processing method and device {NETWORK-ON-CHIP DATA PROCESSING METHOD AND DEVICE}

Related application

For this application, the application number filed on October 18, 2018 is 201811216718.9, the name of the invention is "Network on-chip processing system and network on-chip data processing method", the application number is 201811215820.7, and the name of the invention is "Network on-chip processing system and Network on-chip data processing method", the application number is 201811215978.4, the name of the invention is "Network on-chip processing system and network on-chip data processing method", the application number is 201811216857.1, and the name of the invention is "Network on-chip data processing method, storage medium, Computer equipment and devices", the application number filed on November 21, 2018 is 201811392232.0, the name of the invention is "data processing method, device and related products", the application number is 201811392262.1, and the name of the invention is "data processing method , Device and related products", the application number is 201811392279.7, the name of the invention is "data processing device, method and related products", the application number is 201811393352.2, and the name of the invention is "data processing device, method and related products", The application number is 201811390409.3, the name of the invention is "Data processing device, method and related products", the application number is 201811390428.6, the name of the invention is "Data processing device and related products", the application number is 201811392270.6, and the name of the invention is " It claims the priority of a Chinese patent application for "Data Processing Devices and Related Products", and the entire contents are cited for reference.

The present application relates to the field of information processing technology, and more particularly, to a method and apparatus for processing network on-chip data.

As semiconductor process technology advances, it has already become a reality to integrate hundreds of millions of transistors on a single chip. Network on Chip (NoC) can implement on-chip communication by integrating a large amount of computational resources on a single chip.

In a neural network, a large amount of computation must be performed, so processing such as forward computation, reverse computation, and weight update should be performed in parallel for some computations. In a chip structure with a large number of transistors, the chip design will face problems such as high memory access overhead, large bandwidth blockage, and low data read/write efficiency.

The present application provides an interaction method, an apparatus, and an intelligent terminal in order to at least to some extent solve problems existing in the related art.

According to a first aspect, the present application provides a network on-chip processing system. The system includes a storage device and a plurality of calculation devices installed on the same chip, at least one calculation device being connected to the storage device and at least two or more calculation devices being connected to each other.

In one embodiment, any two of the plurality of computing devices are directly connected.

In one embodiment, the plurality of calculation devices includes a first calculation device and a plurality of second calculation devices, and the first calculation device is connected to the storage device, and at least one of the plurality of second calculation devices The second calculation device is connected to the first calculation device.

In an embodiment, at least two or more of the second calculation devices among the plurality of second calculation devices are connected to each other, and are connected to the storage device through the first calculation device.

In one embodiment, any two second calculation devices of the plurality of second calculation devices are directly connected to the first calculation device.

In one embodiment, each of the plurality of calculation devices is connected to the storage device, and at least two or more calculation devices are connected to each other.

According to a second aspect, an embodiment of the present application provides a neural network computing device including one or more of the computing devices in the network on-chip processing system. The neural network computing device obtains operand input data and control information from another processing device, performs a designated machine learning operation, and transmits the execution result to another processing device through an I/O interface.

When the neural network computing device includes a plurality of the computing devices, the plurality of computing devices may be connected through a specific structure to transmit data.

Here, since the plurality of calculation devices are interconnected through a PCI bus to transmit data, a larger-scale machine learning operation is supported, and the plurality of calculation devices share one control system or have their own control systems. The plurality of computing devices share a memory or have their own memory, and an interconnect method of the plurality of computing devices is an arbitrary interconnect topology.

According to a third aspect, an embodiment of the present application provides an integrated processing device including a machine learning processing device, a universal interconnect interface, and other processing devices according to the second aspect. The neural network computing device and the other processing device interact with each other to jointly perform a computing operation designated by the user. The integrated processing device may further include a storage device connected to the neural network computing device and the other processing device, respectively, to store data of the neural network computing device and the other processing device.

According to a fourth aspect, an embodiment of the present application provides a neural network chip including a computing device of the network on-chip processing system, a neural network computing device according to the second aspect, or an integrated processing device according to the third aspect.

According to a fifth aspect, an embodiment of the present application provides a neural network chip package structure including the neural network chip according to the fourth aspect.

According to a sixth aspect, an embodiment of the present application provides a board card including the neural network chip package structure according to the fifth aspect.

According to a seventh aspect, an embodiment of the present application provides an electronic device including a neural network chip according to the fourth aspect or a board card according to the sixth aspect.

According to the eighth aspect, the embodiment of the present application further provides a network on-chip data processing method for performing machine learning calculation.

The above method,

Accessing the storage device through the first calculation device to obtain first calculation data;

Obtaining a first operation result by performing an operation on the first operation data through the first calculation device; And

And transmitting the first calculation result to a second calculation device.

In one embodiment, the method further includes accessing the storage device through the second computing device to obtain second computational data.

In one embodiment, the method further comprises the step of obtaining a second operation result by calculating on the second operation data and the first operation result through the second calculation device.

The network on-chip processing system includes a storage device installed on the same chip and a plurality of computing device groups, and includes a plurality of computing devices for each computing device group, and at least one computing device group among the plurality of computing device groups is the storage device. Devices are connected, and at least two groups of computing devices are connected to each other.

In one embodiment, any two computing device groups of the plurality of computing device groups are directly connected.

In one embodiment, each said computing device group includes at least one or more computing devices connected to at least one or more computing devices of another said computing device group.

In one embodiment, the plurality of computing device groups are connected to each other through any one computing device in the plurality of computing device groups.

In one embodiment, at least one or more computing devices of each group of computing devices are connected to the storage device and at least two or more computing devices are connected to each other.

In one embodiment, any two computing devices in each of the groups of computing devices are directly connected.

In one embodiment, each said computing device group includes a first computing device and a plurality of second computing devices, wherein the first computing device is connected to the storage device, and at least one of the plurality of second computing devices The above second calculation device is connected to the first calculation device.

In one embodiment, at least two or more second calculation devices among a plurality of second calculation devices in each of the calculation device groups are connected to each other, and are connected to the storage device through the first calculation device.

In one embodiment, any two of a plurality of second computing devices in each said computing device group are directly connected to the first computing device.

In one embodiment, each of the plurality of calculation devices in each of the calculation device groups is connected to the storage device, and at least two or more calculation devices are connected to each other.

In one embodiment, the plurality of calculation device groups includes a main calculation device group and a plurality of sub calculation device groups, and the main calculation device group is connected to the storage device, and at least one of the plurality of sub calculation device groups The sub-computing device group is connected to the main calculating device group.

In one embodiment, at least two or more sub-calculation device groups among the plurality of sub-calculation device groups are connected to each other and to the storage device through the main calculation device group.

In one embodiment, any two sub computing device groups of the plurality of sub computing device groups are directly connected to the main computing device group.

In one embodiment, all of the computing device groups among the plurality of computing device groups are all connected to the storage device, and at least two or more computing device groups are connected to each other.

The embodiment of the present application further provides a method for processing network on-chip data. The above method,

Accessing a storage device through a first computing device group including a plurality of first computing devices to obtain first computational data;

Obtaining, by the first calculation device group, a first calculation result by operating on the plurality of first calculation data; And

And transmitting the result of the first calculation to a second group of calculation devices.

In one embodiment, the method further comprises accessing the storage device through the second computing device group comprising a plurality of second computing devices to obtain second computational data.

In one embodiment, the method further comprises the step of obtaining a second operation result by calculating on the second operation data and the first operation result through the second calculation device group.

In one embodiment, the method further comprises the step of obtaining a second calculation result by calculating on the second calculation data and the first calculation result through the second calculation device group, wherein the step comprises:

And calculating and forwarding the second calculation data and the first calculation result between the plurality of second calculation devices to obtain the second calculation result.

The network on-chip processing system includes a plurality of network on-chip processing modules installed on the same chip and connected to each other, each network on-chip processing module includes at least one storage device and a plurality of computing devices, and each network on-chip processing module At least one or more computing devices are connected to at least one or more storage devices inside the network on-chip processing module, and at least two or more of the plurality of computing devices are connected to each other.

In one embodiment, a plurality of calculation devices among each network on-chip processing module include a first calculation device and a plurality of second calculation devices, and the first calculation device includes at least one storage device inside the network on-chip processing module. And at least one second calculation device of the plurality of second calculation devices is connected to the first calculation device.

In one embodiment, at least two or more second computing devices of each network on-chip processing module are connected to each other and connected to at least one or more storage devices in the network on-chip processing module through the first computing device.

In one embodiment, any two second computing devices of each network on-chip processing module are directly connected to the first computing device.

In an embodiment, each of the computing devices among each network on-chip processing module is connected to at least one or more storage devices inside the network on-chip processing module, and at least two or more computing devices are connected to each other.

In one embodiment, any two computing devices of each network on-chip processing module are directly connected.

In one embodiment, each network on-chip processing module includes a plurality of storage devices, and in the network on-chip processing module at least one computing device is connected to the plurality of storage devices inside the network on-chip processing module.

In one embodiment, all of the computing devices among each network on-chip processing module are connected to the plurality of storage devices in the network on-chip processing module.

In one embodiment, each network on-chip processing module includes at least one or more computing devices and is connected to at least one or more computing devices of other network on-chip processing modules.

In one embodiment, the plurality of network on-chip processing modules are connected to each other through any one computing device of each network on-chip processing module.

In one embodiment, in each network on-chip processing module, each computing device is all connected to a storage device and the distance between each computing device and the storage device is a first communication distance.

In one embodiment, any two network on-chip processing modules are directly connected.

The embodiment of the present application further provides a method for processing network on-chip data. The above method,

Acquiring first operation data through a first network on-chip processing module, wherein the first network on-chip processing module includes a first storage device and a plurality of first calculation devices, and the first operation data is 1 stored on storage device-;

Calculating the first calculation data through a plurality of first calculation devices in the first network on-chip processing module to obtain a first calculation result; And

And transmitting the first operation result to a second network on-chip processing module.

In one embodiment, the method further comprises acquiring second operation data through the second network on-chip processing module, wherein the second network on-chip processing module comprises a second storage device and a plurality of second And a calculation device, wherein the second calculation data is stored in the second storage device.

In one embodiment, the method further includes calculating the second operation data and the first operation result through a plurality of second calculation devices in the second network on-chip processing module to obtain a second operation result. .

In one embodiment, the method,

Calculating the second calculation data and the first calculation result between the plurality of second calculation devices to obtain the second calculation result; And

And storing the second operation result in the second storage device.

In one embodiment, the method,

Accessing the first storage device through a first main computing device in the first network on-chip processing module to obtain the first calculation data;

Forwarding the first calculation data between a first main calculation device and a plurality of first sub calculation devices in the first network on-chip processing module; And

Further comprising the step of calculating the first calculation data through a first main calculation device and a plurality of first sub calculation devices in the first network on-chip processing module to obtain the first calculation result,

Here, the first calculation device includes a first main calculation device and a plurality of first sub calculation devices.

A network on-chip data processing method applied to a network on-chip processing system that includes a storage device and a computing device and performs machine learning calculation,

The above method,

Accessing a storage device in the network on-chip processing system through a first computing device in the network on-chip processing system to obtain first operation data;

Obtaining a first operation result by performing an operation on the first operation data through the first calculation device; And

And transmitting the first calculation result to a second calculation device in the network on-chip processing system.

In one embodiment, the calculation device comprises a calculation unit and a controller unit,

The step of accessing the storage device of the network-on-chip processing system through the first computing device of the network-on-chip processing system to obtain first operation data,

And obtaining, by the controller unit of the first calculation device, the first calculation data and calculation instruction from the storage device.

In one embodiment, the operation unit includes one master processing circuit and a plurality of slave processing circuits.

The step of obtaining a first calculation result by calculating on the first calculation data through the first calculation device,

A plurality of calculation instructions are obtained by analyzing the calculation instructions through a controller unit in the first calculation device, and a controller unit in the first calculation device transmits the plurality of calculation commands and the first calculation data to the first calculation device. Transmitting to a mast processing circuit;

Performing pre-order processing on the first calculation data through a mast processing circuit in the first calculation device, and transmitting data and calculation instructions to a plurality of slave processing circuits among the first calculation devices;

A plurality of slave processing circuits among the first calculation devices perform intermediate operations according to calculation data and calculation instructions transmitted from the mast processing circuit in the first calculation device to obtain a plurality of intermediate results, and obtain a plurality of intermediate results. 1 transmitting to the mast processing circuit of the calculating device; And

And a step of obtaining, by the mast processing circuit of the first calculation device, a subsequent processing on the plurality of intermediate results to obtain a first calculation result of the calculation instruction.

In one embodiment, the step of transmitting the first calculation result to a second calculation device in the network on-chip processing system,

And transmitting, by the controller unit of the first calculation device, the first calculation result to a second calculation device in the network on-chip processing system.

In one embodiment, the machine learning calculation includes an artificial neural network calculation, the first calculation data includes input neuron data and a weight, and the first calculation result is output neuron data.

In one embodiment, the computing device further comprises a storage unit and a direct memory access unit, the storage unit comprising any combination of registers and caches,

The cache stores the first operation data,

The register stores a scalar in the first operation data.

In one embodiment, the controller unit includes an instruction storage unit, an instruction processing unit and a storage queue unit,

The instruction storage unit stores calculation instructions related to the artificial neural network,

The instruction processing unit analyzes the computation instruction to obtain a plurality of computation instructions,

The storage queue unit includes a command queue, and the command queue includes a plurality of calculation commands and/or calculation commands to be executed according to a preceding and subsequent order of the command queue.

In one embodiment, the mast processing circuit includes a dependency relationship processing unit,

The dependency relationship processing unit determines whether there is an association relationship between a first arithmetic instruction and a 0th arithmetic instruction prior to the first arithmetic instruction, and if there is a correlation between the first arithmetic instruction and the 0th arithmetic instruction Caching the first operation instruction in the instruction storage unit, extracting the first operation instruction from the instruction storage unit after performing the 0th operation instruction and transmitting the extracted first operation instruction to the operation unit;

The step of determining whether there is an association relationship between the first operation instruction and a zeroth operation instruction prior to the first operation instruction,

Extracting a first storage address section of data required for the first operation instruction by the first operation instruction, and extracting a zeroth storage address section of data required for the 0th operation instruction by the 0th operation instruction, If there is an overlapped region between the first storage address section and the 0th storage address section, it is determined that there is a correlation between the first operation command and the 0th operation command, and the first storage address section and the first And if there is no overlapped region in the 0 storage address section, determining that there is no correlation between the first operation instruction and the zeroth operation instruction.

In one embodiment, the arithmetic unit includes a tree module, the tree module includes one root port and a plurality of branch ports, the root port of the tree module is connected to the mast processing circuit, and the tree module The plurality of branch ports of are respectively connected to one slave processing circuit among the plurality of slave processing circuits,

The tree module forwards data blocks, weights, and operation instructions between the mast processing circuit and the plurality of slave processing circuits.

In one embodiment, the operation unit includes one or a plurality of branch processing circuits, each branch processing circuit is connected to at least one or more slave processing circuits,

The mast processing circuit determines the input neuron as broadcast data and the weight as distribution data, distributes the distribution data into a plurality of data blocks, and includes at least one data block, broadcast data, and a plurality of data blocks among the plurality of data blocks. Transmits at least one operation instruction of the operation instructions of to the branch processing circuit,

The branch processing circuit forwards data blocks, broadcast data, and operation instructions between the mast processing circuit and the plurality of slave processing circuits,

The plurality of slave processing circuits operate on the data block and broadcast data received by the operation instruction to obtain an intermediate result and transmit the intermediate result to the branch processing circuit,

The mast processing circuit subsequently processes the intermediate result transmitted from the branch processing circuit to obtain a first operation result of the calculation instruction, and transmits the first operation result of the calculation instruction to the controller unit.

In one embodiment, the plurality of slave processing circuits form an array distribution, each slave processing circuit is connected to another adjacent slave processing circuit, and the mast processing circuit processes K slaves among the plurality of slave processing circuits. Connected to the circuit, wherein the K basic circuits are n slave processing circuits in a first row, n slave processing circuits in an mth row, and m slave processing circuits in a first column,

The K slave processing circuits forward data and commands between the mast processing circuit and a plurality of slave processing circuits,

The mast processing circuit determines the input neuron as broadcast data and the weight as distribution data, distributes the distribution data to a plurality of data blocks, and at least one of at least one data block of the plurality of data blocks and a plurality of operation instructions. One or more operation instructions are transmitted to the K slave processing circuits,

The K slave processing circuits forward data between the mast processing circuit and the plurality of slave processing circuits,

The plurality of slave processing circuits operate on the received data block according to the operation instruction to obtain an intermediate result, and transmit the operation result to the K slave processing circuits,

The mast processing circuit subsequently processes intermediate results transmitted by the K slave processing circuits to obtain a first calculation result of the calculation command, and transmits the first calculation result of the calculation command to the controller unit.

In one embodiment, the mast processing circuit combines and sorts intermediate results transmitted from a plurality of processing circuits to obtain a first operation result of the calculation instruction, or

Not only are the intermediate results transmitted from the plurality of processing circuits combined and sorted, but also the first operation result of the calculation instruction is obtained through activation processing.

In one embodiment, the mast processing circuit includes one or any combination of a conversion processing circuit, an activation processing circuit, and an addition processing circuit,

The conversion processing circuit performs pre-order processing on the first operation data, and specifically, compatibility between the first data structure and the second data structure for the data or intermediate result received by the master processing circuit, or The processing circuit performs compatibility between the first data type and the second data type on the received data or intermediate results,

The activation processing circuit performs the subsequent processing, specifically, performs an activation operation of data in the mast processing circuit,

The addition processing circuit performs the subsequent processing, but specifically performs an addition operation or an accumulation operation.

In one embodiment, the slave processing circuit includes a multiplication processing circuit,

The multiplication processing circuit obtains a multiplication result by performing a multiplication operation on the received data block.

In one embodiment, the slave processing circuit includes an accumulation processing circuit, and the accumulation processing circuit obtains the intermediate result by performing an accumulation operation on the multiplication result.

In one embodiment, the tree module is an n-number tree structure, and n is an integer greater than or equal to 2.

In one embodiment, the method further includes accessing a storage device in the network on-chip processing system through a second computing device in the network on-chip processing system to obtain second computational data.

In one embodiment, the method further includes calculating the second operation data and the first operation result through a second calculation device in the network on-chip processing system to obtain a second operation result.

An embodiment of the present application provides a network on-chip data processing apparatus that performs machine learning calculations. The device,

A first arithmetic data acquisition module for acquiring first arithmetic data by accessing a storage device in the network-on-chip processing system through a first computing device in the network-on-chip processing system;

An operation module for obtaining a first operation result by calculating on the first operation data through the first calculation device; And

And a first calculation result transmission module for transmitting the first calculation result to a second calculation device in the network on-chip processing system.

This application provides a data processing method. The above method,

Receiving a data operation signal transmitted from an internal or external device, wherein the data operation signal includes an operation domain and an operation code, the operation code includes a first type flag bit, and the operation domain is 2 type flag bits are included, the first type flag bit indicates whether the data operation signal is an I/O command, and the second type flag bit indicates whether the data operation signal is broadcast in the I/O command or Indicates whether it is a multicast command or not; And

And obtaining necessary input data by performing a corresponding operation on operation-planned data in a memory according to the data operation signal.

In one embodiment, the operation domain further includes a data reception flag bit, and the data reception flag bit represents an apparatus or processing circuit that receives the input data.

In an embodiment, the number of data reception flag bits may represent the number of devices interactable with the memory or the number of processing circuits.

The operation domain further includes information on operation schedule data, and the operation schedule data information includes a source address in the memory, an operation schedule data length, and a data return address after data operation. The step of obtaining necessary input data by performing a corresponding operation on the operation scheduled data in the memory according to the data operation signal,

Starting to read the memory from the source address to obtain input data satisfying the data length;

Determining an apparatus or processing circuit for receiving input data by the data reception flag bit; And

And returning the input data to a storage space corresponding to the data return address in the device or processing circuit by the data return address.

In one embodiment, the operation domain further includes a jump sub-operation domain, the jump sub-operation domain includes a jump width and a jump data length that operates after each jump, and the memory is read from the source address. Thus, the step of obtaining input data satisfying the data length,

Reading the memory from the source address and acquiring first jump data based on a jump data length after a current jump;

Acquiring a last address of the jump data and jumping from the last address to a target jump address according to the jump width;

And obtaining second jump data by the jump data length after jumping from the target jump address until the jump data length obtained after each jump satisfies the data length.

In one embodiment, the jump sub-operation domain includes a stride operation domain and/or a segment operation domain, the stride operation domain represents a jump width of the data operation signal each time, and the segment operation domain is the preset data It represents the size of each division of the motion signal.

In one embodiment, the operation domain represents a processing operation performed on the read data.

In one embodiment, the method,

If the value of the first type flag bit is I/O, determining the data operation signal as an I/O command; And

If the value of the second type flag bit is 1, determining the data operation signal as a broadcast or multicast command among the I/O commands.

In one embodiment, the step of receiving a data operation signal transmitted from the internal or external device,

Analyzing the data operation signal to obtain information on a type flag bit of the data operation signal and operation schedule data;

And performing the analyzed data operation signal according to a command queue indicating an execution order of the data operation signal.

In one embodiment, prior to performing the analyzed data operation signal by the command queue, the method,

Determining a dependence relationship between the analyzed data operation signals adjacent to each other and obtaining a determination result, the dependence relationship is an s-th data operation signal and an s-1th data operation signal before the s-th data operation signal Indicates whether there is an association -; And

If the determination result is that there is a dependency relationship between the s-th data operation signal and the s-1 th data operation signal, the s-th data operation signal is cached, and the s-th data operation signal is After performing the operation, the method further includes extracting the s-th data operation signal.

In one embodiment, the step of determining the dependence of the analyzed data operation signal adjacent to,

A first storage address section for extracting data necessary for the s-th data operation signal based on the s-th data operation signal; And obtaining 0th storage address intervals for extracting data necessary for the s-1th data operation signal according to the s-1th data operation signal.

Determining that the s-th data operation signal and the s-1 th data operation signal have a dependency relationship if there is an overlapped region between the first storage address section and the 0-th storage address section; And

And determining that there is no dependency relationship between the s-th data operation signal and the s-1 th data operation signal if there is no overlapped region between the first storage address section and the 0-th storage address section. .

An embodiment of the present application provides a data processing apparatus including a processor and a memory. When a computer program is stored in the memory, and the processor executes the computer program,

Receiving a data operation signal transmitted from an internal or external device, wherein the data operation signal includes an operation domain and an operation code, the operation code includes a first type flag bit, and the operation domain is 2 type flag bits are included, the first type flag bit indicates whether the data operation signal is an I/O command, and the second type flag bit indicates whether the data operation signal is broadcast in the I/O command or Indicates whether it is a multicast command or not; And

A step of obtaining required input data by performing a corresponding operation on operation-planned data in a memory according to the data operation signal; is implemented.

This application provides a data processing method. The above method,

Receiving a data operation signal transmitted from an internal or external device,-The data operation signal includes a data operation code including the type flag bit, and the type flag bit is a broadcast or multicast command of the operation signal Represents -; And

And obtaining necessary input data by performing a corresponding operation on operation-planned data in a memory according to the data operation signal.

An embodiment of the present application provides a data processing apparatus including a processor and a memory. When a computer program is stored in the memory, and the processor executes the computer program,

Receiving a data operation signal transmitted from an internal or external device,-The data operation signal includes a data operation code including the type flag bit, and the type flag bit is a broadcast or multicast command of the operation signal Represents -; And

A step of obtaining necessary input data by performing a corresponding operation on operation-planned data in a memory according to the data operation signal is implemented.

In one embodiment, the data operation signal further includes an operation domain, the operation domain includes a data reception flag bit, and the data reception flag bit represents an apparatus or a processing circuit that receives the input data.

In an embodiment, the number of data reception flag bits may represent the number of devices interactable with the memory or the number of processing circuits.

In one embodiment, the method,

If the value of the type flag bit is CAST, determining the data operation signal as a broadcast or multicast command.

In one embodiment, the step of receiving a data operation signal transmitted from the internal or external device,

Analyzing the data operation signal to obtain information on a type flag bit of the data operation signal and operation schedule data;

And performing the analyzed data operation signal according to a command queue indicating an execution order of the data operation signal.

A data processing device for processing machine learning data, wherein the data processing device includes a machine learning device, a transmission circuit, and a shared memory, the machine learning device is connected to the transmission circuit, and the transmission circuit is connected to the shared memory. Connected,

The transmission circuit acquires input data required for the machine learning device from the shared memory by a data operation signal sent from the machine learning device, and returns the input data to the machine learning device, and the data operation signal is a data operation. It contains the information of the type flag bit of the signal and the operation schedule data.

In an embodiment, the machine learning apparatus obtains output data by performing a machine learning operation based on the input data.

In one embodiment, the machine learning apparatus stores data by additionally transmitting the output data to the shared memory through the transmission circuit.

In one embodiment, the machine learning device includes at least one machine learning unit,

The data operation signal includes a data reception flag bit, and the data reception flag bit represents a target machine learning unit that has received the input data.

In one embodiment, the value of the type flag bit of the data operation signal includes CAST, and indicates that the data operation signal is a broadcast or multicast command.

In one embodiment, the type flag bit of the data operation signal includes a first type flag bit and a second type flag bit.

Here, the value of the first type flag bit includes I/O, and indicates whether the data operation signal is an I/O command.

The second type flag bit indicates whether the data operation signal is a broadcast or multicast command of the I/O command.

In one embodiment, the information on the operation schedule data includes at least one of a source address in the shared memory, a data length to be operated, and a data return address after data operation is performed.

In one embodiment, the data operation signal further includes jump information, and the jump information includes a jump width and a data length operated after each jump.

In one embodiment, the jump information includes stride jump information and/or segment jump information,

The stride jump information represents a jump width of the data operation signal each time,

The segment operation domain represents a predetermined division size of the data operation signal.

In one embodiment, the data operation signal further includes a function flag bit indicating a processing operation to be performed on the read data.

The embodiment of the present application further provides a data processing method applied to the data processing apparatus. The above method,

Receiving, by a transmission circuit of the data processing apparatus, a data operation signal transmitted from the machine learning apparatus of the data processing apparatus,-the data operation signal includes a type flag bit of a data operation signal and information of operation scheduled data- ;

The machine learning apparatus by determining an operation to be performed by the transmission circuit on data in a shared memory according to the type flag bit of the data operation signal, and performing the operation on the operation scheduled data according to the operation scheduled data information. Obtaining input data necessary for the process and returning the input data to the machine learning device; And

The machine learning apparatus performs a machine learning operation based on the input data to obtain output data, and converts the output data to new input data, and transmits and stores the output data to a shared memory through the transmission circuit.

In one embodiment, the machine learning device includes at least one machine learning unit, the data operation signal includes a data reception flag bit, and returning the input data to the machine learning device,

The transmission circuit determines a target machine learning unit for receiving the input data based on the value of the data reception flag bit, and sends the input data to the target machine learning unit.

In one embodiment, the operation schedule data information includes a source address in the memory, an operation schedule data length, and a data return address after data operation, and the operation schedule data includes the operation schedule data. The step of performing the operation on predetermined data to obtain input data required for the machine learning device and returning the input data to the machine learning device,

The transmission circuit starting to read the shared memory from the source address to obtain the input data satisfying the data length;

And returning the input data to the target machine learning unit by the data return address and the data reception flag bit.

The present application provides a data processing apparatus. The data processing device includes a machine learning device, a transmission circuit, and a shared memory, the machine learning device includes at least one machine learning unit, and a unicast read operation and a broadcast operation performed by the machine learning unit are Share one data receiving interface, the machine learning unit is connected to the transmitting circuit through a transmitting interface and a shared data receiving interface, the transmitting circuit is connected to the shared memory,

The transmission circuit acquires input data necessary for the machine learning device from the shared memory by a data operation signal sent by the machine learning device through the transmission interface, and transmits the input data to the machine through the shared data reception interface. Return to the learning device.

In an embodiment, the machine learning apparatus obtains output data by performing a machine learning operation based on the input data.

In one embodiment, the machine learning apparatus further transmits and stores the output data to the shared memory through the transmission circuit.

In one embodiment, the transmission interface includes a unicast read signal transmission interface and a broadcast signal transmission interface, and the machine learning unit connects the unicast read signal transmission interface and the shared data reception interface to the transmission circuit, respectively. The unicast read operation is implemented by being, and the broadcast operation is implemented by connecting the broadcast signal transmission interface and the shared data reception interface to the transmission circuit, respectively.

In one embodiment, the transmission circuit comprises a second transmission interface, a read/write processing circuit connected to the second transmission interface, and an arbitration circuit connected to the read/write processing circuit,

The read/write processing circuit receives the data operation signal transmitted by the at least one machine learning unit through the transmission interface and the second transmission interface, transmits the data operation signal to the arbitration circuit, and the arbitration circuit Returning the data acquired from the shared memory to a machine learning unit corresponding to the data operation signal through the second transmission interface and the shared data reception interface,

The arbitration circuit mediates the data operation signal received from the read/write processing circuit according to a pre-set arbitration rule, and operates the data in the shared memory according to the data operation signal in which arbitration is successful.

In one embodiment, the read/write processing circuit includes a unicast read processing circuit and a broadcast processing circuit, the unicast read processing circuit processes a unicast read signal, and the broadcast processing circuit And/or process a multicast signal.

In one embodiment, the second transmission interface comprises at least one group of unicast read signal receiving interfaces and unicast read data transmission interfaces connected to the unicast read processing circuit, and at least one group of broadcasts connected to the broadcast processing circuit. A signal reception interface and a broadcast data transmission interface, the unicast read signal reception interface is connected to a unicast read signal transmission interface of the machine learning unit, and the broadcast signal reception interface is a broadcast of the machine learning unit A signal transmission interface, and the unicast read data transmission interface and the broadcast data transmission interface of the transmission circuit are respectively connected to a shared data reception interface of the machine learning unit.

In one embodiment, the read/write processing circuit includes a broadcast processing circuit and a plurality of unicast read processing circuits, and the plurality of unicast read processing circuits and the plurality of machine learning units are connected one-to-one, and the The broadcast processing circuit and the plurality of machine learning units are connected one-to-many.

In one embodiment, the second transmission interface includes a group of broadcast interfaces connected to the broadcast processing circuit, the broadcast interface includes a broadcast signal reception interface and a broadcast data transmission interface, and the plurality of The machine learning unit of is connected to the broadcast processing circuit through the group of broadcast interfaces.

In one embodiment, the second transmission interface includes a plurality of groups of unicast read signal reception interfaces and shared data transmission interfaces connected one-to-one with the plurality of unicast read processing circuits, and a broadcast signal reception connected to the broadcast processing circuit. An interface, wherein the shared data transmission interface is also connected to the broadcast processing circuit, the unicast read signal reception interface is connected to the unicast read signal transmission interface of the machine learning unit, and the broadcast signal reception interface is It is connected to the broadcast signal transmission interface of the machine learning unit, and the shared data transmission interface is connected to the shared data reception interface of the machine learning unit.

The embodiment of the present application further provides a data processing method applied to a data processing device. The data processing device includes a machine learning device, a transmission circuit, and a shared memory, the machine learning device includes at least one machine learning unit, and a unicast read operation and a broadcast operation performed by the machine learning unit are one Share the data receiving interface, the machine learning unit is connected to the transmitting circuit through a transmitting interface and a shared data receiving interface, the transmitting circuit is connected to the shared memory, and the data processing method,

Transmitting, by the machine learning device, a data operation signal to the transmission circuit through the transmission interface; And

And returning, by the transmission circuit, input data required for the machine learning device from the shared memory according to the data operation signal, and returning the input data to the machine learning device through the shared data receiving interface.

In one embodiment, the data operation signal is a broadcast signal and/or a multicast signal, and returning the input data to the machine learning device through the shared data receiving interface,

And transmitting, by the transmission circuit, the input data to a plurality of machine learning units corresponding to the broadcast signal and/or multicast signal through the shared data receiving interface.

The present application provides a data processing apparatus. The data processing device includes a machine learning device, a transmission circuit, and a shared memory, the machine learning device includes at least one machine learning unit, and the machine learning unit includes at least one transmission interface and at least one reception interface. And at least two data operations among a unicast read operation, a unicast write operation and a broadcast operation performed by the machine learning unit share one transmission interface of the machine learning unit, and the machine learning unit transmits the transmission Connected to a circuit, the transfer circuit is connected to the shared memory,

The transmission circuit is configured to obtain input data necessary for the machine learning device from the shared memory according to a data operation signal transmitted by the machine learning device through the at least one transmission interface of the machine learning unit, and provide the receiving interface. The input data is returned to the machine learning device.

In one embodiment, the read/write processing circuit is divided into a plurality of processing circuit groups, one machine learning unit corresponds to one processing circuit group, and the processing circuit group is one unicast read processing circuit, one And a unicast recording processing circuit and one broadcast processing circuit.

In one embodiment, data returned by the unicast read processing circuit and the broadcast processing circuit in the processing circuit group share one shared data receiving interface on the machine learning unit.

In one embodiment, the at least one transmission interface includes a shared signal transmission interface and a unicast read signal transmission interface shared by a unicast write operation and a broadcast operation.

In one embodiment, the second transmission interface includes a plurality of interface groups, the one processing circuit group corresponds to one interface group, and the one interface group is a unit connected to the unicast read processing circuit. A cast read signal reception interface and a unicast read data transmission interface; A unicast read signal receiving interface connected to the unicast write processing circuit; And a broadcast signal reception interface and a loadcast data transmission interface connected to the broadcast processing circuit.

In one embodiment, in the one processing circuit group, the unicast recording processing circuit and the broadcast processing circuit share one shared signal reception interface in the corresponding interface group, and a shared signal reception interface corresponding to the processing circuit group. Is connected to the shared signal transmission interface of the machine learning unit corresponding to the processing circuit group, and the unicast read signal reception interface in the processing circuit group is connected to the unicast read signal transmission interface of the machine learning unit corresponding to the processing circuit group. Connected.

In one embodiment, the unicast read processing circuit and the broadcast processing circuit in the one processing circuit group share one shared data transmission interface in the corresponding interface group, and a shared data transmission interface corresponding to the processing circuit group Is connected to the shared data receiving interface of the machine learning unit corresponding to the processing circuit group.

In one embodiment, the shared signal receiving interface corresponding to the processing circuit group is connected to a unicast recording processing circuit and a broadcast processing circuit in the processing circuit group, respectively, and operation of data transmitted from the shared signal transmission interface of the machine learning unit A signal is received, the data operation signal is divided into two identical data operation signals, and transmitted to the unicast recording processing circuit and the broadcast processing circuit, respectively.

The embodiment of the present application further provides a data processing method applied to a data processing device. The data processing device includes a machine learning device, a transmission circuit, and a shared memory, the machine learning device includes at least one machine learning unit, and the machine learning unit includes at least one transmission interface and at least one reception interface. And at least two data operations among a unicast read operation, a unicast write operation and a broadcast operation performed by the machine learning unit share one transmission interface on the machine learning unit, and the machine learning unit transmits the transmission Connected to a circuit, the transfer circuit is connected to the shared memory, and the method comprises:

Transmitting, by the machine learning device, a data operation signal to the transmission circuit through the at least one transmission interface;

And the transmission circuit obtaining input data necessary for the machine learning apparatus from the shared memory according to the data operation signal, and returning the input data to the machine learning apparatus through the receiving interface.

In one embodiment, the data operation signal is a broadcast signal and/or a multicast signal, and returning the input data to the machine learning device through the receiving interface,

And transmitting, by the transmission circuit, the input data to a plurality of machine learning units corresponding to the broadcast signal and/or multicast signal through the shared data receiving interface.

The present application provides a data processing apparatus. The data processing device includes a machine learning device, a transmission circuit, and a shared memory, the machine learning device is connected to the transmission circuit through a first transmission interface, and the transmission circuit is connected to the shared memory,

The transmission circuit acquires input data necessary for the machine learning device from the shared memory according to a data operation signal sent from the machine learning device, returns the input data to the machine learning device, and shares the data operation signal. Represents an operation method for data in memory.

In one embodiment, the machine learning device comprises at least one machine learning unit, the machine learning unit comprises at least one computation unit, and a controller unit connected to the computation unit, and the computation unit comprises one mast And a processing circuit and a plurality of slave processing circuits, wherein the operation unit is connected to the transmission circuit through the first transmission interface.

The controller unit transmits the data operation signal and the output data to the transmission circuit through a transmission interface of the first transmission interface, and the transmission circuit is obtained from the shared memory through a reception interface of the first transmission interface. One of the input data is received, and the input data is transmitted to the master processing circuit and/or the slave processing circuit.

The mast processing circuit distributes the input data to the plurality of slave processing circuits, and the plurality of slave processing circuits obtain a plurality of intermediate results by performing intermediate operations on the data transmitted by the mast processing circuit. The intermediate result is transmitted to the mast processing circuit.

The mast processing circuit further performs subsequent processing on the plurality of intermediate results to obtain a calculation result.

In one embodiment, the structure of the mast processing circuit and the slave processing circuit includes at least one of H type, systolic array type, and tree type structure.

In one embodiment, the transmission circuit comprises a second transmission interface, at least one read/write processing circuit connected to the second transmission interface, and an arbitration circuit connected to the read/write processing circuit, wherein the at least one machine The connection of the learning unit with the second transmission interface through the first transmission interface implements connection between the at least one machine learning unit and the transmission circuit.

The read/write processing circuit receives the data operation signal transmitted by the at least one machine learning unit through the first transmission interface and the second transmission interface, and transmits the data operation signal to the arbitration circuit, The data read from the shared memory is transmitted to at least one machine learning unit through the second transmission interface.

The arbitration circuit arbitrates the data operation signals received from the at least one read/write processing circuit according to a pre-set arbitration rule, and operates the data in the shared memory according to the successful data operation signal.

In one embodiment, the read/write processing circuit includes at least one of a unicast read processing circuit, a unicast write processing circuit, and a broadcast processing circuit. The data operation signal includes one or more of a unicast read request, a unicast write request, a unicast read command, a unicast write command, a multicast command, and a broadcast command.

Here, the unicast type processing circuit processes a unicast type signal, and the broadcast type processing circuit processes a multicast or broadcast type signal.

In one embodiment, if the data operation signal is a command type signal, the read/write processing circuit specifically analyzes the command type signal to generate a request type signal, and transfers the request type signal to the arbitration circuit. send.

In one embodiment, if the data operation signal is a multicast command, the multicast command includes an identifier of a target machine learning unit of a plurality of received data.

The read/write processing circuit specifically transmits the data obtained by the arbitration circuit from the shared memory to the plurality of target machine learning units.

In one embodiment, if the data operation signal is a broadcast command, the read/write processing circuit specifically transmits the data obtained by the arbitration circuit from the shared memory to all machine learning units.

In one embodiment, the input data includes input neuron data and/or weights, and the output data includes output neuron data.

In one embodiment, the data processing apparatus is divided into at least one or more clusters, and each cluster includes a plurality of machine learning units, one transmission circuit, and at least one shared memory.

The transmission circuit further includes an arbitration circuit in the cluster to which it belongs and a first type direct access controller DMA connected to the shared memory in the cluster, and/or a second type DMA connected to the arbitration circuit in the cluster to which it belongs and the shared memory in another cluster. do.

The first DMA controls data interaction between the arbitration circuit in the cluster and the shared memory in the cluster.

The second DMA controls data interaction between an arbitration circuit in the cluster and a shared memory in another cluster, and data interaction between an arbitration circuit in the cluster and an off-chip memory.

In one embodiment, the transfer circuit further includes a first selective transfer circuit coupled to the first type DMA, and a second selective transfer circuit coupled to the second type DMA.

The first selection transfer circuit is selectively connected to a shared memory in a cluster to which it belongs.

The second selection transfer circuit is selectively connected to a cluster to which it belongs, a shared memory in another cluster, and the off-chip memory.

In one embodiment, the transmission circuit further comprises a caching circuit connected to the arbitration circuit and the shared memory, wherein the arbitration circuit temporarily stores the data obtained from the shared memory and the arbitration circuit writes to the shared memory. Save the data temporarily.

In one embodiment, a transmission bandwidth between the transmission circuit and the shared memory is greater than a transmission bandwidth between the transmission circuit and the machine learning unit.

The present application provides a data processing apparatus. The data processing device performs processing on machine learning data, includes a machine learning device, a transmission circuit and a shared memory, the transmission circuit includes a plurality of read/write processing circuits and one arbitration circuit, and the machine The learning apparatus includes a plurality of machine learning units, each machine learning unit including at least one operation unit, wherein the plurality of machine learning units are connected to the transmission circuit through a first transmission interface, and the transmission circuit Is connected to the shared memory,

The arbitration circuit arbitrates the data operation signals transmitted from the plurality of machine learning units and obtains input data necessary for the machine learning apparatus from the shared memory according to the arbitration successful data operation signals,

The read/write processing circuit determines a target machine learning unit or a target operation unit from the plurality of machine learning units according to address information included in the arbitration successful data operation signal or the type of the data operation signal, and stores the input data. Return to the target machine learning unit or target computation unit.

In one embodiment, the arbitration circuit specifically prioritizes the data operation signals transmitted by the plurality of read/write circuits, and makes the data operation signal having the highest priority as the arbitration successful data operation signal.

In one embodiment, the arbitration circuit, specifically, when the priority of the data operation signals transmitted by the plurality of read/write processing circuits is the same, according to the type of the plurality of data operation signals and the execution condition set in advance. Confirm the data operation signal that the arbitration was successful.

In an embodiment, if the data operation signal is a unicast type signal, the execution condition may be: a channel of a machine learning unit transmitting the unicast type signal is in an idle state; Or, among the machine learning units that transmit the unicast type signal, the channel of the computation unit is in an idle state.

In one embodiment, if the data operation signal is a multicast type signal, the execution condition is that the channel of the machine learning unit transmitting the multicast type signal is in an idle state and the multicast type signal is The channel of the designated target machine learning unit is in an idle state; Or, among the machine learning units that have transmitted the multicast type signal, the channel of the computation unit is in an idle state, and the channel of the target computation unit designated by the multicast type signal is in the idle state.

In one embodiment, if the data operation signal is a broadcast type signal, the execution condition is that the channel of the machine learning unit that transmitted the broadcast type signal is in an idle state and the channels of the other machine learning units are Idle state; Alternatively, among the machine learning units transmitting the broadcast type signal, a channel of the computation unit is in an idle state, and a channel of the computation unit among other machine learning units is in an idle state.

In one embodiment, the transmission circuit further includes a second transmission interface, each interface of the second transmission interface is connected in correspondence with each interface of the first transmission interface, and one machine learning unit is one It is connected correspondingly with the read/write processing circuit of.

In one embodiment, a plurality of calculation units in the one machine learning unit share one transmission interface in the first transmission interface, and each calculation unit corresponds to one data reception interface.

In one embodiment, the plurality of operation units in the one machine learning unit correspond to one transmission interface and one data reception interface in the first transmission interface, respectively.

In one embodiment, the transmission circuit further includes a second transmission interface, and the plurality of machine learning units share one signal reception interface and one data return interface in the second transmission interface.

In one embodiment, the read/write processing circuit includes a signal queue for storing a data operation signal signal transmitted by each machine learning unit.

The read/write processing circuit is further configured to determine whether or not there is a residual space in the request queue when receiving the data operation signal, and if "Yes", the data operation signal is cached in the request queue, and " If no, the data operation signal is blocked.

In one embodiment, if the read/write processing circuit is a broadcast processing circuit, the signal queue includes a command queue and a request queue.

The command queue caches a command type signal received by the broadcast processing circuit.

The request queue caches the signal of the request type obtained after analyzing the signal of the command type.

In one embodiment, the machine learning unit further comprises a controller unit connected to the operation unit, the operation unit comprises one mast processing circuit and a plurality of slave processing circuits, the operation unit is the first transmission interface Is connected to the transmission circuit through.

The controller unit transmits the data operation signal and the output data to the transmission circuit through a transmission interface in the first transmission interface, and the transmission circuit obtained from the shared memory through a reception interface in the first transmission interface. The input neuron data and the weight are obtained, and the input neuron data and the weight are transmitted to the mast processing circuit and/or the slave processing circuit.

The mast processing circuit distributes the input data to the plurality of slave processing circuits, and the plurality of slave processing circuits perform an intermediate operation based on the neuron data and weights transmitted by the mast processing circuit to obtain a plurality of intermediate results. The intermediate result of is transmitted to the mast processing circuit.

The mast processing circuit also performs subsequent processing on the plurality of intermediate results to obtain a calculation result.

In one embodiment, the input data includes input data, and the output data includes output data.

It should be understood that the above-described general description and detailed description to be described later are merely exemplary and are for description only, and not limiting the present application.

The accompanying drawings are included in the specification to form a part of the specification, illustrate embodiments of the present application, and are used in conjunction with the specification to explain the principles of the present application.
1 is a block diagram of a network on-chip processing system 1100 according to an embodiment.
2 is a block diagram of a network on-chip processing system 1200 according to an embodiment.
3 is a block diagram of a network on-chip processing system 1300 according to an embodiment.
4 is a block diagram of a network on-chip processing system 1400 according to an embodiment.
5A is a block diagram of a network on-chip processing system 1500 according to an embodiment.
5B is a block diagram of a network on-chip processing system 15000 according to an embodiment.
6 is a block diagram of a network on-chip processing system 1600 according to an embodiment.
7 is a block diagram of a network on-chip processing system 1700 according to an embodiment.
8 is a block diagram of a network on-chip processing system 1800 according to an embodiment.
9 is a block diagram of a network on-chip processing system 1900 according to an embodiment.
10A is a block diagram of a network on-chip processing system 1910 according to an embodiment.
10B is a block diagram of a network on-chip processing system 19100 according to an embodiment.
11 is a block diagram of a network on-chip processing system 1920 according to an embodiment.
12 is a block diagram of a network on-chip processing system 1930 according to an embodiment.
13 is a block diagram of a calculation device according to an embodiment.
14 is a block diagram of a calculation device according to another embodiment.
15 is a block diagram of a mast processing circuit according to an embodiment.
16 is a block diagram of a calculation device according to another embodiment.
17 is a block diagram of a calculation device according to another embodiment.
18 is a block diagram of a tree module according to an embodiment.
19 is a block diagram of a calculation device according to another embodiment.
Fig. 20 is a block diagram of a calculation device according to another embodiment.
Fig. 21 is a block diagram of a calculation device according to another embodiment.
22 is a block diagram of an integrated processing apparatus according to an embodiment.
23 is a block diagram of an integrated processing apparatus according to another embodiment.
24 is a configuration diagram of a board card according to an embodiment.
25 is a flowchart of a method of processing network on-chip data according to an embodiment.
26 is a flowchart of a method for processing network on-chip data according to another embodiment.
27 is a flowchart of a method of processing network on-chip data according to another embodiment.
28 is a flowchart of a method of processing network on-chip data according to another embodiment.
29 is a flowchart of a method of processing network on-chip data according to another embodiment.
30 is a flowchart of a method of processing network on-chip data according to another embodiment.
31 is a schematic diagram of an application environment of a data processing method according to an embodiment.
32 is a flowchart of a data processing method according to an embodiment.
33 is a flowchart of a data processing method according to an embodiment.
34 is a flowchart of a data processing method according to an embodiment.
35 is a flowchart of a data processing method according to an embodiment.
36 is a flowchart of a data processing method according to an embodiment.
37 is a flowchart of a data processing method according to an embodiment.
38 is a flowchart of a data processing method according to an embodiment.
39 is a flowchart of a data processing method according to an embodiment.
40 is a flowchart of a data processing method according to an embodiment.
41 is a flowchart of a data processing method according to an embodiment.
42 is a flowchart of a data processing method according to an embodiment.
43 is a configuration diagram of a data processing apparatus according to an embodiment.
44 is a block diagram of a machine learning unit according to an embodiment.
45 is a configuration diagram of a data processing apparatus according to an embodiment.
46 is a configuration diagram of a data processing apparatus according to an embodiment.
47 is a configuration diagram of a data processing apparatus according to an embodiment.
48 is a configuration diagram of a data processing apparatus according to an embodiment.
49 is a block diagram of a machine learning unit according to an embodiment.
50 is a block diagram of a data processing apparatus according to an embodiment.
51 is a configuration diagram of a data processing apparatus according to an embodiment.
52 is a configuration diagram of a data processing apparatus according to an embodiment.
53 is a configuration diagram of a data processing apparatus according to an embodiment.
54 is a block diagram of a data processing apparatus according to an embodiment.
55 is a configuration diagram of a data processing apparatus according to an embodiment.
56 is a configuration diagram of a data processing apparatus according to an embodiment.
56A is a block diagram of a machine learning apparatus according to an embodiment.
57 is a block diagram of a transmission circuit according to an embodiment.
57A is a block diagram of a transmission circuit according to an embodiment.
57B is a block diagram of a transmission circuit according to an embodiment.
58 is a block diagram of a transmission circuit in one cluster according to an embodiment.
59 is a block diagram of a transmission circuit in another cluster according to an embodiment.
60 is a block diagram of another transmission circuit according to an embodiment.
61 is a configuration diagram of a data processing apparatus according to an embodiment.
62 is a block diagram of a machine learning unit according to an embodiment.
63 is a configuration diagram of a data processing apparatus according to an embodiment.
64 is a configuration diagram of a data processing apparatus according to an embodiment.
65 is a configuration diagram of a data processing apparatus according to an embodiment.
66 is a configuration diagram of a data processing apparatus according to an embodiment.

Here, exemplary embodiments are described in detail and examples thereof are shown in the drawings. In the following description of the drawings, the same reference numerals in different drawings indicate the same or similar elements unless otherwise specified. The embodiments described in the exemplary embodiments below are not representative of all embodiments as in the present application. Conversely, the embodiments are merely examples of apparatus and methods consistent with aspects of the present application described in the appended claims.

In one embodiment, the network on-chip processing system according to the present application includes a storage device and a plurality of calculation devices, the storage device and the plurality of calculation devices are installed on the same chip, and at least one calculation device is the storage device And at least two computing devices are connected to each other.

Here, a network on chip (NoC) refers to an on-chip communication network that integrates a large amount of computational resources on a single chip and connects these resources. Alternatively, each computing device on a chip may access the network on-chip through a respective interface and communicate using a target module that wishes to communicate with shared network resources. Specifically, when the storage device and the plurality of calculation devices are installed on the same chip, it means that the storage device and the plurality of calculation devices are integrated on one chip. The processor core and the off-chip storage device in the computing unit are connected through the NoC, and the NoC also supports communication between the multiple cores of the processor.

The network on-chip processing system according to the embodiment of the present application implements on-chip communication based on NoC. Meanwhile, the network on-chip processing system according to the exemplary embodiment of the present application is capable of off-chip storage as well as on-chip storage. In other words, in the process of the neural network processor, operation data may be stored in an on-chip storage device or an off-chip storage device. Since the on-chip storage capacity of the network on-chip processing system is limited, operation data and intermediate results generated during the operation process can be temporarily stored in an off-chip storage device and then read from the off-chip storage device to NoC if necessary. In the embodiment of the present application, all storage devices in the network on-chip processing system mean on-chip storage devices, and the computing devices in the network on-chip processing system include a neural network processor.

In one embodiment, the present application further provides a network on-chip processing system. The system includes a storage device and a plurality of calculation devices, the plurality of calculation devices includes a first calculation device and a plurality of second calculation devices, and the storage device and a plurality of the calculation devices are installed on the same chip. . Here, the first calculation device is connected to the storage device, and at least one second calculation device of the plurality of second calculation devices is connected to the first calculation device.

In one embodiment, a neural network chip is provided. The chip includes a storage device, a plurality of computing devices, a first interconnect device, and a second interconnect device, wherein at least one computing device and the storage device are connected through the first interconnect device, and the plurality of computing devices are the It is connected through a second interconnect device. Furthermore, the computing device may implement a read/write operation for the storage device through the first interconnect device, and may transmit data between the plurality of computing devices through the second interconnect device.

Hereinafter, a network on-chip processing system and a neural network chip will be described, respectively.

1, FIG. 1 is a network on-chip processing system 1100 according to an embodiment. The network on-chip processing system 1100 includes a storage device 1101, a first calculation device 1102, a second calculation device 1103 and a second calculation device 1104, and includes a storage device 1101, a first calculation device. The device 1102, the second computing device 1103 and the second computing device 1104 are installed on the same chip of the network on-chip processing system 1100. Here, the first calculation device 1102 is connected to the storage device 1101, the second calculation device 1103 is connected to the first calculation device 1102, and the second calculation device 1103 is also a second calculation device. Connected to (1104). Only the first computing device 1102 can access the storage device 1101. In other words, only the first calculation device 1102 can read/write data from the storage device 1101. The first calculation device 1102, the second calculation device 1103, and the second calculation device 1104 may transmit data to each other.

Specifically, when the second calculation device 1104 wants to read data, the first calculation device 1102 accesses the storage device 1101 to provide the data required for the second calculation device 1104 from the storage device 1101 Is read, and after transmitting the data to the second calculation device 1103, the second calculation device 1103 transmits the data to the second calculation device 1104 again. Alternatively, a second calculation device 1103 and a second calculation device 1104 other than the first calculation device 1102 may also be connected to the storage device 1101, and the first calculation device 1102, the second calculation device At least one of the device 1103 and the second computing device 1104 only needs to be connected to the storage device 1101. Here, there is no particular limitation on this. Alternatively, the second calculation device 1103 may be connected to each other to the second calculation device 1104 or may be connected to each other to the first calculation device 1102. At least two or more of the first calculation device 1102, the second calculation device 1103, and the second calculation device 1104 need only be connected to each other. Here, there is no particular limitation on this.

As shown in FIG. 2, FIG. 2 is a network on-chip processing system 1200 according to an embodiment. The network on-chip processing system 1200 includes a storage device 1201, a first calculation device 1202, a second calculation device 1203, and a second calculation device 1204, and includes a storage device 1201, a first calculation device, and The device 1202, the second computing device 1203 and the second computing device 1204 are installed on the same chip of the network on-chip processing system 1200. Here, the first calculation device 1202 is connected to the storage device 1201, and the second calculation device 1203 and the second calculation device 1204 are directly connected to the first calculation device 1202. That is, since the second calculation device 1204 is not only connected to the second calculation device 1203 but also is connected to the first calculation device 1201, it is connected to the first calculation device 1201 through the second calculation device 1203. There is no need to configure it. Only the first computing device 1202 can access the storage device 1201. In other words, only the first calculation device 1202 can read/write data from the storage device 1201. The first calculation device 1202, the second calculation device 1203, and the second calculation device 1204 may transmit data to each other.

Specifically, when the second calculation device 1204 wants to read data, the first calculation device 1202 accesses the storage device 1201 to provide data required for the second calculation device 1204 from the storage device 1201. After reading the data, the data is directly transmitted to the second calculation device 1204, so there is no need to forward the data through the second calculation device 1203. Alternatively, the first calculating device 1202, the second calculating device 1203 and the second calculating device 1204 can all be connected to the storage device 1201, and the first calculating device 1202, the second calculating device At least one of the calculation devices 1203 and the second calculation device 1204 need only be connected to the storage device 1201. Here, there is no particular limitation on this. Alternatively, the second calculation device 1203 may be connected to the second calculation device 1204 or may be connected to the first calculation device 1202. At least two or more of the first calculation device 1202, the second calculation device 1203, and the second calculation device 1204 need only be connected to each other. Here, there is no particular limitation on this.

In the network on-chip processing system, a plurality of calculation devices installed on the same chip are configured to be connected to each other, so that a plurality of calculation devices can transmit data to each other, so that when all of the plurality of calculation devices read data from a storage device, Not only can the bandwidth overhead be prevented from occurring, but also data reading/writing efficiency is improved.

In one embodiment, the present application provides a network on-chip processing system. The system includes a storage device and a plurality of calculation devices, and the storage device and a plurality of calculation devices are installed on the same chip. Here, each of the calculation devices in the plurality of calculation devices is connected to the storage device, and at least two or more calculation devices are connected to each other.

3, FIG. 3 is a network on-chip processing system 1300 according to an embodiment. The network on-chip processing system 1300 includes a storage device 1301, a calculation device 1302, a calculation device 1303 and a calculation device 1304, and includes a storage device 1301, a calculation device 1302, and a calculation device ( 1303) and the calculating device 1304 are installed on the same chip of the network on-chip processing system 1300, the calculating device 1302, the calculating device 1303 and the calculating device 1304 are all connected to the storage device 1301 , The calculation device 1302 and the calculation device 1303 are connected to each other, and at the same time, the calculation device 1303 and the calculation device 1304 are connected to each other. The calculation device 1202, the calculation device 1203, and the calculation device 1304 all have access to the storage device 1201, and the calculation device 1302 and the calculation device 1303 can transmit data to each other and calculate The device 1303 and the computing device 1304 may also transmit data to each other.

Specifically, when the calculation device 1304 wants to read data, the calculation device 1304 may directly access the storage device 1301, or the calculation device 1303 accesses the storage device 1301 After reading data necessary for the calculation device 1304 from 1301, the data may be transmitted to the calculation device 1304. In addition, the calculation device 1302 accesses the storage device 1301 to read the data required for the calculation device 1304 from the storage device 1301, and then transmits the data to the calculation device 1303, and the calculation device 1303 It may be transmitted back to the computing device 1304. Alternatively, only at least one of the computing device 1302, the computing device 1303 and the computing device 1304 need only be connected to the storage device 1301. Here, there is no particular limitation on this. Alternatively, at least two or more of the calculation device 1302, the calculation device 1303 and the calculation device 1304 may be connected to each other. Here, there is no particular limitation on this.

In the network on-chip processing system, by configuring a plurality of computing devices installed on the same chip to be connected to each other, all data required for any one computing device can be transmitted between a plurality of computing devices, so the system reads the storage device interface at the same time It can reduce computational equipment and reduce bandwidth clogging.

As shown in FIG. 4, FIG. 4 is a network on-chip processing system 1400 according to an embodiment. The network on-chip processing system 1400 includes a storage device 1401, a calculation device 1402, a calculation device 1403 and a calculation device 1404, and includes a storage device 1401, a calculation device 1402, and a calculation device ( 1403 and the computing device 1404 are installed on the same chip of the network on-chip processing system 1400. Here, the calculation device 1402, the calculation device 1403, and the calculation device 1404 are all connected to the storage device 1401, and the calculation device 1402, the calculation device 1403, and the calculation device 1404 are connected to each other. Has been. The computing device 1402, the computing device 1403 and the computing device 1404 are all accessible to the storage device 1401, and the computing device 1402, the computing device 1403, and the computing device 1404 are mutually Can be transmitted.

Specifically, when the computing device 1404 wants to read data, it may directly access the storage device 1401, or the computing device 1403 accesses the storage device 1401 to access the storage device 1401 from the storage device 1401. After reading the data required by 1404, the data may be transmitted to the computing device 1404. In addition, since the calculation device 1402 can access the storage device 1401 and read the data required for the calculation device 1404 from the storage device 1401, the data can be immediately transmitted to the calculation device 1404. 1403), data can be forwarded. Alternatively, at least one of the computing device 1402, the computing device 1403, and the computing device 1404 may be connected to the storage device 1401. Here, there is no particular limitation on this. Alternatively, at least two or more of the calculation device 1402, the calculation device 1403, and the calculation device 1404 may be connected to each other. Here, there is no particular limitation on this.

In the network on-chip processing system, a plurality of computing devices installed on the same chip are configured to be directly connected without passing through a storage device, so that the two computing devices can directly transmit data to each other, thereby improving data reading/writing efficiency. .

In one embodiment, the present application further provides a network on-chip processing system. The system includes a storage device and a plurality of computing device groups, wherein the storage device and a plurality of computing device groups are installed on the same chip, and each computing device group includes a plurality of computing devices. Here, in the plurality of computing device groups, at least one computing device group is connected to the storage device, and at least two or more computing device groups are connected to each other.

In one embodiment, the present application further provides a neural network chip. The chip includes a storage device, a plurality of computing device groups, a first interconnect device, and a second interconnect device, and at least one computing device group among the plurality of computing device groups is connected to the storage device through the first interconnect device. And the plurality of computing device groups are connected through the second interconnect device. Furthermore, the computing device group may implement a read/write operation for the storage device through the first interconnect device, and data may be transmitted between the plurality of computing device groups through the second interconnect device. Here, the plurality of calculation devices may be divided into a plurality of groups, and the number of calculation devices in each group is not particularly limited. As an example, one group includes four computing devices.

As shown in Fig. 5A, one embodiment provides a network on-chip processing system 1500. The network-on-chip processing system 1500 includes a storage device 1501 and six calculation devices (calculation device 1502 to calculation device 1507), and includes a storage device 1501 and six calculation devices (calculation device 1502). ) To computing device 1507 are installed on the same chip of the network on-chip processing system 1500. For example, the six calculation devices can be divided into three groups, and each group includes two calculation devices. For example, the calculation device 1502 and the calculation device 1503 belong to the first calculation device group (cluster1), the calculation device 1504 and the calculation device 1505 belong to the second calculation device group (cluster2), The calculation device 1506 and the calculation device 1507 belong to the third calculation device group cluster3. cluster1 is the main computing device group, and cluster2 and cluster3 are the sub computing device groups. Here, only cluster1 is connected to the storage device 1501, and cluster1, cluster2, and cluster3 are connected to each other. The calculation device 1502 in cluster1 is connected to the storage device 1501, the calculation device 1503 in cluster1 and the calculation device 1504 in cluster2 are connected to each other, and the calculation device 1505 in cluster2 and the calculation device in cluster3 ( 1507) are connected to each other.

Specifically, when cluster3 wants to read data, cluster1 accesses the storage device 1501, reads the data required for cluster3 from the storage device 1501, and then transmits the data to cluster2 and then transmits the data from cluster2 to cluster3. . Here, the plurality of calculation devices may be divided into a plurality of groups, and the number of calculation devices in each group is not particularly limited. It is preferred to include four computing devices in one group.

Alternatively, not all of the plurality of computing devices need to be connected to the storage device 1501, and only one or more of the two computing device groups need be connected to the storage device 1501. Here, there is no particular limitation on this. Alternatively, cluster1 may be connected to cluster2 or cluster3, so at least two or more of the three computing device groups may be connected to each other. Here, there is no particular limitation on this. Alternatively, at least one or more computing devices per said computing device group are connected to at least one or more computing devices within said different computing device groups. In other words, each computing device in cluster1 may be connected to the second device group, and at least one computing device in cluster1 may be connected to at least one computing device in cluster2. Here, there is no particular limitation on this. Alternatively, the plurality of computing device groups are connected to each other via any one computing device in the plurality of computing device groups. In other words, any one computing device in cluster1 can be connected to any one computing device in cluster2. Here, there is no particular limitation on this.

5B, FIG. 5B is a network on-chip processing system 15000 according to an embodiment. The network-on-chip processing system 15000 includes a storage device 15010 and six calculation devices (calculation devices 15020 to 15070), and a storage device 15010 and six calculation devices (calculation devices 15020). ) To computing devices 15070) are installed on the same chip of the network on-chip processing system 15000, and the six computing devices are also divided into 3 groups. The calculation device 15020 and the calculation device 15030 belong to the first calculation device group (cluster1), the calculation device 15040 and the calculation device 15050 belong to the second calculation device group (cluster2), and the calculation device 15060 ) And the computing device 15070 belong to the third computing device group cluster3. cluster1 is the main computing device group, and cluster2 and cluster3 are the sub computing device groups. Here, only cluster1 is connected to the storage device 15010, and cluster1, cluster2, and cluster3 are connected to each other. The calculation device 15020 of cluster1 is connected to the storage device 15010, the calculation device 15030 of cluster1 is connected to the calculation device 15040 of cluster2, and the calculation device 15050 of cluster2 is a calculation device 15070 of cluster3. ), and the computing device 15060 of cluster3 is connected to the computing device 15020 of cluster1.

Specifically, when cluster3 wants to read data, cluster1 may access the storage device 1501, read data necessary for cluster3 from the storage device 1501, and then directly transmit the data to cluster3. Here, the plurality of calculation devices may be divided into a plurality of groups, and the number of calculation devices in each group is not particularly limited. It is preferred to include four computing devices in one group.

Alternatively, all of the plurality of computing devices need not be connected to the storage device 15010, and at least one group of computing devices in the two computing device groups may be connected to the storage device 15010. Here, there is no particular limitation on this. Alternatively, cluster1 may be connected to cluster2 or cluster3, so at least two or more of the three computing device groups may be connected to each other. Here, there is no particular limitation on this. Alternatively, at least one or more computing devices in each said computing device group are connected to at least one or more computing devices in another said computing device group. In other words, each computing device in cluster1 can be connected to the second device group, and at least one computing device in cluster1 may be connected to at least one computing device in cluster2. Here, there is no particular limitation on this. Alternatively, the plurality of computing device groups are connected to each other via any one computing device in the plurality of computing device groups. That is, an arbitrary computing device in cluster1 is connected to an arbitrary computing device in cluster2. Here, there is no particular limitation on this.

In the network on-chip processing system, by configuring a plurality of computing device groups installed on the same chip to be connected to each other, intergroup communication is possible between a plurality of computing device groups. The system can reduce the number of computing devices that simultaneously read the storage device interface through data transmission between groups, and reduce energy overhead due to memory access. At the same time, communication is implemented between groups of a plurality of computing devices installed on the same chip through a variety of connection methods, so by establishing multiple communication channels between a plurality of computing devices, the most desirable channel is selected according to the current network congestion. As data can be transmitted, energy consumption is reduced and data processing efficiency is improved.

In one embodiment, a network on-chip processing system is provided. The system includes a storage device and a plurality of calculation device groups, the storage device and a plurality of calculation device groups are installed on the same chip, each calculation device group includes a plurality of calculation devices, and the plurality of calculation devices At least one computing device group among the device groups is connected to the storage device, and the plurality of computing device groups are connected to each other.

6, FIG. 6 is a network on-chip processing system 1600 according to an embodiment. The network on-chip processing system 1600 includes a storage device 1601 and six calculation devices (calculation devices 1602 to 1607), and includes a storage device 1601 and six calculation devices (calculation devices 1602). ) To computing device 1607) are installed on the same chip of the network on-chip processing system 1600. For example, six calculation devices may be divided into three groups, and the calculation device 1602 and the calculation device 1603 belong to the first calculation device group cluster1, and the calculation device 1604 and the calculation device 1605 ) Belongs to the second calculation device group cluster2, and the calculation device 1606 and the calculation device 1607 belong to the third calculation device group cluster3. Here, cluster1, cluster2, and cluster3 are all connected to the storage device 1601, cluster1 and cluster2 are connected to each other, and cluster2 and cluster3 are connected to each other. All of the calculation devices 1602 to 1607 are connected to the storage device 1601, the calculation device 1603 of cluster1 and the calculation device 1604 of cluster2 are connected to each other, and the calculation device 1604 of cluster2 and The computing devices 1607 of cluster3 are connected to each other.

Specifically, when cluster3 attempts to read data, cluster2 may access the storage device 1601, read data necessary for cluster3 from the storage device 1601, and transmit the data to cluster3. In addition, after cluster1 accesses the storage device 1601 and reads data necessary for cluster3 from the storage device 1601, the data may be transmitted to cluster2, and then transmitted from cluster2 to cluster3. Here, the plurality of calculation devices may be divided into a plurality of groups, and the number of calculation devices in each group is not particularly limited. For example, four calculation devices are included in one group.

Alternatively, all of the plurality of computing devices need not be connected to the storage device 1601, and only at least one or more of the two computing device groups need to be connected to the storage device 1601. Here, there is no particular limitation on this. Alternatively, each computing device of cluster1 may be connected to the second unit group and/or cluster3, and at least one computing device of cluster1 may be connected to at least one or more computing devices of cluster2 and/or cluster3. Here, there is no particular limitation on this. Alternatively, any one computing device in cluster1 may be connected to any one computing device in cluster2 and/or cluster3. Here, there is no particular limitation on this.

In the network on-chip processing system, by configuring a plurality of computing device groups installed on the same chip to be connected to each other, all data required for an arbitrary computing device group can be transmitted between the plurality of computing device groups. It is possible to reduce the number of computational devices that simultaneously read s and reduce bandwidth clogging.

In one embodiment, a network on-chip processing system is provided. The system includes a storage device and a plurality of calculation device groups, the storage device and a plurality of calculation device groups are installed on the same chip, each calculation device group includes a plurality of calculation devices, and the plurality of calculation devices At least one computing device group in the device group is connected to the storage device, and any two computing device groups among the plurality of computing device groups are directly connected.

As shown in FIG. 7, FIG. 7 is a network on-chip processing system 1700 according to an embodiment. The network on-chip processing system 1700 includes a storage device 1701 and six calculation devices (calculation devices 1702 to 1707), and a storage device 1701 and six calculation devices (calculation devices 1702). ) To calculation device 1707) are installed on the same chip of the network on-chip processing system 1700, the six calculation devices are divided into three groups, and the calculation device 1702 and the calculation device 1703 are a first calculation device group. (cluster1), the calculation device 1704 and the calculation device 1705 belong to a second calculation device group (cluster2), and the calculation device 1706 and the calculation device 1707 belong to a third calculation device group (cluster3). Belongs to. Here, cluster1, cluster2, and cluster3 are all connected to the storage device 1701, and the three computing device groups of cluster1, cluster2, and cluster3 are connected to each other. All of the calculation devices 1702 to 1707 are connected to the storage device 1701, the calculation device 1703 of cluster1 and the calculation device 1704 of cluster2 are connected to each other, and the calculation device 1704 of cluster2 and The computing device 1707 of cluster3 is connected to each other, and the computing device 1702 of cluster1 and the computing device 1706 of cluster3 are connected to each other.

Specifically, when cluster3 attempts to read data, cluster2 may access the storage device 1701 to read data necessary for cluster3 from the storage device 1701 and then transmit the data from cluster2 to cluster3. In addition, after cluster1 accesses the storage device 1701 to read data necessary for cluster3 from the storage device 1701, the data may be directly transmitted from cluster1 to cluster3. The plurality of calculation devices may be divided into a plurality of groups, and the number of calculation devices in each group is not particularly limited. It is preferred to include four computing devices in one group.

In the plurality of computing devices, it is not necessary for all of the computing devices to be connected to the storage device 1701, and at least one or more computing device groups in the two computing device groups may be connected to the storage device 1701. Here, there is no particular limitation on this. Alternatively, each computing device in cluster1 may be connected to the second unit group and/or cluster3, and at least one computing device in cluster1 may be connected to at least one computing device in cluster2 and/or cluster3. Here, there is no particular limitation on this. Alternatively, any one computing device in cluster1 may be connected to any one computing device in cluster2 and/or cluster3. Here, there is no particular limitation on this.

In the network on-chip processing system, since a plurality of groups of computing devices installed on the same chip are directly connected to each other, data reading/writing efficiency can be improved.

In one embodiment, the present application provides a network on-chip processing system. The system includes a storage device and a plurality of calculation device groups, the storage device and a plurality of calculation device groups are installed on the same chip, each calculation device group includes a plurality of calculation devices, and the plurality of calculation devices At least one computing device group among the device groups is connected to the storage device, at least two or more computing device groups are connected to each other, and a plurality of computing devices of each of the computing device groups are connected to each other.

As shown in FIG. 8, FIG. 8 is a network on-chip processing system 1800 according to an embodiment. The network on-chip processing system 1800 includes a storage device 1801 and six calculation devices (calculation devices 1802 to 1807), and includes a storage device 1801 and six calculation devices (calculation devices 1802). ) To calculation device 1807) are installed on the same chip of the network on-chip processing system 1800, the six calculation devices are divided into two groups, and the calculation device 1802, the calculation device 1803 and the calculation device 1804 A belongs to the first calculation device group cluster1, and the calculation device 1805, the calculation device 1806, and the calculation device 1807 belong to the second calculation device group cluster2. Both cluster1 and cluster2 are connected to the storage device 1801, cluster1 and cluster2 are connected to each other, three computing devices in cluster1 are connected to each other, and three computing devices in cluster2 are connected to each other. All of the calculation devices 1802 to 1807 are connected to the storage device 1801, the calculation device 1802 of cluster1 and the calculation device 1805 of cluster2 are connected to each other, and the calculation device 1803 is a calculation device. 1802 and a calculation device 1804 are connected, and the calculation device 1806 is connected to a calculation device 1805 and a calculation device 1807. Here, a connection method between the plurality of computing devices of each computing device group may refer to a connection method of the network on-chip processing system 1100 to the network on-chip processing system 1400. It is not repeated here.

Specifically, when cluster2 attempts to read data, it may directly access the storage device 1801. In addition, cluster1 may access the storage device 1801 to read data necessary for cluster2 from the storage device 1801 and then transmit the data to cluster2. At the same time, the second computing device can also transmit data within the group. When cluster2 attempts to read data, the computing device 1805, the computing device 1806, and the computing device 1807 of cluster2 can access the storage device 1801 at the same time. Here, the calculation device 1805, the calculation device 1806, and the calculation device 1807 read some data required for cluster2, respectively, and such data can be transmitted within cluster2. The plurality of calculation devices may be divided into a plurality of groups, and the number of calculation devices in each group is not particularly limited. It is preferred to include four computing devices in one group.

Alternatively, not all of the plurality of computing devices need to be connected to the storage device 1801, and at least one of the two computing device groups may be connected to the storage device 1801. Here, there is no particular limitation on this. Alternatively, all of the computing devices in cluster1 may be connected to the second unit group, and at least one computing device in cluster1 may be connected to at least one computing device in cluster2. Here, there is no particular limitation on this. Alternatively, any one computing device in cluster1 may be connected to any one computing device in cluster2. Here, there is no particular limitation on this.

In the network on-chip processing system, a plurality of computing device groups installed on the same chip are configured to be connected to each other, and at the same time, a plurality of computing devices of each computing device group are configured to be connected to each other to communicate intra-group communication between a plurality of computing devices. In addition, communication between groups becomes possible. Therefore, the system can reduce energy overhead due to memory access and improve data reading efficiency.

In one embodiment, the present application further provides a network on-chip processing system. The system includes a plurality of network on-chip processing modules connected to each other, the plurality of network on-chip processing modules are installed on the same chip, and each network on-chip processing module includes at least one storage device and a plurality of computing devices, In each network on-chip processing module, at least one or more computing devices are connected to at least one or more storage devices inside the network processing module, and at least two or more of the plurality of computing devices are connected to each other.

In one embodiment, a neural network chip is provided. The chip includes a plurality of network on-chip processing modules connected to each other, and each network on-chip processing module includes at least one storage device, a plurality of computing devices, a first interconnect device and a second interconnect device, and each network on-chip In the processing module, at least one computing device and at least one storage device inside the network on-chip processing module are connected through the first interconnect device, and the plurality of computing devices are connected to each other through the second interconnect device. Further, the computing device implements a read/write operation for a storage device inside the network on-chip processing module belonging to the first interconnect device, and the plurality of computing devices may also transmit data through the second interconnect device.

9, FIG. 9 is a network on-chip processing system 1900 according to an embodiment. The network on-chip processing system 1900 includes four network on-chip processing modules connected to each other, and the four network on-chip processing modules are installed on the same chip of the network on-chip processing system 1900, and each network on-chip processing module is one It includes a storage device 1901 and four calculation devices (calculation device 1902 to calculation device 1905). Here, in each network on-chip processing module, the computing device 1902 is connected to the storage device 1901 inside the corresponding network on-chip processing module, and the four computing devices inside each network on-chip processing module are connected to each other.

Specifically, all data to be processed by each network on-chip processing module is stored in a storage device inside the network on-chip processing module. In other words, the plurality of computing devices in each network on-chip processing module can only access the storage device inside the network on-chip processing module to which they belong, and can only read/write data from the storage device inside the network on-chip processing module to which they belong. .

Alternatively, the number of storage devices in each network on-chip processing module is not limited to one, and may be two, three, or a plurality. Here, although not particularly limited thereto, four are preferred. Alternatively, in each network on-chip processing module, the plurality of computing devices are connected to each other to form a computing device network, and the connection method between the plurality of computing devices in each network on-chip processing module is a network on-chip processing system (1100). To a connection method of the network on-chip processing system 1400 may be referred to. This is not repeated. Alternatively, it is not necessary that all of the plurality of computing devices in each network on-chip processing module be connected to the storage device 1901, and at least one computing device of each network on-chip processing module only needs to be connected to the storage device 1901. . Here, there is no particular limitation on this.

Alternatively, each computing device in each network on-chip processing module may be connected to a different network on-chip processing module, and at least one computing device in each network on-chip processing module is connected to at least one or more computing devices in another network on-chip processing module. It just needs to be done. Here, there is no particular limitation on this. Alternatively, the plurality of network on-chip processing modules are connected to each other through any one of each of the network on-chip processing modules. In other words, any one computing device of each network on-chip processing module may be connected to any one computing device of another network on-chip processing module. Here, there is no particular limitation on this.

In the network-on-chip processing system, a plurality of network-on-chip processing modules installed on the same chip are configured to be connected to each other, and a plurality of computing devices of each network on-chip processing module are configured to be connected to each other. As communication between modules as well as internal communication is possible, energy overhead due to memory access can be reduced and data reading efficiency can be improved. At the same time, multiple network on-chip processing modules installed on the same chip implement module-to-module communication in a variety of ways, so establish multiple communication channels between multiple computing devices and select the most desirable channel according to the current network congestion. By transmitting data, energy is saved and data processing efficiency is improved.

In one embodiment, the present application provides a network on-chip processing system. The system includes a plurality of network on-chip processing modules connected to each other, the plurality of network on-chip processing modules are installed on the same chip, each network on-chip processing module includes a plurality of storage devices, in the network on-chip processing module At least one or more computing devices are connected to the plurality of storage devices inside the network on-chip processing module, and at least two or more of the plurality of computing devices are connected to each other.

10A, FIG. 10A is a network on-chip processing system 1910 according to an embodiment. The network on-chip processing system 1910 includes four network on-chip processing modules connected to each other, the four network on-chip processing modules are installed on the same chip of the network on-chip processing system 1910, and each network on-chip processing module, A storage device 1911, a storage device 1916, and four calculation devices (calculation devices 1912 to 1915) are included, and in each network on-chip processing module, the calculation device 1912 is It is connected to the storage device 1911 and the storage device 1916 inside the processing module, and the four computing devices inside each network on-chip processing module are connected to each other.

Specifically, all data to be processed by each network on-chip processing module is stored in a storage device inside the network on-chip processing module. In other words, since the plurality of computing devices in each network on-chip processing module can only access the storage device inside the network on-chip processing module to which they belong, data can only be read/written from the storage device inside the network on-chip processing module to which they belong. . At least one or more computing devices in each network on-chip processing module are connected to all storage devices in the network on-chip processing module. In other words, a computing device in each network on-chip processing module can access all storage devices in the network on-chip processing module. Here, the number of storage devices in each network on-chip processing module is not limited to two, but may be three, four, or a plurality. Here, although not particularly limited thereto, four are preferred. Therefore, it is possible to efficiently process data while saving the occupied space.

Specifically, the computing devices in each network on-chip processing module preferentially access adjacent storage devices. Here, the adjacent storage device refers to a storage device having the shortest communication distance among a plurality of storage devices connected to the computing device. In other words, the storage device with the shortest communication distance has higher access priority than other storage devices.

Alternatively, in each network on-chip processing module, the plurality of computing devices are connected to each other to form a computing device network, and a connection method between the plurality of computing devices in each network on-chip processing module is a network on-chip processing system (1100). ) To a connection method of the network on-chip processing system 1400 may be referred to. It is not repeated here. Alternatively, not all of the computing devices need to be connected to the storage device 1911 in the plurality of computing devices in each network on-chip processing module, and at least one computing device in each network on-chip processing module is connected to the storage device 1911. Just do it. Here, there is no particular limitation on this.

Alternatively, each of the computing devices in each network on-chip processing module may be connected to a different network on-chip processing module, and at least one computing device of each network on-chip processing module may calculate at least one of the other network on-chip processing modules. Just connect to the device. Here, there is no particular limitation on this. Alternatively, the plurality of network on-chip processing modules are connected to each other through a computing device among each of the network on-chip processing modules. In other words, any one computing device of each network on-chip processing module may be connected to any one computing device of the other network on-chip processing module. Here, there is no particular limitation on this.

In the network on-chip processing system, multiple communication channels can be provided to transmit data by accessing all storage devices in the network on-chip processing module to which each computing device belongs, thereby improving data reading/writing efficiency. have. In addition, since each of the computing devices of the system accesses adjacent storage devices preferentially, overhead due to memory access can be reduced and a certain degree of flexibility can be ensured.

In one embodiment, the network on-chip processing system 19100 illustrated in FIG. 10B stores all data that each network on-chip processing module needs to process in a storage device inside the network on-chip processing module. In other words, the plurality of computing devices in each network on-chip processing module can only access the storage devices inside the network on-chip processing module to which they belong, so they can only read/write data from the storage devices inside the network on-chip processing module to which they belong. . At least one computing device in each network on-chip processing module is configured to be connected to all storage devices in the network on-chip processing module. In other words, a computing device in each network on-chip processing module can access all storage devices in that network on-chip processing module. Here, the number of storage devices in each network on-chip processing module is not limited to two, but may be three, four, or a plurality. Here, although not particularly limited thereto, four are preferred.

Specifically, in each network on-chip processing module, each computing device is connected to a storage device at a first communication distance. Here, the first communication distance indicates the shortest communication distance. In other words, the computing devices in each network on-chip processing module can only access adjacent storage devices. That is, the computing device in each network on-chip processing module can access only the storage device having the shortest communication distance with it. For example, the computing device 19120 can access only the storage device 19110 that is adjacent to it, and cannot access the storage device 19160. Computing device 19130 can only access adjacent storage device 19160 and cannot access storage device 19110. When data to be read by the calculation device 19120 is stored in the storage device 19160, the data is first read from the storage device 19160 through the calculation device 19130 and then the data is calculated by the calculation device 19120. Transfer to.

Alternatively, in each network on-chip processing module, the plurality of computing devices are connected to each other to form a computing device network, and the connection method between the plurality of computing devices in each network on-chip processing module is a network on-chip processing system (1100). To a connection method of the network on-chip processing system 1400 may be referred to. It is not repeated here. Alternatively, all of the plurality of computing devices in each network on-chip processing module need not be connected to the storage device 19110, and at least one computing device of each network on-chip processing module may be connected to the storage device 19110. Here, there is no particular limitation on this.

Alternatively, each computing device in each network on-chip processing module can be configured to be connected to another network on-chip processing module, and at least one computing device in each network on-chip processing module is at least It just needs to be connected to more than one computing device. Here, there is no particular limitation on this. Alternatively, the plurality of network on-chip processing modules are connected to each other through any one computing device in each network on-chip processing module. In other words, any one computing device in each network on-chip processing module may be connected to any one computing device in another network on-chip processing module. Here, there is no particular limitation on this.

In the network on-chip processing system, multiple communication channels can be provided to transmit data by accessing all storage devices in the network on-chip processing module to which each computing device belongs, thereby improving data reading/writing efficiency. have. The system can access only the storage devices to which each computing device is adjacent to each other, thereby maximizing overhead due to memory access.

In one embodiment, the present application provides a network on-chip processing system. The system includes any two network on-chip processing modules directly connected, any two network processing modules are installed on the same chip, and each network on-chip processing module includes at least one storage device and a plurality of computing devices, In each network on-chip processing module, at least one or more computing devices are connected to at least one or more storage devices inside the network processing module, and at least two or more of the plurality of computing devices are connected to each other.

As shown in FIG. 11, FIG. 11 is a network on-chip processing system 1920 according to an embodiment. The network on-chip processing system 1920 includes four network on-chip processing modules connected to each other, and the four network on-chip processing modules are installed on the same chip of the network on-chip processing system 1920, among the four network on-chip processing modules. Any two network on-chip processing modules are directly connected to each other, and each network on-chip processing module includes one storage device 1921 and four calculation devices (calculation device 1922 to calculation device 1925). . Here, in each network-on-chip processing module, the computing device 1922 is connected to the storage device 1921 inside the network-on-chip processing module, and the four computing devices inside each network-on-chip processing module are connected to each other.

Specifically, all data to be processed by each network on-chip processing module is stored in a storage device inside the network on-chip processing module. In other words, the plurality of computing devices in each network on-chip processing module can only access the storage devices inside the network on-chip processing module to which they belong, and can only read/write data from the storage devices inside the network on-chip processing module to which they belong. .

Alternatively, the number of storage devices in each network on-chip processing module is not limited to one, and may be two, three, or a plurality. Here, although not particularly limited thereto, four are preferred. Alternatively, in each network on-chip processing module, the plurality of computing devices are connected to each other to form a computing device network, and the connection method between the plurality of computing devices in each network on-chip processing module is a network on-chip processing system (1100). To a connection method of the network on-chip processing system 1400 may be referred to. It is not repeated here. Alternatively, the plurality of computing devices in each network on-chip processing module do not all need to be connected to the storage device 1921, and at least one computing device of each network on-chip processing module only needs to be connected to the storage device 1921. . Here, there is no particular limitation on this.

Alternatively, each computing device in each network on-chip processing module may be configured to be connected to another network on-chip processing module, and at least one computing device in each network on-chip processing module may be configured to be connected to another network on-chip processing module. It only needs to be connected to at least one computing device. Here, there is no particular limitation on this. Alternatively, the plurality of network on-chip processing modules are connected to each other through any one computing device in each network on-chip processing module. That is, any one computing device in each network on-chip processing module may be connected to any one computing device in another network on-chip processing module. Here, there is no particular limitation on this.

In the network on-chip processing system, a plurality of network on-chip processing modules installed on the same chip are configured to be connected to each other, and at the same time, a plurality of computing devices in each network on-chip processing module are configured to be connected to each other. As intra-module communication is possible and direct communication between modules is possible between any two network on-chip processing modules at the same time, the system can reduce the number of computational devices that simultaneously read the storage device interface, reduce bandwidth clogging, and reduce data transmission between modules. As a result, data reading/writing efficiency can be improved.

In one embodiment, the present application provides a network on-chip processing system. The system includes any two network on-chip processing modules directly connected, any two network processing modules are installed on the same chip, each network on-chip processing module includes a plurality of storage devices, and the network on-chip processing In the module, at least one or more calculation devices are connected to the plurality of storage devices inside the network processing module, and at least two or more of the plurality of calculation devices are connected to each other.

As shown in FIG. 12, FIG. 12 is a network on-chip processing system 1930 according to an embodiment. The network on-chip processing system 1930 includes four network on-chip processing modules connected to each other, and the four network on-chip processing modules are installed on the same chip of the network on-chip processing system 1920, and in the four network on-chip processing modules. Any two network on-chip processing modules are directly connected to each other, and each network on-chip processing module includes a storage device 1931, a storage device 1936, and four calculation devices (calculation device 1932 to calculation device 1935). ). In each network on-chip processing module, the computing device 1932 is connected to the storage device 1931 and the storage device 1936 inside the corresponding network on-chip processing module, and the four computing devices inside each network on-chip processing module are mutually connected.

Specifically, all data to be processed by each network on-chip processing module is stored in a storage device inside the network on-chip processing module. In other words, the plurality of computing devices in each network on-chip processing module can only access the storage devices inside the network on-chip processing module to which they belong, so they can only read/write data from the storage devices inside the network on-chip processing module to which they belong. . The computing devices in each network on-chip processing module preferentially access adjacent storage devices.

Alternatively, the number of storage devices in each network on-chip processing module is not limited to two, but may be three, four, or a plurality. Here, although not particularly limited thereto, four are preferred. Specifically, at least one or more computing devices in each network on-chip processing module are configured to be connected to all storage devices in the network on-chip processing module. In other words, the computing device in each network on-chip processing module can access all storage devices in the network on-chip processing module.

Alternatively, in each network on-chip processing module, the plurality of computing devices are connected to each other to form a computing device network, and a connection method between the plurality of computing devices in each network on-chip processing module is a network on-chip processing system (1100). ) To a connection method of the network on-chip processing system 1400 may be referred to. It is not repeated here. Alternatively, in a plurality of computing devices in each network on-chip processing module, all computing devices do not need to be connected to the storage device 1931, and at least one of the network on-chip processing modules is connected to the storage device 1931. Just do it. Here, there is no particular limitation on this.

Alternatively, each computing device in each network on-chip processing module may be connected to a different network on-chip processing module, and at least one computing device in each network on-chip processing module may be at least one computing device in a different network on-chip processing module. Just connect to. Here, there is no particular limitation on this. Alternatively, the plurality of network on-chip processing modules are connected to each other through any one computing device in each network on-chip processing module. In other words, any one computing device in each network on-chip processing module may be connected to any one computing device in another network on-chip processing module. Here, there is no particular limitation on this.

In the network on-chip processing system, each computing device can access all storage devices in the network on-chip processing module to which it belongs, and at the same time, any two network on-chip processing modules enable direct communication between the modules. Therefore, the system can provide multiple communication channels to transmit data, thereby improving data read/write efficiency. Each of the computing devices in the above systems preferentially accesses adjacent storage devices, so that overhead due to memory X can be saved and a certain degree of flexibility can be ensured.

In one embodiment, as shown in FIG. 13, the computing device of the network on-chip processing system shown in FIG. 13 is configured to perform machine learning calculation, and the computing device comprises a controller unit 11 and a computing unit 12. The controller unit 11 and the operation unit 12 are connected, and the operation unit 11 includes one mast processing circuit and a plurality of slave processing circuits.

The controller unit 11 is used to obtain input data and calculation instructions. In one selectable method, the acquisition of input data and calculation command may be specifically implemented through a data input/output unit, and the data input/output unit may specifically be one or more data I/O interfaces or I/O pins.

The calculation instruction may include a forward arithmetic instruction or a reverse training instruction; Or another neural network operation instruction such as a convolution operation instruction, but is not limited thereto. The specific embodiment of the present application does not limit the specific expression format of the calculation instruction.

Specifically, the controller unit 11 is further configured to analyze the calculation command to obtain a plurality of calculation commands, and transmits the plurality of calculation commands and the input data to the mast processing circuit.

The operation unit 12 includes one mast processing circuit 101 and a plurality of slave processing circuits 102. Here, the mast processing circuit 101 performs pre-order processing on the input data, and transmits data and operation instructions to the plurality of slave processing circuits;

The plurality of slave processing circuits obtain a plurality of intermediate results by performing intermediate operations in parallel according to the data and operation instructions transmitted from the mast processing circuit, and then transmit the plurality of intermediate results to the mast processing circuit;

The mast processing circuit 101 performs subsequent processing on the plurality of intermediate results to obtain a calculation result of the calculation instruction.

The technical solution provided in the present application installs the operation unit in a one-master-multiple-slave structure and divides data according to a calculation instruction for forward operation. Accordingly, by performing parallel operation on a portion with a large amount of calculation through a plurality of slave processing circuits, it is possible to improve the operation speed and save the operation time, thereby lowering the power consumption.

Alternatively, the computing device further includes a corresponding storage unit 10 and a direct memory access unit 50, and the storage unit 10 may include one of a register, a cache, or any combination. Specifically, the cache stores the calculation instruction, the register stores the input data and scalar, and the cache is a fast temporary storage cache. The direct memory access unit 50 reads or stores data from the storage unit 10.

Alternatively, the controller unit includes an instruction storage unit 110, an instruction processing unit 111 and a storage queue unit 113.

The instruction storage unit 110 stores a calculation instruction related to the artificial neural network,

The instruction processing unit 111 analyzes the computation instruction to obtain a plurality of computation instructions,

The storage queue unit 113 stores a command queue, and the command queue includes a plurality of calculation commands and/or calculation commands to be executed according to the order of before and after the queue.

For example, in one selectable technical solution, the mast operation processing circuit may further include one controller unit. The controller unit may include a master instruction processing unit and is specifically used to decode instructions into micro instructions. In another selectable technical solution, the slave operation processing circuit may further include another controller unit, and the other controller unit includes a slave command processing unit, specifically for receiving and processing micro-instructions. Used. The micro-instruction may be a next level instruction of the instruction. The micro-command may be obtained by dividing or decoding the command, and may be further decoded as a control signal of each member, each unit or each processing circuit.

In one selectable scheme, the structure of the calculation instruction is as shown in Table 1.

Table 1

The ellipsis in the above table means that a plurality of registers or immediate values can be included.

In another selectable method, the calculation instruction may include one or more operation domains and one operation code. The calculation command may include a neural network calculation command. For example, as an example of a neural network operation command, as shown in Table 2, register number 0, register number 1, register number 2, register number 3, and register number 4 may be operation domains. Here, each of the register number 0, register number 1, register number 2, register number 3, and register number 4 may be a number of one or a plurality of registers.

Table 2

The register may be an off-chip memory, or of course, an on-chip memory in an actual application, and is used to store data, and the data may specifically be n-dimensional data, and n is an integer greater than or equal to 1. For example, when n=1, it is one-dimensional data, that is, it is a vector; When n=2, it is two-dimensional data, that is, it is a matrix; When n=3 or 3 or more, it is a multidimensional tensor.

Alternatively, the controller unit,

When there are a plurality of arithmetic instructions, it is determined whether a first arithmetic instruction and a 0th arithmetic instruction prior to the first arithmetic instruction have a correlation relationship, and if the first arithmetic instruction and the 0-th arithmetic instruction have an association relationship A dependency relationship processing unit 112 that caches the first operation instruction in the instruction storage unit, and extracts the first operation instruction from the instruction storage unit after performing the 0th operation instruction and transmits it to the operation unit. Including,

The step of determining whether the first operation instruction and the zeroth operation instruction prior to the first operation instruction have an association relationship,

Extracting a first storage address section of data (e.g., a matrix) required for the first operation command by the first operation command, and storing the zeroth matrix required for the 0th operation command by the 0th operation command If an address section is extracted, but if there is an overlapped region between the first storage address section and the 0th storage address section, it is determined that the first operation instruction and the zeroth operation instruction are related, and the first storage And determining that the first operation instruction and the zeroth operation instruction have no relationship if there is no overlapped region between the address section and the zeroth storage address section.

In another embodiment, as shown in FIG. 14, the operation unit 12 may include one mast processing circuit 101 and a plurality of slave processing circuits 102. In one embodiment, as shown in FIG. 14, a plurality of slave processing circuits form an array distribution, each slave processing circuit is connected to another adjacent slave processing circuit, and the mast processing circuit is used to process the plurality of slaves. It is connected to K slave processing circuits among the circuits, wherein the K slave processing circuits are n slave processing circuits in a first row, n slave processing circuits in an mth row, and m slave processing circuits in a first column. Note that, as shown in Fig. 14, the K slave processing circuits only include n slave processing circuits in the first row, n slave processing circuits in the mth row, and m slave processing circuits in the first column. That is, the K slave processing circuits are slave processing circuits directly connected to the mast processing circuit among a plurality of slave processing circuits.

The K slave processing circuits forward data and commands between the mast processing circuit and a plurality of slave processing circuits.

Alternatively, as shown in Fig. 15, the mast processing circuit includes one or any combination of a conversion processing circuit 110, an activation processing circuit 111, and an addition processing circuit 112,

The conversion processing circuit 110 performs compatibility (for example, conversion of continuous data and discrete data) between the first data structure and the second data structure with respect to the data block or intermediate result received by the mast processing circuit, or The master processing circuit performs compatibility between the first data type and the second data type (e.g., fixed-point type and floating-point type conversion) on the data block or intermediate result received,

The activation processing circuit 111 performs an activation operation of data in the mast processing circuit,

The addition processing circuit 112 performs an addition operation or an accumulation operation.

The mast processing circuit determines the input neuron as broadcast data and the weight as distribution data, distributes the distribution data into a plurality of data blocks, and includes at least one data block among the plurality of data blocks and a plurality of operation instructions. Transmits at least one operation command to the slave processing circuit,

The plurality of slave processing circuits operate on the data blocks already received by the operation instruction to obtain intermediate results, and transmit the operation results to the mast processing circuit,

The mast processing circuit processes intermediate results transmitted by a plurality of slave processing circuits to obtain a result of the calculation command, and transmits the result of the calculation command to the controller unit.

The slave processing circuit includes a multiplication processing circuit,

The multiplication processing circuit multiplies the received data block to obtain a multiplication result,

Alternatively, the slave processing circuit further includes a forwarding processing circuit, which forwarding processing circuitry forwards the received data block or multiplication result.

Alternatively, the slave processing circuit further includes an accumulating processing circuit, wherein the accumulating processing circuit accumulates the multiplication result to obtain the intermediate result.

In another embodiment, the operation instruction is a calculation instruction such as a matrix multiplied by a matrix, an accumulation instruction, and an activation instruction.

Hereinafter, a detailed calculation method of the calculation device shown in FIG. 1 will be described through a neural network operation command. For a neural network operation instruction, the actual formula to be performed is:

이고, 여기서, 가중치

에 입력 데이터

를 곱하고 합을 구한 다음 오프셋 b 를 더하고 활성화 연산

을 진행하여, 마지막 출력결과 s 를 얻는다.

Is, where, the weight

Input data to

Multiply and sum, then add offset b and activate

To get the final output s.

In one selectable scheme, as shown in Fig. 16, the calculation unit includes a tree module 40, the tree module includes a root port 401 and a plurality of branch ports 404, and , A root port of the tree module is connected to the mast processing circuit, and a plurality of branch ports of the tree module are connected to one slave processing circuit among a plurality of slave processing circuits, respectively,

The tree module has a transmission/reception function. For example, the tree module shown in FIG. 16 performs a transmission function, and the tree module shown in FIG. 17 performs a reception function.

The tree module forwards data blocks, weights, and operation instructions between the mast processing circuit and the plurality of slave processing circuits.

Alternatively, the tree module may include nodes of at least one layer as an optional result of the computing device, the node is a linear structure having a transmission function, and the node itself may not have a calculation function. If the tree module has a node of layer 0, the tree module is not needed.

Alternatively, the tree module may have an n-number tree structure, for example, a two-number tree structure shown in FIG. 18. However, a 3-number tree structure is also possible. N is an integer greater than or equal to 2. The specific embodiment of the present application does not limit the specific value of n. The number of layers may also be 2, and the slave processing circuit may be connected to a node of a layer other than the node of the second layer below. For example, it may be connected to the node of the first layer from the bottom shown in FIG. 18.

Alternatively, the operation unit may have a separate cache, and may include a neuron cache unit, as shown in FIG. 19. The neuron cache unit 63 caches input neuron vector data and output neuron value data of the slave processing circuit.

As shown in FIG. 20, the calculation unit includes a weight cache unit 64 for caching weights required for calculation by the slave processing circuit.

In one selectable embodiment, as shown in Fig. 21, the arithmetic unit 12 may include a branch processing circuit 103 and its specific connection structure is as shown in Fig. 21.

Here, the mast processing circuit 101 is connected to the branch processing circuit 103 (one or more), and the branch processing circuit 103 is connected to one or a plurality of slave processing circuits 102;

The branch processing circuit 103 forwards data or commands between the mast processing circuit 101 and the slave processing circuit 102.

This application includes one or more calculation devices mentioned in the present application, acquires operand data and control information from other processing devices, performs a designated machine learning operation, and transmits the execution result through an I/O interface to a peripheral device. A neural network computing device that transmits to is additionally disclosed. Peripheral devices are, for example, cameras, displays, mice, keyboards, network cards, wifi interfaces and servers. When including at least one or more computing devices, the computing devices are connected through a specific structure to transmit data. For example, it is interconnected over the PCIE bus to transmit data, enabling larger scale machine learning operations. In this case, one control system may be shared or each may have an independent control system, and each may share a memory or each accelerator may have its own memory. Meanwhile, the interconnect scheme may be any interconnect topology.

Since the neural network computing device has high compatibility, it can connect to various types of servers through a PCIE interface.

The present application further discloses an integrated processing device including the neural network computing device, a general-purpose interconnect interface, and other processing devices. The neural network computing device interacts with other processing devices to jointly perform a user-specified operation. 22 is a schematic diagram of an integrated processing device.

Other processing units include one or more processors among general-purpose/dedicated processors such as a central processor CPU, a graphics processor GPU, and a neural network processor. The number of processors included in other processing units is not limited. Another processing device is an interface between the neural network computing device and external data and control, and includes data transport and performs basic control such as start and stop of the neural network computing device. Other processing units may also collaborate with neural network computing units to jointly complete computational tasks.

The general-purpose interconnect interface transmits data and control commands between the neural network computing device and other processing devices. The neural network computing device may acquire necessary input data from another processing device and record it in a storage device of the neural network computing device, obtain a control command from another processing device and record it in the control cache of the neural network computing device. Data in the storage module can be read and transmitted to other processing devices.

Alternatively, as shown in FIG. 23, the structure may further include a storage device connected to the neural network computing device and the other processing device, respectively. The storage device stores data of the neural network computing device and the other processing device, and is particularly used for data that cannot be stored in an internal storage of the neural network computing device or another processing device in the data requiring calculation.

The integrated processing unit can be used as an SOC-on-chip system for devices such as mobile phones, robots, DRAM, and video surveillance devices, and can effectively reduce a core area of a control part, increase processing speed, and reduce overall power consumption. In this situation, the universal interconnect interface of the integrated processing unit is connected to some member of the unit. Some members are, for example, a camera, display, mouse, keyboard, network card, wifi interface.

In some embodiments, the present application further provides a chip including the neural network computing device or the integrated processing device.

In some embodiments, the present application further provides a chip package structure including the chip.

In some embodiments, the present application further provides a board card including the chip package structure. Referring to FIG. 24, FIG. 24 provides a board card. The board card may further include an auxiliary member other than the chip 389, and the auxiliary member includes, but is not limited to, a memory device 390, an interface device 391, and a controller device 392. .

The memory device 390 is connected to a chip in the chip package structure through a bus to store data. The memory device may include a plurality of groups of memory cells 393. The memory cells of each group are connected to the chip through a bus. It can be understood that the memory cells of each group may be DDR SDRAM (Double Data Rate SDRAM, Double Data Rate Synchronous Dynamic Random Memory).

DDR can double the speed of SDRAM without increasing the clock frequency. DDR allows data to be read on the rising and falling edges of a clock pulse. The speed of DDR is twice that of standard SDRAM. In an embodiment, the memory device may include four groups of the memory cells. Each group of memory cells may include a plurality of DDR4 particles (chips). In one embodiment, four 72-bit DDR4 controllers may be included in the chip, and 64 bits of the 72-bit DDR4 controllers are used for data transmission, and 8 bits are used for ECC verification. When DDR4-3200 particles are used for each group of memory cells, it can be understood that the theoretical bandwidth of data transmission can reach 25600MB/s.

In one embodiment, each group of the memory cells includes a plurality of DDR SDRAMs installed in parallel. DDR can transfer data twice within one clock cycle. A controller for controlling DDR is installed in the chip to control data transmission and data storage of each of the memory cells.

The interface device is electrically connected to a chip in the chip package structure. The interface device implements data transmission between the chip and an external device (eg, a server or a computer). For example, in one embodiment, the interface device may be a standard PCIE interface. For example, data to be processed is transferred from the server to the chip through a standard PCIE interface, through which data transfer is implemented. Alternatively, when transmitting over the PCIE 3.0 X 16 interface, the theoretical bandwidth can reach 16000MB/s. In another embodiment, the interface device may be a different interface. The interface unit of the present application is not limited to the specific expression format of the other interface, and only needs to be able to implement a connection function. Meanwhile, the calculation result of the chip is also transmitted to an external device (eg, a server) through the interface device.

The control device is electrically connected to the chip. The control device monitors the state of the chip. Specifically, the chip and the control device may be electrically connected through an SPI interface. The control device may include a micro controller unit (MCU). If the chip includes a plurality of processing chips, a plurality of processing cores, or a plurality of processing circuits, a plurality of loads can be driven. Thus, the chip can be in different operating states, such as high load and low load. The control device can control operating states of a plurality of processing chips, a plurality of processing cores, or a plurality of processing circuits in the chip.

In some embodiments, the present application provides an electronic device including the board card.

Electronic devices include data processing devices, robots, computers, printers, scanners, tablets, smart terminals, mobile phones, driving recorders, navigators, sensors, cameras, servers, cloud servers, cameras, video cameras, projectors, watches, headphones, portable devices. It includes storage devices, wearable devices, means of transportation, home appliances and/or medical equipment.

The means of transportation includes an airplane, a ship and/or a vehicle, and the home appliance includes a television, an air conditioner, a microwave oven, a refrigerator, an electric rice cooker, a humidifier, a washing machine, a lamp, a gas stove, and a range hood, and the medical equipment is Nuclear magnetic resonance equipment, B-ultrasonic equipment and/or electrocardiogram equipment.

In one embodiment, as shown in Fig. 25, a network on-chip data processing method is provided. The method includes the following steps.

Step 202: The storage device is accessed through the first calculation device to obtain first calculation data.

Here, the first calculation device includes an arithmetic unit and a controller unit, and the arithmetic unit includes one mast processing circuit and a plurality of slave processing circuits. Specifically, the controller unit in the first calculation device obtains the first calculation data and calculation instruction from the storage device.

Step 204: A first calculation result is obtained by performing an operation on the first calculation data through the first calculation device.

Here, the first calculation data read from the storage device is calculated by the first calculation device according to a corresponding calculation instruction to obtain a first calculation result.

Step 206: The first calculation result is transmitted to a second calculation device.

Here, the first calculation device transmits the first calculation result to the second calculation device by the controller unit in the first calculation device through a communication channel set with the second calculation device. Alternatively, the result of the first operation may be transmitted to the second computing device as well as to the storage device.

Furthermore, the network on-chip data processing method according to the present embodiment can be applied to any one network on-chip processing system shown in FIGS. 1-5.

The network-on-chip data processing method can implement data transmission between a plurality of calculation devices by transmitting a first calculation result in a first calculation device to a second calculation device, and at the same time, the calculation device stores the calculation data through recycling. The bandwidth overhead caused by multiple accesses to the device can be avoided. The method can reasonably use calculation data and intermediate calculation results, thereby improving data processing efficiency.

In one embodiment, as shown in Fig. 26, a network on-chip data processing method is provided. The method includes the following steps.

Step 302: The storage device is accessed through the first calculation device to obtain first calculation data.

Here, the first calculation device includes an arithmetic unit and a controller unit, and the arithmetic unit includes one mast processing circuit and a plurality of slave processing circuits. Specifically, the controller unit in the first calculation device obtains the first calculation data and calculation instruction from the storage device.

Step 304: A first calculation result is obtained by performing calculation on the first calculation data through the first calculation device.

Here, the first calculation data read from the storage device is calculated by the first calculation device according to a corresponding calculation instruction to obtain a first calculation result.

Step 306: The first calculation result is transmitted to a second calculation device.

Here, the first calculation device transmits the first calculation result to the second calculation device by the controller unit in the first calculation device through a communication channel set with the second calculation device.

Step 308: The storage device is accessed through the second calculation device to obtain second calculation data.

Here, the second calculation device includes an arithmetic unit and a controller unit, and the arithmetic unit includes one mast processing circuit and a plurality of slave processing circuits. Specifically, the controller unit in the second calculation device obtains the second calculation data and calculation instruction from the storage device.

Step 310: A second calculation result is obtained by performing an operation on the second calculation data and the first calculation result through the second calculation device.

Here, the second calculation data read from the storage device and the first calculation result received from the first calculation device are calculated by the first calculation device according to the corresponding calculation instruction to obtain a second calculation result.

Furthermore, the network on-chip data processing method according to the present embodiment can be applied to any one network on-chip processing system shown in FIGS. 1-5.

In the network-on-chip data processing method, after transmitting a first calculation result in a first calculation device to a second calculation device, the second calculation device performs re-calculation using the first calculation result, thereby implementing the recycling of calculation data. I can. The method can reasonably use calculation data and intermediate calculation results, thereby improving data processing efficiency.

In one embodiment, the network on-chip data processing method shown in FIG. 26 is applied to the network on-chip processing system 1900 shown in FIG. 9. Here, the calculation device 1902 to the calculation device 1905 are all connected to the storage device 1901 in the network on-chip processing module to which they belong, and any two calculation devices of the calculation device 1902 to 1905 are mutually It is directly connected.

, 행렬B =

, 행렬C = A * B =

을 계산한다.For example, compute one matrix multiplication, and matrix A =

, Matrix B =

, Matrix C = A * B =

Calculate

=

+

；here,

=

+

；

=

+

；

=

+

；

=

+

；

=

+

；

=

+

,

=

+

,

First, dividing the time to obtain 3 time intervals.

Subsequently, in the first time interval, the computing devices 1902 to 1905 access the storage device 1901 in the network on-chip processing module to which they belong.

및

; 계산 장치(1903)가 저장 장치(1901)로부터 제1 연산 데이터

및

를 판독하고, 계산 장치(1904)가 저장 장치(1901)로부터 제1 연산 데이터

및

를 판독하며, 계산 장치(1905)가 저장 장치(1901)로부터 제1 연산 데이터

및

를 판독한다.Specifically, the calculation device 1902 is the first calculation data from the storage device 1901

And

; The calculation device 1903 is the first calculation data from the storage device 1901

And

Is read, and the calculation device 1904 transmits the first calculation data from the storage device 1901

And

Is read, and the calculation device 1905 is configured to generate the first calculation data from the storage device 1901.

And

Read.

및

에 대해 연산하여 제1 연산 결과

를 얻고, 계산 장치(1903)에서 판독된 제1 연산 데이터

및

에 대해 연산하여 제1 연산 결과

를 얻으며, 계산 장치(1904)에서 판독된 제1 연산 데이터

및

에 대해 연산하여, 제1 연산 결과

를 얻으며, 계산 장치(1905)에서 판독된 제1 연산 데이터

및

에 대해 연산하여 제1 연산 결과

를 얻는다.Furthermore, the first calculation data read from the calculation device 1902

And

The first operation result by calculating on

Is obtained, and the first calculation data read in the calculation device 1903

And

The first operation result by calculating on

Is obtained, and the first calculation data read by the calculation device 1904

And

And the first operation result

Is obtained, and the first calculation data read by the calculation device 1905

And

The first operation result by calculating on

Get

를 판독하고, 계산 장치(1904)로부터 제1 연산 데이터

를 판독하여 연산을 통해 제2 연산 결과

를 얻고, 계산 장치(1903)가 각각 계산 장치(1902)로부터 제1 연산 데이터

를 판독하고, 계산 장치(1905)로부터 제1 연산 데이터

를 판독하여 연산을 통해 제2 연산 결과

를 얻으며, 계산 장치(1904)가 각각 계산 장치(1905)로부터 제1 연산 데이터

를 판독하고, 계산 장치(1902)로부터 제1 연산 데이터

를 판독하여 연산을 통해 제2 연산 결과

를 얻으며, 계산 장치(1905)가 각각 계산 장치(1904)로부터 제1 연산 데이터

를 판독하고 계산 장치(1903)로부터 제1 연산 데이터

를 판독하여 연산을 통해 제2 연산 결과

를 얻는다.Subsequently, in the second time interval, the calculation devices 1902 each receive the first calculation data from the calculation devices 1903.

Is read, and the first calculation data from the calculation device 1904

By reading the result of the second operation

Is obtained, the calculation device 1902 is the first calculation data from the calculation device 1902, respectively

Is read, and the first calculation data from the calculation device 1905

By reading the result of the second operation

Is obtained, and the calculation device 1904 is the first calculation data from the calculation device 1905, respectively.

Is read, and the first calculation data from the calculation device 1902

By reading the result of the second operation

Is obtained, and the calculation device 1905 receives the first calculation data from the calculation device 1904, respectively.

And the first calculation data from the calculation device 1903

By reading the result of the second operation

Get

및 제2 연산 결과

에 대해 연산하여 제3 연산 결과

=

+

를 얻은 후, 제3 연산 결과

를 저장 장치(1901)에 송신하며; 계산 장치(1903)가 제1 연산 결과

및 제2 연산 결과

에 대해 연산하여 제3 연산 결과

=

+

를 얻은 후, 제3 연산 결과

를 저장 장치(1901)에 송신하고; 계산 장치(1904)가 제1 연산 결과

및 제2 연산 결과

에 대해 연산하여 제3 연산 결과

=

+

를 얻은 후 제3 연산 결과

를 저장 장치(1901)로 송신하고; 계산 장치(1905)가 제1 연산 결과

및 제2 연산 결과

에 대해 연산하여 제3 연산 결과

=

+

를 얻은 후, 제3 연산 결과

를 저장 장치(1901)에 송신한다.Subsequently, in the third time interval, the calculation device 1902

And the result of the second operation

The result of the third operation by calculating on

=

+

After obtaining, the third operation result

To the storage device 1901; The calculation device 1903 is the first calculation result

And the result of the second operation

The result of the third operation by calculating on

=

+

After obtaining, the third operation result

To the storage device 1901; The calculation device 1904 is the first calculation result

And the result of the second operation

The result of the third operation by calculating on

=

+

After obtaining the result of the third operation

To the storage device 1901; The calculation device 1905 is the first calculation result

And the result of the second operation

The result of the third operation by calculating on

=

+

After obtaining, the third operation result

Is transmitted to the storage device 1901.

In one embodiment, as shown in Fig. 27, a network on-chip data processing method is provided. The method includes the following steps.

Step 402: The storage device is accessed through the first computing device group to obtain first calculation data. Here, the first calculation device group includes a plurality of first calculation devices.

Here, each of the first calculation devices in the first calculation device group cluster1 includes a calculation unit and a controller unit, and the calculation unit includes one mast processing circuit and a plurality of slave processing circuits. Specifically, the controller unit in cluster1 obtains the first calculation data and calculation instruction from the storage device.

Alternatively, a plurality of first computing devices in cluster1 simultaneously access the storage device, and each first computing device reads some data required for cluster1 from the storage device. These data are transferred within cluster1. Alternatively, one or more first computing devices in cluster1 are designated to be able to access the storage device, and the remaining first computing devices are allowed to communicate only within the group.

Step 404: A first calculation result is obtained by performing calculation on the plurality of first calculation data through the first calculation device group.

Here, the plurality of first calculation data is calculated and forwarded between the plurality of first calculation devices according to a corresponding calculation instruction to obtain a first calculation result.

Step 406: Send the first calculation result to the second computing device group.

Here, cluster1 transmits the first operation result to cluster2 by the controller unit in cluster1 through a communication channel established with the second computing device group cluster2.

Alternatively, the first operation result may be transmitted to cluster2 as well as the first operation result may be transmitted to the storage device. Alternatively, the first calculation result is transmitted to cluster2 through an arbitrary first calculation device of cluster1 in which a communication channel is established with cluster2. Alternatively, cluster1 may transmit the first calculation result to any one second computing device of cluster2 in which cluster1 and a communication channel are established.

Further, the network on-chip data processing method according to the present embodiment can be applied to any one network on-chip processing system shown in FIGS. 6-8.

The network-on-chip data processing method can implement intra-group communication as well as inter-group data transmission between a plurality of groups of computing devices, so that calculation data and intermediate calculation results can be reasonably used, thereby improving data processing efficiency.

In one embodiment, as shown in Fig. 28, a network on-chip data processing method is provided. The method includes the following steps.

Step 502: Acquire first computational data by accessing the storage device through the first computing device group. Here, the first calculation device group includes a plurality of first calculation devices.

Here, each of the first calculation devices in the first calculation device group cluster1 includes a calculation unit and a controller unit, and the calculation unit includes one mast processing circuit and a plurality of slave processing circuits. Specifically, the controller unit in cluster1 obtains the first calculation data and calculation instruction from the storage device.

Alternatively, a plurality of first computing devices in cluster1 simultaneously access the storage device, and each first computing device reads some data required for cluster1 from the storage device. These data are transferred within cluster1. Alternatively, one or more first computing devices in cluster1 are designated to allow access to the storage device, and the remaining first computing devices are allowed to communicate only within the group.

Step 504: A first calculation result is obtained by performing calculation on the plurality of first calculation data through the first calculation device group.

Here, the plurality of first calculation data is calculated and forwarded between the plurality of first calculation devices according to a corresponding calculation instruction to obtain a first calculation result.

Step 506: Send the first calculation result to the second computing device group.

* Here, cluster1 transmits the first calculation result to cluster2 by the controller unit in cluster1 through a communication channel established with the second computing device group (cluster2).

Alternatively, the first calculation result is transmitted to cluster2 through an arbitrary first calculation device in cluster1 in which cluster2 and a communication channel are established. Alternatively, cluster1 may transmit the first calculation result to any one second computing device in cluster2 in which cluster1 and a communication channel are established.

Step 508: Access the storage device through the second computing device group to obtain second calculation data. Here, the second calculation device group includes a plurality of second calculation devices.

Here, each of the first calculation devices in cluster2 includes a calculation unit and a controller unit, and the calculation unit includes a single mast processing circuit and a plurality of slave processing circuits. Specifically, the controller unit in cluster2 obtains the second operation data and the calculation instruction from the storage device.

Alternatively, a plurality of second computing devices in cluster2 simultaneously access the storage device, and each second computing device reads some data required for cluster2 from the storage device. These data are transferred within cluster2. Alternatively, one or more second computing devices in cluster2 are designated to allow access to the storage devices, and the remaining second computing devices are allowed to communicate only within the group.

Step 510: A second operation result is obtained by performing an operation on the second operation data and the first operation result through the second calculation device group.

Here, the second calculation data read from the storage device and the first calculation result received from the first calculation device group are calculated and forwarded between the plurality of second calculation devices according to a corresponding calculation instruction to obtain the second calculation result. Get

Further, the network on-chip data processing method according to the present embodiment can be applied to any one network on-chip processing system shown in FIGS. 6-8.

In the network-on-chip data processing method, after transmitting the first calculation result in the first calculation device group to the second calculation device group, the second calculation device group performs recalculation using the first calculation result. Can be implemented. The above method can reasonably use the operation data and intermediate operation results, thereby improving data processing efficiency.

In one embodiment, as shown in Fig. 29, a network on-chip data processing method is provided. The method includes the following steps.

Step 602: Acquire first operation data through a first network on-chip processing module. Here, the first network on-chip processing module includes a first storage device and a plurality of first calculation devices, and stores first operation data in the first storage device.

Here, each of the first calculation devices in the first network on-chip processing module includes an operation unit and a controller unit, and the operation unit includes one mast processing circuit and a plurality of slave processing circuits. Specifically, the controller unit in the first network on-chip processing module obtains the first operation data and the calculation instruction from the first storage device.

Alternatively, a plurality of first computing devices in a first network on-chip processing module simultaneously access a first storage device, and each of the first computing devices may receive some data required for the first network on-chip processing module from the first storage device. Read. This data is transmitted within the first network on-chip processing module.

Alternatively, one or a plurality of first computing devices in the first network on-chip processing module are designated to allow access to the first storage device and the remaining first computing devices are only capable of intra-group communication. Specifically, all computational data to be processed by the first network on-chip processing module are stored in the first storage device.

Step 604: A first calculation result is obtained by performing calculation on the first calculation data through a plurality of first calculation devices in the first network on-chip processing module.

Here, the plurality of first calculation data is calculated and forwarded between the plurality of first calculation devices according to a corresponding calculation instruction to obtain a first calculation result.

Step 606: The first operation result is transmitted to the second network on-chip processing module.

Here, the first network on-chip processing module transmits the first operation result to the second network on-chip processing module by the controller unit in the first network on-chip processing module through a communication channel established with the second network on-chip processing module.

Alternatively, the first operation result may be transmitted to the second network on-chip processing module as well as the first operation result may be transmitted to the first storage device. Alternatively, the first calculation result is transmitted to the second network on-chip processing module through any one of the first network on-chip processing modules in which the second network on-chip processing module and the communication channel are established. Alternatively, the first network on-chip processing module transmits the first operation result to any one of the first network on-chip processing module and the second network on-chip processing module in which a communication channel is established.

Further, the network on-chip data processing method according to the present embodiment can be applied to any one network on-chip processing system shown in FIGS. 9-12.

The network-on-chip data processing method can implement intra-group communication as well as inter-group data transmission between a plurality of computing device groups, so that operation data and intermediate operation results can be reasonably used, thereby improving data processing efficiency.

In one embodiment, as shown in Fig. 30, a network on-chip data processing method is provided. The method includes the following steps.

Step 702: Acquire first operation data through a first network on-chip processing module. Here, the first network on-chip processing module includes a first storage device and a plurality of first calculation devices, and the first operation data is stored in the first storage device.

Here, each of the first calculation devices in the first network on-chip processing module includes an arithmetic unit and a controller unit, and the arithmetic unit includes one mast processing circuit and a plurality of slave processing circuits. Specifically, the controller unit in the first network on-chip processing module obtains the first operation data and the calculation instruction from the first storage device.

Alternatively, a plurality of first computing devices in a first network on-chip processing module simultaneously access a first storage device, and each of the first computing devices may receive some data required for the first network on-chip processing module from the first storage device. Read. This data is transmitted within the first network on-chip processing module.

Alternatively, one or a plurality of first computing devices in the first network on-chip processing module are designated to allow access to the first storage device and the remaining first computing devices are only capable of intra-group communication. Specifically, all operation data to be processed by the first network on-chip processing module is stored in the first storage device.

Step 704: A first calculation result is obtained by performing calculation on the first calculation data through a plurality of first calculation devices in the first network on-chip processing module.

Here, the plurality of first calculation data is calculated and forwarded between the plurality of first calculation devices according to a corresponding calculation instruction to obtain a first calculation result.

Step 706: The first operation result is transmitted to the second network on-chip processing module.

Here, the first network on-chip processing module transmits the first operation result to the second network on-chip processing module by the controller unit in the first network on-chip processing module through a communication channel established with the second network on-chip processing module.

Alternatively, the first calculation result is transmitted to the second network on-chip processing module through any one of the first network on-chip processing modules in which the second network on-chip processing module and the communication channel are established. Alternatively, the first network on-chip processing module transmits the first operation result to any one of the first network on-chip processing module and the second network on-chip processing module in which a communication channel is established.

Step 708: Acquire second operation data through the second network on-chip processing module. Here, the second network on-chip processing module includes a second storage device and a plurality of second calculation devices, and stores the second operation data in the second storage device.

Here, each second calculation device in the second network on-chip processing module includes an operation unit and a controller unit, and the operation unit includes one mast processing circuit and a plurality of slave processing circuits. Specifically, the controller unit in the second network on-chip processing module obtains the second operation data and the calculation instruction from the second storage device.

Alternatively, a plurality of second computing devices in the second network on-chip processing module simultaneously access a second storage device, and each second computing device may receive some data required for the second network on-chip processing module from the second storage device. Get This data is transmitted within the second network on-chip processing module.

Alternatively, one or a plurality of second computing devices in the second network on-chip processing module are designated to allow access to the second storage device and the remaining second computing devices are only capable of intra-group communication. Specifically, all operation data to be processed by the second network on-chip processing module is stored in the second storage device.

Step 710: A second calculation result is obtained by performing calculation on the second calculation data and the first calculation result through a plurality of second calculation devices in the second network on-chip processing module.

Here, step 710 specifically includes the following steps.

Step 7102: Calculate the second calculation data and the first calculation result between the plurality of second calculation devices to obtain the second calculation result.

Specifically, each second calculation device obtains a plurality of intermediate results by calculating on the second calculation data and the first calculation result according to the corresponding calculation instruction. Subsequently, a second operation result is obtained by performing an operation on a plurality of intermediate results according to a corresponding calculation instruction.

Step 7104: The second operation result is stored in the second storage device.

Further, the network on-chip data processing method according to the present embodiment can be applied to any one network on-chip processing system shown in FIGS. 9-12.

In the network-on-chip data processing method, after transmitting the first operation result in the first network on-chip processing system to the second network on-chip processing system, the second network on-chip processing system performs calculation again using the first operation result. Data can be recycled. The above method can reasonably use the operation data and intermediate operation results, thereby improving data processing efficiency.

The network-on-chip processing method according to the embodiment of the present application is used for machine learning calculation, and specifically, may be used for artificial neural network calculation. Here, the operation data in the network on-chip processing system may specifically include input neuron data and weights, and the operation result in the network on-chip processing system may specifically be a result of an artificial neural network operation, that is, output neuron data.

The operation in the neural network may be a one-layer operation in the neural network, and in a multi-layer neural network, the implementation process is as follows. In the forward operation, after the execution of the artificial neural network of the previous layer is finished, the operation instruction of the next layer uses the output neuron calculated from the computation unit as the next layer input neuron (or performs some operation on the output neuron, and then returns the next layer input neuron. At the same time, the weight is also replaced with the next layer weight. In the reverse operation, after the backward operation of the previous layer artificial neural network is performed, the next layer operation instruction calculates the input neuron gradient calculated from the operation unit as the output neuron gradient of the next layer (or after some operation on the input neuron gradient. The output neuron of the next layer is tilted again), and the weight is replaced with the weight of the next layer.

The machine learning calculation may further include a support vector machine operation, a k-adjacent (k-nn) operation, a k-means (k-means) operation, a principal component analysis operation, and the like. For convenience of explanation, a specific method of machine learning calculation will be described below by taking an artificial neural network operation as an example.

층으로 설정하고,

. 제

층 및 제

층에 대해, 제

층을 입력층으로 부르되, 그 중의 뉴런은 상기 입력 뉴런이고, 제

층을 출력층으로 부르되, 그 중의 뉴런은 상기 출력 뉴런이다. 꼭대기 층을 제외한 각층을 모두 입력층으로 하고 그 다음 층이 대응하는 출력층일 수 있다.For an artificial neural network operation, if the artificial neural network operation has a multilayer operation, the input neurons and output neurons of the multilayer operation do not mean the neurons of the input layer and the neurons of the output layer of the entire neural network, but an arbitrary adjacent 2 of the neural network. In each layer, the neurons in the next layer of neural network forward operation are the input neurons, and the neurons in the previous layer of the neural network forward operation are the output neurons. Take a convolutional neural network, for example, a convolutional neural network

Set it as a layer,

. My

Floor and Article

About the floor, my

The layer is called the input layer, but the neuron among them is the input neuron,

The layer is called the output layer, and the neurons among them are the output neurons. Each layer except the top layer may be an input layer, and the next layer may be a corresponding output layer.

In one selectable embodiment, taking a complete connection operation among neural network operations as an example, the process may be y=f(wx+b), where x is an input neuron matrix, w is a weight matrix, and b is an offset scalar. And f is the activation function. Specifically, it may be any one of the sigmoid function, tanh, relu, and softmax function. Here, assuming that it has a 2-binary tree structure and has 8 slave processing circuits, the implementation method is as follows.

The controller unit obtains the input neuron matrix x, the weight matrix w, and the fully connected operation instruction from the storage unit, and then transmits the input neuron matrix x, the weight matrix w, and the fully connected operation instruction to the mast processing circuit,

The mast processing circuit determines the input neuron matrix x as broadcast data and the weight matrix w as distribution data, divides the weight matrix w into 8 child matrices, and processes 8 child matrices into 8 slaves through a tree module. Allocate into circuits, broadcast the input neuron matrix x to 8 slave processing circuits,

The slave processing circuit performs multiplication and accumulation operations of 8 sub-matrices and input neuron matrix x in parallel to obtain 8 intermediate results, and then transmits 8 intermediate results to the mast processing circuit.

The mast processing circuit sorts the eight intermediate results to obtain the operation result of wx, performs the operation of offset b on the operation result, then activates the activation operation to obtain the final result y, and sends the final result y to the controller unit. Thus, the final result y is output or stored to a storage unit by the controller unit.

A method in which the computing device illustrated in FIG. 1 performs a neural network forward operation instruction is specifically as follows.

The controller unit extracts a neural network forward operation instruction, an operation domain corresponding to the neural network operation instruction, and at least one operation code from the instruction storage unit, and the controller unit transmits the operation domain to the data access unit and transmits the at least one operation code. Send to the calculation unit.

The controller unit extracts the weight w and offset b corresponding to the operation domain from the storage unit (if b is 0, it is not necessary to extract the offset b), and transmits the weight w and the offset b to the mast processing circuit of the computation unit. Then, the controller unit extracts the input data Xi from the storage unit and transmits the input data Xi to the mast processing circuit.

The mast processing circuit is determined by a multiplication operation by the at least one operation code, determines input data Xi as broadcast data, weights as distribution data, and divides the weight w into n data blocks,

The instruction processing unit of the controller unit determines a multiplication instruction, an offset instruction, and an accumulation instruction according to the at least one operation code, and transmits a multiplication instruction, an offset instruction, and an accumulation instruction to the mast processing circuit, and the mast processing circuit performs the multiplication instruction. , The input data Xi is transmitted to a plurality of slave processing circuits in a broadcast manner, and the n data blocks are transmitted to the plurality of slave processing circuits (e.g., having n slave processing circuits, each slave processing circuit is one Data blocks are transmitted), and the plurality of slave processing circuits multiply the input data Xi and the received data block by the multiplication instruction to obtain an intermediate result, and then mast processing the intermediate result. Transmission to the circuit, and the mast processing circuit accumulates the intermediate result transmitted from the plurality of slave processing circuits according to the accumulation instruction to obtain an accumulation result, and adds an offset b to the accumulation result according to the offset instruction After obtaining the result, the final result is transmitted to the controller unit.

Meanwhile, the order of the addition operation and the multiplication operation may be changed.

The technical solution provided in this application implements the multiplication operation and offset operation of a neural network by one instruction, that is, a neural network operation instruction, and it is not necessary to store or extract the intermediate result calculated by the neural network. Since can be omitted, there is an advantage of improving the computational effect of the neural network by omitting the corresponding operation step.

As information technology is constantly evolving and improving day by day, the demands of people for data access and data processing are increasing, and the demands for processors that process and access some data are also becoming increasingly stringent. Taking a general-purpose processor as an example, a multi-core processor composed of a plurality of general-purpose processor cores (eg, CPU cores) has become a major trend due to its strong parallel computing power.

However, as artificial neural networks are constantly developing, machine learning chips with more and more configurations are gradually being produced. These machine learning chips need to access data according to commands or process data in shared storage while driving. When there is a large amount of data being accessed or shared and stored, the instruction of the machine learning chip gradually becomes complex, which affects the speed of reading the data being shared and stored by the instruction, thereby reducing the processing efficiency of neuron data.

Therefore, improving the speed at which a machine learning chip accesses data when accessing data is a technical problem that a person skilled in the art needs to solve urgently.

In order to solve the above problem, the present application provides the following technical solutions.

The data processing method according to the present application can be applied to the hardware circuit shown in FIG. 31. The circuit includes a machine learning device 11, a transfer circuit 12 and a shared memory 13, and the machine learning device 11 and the transfer circuit 12, a transfer circuit 12 and a shared memory 13 Are all connected through an interface, and the machine learning device 11, the transmission circuit 12 and the shared memory 13, and the interface may all be implemented by a hardware circuit method. For example, the machine learning device may be a device with an operation function composed of a plurality of machine learning units (Machine Learning Unit, abbreviated as MLU), and the transmission circuit may be a broadcast bus, and a shared memory May be a non-volatile and/or volatile memory, and includes, but is not limited to, random access memory (RAM), cache memory, and the like. This embodiment does not specifically limit the hardware format. Here, the transmission circuit 12 acquires input data necessary for the machine learning device 11 from the shared memory 13 according to the data operation signal sent from the machine learning device 11 and transfers the input data to the machine learning device 11. Used to return. The machine learning apparatus 11 performs a machine learning operation according to the input data to obtain output data, uses the output data as new input data, transmits the output data to the shared memory 13 through the transmission circuit 12, and stores the data.

In one embodiment, Fig. 32 provides a data processing method. In this embodiment, the transmission circuit determines the type of the data operation signal according to the first type flag bit and the second type flag bit of the data operation signal, and then obtains necessary data from the memory according to the operation corresponding to the determined type and then accesses it. It relates to a specific process of improving speed. As shown in Fig. 2, the method includes the following steps.

S2101: Receive a data operation signal transmitted from an internal or external device. The data operation signal includes an operation domain and an operation code, the operation code includes a first type flag bit, the operation domain includes a second type flag bit, and the first type flag bit is the data operation It indicates whether a signal is an I/O command, and the second type flag bit indicates whether the data operation signal is a broadcast or multicast command among the I/O commands.

In this embodiment, the transmission circuit receives a data operation signal transmitted from the internal or external device. The data operation signal has a first type flag bit and a second type flag bit of the data operation signal. Here, the internal or external device may be a machine learning device connected to a transmission circuit through an interface, and the machine learning device may be of any hardware type, for example, a device having an operation function consisting of a plurality of MLUs. . Here, the transmission circuit determines whether the value of the data operation signal is an I/O command according to the first type flag bit of the data operation signal of the data operation signal, and the second type flag bit determines whether the data operation signal is You can determine whether it is a specific type within the I/O command. For example, if the value of the first type flag bit of the data operation signal is an I/O command and the value of the second type flag bit is 1, the data operation signal is a broadcast or multicast command among I/O commands. .

S2102: Perform a corresponding operation on the operation scheduled data in the memory according to the data operation signal to obtain necessary input data.

As the transmission circuit receives the data operation signal transmitted from the internal or external device in step S2101, the transmission circuit performs an operation corresponding to the operation scheduled data in the memory according to the type flag bit of the data operation signal to be required. Input data, e.g. neuron data and weights are obtained. Here, neuron data and weights are data required for internal or external devices. For example, when the internal or external device is a machine learning device, the neuron data and weight are data that the machine learning device needs to input during a machine learning operation. The abnormal data may be data previously stored in the memory, or may be data that is output after the machine learning apparatus performs a machine learning operation. This embodiment is not limited to this.

In the data processing method according to the present embodiment, the transmission circuit performs an operation corresponding to the operation scheduled data in the memory according to a data operation signal transmitted from an internal or external device and having a first type flag bit and a second type flag bit. To get the required input data. In this embodiment, since the data operation signal has a first type flag bit and a second type flag bit, after receiving the data operation signal, the transmission circuit receives the first type flag bit and the second type of the data operation signal therein. A specific type of the data operation signal is determined according to a flag bit, and a corresponding operation is performed on the operation scheduled data in the memory. This simplifies data accessing logic and improves data access efficiency by first classifying data according to the type flag bit of the data operation signal and quickly mapping it to the corresponding operation, greatly improving the access speed for machine learning chips to access data. Made it.

The operation codes and operation domains, and the relationship between the operation codes and operation domains, and the type flag bits of the data operation signals, the operation schedule data information, and the data reception flag bits, respectively, will be described through several embodiments below.

In one embodiment, the operation domain further includes a data reception flag bit, and the data reception flag bit represents an apparatus or processing circuit that receives the input data. Alternatively, the number of data reception flag bits represents the number of devices or processing circuits capable of interacting with the memory. Alternatively, if the value of the first type flag bit is I/O, the data operation signal is determined as an I/O command, and if the value of the second type flag bit is 1, the data operation signal is set to the I/O. Confirm with the broadcast or multicast command in the /O command.

In this embodiment, the operation code of the data operation signal indicates the operation type of the data operation signal, includes a first type flag bit of the data operation signal, and the operation domain is the data required for the data operation signal Stores information and includes a second type flag bit. Exemplarily, if the value of the first type flag bit of the data operation signal in the operation code is I/O, it indicates that the data operation signal is an I/O command, and if the value of the second type flag bit in the operation domain is 1, Indicates that the data operation signal is a broadcast or multicast command among I/O commands. Note that when the second type flag bit described in the present embodiment is 1, the data operation signal is determined as a broadcast or multicast command among I/O commands. This is only one embodiment, and when the second type flag bit is 0 or other identifier according to the actual demand of the user, the data operation signal may be determined as a broadcast or multicast command among I/O commands. This embodiment is not limited to this. Here, the data reception flag bit represents a device or a processing circuit capable of receiving input data (eg, input neuron data and weight) among internal or external devices. Here, the device may be a machine learning device or an MLU, and the processing circuit may be an operation unit or a mast processing circuit or a slave processing circuit of the operation unit, and the present embodiment is not limited thereto. Here, the number of data reception flag bits indicates the number of devices capable of interacting with the memory or the number of processing circuits. Exemplarily, if the flag bits of three MLUs (machine learning units) are 1 in the data reception flag bit in the operation domain, it indicates that the three MLUs can receive data, and if one MLU flag bit is 0 , It indicates that the one MLU cannot receive data. Note that, marking the MLU capable of receiving data as 1 is only an example, and the user may indicate the MLU capable of receiving data as 0 or another identifier according to actual demand. This embodiment is not limited to this.

In this embodiment, the transmission circuit determines a specific type of the data operation signal according to the first type flag bit and the second type flag bit of the data signal, maps it to a corresponding operation, and performs an operation according to the data reception flag bit. Since the data access logic is simplified and data access efficiency is improved by determining the target device for transmitting the data after data transmission, the machine learning chip has greatly improved the access speed at the time of data access.

In another embodiment, the operation domain further includes information on operation schedule data, and the operation schedule data information includes a source address of the operation schedule data in the memory, an operation schedule data length, and data return after operation of the data. Include your address. As shown in Fig. 33, a data processing method is provided, and in this embodiment, a transmission circuit reads data from a memory according to the data information possessed by a data operation signal, and then stores the read data according to the data operation information. Or it relates to a specific process of returning to the processing circuit. The S2102 includes the following steps.

S2201: Start reading the memory from the source address to obtain input data satisfying the data length.

In this embodiment, since the operation scheduled data includes a source address in a memory, a data length to be operated, and a data return address after operating the data in the information of the operation scheduled data of the data operation signal, the transmission circuit Data starts to be read, and reads as long as it satisfies the data length scheduled for operation according to a preset rule. Here, the operation scheduled data length is set by the user according to an actual situation, and the present embodiment is not limited thereto. Here, when the transmission circuit acquires input data and data satisfying the data length, it reads data satisfying the data length from the memory according to a preset rule. Here, the preset rule is also a rule set by the user according to an actual situation. The present embodiment is not limited to this, for example, reading from the source address one by one until the read data length satisfies the corresponding data length.

S2202: Determine an apparatus or processing circuit for receiving input data according to the data reception flag bit.

In step S2201, the transmission circuit acquires input data satisfying the data length, and the transmission circuit receives an apparatus or processing circuit that returns data according to a data reception flag bit in the data signal. For example, when the device is a machine learning device, the transmission circuit determines that data is returned to one or more target machine learning units in the machine learning device according to the data reception flag bit.

S2203: Return the input data to a storage space corresponding to the data return address in the device or processing circuit according to the data return address.

In this step, by the device or processing circuit that returns the data determined in the above step, the transmission circuit returns the input data among the device or processing circuit according to the data return address among the information of the operation scheduled data of the data operation signal. Return to the storage space corresponding to the address. Here, the data return address in the information on the operation schedule data may be an address in a plurality of target machine learning units of the machine learning apparatus.

Illustratively, as shown in Table 3, based on the above embodiment, this embodiment can be exemplified as follows. When the value of the first type data flag bit in the operation code is I/O, it indicates that the data operation signal is an I/O command, and when the value of the second type data flag bit in the operation domain is 1, the data operation signal is I/O. Indicates that this is a broadcast or multicast command in the /O command. Therefore, when the value of the second type data flag bit is 0, it indicates that the data operation signal is not a broadcast or multicast command. The data information on the operation schedule in the operation domain includes a source address 0x110011, a target address 0x000100, and a data length 0x0100. The data length is a length set by the user himself, and the user may set the set length to one value and may set the set length to a plurality of values. This embodiment is not limited to this. When three MLU flags in the data reception flag bit in the operation domain are 1, it indicates that the three MLUs can receive data, and when one MLU flag is 0, the one MLU can receive data. Indicates none. Specifically, the transmission circuit reads data of a length of 0x0100 from the address 0x110011 in the shared memory according to the data operation signal, and then writes it to addresses 0x000100 of MLU3, MLU1, and MLU0 of the machine learning device.

Table 3

In the data processing method according to the present embodiment, the transmission circuit starts reading a memory from a source address according to a data operation signal to obtain input data satisfying the data length, and receives the input data according to the data reception flag bit, or After determining the processing circuit, the input data is returned to the device or the storage space corresponding to the data return address in the processing circuit according to the data return address. In this embodiment, when the transmission circuit acquires input neuron data and weights satisfying the data length, data is read according to a reading rule indicated by data operation information among the data operation signals, thus simplifying the data reading logic of the transmission circuit. Since the data access efficiency is increased, the machine learning chip has greatly improved the access speed during data access.

Alternatively, in the embodiment shown in FIG. 33, the device includes at least one machine learning unit, and each machine learning unit includes a mast processing circuit and a plurality of slave processing circuits. Here, the data signal operation performed by at least one machine learning unit (i.e., MLU) included in the machine learning device may share one data reception interface, and the machine learning unit may share a transmission interface or a shared data reception interface. Can be connected to the transmission circuit through. Note that both the transmission interface and the shared data reception interface may be implemented in a hardware circuit manner. This embodiment does not limit the types of the transmission interface and the shared data reception interface. Here, each machine learning unit includes a mast processing circuit and a plurality of slave processing circuits, and the mast processing circuit is used to distribute input data (neuron data and weights) to a plurality of slave processing circuits; The plurality of slave processing circuits are used to obtain a plurality of intermediate results by performing intermediate operations in parallel with input data transmitted from the mast processing circuit, and then to transmit the plurality of intermediate results to the mast processing circuit. In this case, the device can be allocated to each machine learning unit to process the neurons in it, and by correspondingly outputting the corresponding output data, it is possible to perform further and further neural network parallel computation to implement parallel processing of neural network computation. Treatment efficiency can be improved.

Based on the above embodiment, the operation domain further includes a jump sub-operation domain, and the jump sub-operation domain includes a jump width and a jump data length that operates after each jump. As shown in Fig. 34, a data processing method is provided, and this embodiment relates to a specific process in which a transmission circuit reads data in a memory according to a jump sub-operation domain in an operation domain. The S2201 includes the following steps.

S2301: Start reading the memory from the source address and acquire first jump data according to the jump data length after the current jump.

In this embodiment, the operation domain of the data operation signal includes a jump sub-operation domain, and the jump sub-operation domain reads operation scheduled data information according to the data operation signal according to the rules of the sub-operation domain. It is used to indicate the transmission circuit. Alternatively, the jump sub-operation domain includes a stride operation domain and/or a segment operation domain, the stride operation domain represents a width at which the data operation signal jumps each time, and the segment operation domain is the preset data It represents the size divided by the operation signal each time. Note that the length and name of the stride operation domain and the segment operation domain in the embodiment of the present application are only examples, and the embodiment of the present application is not limited thereto. Here, the jump sub-action domain includes a jump width and a jump data length that operates after each jump, and the jump data length may be a preset data length. Specifically, the transmission circuit starts reading the memory from the source address in the operation-planned data information and determines the data of the read jump data length as the first jump data after this jump, and the first jump data It indicates the data acquired after jumping as much as the preset length data when reading, and the preset length is set by the user according to the actual situation. This embodiment is not limited to this.

S2302: Acquire the last address of the jump data and jump from the last address to the target jump address according to the jump width.

According to the first jump data read in step S2301, the transmission circuit obtains the last address of the first jump data, and the first jump data according to the jump width (eg, stride width) in the jump sub-operation domain. Jumps to the target jump address by the length of the jump width from the last address of. It should be understood that the length between the last address of the first jump data and the target jump address is a jump width in the jump sub-operation domain.

S2303: From the target jump address until the length of jump data obtained after each jump satisfies the data length, second jump data is acquired according to the jump data length after jumping.

In this step, when reading data, the transmission circuit jumps data of a preset length from the target jump address determined in step S2302 and then determines the jumped data by the preset length as second jump data. When the length between the address of the second jump data and the source address of the jump start satisfies the data length of the data required for the machine learning device, it indicates that the data required for the machine learning device has been read, and the second jump data If the length between the address and the source address of the jump start does not satisfy the data length of data required by the machine learning device, the length between the address of the second jump data and the source address of the jump start is data required by the machine learning device. The data is read by jumping from the last address of the second jump data according to the jump sequence of steps S2301 to S2303 until the data length of is satisfied, that is, reading of data required for the machine learning device is completed.

Illustratively, as shown in Table 4, a process in which the transmission circuit reads data in this embodiment is as follows. If the operation domain further includes a jump sub-operation domain stride operation domain, the transmission circuit reads data in the shared memory from the source address 0x110011 of the data information, but data of a preset length (the preset length is the data information in the table below). The data length of the inside is smaller than 0x0100), and then the address of the stride length (0x0008) is jumped to read the data of the preset length again. The data is continuously read in this order, but when the total length of the read data becomes 0x0100 of the data length of the data information in Table 4, it indicates that the data reading is complete. If the operation domain further includes the segment operation domain of the jump sub operation domain, the transmission circuit starts reading data in the shared memory from the source address 0x110011 of the data information, but first reads the data of the segment length (0x0010) and then the stride length. It jumps the address of (0x0008) and reads the data of segment length (0x0010) again. The data is continuously read in this order, but when the total length of the read data reaches the data length 0x0100 of the data information in Table 3, it indicates that the data reading is complete. Note that, when there is only a segment operation domain and no stride operation domain in the jump sub operation domain, the transmission circuit reads data of segment length (0x0010) from the source address 0x110011 when reading data, When the total length becomes the data length 0x0100 of the data information in Table 4 below, it indicates that the data reading is complete.

Table 4

In the data processing method according to the present embodiment, the transmission circuit starts reading the shared memory from the source address, acquires first jump data according to the jump data length after the current jump, and jumps from the last address of the first jump data. After jumping to the target jump address according to the width, second jump data is obtained according to the jump data length after jumping from the target jump address until the length of the jump data obtained after each jump satisfies the data length. Accordingly, when the jump sub-operation domain is included in the operation domain, the transmission circuit reads data according to the jump rule of the sub-operation domain, thereby simplifying the data reading logic of the transmission circuit and increasing the access efficiency of the data so that the machine learning chip The access speed at the time of access is greatly improved.

When the transmission circuit operates according to the received data operation signal, the data operation signal it starts to receive is one coded instruction, and it must first be decoded and analyzed for the data operation signal. Provides data processing methods. As shown in Fig. 35, the step of receiving, by a transmission circuit in the data processing device, a data operation signal transmitted from a machine learning device in the data processing device, includes the following steps.

S2401: By analyzing the data operation signal, information on the type flag bit and operation schedule data of the data operation signal is obtained.

Note that, in general, the number of data operation signals in a data processing process is relatively large, and when a transmission circuit processes one data operation signal, the other signal must be stored. Specifically, the transmission circuit analyzes the data operation signal, and specifically, analyzes data information included in the data operation signal and a type flag bit of the data operation signal. Here, the data operation information may include information such as an operation scheduled data length, a target address, and an initial address. This embodiment is not limited to this.

S2402: Execute the analyzed data operation signal according to a command queue, and the command queue indicates an execution order of the data operation signal.

The data operation signals should be sequentially processed according to the order of execution, and based on the data operation information and type flag bits obtained by the transmission circuit analyzing the data operation signal in step S401, the transmission circuitry, according to the command queue, the Perform the analyzed data motion signal.

In the data processing method according to the present embodiment, the data operation signal is analyzed through a transmission circuit to obtain information on the type flag bit of the data operation signal and the operation scheduled data, and then the transmission circuit performs the data operation signal analyzed by the command queue. do. Accordingly, before performing the data operation signal, the data operation signal is analyzed first, and then sequentially performed, thereby greatly improving the speed at which the transmission circuit operates according to the data operation signal.

When the transmission circuit performs data operation signals according to the order in the queue, the embodiment of the present application provides another embodiment in consideration of the fact that the targets to be executed are data operation signals related to each other. As shown in Fig. 36, before the transmitting circuit performs the analyzed data operation signal according to the command queue, the method further includes the following steps.

S2501: A determination result is obtained by determining a dependence relationship between the analyzed data operation signals adjacent to each other. The dependency relationship indicates whether there is a correlation between the s-th data operation signal and the s-1 th data operation signal before the s-th data operation signal.

Here, the transmission circuit must determine a dependency relationship between the analyzed data operation signals adjacent to each other, and determine that there must be a relationship between two adjacent data operation signals processed according to the determination result. Here, the s-th data operation signal indicates only one of the data operation signals, but does not display any specific signal. The s-1th data operation signal represents a signal immediately before the sth data operation signal.

Alternatively, an implementation method in which the transmission circuit determines a dependency relationship between the analyzed data operation signals adjacent to each other is as follows. That is, a first storage address section for extracting the data necessary for the s-th data operation signal by the s-th data operation signal, and the s-1 th data operation by the s-1 th data operation signal 0th storage address intervals for extracting data necessary for a signal are each acquired, and if there is an overlapped region between the first storage address interval and the 0th storage address interval, the sth data operation signal and the s-1th It is confirmed that the second data operation signal has a dependency relationship. If there is no overlapping region between the first storage address section and the 0th storage address section, it is determined that the sth data operation signal and the s-1th data operation signal have no dependency relationship. Here, the transmission circuit determines the dependency relationship between the adjacent analyzed data operation signals according to the relationship between the first address storage period of the s-th data operation signal and the 0th storage address period of the s-1th data operation signal. And the method of judgment is as follows. That is, if there is no overlapping region between the first storage address section and the 0th storage address section, the sth data operation signal and the s-1th data operation signal have no dependency relationship, and the first storage address section and the If there is an overlapped region in the 0th storage address section, it indicates that there is a dependency relationship between the sth data operation signal and the s-1th data operation signal.

S2502: If the determination result has a dependency relationship between the s-th data operation signal and the s-1 th data operation signal, the s-th data operation signal is cached, and the s-1 th data operation signal After performing, the s-th data operation signal is extracted.

According to the dependence of the two adjacent data operation signals determined by the transmission circuit in the above step, the data operation signals are performed in sequence, and the determination result is between the s-th data operation signal and the s-1th data operation signal. If there is a dependency relationship, the transmission circuit first caches the s-th data operation signal, and after completion of the execution of the s-1 th data operation signal, extracts the s-th data operation signal again.

In the data processing method according to this embodiment, the transmission circuit first determines the relationship between two adjacent data operation signals to ensure consistency of the data operation signals. In this way, the order of performing corresponding operations according to the subsequent data operation signals is ensured by the orderly preparatory work at the front stage, thereby increasing data access efficiency and greatly improving the access speed when the machine learning chip accesses data.

In addition, considering that the data read by the transmission circuit according to the data operation signal is not in a format suitable for the machine learning device, the transmission circuit must perform certain processing on the read data and then transmit it back to the machine learning device, Alternatively, the operation domain further includes a function flag bit indicating a processing operation to be performed on the read data. Here, the operation domain of the data operation signal includes a function flag bit, and the function flag bit indicates that the transmission circuit performs a corresponding process on the read data of the function flag bit. The number of function flag bits included in the operation domain may be one or more. This embodiment is not limited to this. Exemplarily, the function flag bit is a decompression flag bit, and if the flag is 1, after reading the data, the transmission circuit decompresses the data and retransmits it to the MLU designated in the machine learning device. Alternatively, if the function flag bit is an encryption flag bit and the encrypted flag bit is 1, after reading the data, the transmission circuit decompresses the data and retransmits it to the MLU designated in the machine learning device. In this embodiment, the transmission circuit first performs a corresponding process on the read data according to the function flag bit in the operation domain of the data operation signal and transmits the data to the machine learning device, so that the machine learning device receives the data. Since it is immediately identified and calculated, data processing efficiency is improved, and the access speed of the machine learning chip when accessing data is greatly improved.

In one embodiment, the embodiment of the present application further provides a data processing device. The data processing apparatus includes a processor and a memory, a computer program is stored in the memory, and the processor executes the computer program,

Receiving a data operation signal transmitted from an internal or external device; And

The step of obtaining desired input data by performing a corresponding operation on the operation scheduled data in the memory according to the data operation signal; proceeds. Here, the data operation signal includes a type flag bit indicating the data operation signal broadcast or multicast command.

The implementation principles and technical effects of the data processing apparatus according to the present embodiment are similar to those of the above data processing method, and thus will not be described herein again.

In one embodiment, Fig. 37 provides a data processing method. The present embodiment relates to a specific process in which the transmission circuit determines the type of the data operation signal according to the type flag bit of the data operation signal and acquires necessary data from the memory according to the operation corresponding to the determined type, thereby improving the access speed. . As shown in Fig. 37, the method includes the following steps.

S3101: Receive a data operation signal transmitted from an internal or external device. Here, the data operation signal includes an operation code, and the operation code includes the type flag bit indicating the data operation signal broadcast or multicast command.

In this embodiment, the transmission circuit receives a data operation signal transmitted from an internal or external device, and the operation code of the data operation signal is used to indicate the operation type of the data operation signal, and the type flag of the data operation signal Includes bits. Here, the internal or external device may be a machine learning device connected to a transmission circuit through an interface, and the machine learning device may be of any type of hardware, such as a device having an operation function consisting of a plurality of MLUs. Here, the transmission circuit may determine the type of the data operation signal according to the type flag bit of the data operation signal included in the data operation signal. For example, if the value of the type flag bit of the data operation signal is 1, the data operation signal is a broadcast or multicast command.

S3102: Perform a corresponding operation on the operation scheduled data in the memory according to the data operation signal to obtain necessary input data.

When the transmission circuit receives the data operation signal transmitted from the internal or external device in step S3101, the transmission circuit performs a corresponding operation on the operation scheduled data in the memory according to the type flag bit of the data operation signal, Input data, e.g. neuron data and weights are obtained. Here, the neuron data and weight are data required for an internal or external device. For example, when the internal or external device is a machine learning device, the neuron data and weight are data that the machine learning device needs to input during a machine learning operation. The data may be data previously stored in a memory, or may be data output after a machine learning operation is performed by a machine learning device. This embodiment is not limited to this.

In the data processing method according to the present embodiment, the transmission circuit performs a corresponding operation on the data to be operated in the memory according to the data operation signal transmitted from an internal or external device and has a type flag bit of the data operation signal to provide necessary input. Get the data In this embodiment, since the data operation signal has a type flag bit of the data operation signal, the transmission circuit determines a specific type of the data operation signal according to the type flag bit of the data operation signal therein after receiving the data operation signal. Afterwards, a corresponding operation is performed on the operation scheduled data in the memory. This simplifies data access logic and improves data access efficiency by classifying data according to the type flag bit of the data operation signal and quickly mapping it to the corresponding operation, thereby increasing the access speed of the machine learning chip when accessing data. Improved.

The operation codes and operation domains, and the relationship between the operation codes and operation domains, and the type flag bits of the data operation signals, the operation schedule data information, and the data reception flag bits, respectively, will be described through several embodiments below.

In one embodiment, the data operation signal further includes an operation domain, and the operation domain includes a data reception flag bit indicating a device or processing circuit that has received the input data. Alternatively, the number of data reception flag bits represents the number of devices or processing circuits capable of interacting with the memory. Alternatively, if the value of the type flag bit is CAST, the data operation signal is determined as a broadcast or multicast command.

In this embodiment, the operation code of the data operation signal is used to indicate the operation type of the data operation signal and includes a type flag bit of the data operation signal. For example, if the type flag bit of the data operation signal in the operation code is CAST, it indicates that the data operation signal is a broadcast or multicast command. The operation domain may store data information required for a process in which the data operation signal is performed, and may include a data reception flag bit. The data reception flag bit represents a device or a processing circuit capable of receiving input data among internal or external devices. Here, the device may be a machine learning device or an MLU, and the processing circuit may be an operation unit or a mast processing circuit or a slave processing circuit of the operation unit, and the present embodiment is not limited thereto. Here, the number of data reception flag bits indicates the number of devices capable of interacting with the memory or the number of processing circuits. Exemplarily, if the flag bits of three MLUs (machine learning units) are 1 in the data reception flag bit in the operation domain, it indicates that the three MLUs can receive data, and if one MLU flag bit is 0 , Indicates that the one MLU cannot receive data. It should be noted that the method of indicating the MLU capable of receiving data as 1 is only an example, and the user may indicate the MLU capable of receiving data as 0 or another identifier according to actual demand. This embodiment is not limited to this.

In this embodiment, the transmission circuit determines a specific type of the data operation signal according to the type flag bit of the data signal, maps it to a corresponding operation, and transmits data after performing the operation according to the data reception flag bit. By determining the target device, the data access logic is simplified and the data access efficiency is increased, and the access speed when the machine learning chip accesses the data is greatly improved.

Illustratively, as shown in Table 5, the present embodiment based on the above embodiment can be given the following examples. That is, if the type flag bit of the data operation signal in the operation code is CAST, it indicates that the data operation signal is a broadcast or multicast command, and the source address 0x110011, the target address 0x000100, and the data length 0x0100 are included in the operation scheduled data information in the operation domain. Included. Here, the data length is a length set by the user himself, and the user may set the set length as a single value or a plurality of values. The present embodiment is not limited to the specific value and number of the set length. If the flags of the three MLUs are 1 in the data reception flag bit in the operation domain, it indicates that the three MLUs can receive data, and if one MLU flag is 0, the one MLU cannot receive data. Represents. Specifically, the transmission circuit reads data of 0x0100 path from address 0x110011 in the shared memory according to the data operation signal and writes data to addresses 0x000100 of MLU3, MLU1 and MLU0 in the machine learning device, respectively.

Table 5

In the data processing method according to the present embodiment, the transmission circuit starts reading a memory from a source address according to a data operation signal to obtain input data satisfying the data length, and receives the input data according to the data reception flag bit, or After determining the processing circuit, the input data is returned to the device or the storage space corresponding to the data return address in the processing circuit according to the data return address. In this embodiment, when the transmission circuit acquires input data that satisfies the data length, data is read according to a read rule indicated by data operation information among the data operation signals, thereby simplifying data reading logic of the transmission circuit and accessing data. Increased efficiency, and the machine learning chip significantly improved the access speed when accessing data.

In one embodiment, as shown in FIG. 31, the data processing apparatus according to the embodiment of the present application may be a part or all of FIG. 31, and may be implemented by software, hardware, or a combination thereof. The data processing device 10 is used to process machine learning data, and the data processing device 10 includes a machine learning device 11, a transmission circuit 12 and a shared memory 13, and the machine learning The device 11 is connected to the transmission circuit 12, the transmission circuit 12 is connected to the shared memory 13, the transmission circuit 12 is the data sent from the machine learning device 11 Input data required for the machine learning device 11 is obtained from the shared memory 13 according to an operation signal, and the input data is returned to the machine learning device 11. The data operation signal has a type flag bit of the data operation signal and information on operation schedule data. Alternatively, the machine learning device 11 is used to obtain output neuron data by performing a machine learning operation according to the input data. Alternatively, the machine learning device 11 is also used to store the output neuron data as new input neuron data and transmit it to the shared memory 13 through the transmission circuit 12.

Note that the machine learning apparatus, the transmission circuit, and the shared memory may all be implemented in a hardware circuit manner. Exemplarily, the machine learning device is a device having an arithmetic function composed of a plurality of machine learning units (Machine Learning Unit, abbreviated as MLU), and the transmission circuit may be a broadcast bus, and the shared memory is non- -It may be volatile and/or volatile memory, and includes, but is not limited to, random access memory (RAM), cache memory, and the like. Here, data is transmitted through an interface between the machine learning device, a transmission circuit, and a shared memory. For example, the machine learning apparatus may transmit a data operation signal through the interface, and may transmit or receive data through the interface. Accordingly, the interface may be a receiving interface as well as a transmitting interface. That is, when the interface is a transmission interface, the machine learning device may transmit a data operation signal or data to the transmission circuit, and when the interface is a reception interface, the machine learning device transmits the data operation signal or data transmitted from the transmission circuit. Can receive. Here, the interface may be various interfaces, and all of the various interfaces may be implemented through a hardware circuit. The present embodiment is not limited to specific hardware types of the various interfaces. The interface may be any one capable of implementing data signal interaction between the machine learning device, the transmission circuit and the shared memory. Here, the input data is data to be input when the machine learning apparatus performs a machine learning operation, for example, input neuron data and weights. The data may be data previously stored in the shared memory, or may be data output after the machine learning apparatus performs a machine learning operation. Alternatively, the machine learning apparatus may acquire the data by directly connecting to the shared memory through a plurality of data I/O interfaces or I/O pins. Alternatively, the machine learning apparatus may also obtain the data by connecting to the transmission circuit through a plurality of data I/O interfaces or I/O pins and then connecting to the shared memory through the transmission circuit again.

Here, the data operation signal may indicate a read operation for data in the shared memory by the transmission circuit and may indicate a write operation for data in the shared memory. When the data operation signal sent by the machine learning device is a read operation, the transmission circuit finds and reads the input data corresponding to the corresponding address in the shared memory, then returns the data to the machine learning device that sent the data operation signal, and When the data operation signal sent from the learning device is a write operation, the transmission circuit may write the write data output from the machine learning device to the shared memory. Here, the data operation signal includes a type flag bit of the data operation signal and information on operation scheduled data, and the type flag bit of the data operation signal indicates a type of the data operation signal. For example, if the type flag bit of the data operation signal is CAST, it indicates that the type of the data operation signal is a broadcast or multicast command. The information on the operation schedule data represents data required when a transmission circuit performs a corresponding operation according to the data operation signal, and in the present embodiment, specific data of the type flag bit and operation schedule information of the data operation signal The information is not limited and can be determined according to the actual situation.

Note that the data processing apparatus according to the present application is applied to a machine learning operation, and the machine learning operation includes a neural network operation, a k-means operation, a support vector machine operation, and the like. Taking the neural network operation as an example, the operation in the neural network performed by the machine learning device may be a single-layer operation in the neural network, and the implementation process of the multi-layer neural network is as follows. In the forward operation, after the execution of the artificial neural network of the previous layer is finished, the operation instruction of the next layer uses the output neuron data calculated by the operation unit as the input neuron data of the next layer (or partially operates on the output neuron data and returns to the next layer. At the same time, the weight is replaced by the weight of the next layer, and after the reverse operation of the artificial neural network of the previous layer is completed, the operation instruction of the next layer is the input neuron slope calculated by the computation unit (similarly, input (Can be used as the output neuron data of the next layer) as the output neuron gradient of the next layer (the same can be used as the output neuron data) (or some operation is performed on the input neuron gradient, and then the output neuron of the next layer is tilted again. At the same time, the weight is replaced with the weight of the next layer. Alternatively, the neural network according to the embodiment of the present application may be a pulsed neural network as well as an artificial neural network, and the present embodiment is not limited thereto. The machine learning apparatus according to the present embodiment performs a machine learning operation according to the input data. For example, in the machine learning operation, the machine learning apparatus may calculate neuron data output from the neural network of each layer for a multilayer neural network. A set of operations included in a series of machine learning operations such as multiplication operation, summation operation, and function operation may be performed on a plurality of input data corresponding to the input terminals of the neural network of each layer. After the machine learning apparatus obtains output neuron data of the current layer through a machine learning operation, the machine learning operation is performed again by using the output neuron data as input neuron data of the neural network of the next layer. Prior to that, the output neuron data of the current layer may be first written to and stored in a shared memory through a transmission circuit so that the machine learning apparatus frequently reads the data and performs a machine learning operation.

Specifically, in an actual application, the transmission circuit obtains input data necessary for the machine learning device from the shared memory according to the data operation signal sent from the machine learning device, and then returns the input data to the machine learning device through the receiving interface. Thereafter, the machine learning apparatus performs a machine learning operation based on the input data to obtain output data, uses the output data as new input data, and transmits and stores the output data to a shared memory through a transmission circuit. In this embodiment, since the data operation signal includes the type flag bit of the data operation signal and the operation scheduled data information, the transmission circuit receives the data operation signal and then the data operation signal according to the type flag bit of the data operation signal therein. After determining the type of, the operation is performed by combining it with operation schedule data information included in the data operation signal. This simplifies data access logic and improves data access efficiency by first classifying data according to the type flag bit of the data operation signal and quickly mapping it to the corresponding operation, so that the machine learning chip speeds up accessing data. Greatly improved.

In one embodiment, as shown in FIG. 38, in the data processing apparatus according to the embodiment of the present application, the machine learning apparatus 11 includes at least one machine learning unit 14, and the data operation The signal further includes a data reception flag bit indicating a target machine learning unit receiving the input data.

Here, the data signal operation performed by at least one machine learning unit (i.e., MLU) included in the machine learning device may share one data reception interface, and the machine learning unit may share a transmission interface or a shared data reception interface. Can be connected to the transmission circuit through. Note that both the transmission interface and the shared data reception interface may be implemented in a hardware circuit manner. This embodiment does not limit the types of the transmission interface and the shared data reception interface. Here, the data operation signal includes a data reception flag bit indicating a target machine learning unit capable of receiving input data. In the notation method of the data reception flag bit, for example, a target machine learning unit capable of receiving input data is indicated by 1, and a target machine learning unit that cannot receive input data is indicated by 0. In this case, the target machine learning unit that can receive data is marked as 1 as 1, but the target machine learning unit that can receive data in an actual application is marked as 0, and the target machine learning unit that cannot receive data is marked as 1 It can also be expressed as In this embodiment, a specific format of the data reception flag bit is not limited.

In the present embodiment, a target machine learning unit capable of receiving input data may be determined by the machine learning apparatus according to the marking status of the data reception flag bit included in the data operation signal. Accordingly, when each machine learning unit of the machine learning device receives data, it is determined by the data reception flag bit in the data operation signal, thereby simplifying the access logic in the memory access process of data and increasing the access efficiency of the data. The learning chip has greatly improved the access speed when accessing data.

The relationship between the type flag bit of the data operation signal, the operation scheduled data information, and the data reception flag bit according to the present embodiment will be introduced through several embodiments below.

In an embodiment, when CAST is included in the value of the type flag bit of the data operation signal, it indicates that the data operation signal is a broadcast or multicast command. Alternatively, the operation schedule data information includes a source address of the operation schedule data in the shared memory, an operation schedule data length, and a data return address after data operation is performed.

In this embodiment, the type flag bit of the data operation signal indicates the operation type of the data operation signal. For example, as shown in Table 6, if the type flag bit of the data operation signal is CAST, it indicates that the data operation signal is a broadcast or multicast command, and the source address 0x110011, the target address 0x000100, and data It includes length 0x0100. The data length is a length set by the user himself, and the user may set the set length to one value or to a plurality of values. The present embodiment is not limited to the specific value and number of the set length. If three MLU flags among data reception flag bits are 1, it indicates that the three MLUs can receive data, and if one MLU flag is 0, it indicates that one MLU cannot receive data. Specifically, the transmission circuit reads data of a length of 0x0100 from the address 0x110011 in the shared memory according to the data operation signal, and then writes data to the addresses 0x000100 of MLU3, MLU1 and MLU0 in the machine learning device, respectively.

Table 6

In another embodiment, the type flag bit of the data operation signal may include a first type flag bit and a second type flag bit. Alternatively, when the value of the first type flag bit includes I/O, it indicates that the data operation signal is an I/O command, and the second type flag bit indicates that the data operation signal is in the I/O command. Indicates that this is a broadcast or multicast command.

In this embodiment, the data operation signal includes two data type data flag bits. Here, the first type data flag bit indicates a type of the data operation signal, and the second type data flag bit is included in operation information of the data operation signal to indicate a specific subtype of the data operation signal. Here, the data reception flag bit represents a target machine learning unit capable of receiving input data in the same manner as in the present embodiment. For example, as shown in Table 7, when the value of the first type data flag bit is I/O, it indicates that the data operation signal is an I/O command, and when the value of the second type data flag bit is 1 , Indicates that the data operation signal is a broadcast or multicast command among I/O commands. Therefore, when the value of the second type data flag bit is 0, it indicates that the data operation signal is not a broadcast or multicast command. The operation schedule data information includes a source address 0x110011, a target address 0x000100, and a data length 0x0100. The data length is a length set by the user himself, and the user may set the set length to one value and may set the set length to a plurality of values. This embodiment is not limited to this. When three MLU flags are 1 in the data reception flag bit, it indicates that the three MLUs can receive data, and when one MLU flag is 0, it indicates that one MLU cannot receive data. Specifically, the transmission circuit reads data of a length of 0x0100 from the address 0x110011 in the shared memory according to the data operation signal, and then writes data to the addresses 0x000100 of MLU3, MLU1, and MLU0 in the machine learning device, respectively.

Table 7

In another embodiment, on the basis of Table 1 or Table 2, the data operation signal may further include jump information. The jump information includes a jump width and a data length that operates after each jump. Alternatively, the jump information includes stride jump information and/or segment jump information.

In this embodiment, the jump information included in the data operation signal instructs the transmission circuit to read according to the rule of the jump information when reading operation-planned data information according to the data operation signal. The specific reading method is as follows. After the transmission circuit starts reading data in the shared memory from the source address in the operation scheduled data information and jumps this time, first, the data of the read jump data length is determined as the first jump data, and then the transmission circuit determines the first jump data. The data of the jump width length is jumped from the last address of the first jump data to the target jump address according to the jump width in jump information. Here, it should be understood that the length between the last address of the first jump data and the target jump address is the jump width in the jump information. Subsequently, the transmission circuit again jumps data of a preset length from the target jump address to determine the data jumped by the preset length as second jump data, and the address of the second jump data and the source of the jump start When the length between addresses satisfies the data length of data required for the machine learning device, it indicates that reading of the data required for the machine learning device is completed, and the length between the address of the second jump data and the source address of the jump start is If the data length of the data required for the machine learning device is not satisfied, the jump starts from the last address of the second jump data according to the jump order, and the length between the address of the second jump data and the source address of the jump start is The data is read until the data length of the data required for the machine learning device is satisfied, that is, the reading of the data required for the machine learning device is completed.

In general, the data processing apparatus according to the exemplary embodiment of the present application must analyze the data operation signal before performing a read/write operation on the data operation signal. Alternatively, the transmission circuit includes an instruction storage unit for storing the data operation signal; An instruction processing unit configured to analyze the data operation signal to obtain information on a type flag bit of the data operation signal and operation schedule data; It includes a storage queue unit that stores the command queue. Here, the command queue includes a plurality of the data operation signals to be executed according to the preceding and subsequent order of the command queue. Here, in a general data processing process, the number of data operation signals is relatively large, and when processing one of the data operation signals, another data operation signal must be stored in the instruction storage unit. The command processing unit analyzes data information included in the data operation signal in the process of analyzing the data operation signal. Meanwhile, since the process of acquiring, decoding, and sending the value of the data operation signal is a series of procedural operations, and all the data operation signals have to perform the above process sequentially in order, the command queue is stored through the storage queue unit.

In addition, since the command processing unit can process the next data operation signal in the queue only after processing one data operation signal, there must be a relationship between the data operation signal for which the current processing is completed and the next data operation signal. Alternatively, the transmission circuit further comprises a dependency relationship processing unit for determining whether there is a correlation between the s-th data operation signal and the s-1 th data operation signal before the s-th data operation signal. do. If there is a correlation between the s-th data operation signal and the s-1 th data operation signal, the s-th data operation signal is cached in the command storage unit, and the s-th data operation signal After performing, the s-th data operation signal is extracted from the command storage unit and transmitted to the command processing unit. Here, the step of determining whether there is a correlation between the s-th data operation signal and the s-1 th data operation signal before the s-th data operation signal includes the following process. That is, a first storage address section of data required for the s-th data operation signal is extracted by the s-th data operation signal, and the s-1 th data operation is performed by the s-1 th data operation signal. The 0th storage address section of the data required for the signal is extracted. If there is an overlapping region between the first storage address section and the 0th storage address section, it is determined that there is a correlation between the s-th data operation signal and the s-1 th data operation signal, and the first If there is no overlapped region between the storage address section and the 0-th storage address section, it is determined that there is no correlation between the s-th data operation signal and the s-1 th data operation signal.

In this embodiment, the unused data processing signals are sequentially stored before the data manipulation device operates according to the data processing signals, and when used, the unused data processing signals are sequentially analyzed and decoded. In the process of analyzing and decoding, the relationship between two adjacent data operation signals is determined to ensure consistency of data operation signals. Accordingly, the order of performing the corresponding operation according to the subsequent data operation signal is ensured by the orderly preparatory work at the front stage, thereby improving data access efficiency, thereby greatly improving the access speed when accessing data by the machine learning chip.

The embodiment of the present application further provides a data processing method. The above method can be applied to the hardware circuit shown in FIG. 31. The circuit includes a machine learning device 11, a transfer circuit 12 and a shared memory 13, and the machine learning device 11 and the transfer circuit 12, a transfer circuit 12 and a shared memory 13 All are connected through an interface, and the interface may be implemented by a hardware circuit method. This embodiment is not limited to specific hardware types of the various interfaces. Here, the transmission circuit 12 acquires input data necessary for the machine learning device 11 from the shared memory 13 according to the data operation signal sent from the machine learning device 11 and transfers the input data to the machine learning device 11. Used to return. The machine learning device 11 performs a machine learning operation according to the input data to obtain output neuron data, uses the output neuron data as new input neuron data, transmits it to the shared memory 13 through the transmission circuit 12, and stores it. do.

Hereinafter, the present application will be described in more detail by combining the accompanying drawings and embodiments so that the purpose, technical solution, and advantages of the present application may become more clear. The specific embodiments described herein are only for describing the present application, and do not limit the present application. The data processing method according to the exemplary embodiment of the present application is to solve a technical problem in which a machine learning chip improves an access speed when accessing data when there is a large amount of data being accessed or shared and stored. It will be described in detail how to solve the above technical problem through the technical solution of the present application and the technical solution of the present application by combining the following embodiments and the accompanying drawings. The following embodiments may be combined, and the same or similar concepts or processes are not described redundantly in some embodiments. It should be noted that in the data processing method according to the present application, the performing entity is a transmission circuit, and the performing entity may be a data processing device. The device may implement part or all of the data analysis terminal using software, hardware, or a combination thereof.

In one embodiment, Fig. 38 shows a data processing method. In this embodiment, the transmission circuit determines the type of the data operation signal according to the type flag bit of the data operation signal, maps it to a corresponding operation, and then acquires data necessary for the machine learning device from the shared memory according to the operation to increase the access speed. It is about the specific process of improving. As shown in Fig. 38, the method includes the following steps.

S4101: The transmission circuit in the data processing device receives the data operation signal transmitted by the machine learning device in the data processing device. Here, the data operation signal includes a type flag bit of the data operation signal and information on operation schedule data.

Here, the machine learning device may be a device having an operation function composed of a plurality of MLUs, the transmission circuit may be a broadcast bus, the shared memory may be a non-volatile and/or volatile memory, and a random access memory (RAM ), high-speed cache memory, and the like, but are not limited thereto. In this embodiment, the transmission circuit in the data processing device receives the data operation signal transmitted by the machine learning device in the data processing device. Here, the data operation signal includes a type flag bit of the data operation signal and information on operation schedule data. Here, transmission of a data operation signal between the transmission circuit and the machine learning device may be performed through an interface. The transmission circuit may determine the type of the data operation signal and data information required for operation according to the type flag bit of the data operation signal included in the data operation signal and operation schedule data information.

S4102: The transmission circuit determines an operation to be performed on data in the shared memory according to the type flag bit of the data operation signal, and performs the operation on the operation schedule data based on the operation schedule data information Input data required for a learning device is obtained, and the input data is returned to the machine learning device.

In response to the data operation signal transmitted from the machine learning apparatus received by the transmission circuit in step S4101, the transmission circuit determines an operation to be performed on data in the shared memory by the type flag bit of the data operation signal, and the data operation It is determined which data in the shared memory to perform the operation (this data is the operation-planned data) based on the information of the operation-planned data among the signals. Then, input data required by the machine learning device is obtained and the input data is returned to the machine learning device. Here, the input data is input data required by the machine learning apparatus for machine learning operation. The data may be data previously stored in the shared memory, or may be data output after the machine learning apparatus performs a machine learning operation.

S4103: The machine learning apparatus performs a machine learning operation based on the input data to obtain output data, uses the output data as new input data, and transmits and stores the output data to the shared memory through the transmission circuit.

In this step, the machine learning apparatus obtains output data by performing a machine learning operation according to the input data transmitted by the transmission circuit in step S4102. Then, the output data is used as new input data and is transmitted to the shared memory through a transmission circuit and stored. Here, the operation performed by the machine learning apparatus is a neural network operation as an example. The neural network operation may be a single layer operation in a neural network, and an implementation process for a multilayer neural network is as follows. In the forward operation, after the execution of the artificial neural network of the previous layer is finished, the operation instruction of the next layer calculates the output neuron data calculated by the operation unit as the input neuron data of the next layer (or performs some operations on the output neuron data, At the same time, it is used as the input neuron data of the layer), and the weight is also replaced with the weight of the next layer, and after the reverse operation of the artificial neural network of the previous layer in the reverse operation is completed, the operation instruction of the next layer is the input neuron gradient calculated by the operation unit ( Likewise, the input neuron data can be used as the output neuron gradient of the next layer (the same can be used as the output neuron data) (or some operation is performed on the input neuron gradient and then the next layer And at the same time replace the weights with the weights of the next layer. Alternatively, the neural network according to the embodiment of the present application may be a pulsed neural network as well as an artificial neural network, and the present embodiment is not limited thereto. The machine learning apparatus according to the present embodiment may perform a machine learning operation based on input data. For example, for a multilayer neural network of machine learning operation, the machine learning device can calculate neuron data output by each layer of neural network, and multiplication operation, summation operation, and function for a plurality of input data corresponding to each layer neural network input terminal. A set of operations included in a series of machine learning operations, such as operations, can be performed. After the machine learning apparatus obtains output neuron data of the current layer through a machine learning operation, the machine learning operation may be performed again by using the output neuron data as input neuron data of the neural network of the next layer. Prior to that, the output neuron data of the current layer may be first written to and stored in a shared memory through a transmission circuit so that the machine learning apparatus frequently reads the data and performs machine learning operation.

In the data processing method according to the present embodiment, the transmission circuit performs machine learning from the shared memory according to the data operation signal including the type flag bit of the data operation signal sent by the machine learning device through the transmission interface and the operation scheduled data information. The input data required for the device is acquired, and the input data is returned to the machine learning device through the receiving interface. Then, the machine learning apparatus performs a machine learning operation according to the input data to obtain output data, uses the output data as new input data, and transmits the output data to a shared memory through a transmission circuit and stores it. In this embodiment, since the data operation signal includes the type flag bit and the operation scheduled data information, the transmission circuit determines the type of the data operation signal according to the type flag bit of the data operation signal therein after receiving the data operation signal. Thereafter, operation schedule data information included in the data operation signal is combined to perform a corresponding operation. Accordingly, it is classified according to the type flag bit of the data operation signal, and is quickly mapped to the corresponding operation, thereby simplifying the logic for accessing data and increasing the data access efficiency, thereby increasing the access speed when the machine learning chip accesses data. Improved.

In one embodiment, the machine learning device includes at least one machine learning unit, the data operation signal further includes a data reception flag bit, and the process of returning the input data to the machine learning device includes the transmission And a circuit determining a target machine learning unit for receiving the input data according to the value of the data reception flag bit and transmitting the input data to the target machine learning unit.

In this embodiment, a data signal operation performed by at least one machine learning unit (ie, MLU) included in the machine learning device may share one data reception interface. The MLU transmits signals or data with a transmission circuit through a transmission interface or a shared data reception interface. Note that both the transmission interface and the shared data reception interface may be implemented in a hardware circuit manner, and in this embodiment, the types of the transmission interface and the shared data reception interface are not limited. Here, the data operation signal includes a data reception flag bit, and the data reception flag bit indicates a target machine learning unit capable of receiving input data. In the notation method of the data reception flag bit, for example, a target machine learning unit capable of receiving input data is indicated by 1. It can be understood that the target machine learning unit capable of receiving input data is indicated by 1 as one method, and the target machine learning unit capable of receiving data in an actual application may be indicated by 0. In this embodiment, a specific format of the data reception flag bit is not limited. Specifically, the transmission circuit determines a target MLU for receiving input data according to the value of the data reception flag bit in the data operation signal, and transmits the input data to the target MLU. In the present embodiment, the transmission circuit may determine a target machine learning unit capable of receiving input data from the machine learning apparatus according to the marking situation of the data reception flag bit included in the data operation signal. Accordingly, when each machine learning unit of the machine learning device receives data, it is determined by the data reception flag bit in the data operation signal, thereby simplifying the memory access logic in the memory access process of data, and data access efficiency. To increase, greatly improve the access speed when the machine learning chip accesses data.

Alternatively, when the value of the type flag bit of the data operation signal is CAST, the transmission circuit determines the data operation signal as a broadcast or multicast command. In one optional method, the type flag bit of the data operation signal of the data operation signal indicates the operation type of the data operation signal, and when the type flag bit of the data operation signal is CAST, the data operation signal is broadcast. Or, it indicates that it is a multicast command. Here, it should be understood that indicating the broadcast or multicast command with CAST belongs to only one embodiment, and the user can redefine the data type flag bit according to the actual situation. This embodiment is not limited to this.

Alternatively, the type flag bit of the data operation signal may include a first type flag bit and a second type flag bit, the first type flag bit indicates whether the data operation signal is an I/O command, The second type flag bit indicates whether the data operation signal is a broadcast or multicast command among the I/O commands. Therefore, when the value of the first type flag bit is I/O, the transmission circuit determines the data operation signal as an I/O command, and when the value of the second type flag bit is 1, the transmission circuit The data operation signal is determined by a broadcast or multicast command in the I/O command.

In the selectable manner, the data operation signal comprises data flag bits of two data types, wherein a first type data flag bit indicates a type of the data operation signal, and the second type data flag bit is It is set in the operation information of the data operation signal to indicate a specific subtype of the data operation signal. Specifically, when the value of the first type flag bit in the data operation signal is I/O, the transmission circuit determines the data operation signal as an input/output command, and the value of the second type flag bit in the data operation signal When this is 1, the transmission circuit determines the data operation signal as a broadcast or multicast command in the input/output name.

In one embodiment, Fig. 39 provides a data processing method. The present embodiment relates to a specific process in which a transmission circuit reads data from a shared memory based on data information included in a data operation signal and returns the read data to a target machine learning unit according to the data operation information. As shown in FIG. 39, when the operation schedule data information includes a source address in the shared memory, an operation schedule data length, and a data return address after data operation, S4103 performs the next step. Include.

S4201: The transmission circuit reads the shared memory from the source address to obtain the input data satisfying the data length.

In this embodiment, since the operation scheduled data includes a source address in the memory, a data length to be operated, and a data return address after the operation of the data in the information of the operation scheduled data of the data operation signal, the transmission circuit is the source address in the shared memory. Data is read from and a data length that satisfies the operation waiting according to a preset rule is read. Here, the length of the data to be operated is set by the user according to the actual situation. This embodiment is not limited to this. The process of acquiring input neuron data and data satisfying the data length by the transmission circuit is to read data satisfying the data length from the shared memory according to a preset rule, and the preset rule is The rules are determined according to the actual situation, and the present embodiment is not limited thereto. For example, it may be a method of reading one by one from the source address, but reading until the length of the read data satisfies the length of the data.

S4202: The transmission circuit returns the input data to the target machine learning unit according to the data return address and the data reception flag bit.

In step S4201, the transmission circuit acquires input data that satisfies the data length and returns the data to a data return address in the information of operation scheduled data. Here, the data return address in the information on the operation schedule data may be an address in a plurality of target machine learning units of the machine learning apparatus. Here, the transmission circuit determines that data is returned to the target machine learning unit in the learning device according to the data reception flag bit included in the data operation signal.

In the data processing method according to the present embodiment, the transmission circuit reads the shared memory from the source address to obtain input data satisfying the data length, and sends the input data according to the data return address and the data reception flag bit to the target machine learning unit. Returns to When the transmission circuit acquires input data that satisfies the data length, data is read according to the reading rule indicated by the data operation information in the data operation signal, thereby simplifying the data reading logic of the transmission circuit and improving the access efficiency of data. In addition, the machine learning chip has greatly improved the access speed when accessing data.

In one embodiment, Fig. 40 provides a data processing method. According to the arbitrary embodiment, the motion information in the embodiment may include jump information, and the jump information includes a jump width and a jump data length that operates after each jump. The above embodiment relates to a specific process in which a transmission circuit reads data in a shared memory based on jump information in operation information. As shown in FIG. 40, S4201 includes the following steps.

S4301: The transmission circuit reads the shared memory from the source address, and acquires first jump data based on the jump data length after the current jump.

In this embodiment, the operation information of the data operation signal includes jump information, and the jump information comprises the transmission circuit to read according to the rule of the jump information when reading the operation scheduled data information according to the data operation signal. Instruct. Here, the jump information includes a jump width and a jump data length that operates after each jump, and the jump data length may be a preset data length. Alternatively, the jump information includes stride jump information and/or segment jump information, and the stride jump information indicates a jump width of the data operation signal, and the segment jump information is It indicates the size of each division.

Specifically, the transmission circuit reads the shared memory from the source address in the operation-planned data information, and after this jump, determines the data of the read jump data length as the first jump data, and the first jump data When reading data, the acquired data is displayed after jumping the preset length data. Here, the preset length is set by the user according to an actual situation, and the present embodiment is not limited thereto.

S4302: The transmission circuit acquires the last address of the first jump data, and jumps from the last address to a target jump address according to the jump width.

According to the first jump data read in step S4301, the transmission circuit acquires the last address of the first jump data, and the end of the first jump data according to the jump width (eg, stride width) in jump information. The length of the jump width is jumped from the address to the target jump address. It should be understood that the length between the last address of the first jump data and the target jump address is a jump width in jump information.

S4303: The transmission circuit acquires second jump data according to the jump data length after the jump from the target jump address, until the jump data length obtained after each jump satisfies the data length.

In this step, when the transmission circuit reads the data, it starts from the target jump address determined in step S4302, jumps the data of a preset length, and then determines the jumped data by the preset length as the second jump data. do. When the length between the address of the second jump data and the source address of the jump start satisfies the data length of the data required for the machine learning device, this indicates that reading of the data required for the machine learning device is complete, and the second jump data If the length between the address of the jump start and the source address of the jump start does not satisfy the data length of data required for the machine learning apparatus, the data is jumped from the last address of the second jump data according to the jump order in steps S4301 to S4303. However, until the length between the address of the second jump data and the source address of the jump start satisfies the data length of data required for the machine learning device, that is, when reading of data required for the machine learning device is completed To read.

Since the implementation principle and technical effect of the data processing method according to the present embodiment are similar to those of the data processing apparatus, the description will not be repeated here. In the data processing method according to the present embodiment, the transmission circuit starts reading the shared memory from the source address, acquires first jump data according to the jump data length after the current jump, and jumps from the last address of the first jump data. After jumping to the target jump address according to the width, the second jump data is acquired according to the jump data length after jumping from the target jump address, but until the length of the jump data obtained after each jump satisfies the data length. Accordingly, when jump information is applied to the motion information, the transmission circuit reads data according to the jump rule of the jump information, thereby simplifying the data reading logic of the transmission circuit, increasing the data access efficiency, and allowing the machine learning chip to access data. Improves on its access speed.

When the transmission circuit operates according to the received data operation signal, the received data operation signal is one coded instruction and must first be decoded and analyzed for the data operation signal. Therefore, as shown in FIG. 41, in the data processing method according to the embodiment of the present application, the step of receiving the data operation signal transmitted from the machine learning device in the data processing device by the transmission circuit in the data processing device includes the following steps Include.

S4401: The transmission circuit analyzes the data operation signal to obtain information on a type flag bit of the data operation signal and operation schedule data.

Note that, in a general data processing process, the number of data operation signals is large, and when a transmission circuit processes one of the data operation signals, other signals must be stored. Specifically, the transmission circuit analyzes the data operation signal, and analyzes data information included in the data operation signal and a type flag bit of the data operation signal. Here, the data operation information includes information such as an operation scheduled data length, a target address, and a source address, and the present embodiment is not limited thereto.

S4402: The transmission circuit performs the analyzed data operation signal by a command queue. Here, the command queue represents an execution order of the data operation signal.

The data operation signals must be sequentially performed according to the order of execution, and the transmission circuit is analyzed according to the command queue based on the data operation information and type flag bits obtained by the transmission circuit analyzing the data operation signal in step S4401 It should be understood that it carries the data action signal.

In the data processing method according to the present embodiment, the data operation signal is analyzed through a transmission circuit to obtain information on the type flag bit of the data operation signal and the operation scheduled data, and then the transmission circuit operates the previously analyzed data according to the command queue. Carry the signal. Accordingly, the data operation signal is first analyzed before performing the data operation signal, and then the data operation signal is performed in an orderly manner, thereby greatly improving the speed at which the transmission circuit performs an operation according to the data operation signal.

When the transmission circuit performs the data operation signals according to the order in the queue, considering that it must perform the data operation signals related to each other, the embodiment of the present application provides another embodiment. As shown in Fig. 42, before the transmitting circuit performs the analyzed data operation signal according to the command queue, the method further includes the following steps.

S4501: The transmission circuit determines a dependency relationship between the analyzed data operation signal adjacent to it, and obtains a determination result. The dependence relationship indicates whether there is a correlation between the s-th data operation signal and the protocol s-1 th data operation signal of the s-th data operation signal.

Here, the transmission circuit must determine a dependency relationship between the analyzed data operation signals adjacent to each other, and determine that there must be a relationship between two adjacent data operation signals processed according to the determination result. Here, the s-th data operation signal indicates only one of the data operation signals, but does not display any specific signal. The s-1th data operation signal represents a signal immediately before the sth data operation signal.

Alternatively, an implementation method of determining the dependence relationship of the analyzed data operation signal adjacent to the transmission circuit is as follows. The transmission circuit extracts data necessary for the s-th data operation signal by the s-th data operation signal, and the s-1 th data operation signal according to the s-1 th data operation signal. When each of the 0th storage address intervals for extracting data necessary for the second data operation signal is obtained, and there is an overlapped region between the first storage address interval and the 0th storage address interval, the transmission circuit performs the first data operation When it is determined that there is a dependency relationship between the signal and the s-1th data operation signal, and there is no overlapped region between the first storage address section and the 0th storage address section, the transmission circuit performs the sth data operation It is determined that there is no dependency relationship between the signal and the s-1th data operation signal. Here, the transmission circuit is a dependency relationship between the analyzed data operation signals adjacent to each other according to the relationship between the s-th data operation signal of the s-th data operation signal and the 0th storage address section of the s-1th data operation signal. Is judged, and the judgment method is as follows. That is, if there is no overlapping region between the first storage address section and the 0th storage address section, the sth data operation signal and the s-1th data operation signal have no dependency relationship, and the first storage address section and the If there is an overlapped region in the 0th storage address section, it indicates that there is a dependency relationship between the sth data operation signal and the s-1th data operation signal.

S4502: If the determination result is that the s-th data operation signal and the s-1 th data operation signal have a dependency relationship, the s-th data operation signal is cached and the s-1 th data operation After performing the signal, the s-th data operation signal is extracted.

The data operation signals are performed in order based on the dependency relationship between the two adjacent data operation signals determined by the transmission circuit in the above step, but the determination result is between the s-th data operation signal and the s-1th data operation signal. If there is a dependency relationship, the transmission circuit first caches the s-th data operation signal, and after completion of the execution of the s-1 th data operation signal, extracts the s-th data operation signal again.

In the data processing method according to this embodiment, the transmission circuit first determines the relationship between two adjacent data operation signals to ensure consistency of the data operation signals. As described above, the order of performing the corresponding operation according to the subsequent data operation signal is ensured by the orderly preparation work at the front stage, thereby increasing data access efficiency and greatly improving the access speed when the machine learning chip accesses data.

As information technology is constantly evolving and improving day by day, the demands of people for data access and data processing are increasing, and the demands for processors that process and access some data are also becoming increasingly stringent. Taking a general-purpose processor as an example, a multi-core processor composed of a plurality of general-purpose processor cores (eg, CPU cores) has become a major trend due to its strong parallel computing power.

However, as machine learning algorithms are constantly being developed, more and more machine learning chips have been created. Since such machine learning chips need to frequently access or process data being stored and stored through various methods such as unicast reading and broadcast, a plurality of transmission interfaces corresponding thereto must be installed. This increased the area of the machine learning chip.

Therefore, simplifying the transmission interface of the machine learning chip in order to reduce the area of the machine learning chip is a technical problem that is urgently needed by those skilled in the art.

In order to solve the above problem, the present application provides the following technical solutions.

The data processing apparatus according to the embodiment of the present application may be implemented through software, hardware, or a combination thereof. The data processing device may be part or all of the data processing device shown in FIG. 43. The data processing device may include a machine learning device 11, a transmission circuit 12, and a shared memory 13, and the machine learning device 11 may include at least one machine learning unit 15. In addition, the unicast read operation and broadcast operation performed by the machine learning unit 15 share one data reception interface 142, and the machine learning unit shares a transmission interface 141 and a shared data reception interface 142 ) Is connected to the transmission circuit 12, and the transmission circuit 12 is connected to the shared memory 13. The transmission circuit 12 acquires input data necessary for the machine learning device from the shared memory 13 according to the data operation signal sent by the machine learning device 11 through the transmission interface 141, and the sharing The input data is returned to the machine learning device 11 through the data receiving interface 142. Note that the machine learning unit 15 may include a first transmission interface 14 (not shown), and the first transmission interface may include a transmission interface 141 and a shared data reception interface 142.

Alternatively, the machine learning apparatus 11 may obtain output data by performing a machine learning operation based on input data. Alternatively, the machine learning device 11 may be used to transmit and store output data to the shared memory 13 through the transmission circuit 12. Specifically, when the machine learning device 11 is used to perform a neural network operation, the machine learning device 11 obtains output neuron data by performing an artificial neural network operation based on input neuron data and weights, and obtains output neuron data. As new input neuron data, it is transmitted to the shared memory 13 through the transmission circuit 12 and stored.

Note that the machine learning unit, transmission circuit, shared memory, and various interfaces may all be implemented through a hardware circuit. Illustratively, the transmission circuit may be a broadcast bus. The shared memory may be non-volatile and/or volatile memory, and includes, but is not limited to, random access memory (RAM), high-speed cache memory, and the like. Each type of interface may correspond to one or a plurality of data I/O (in/out, lead-in, read-out) interfaces or I/O pins.

The data processing apparatus according to the present application may be applied to a machine learning operation, and the machine learning operation includes a neural network operation, a k-means operation, a support vector machine operation, and the like. Alternatively, when the machine learning device performs neural network calculation, the input data may include input neuron data and/or weight, and the input neuron data and weight are the machine learning device to perform artificial neural network calculation. This is the data you need to enter. Accordingly, the output data may include output neuron data, and the output neuron data is an intermediate result or a final result output when the machine learning apparatus performs an artificial neural network operation. Since the weights and neuron data can be recycled, it should be understood that the input data does not necessarily include the input neuron data and the weight in the calculation process, and can include only the input neuron data or only the weight.

Taking a neural network operation as an example (unless otherwise specified, all neural network operations will be described in this embodiment as an example), the data processing apparatus according to the present application can perform a single layer operation of a neural network, and a multilayer operation of a neural network You can also do The implementation process for a multilayer neural network is as follows. In the forward operation, after the execution of the artificial neural network of the previous layer is finished, the operation instruction of the next layer uses the output neuron data calculated by the operation unit as the input neuron data of the next layer (or partially operates on the output neuron data and returns to the next layer. At the same time, the weight is replaced by the weight of the next layer, and after the reverse operation of the artificial neural network of the previous layer is completed, the operation instruction of the next layer is the input neuron slope calculated by the computation unit (similarly, input (Can be used as the output neuron data of the next layer) as the output neuron gradient of the next layer (the same can be used as the output neuron data) (or some operation is performed on the input neuron gradient, and then the output neuron of the next layer is tilted again. At the same time, the weight is replaced with the weight of the next layer.

Referring to FIG. 1, in one selectable technical solution, the machine learning apparatus 11 may include a plurality of machine learning units 15. With regard to the operation of a multilayer neural network, the calculation of one layer neural network during forward operation will be described as an example. In an embodiment, the machine learning device may concurrently calculate output neuron data of all neurons in a corresponding layer among a neural network through a plurality of machine learning units (MLUs). For example, if the machine learning device includes 4 machine learning units and a neural network of a corresponding layer has 100 neurons, each machine learning unit may be allocated to process 25 neurons, and the corresponding operation instruction It can be set and implemented. In the above process, each machine learning unit obtains input neuron data and weights respectively corresponding to the 25 neurons of the layer distributed from the shared memory through the transmission circuit, and output neuron data of the 25 neurons of the layer distributed. Is calculated, and output neuron data of 25 neurons of the layer distributed through the transmission circuit can be transmitted and stored in the shared memory. When each of the machine learning units processes data of a plurality of neurons in a corresponding layer, it can be understood that they can be processed through parallel calculation. Since the parallel computation of the neural network of one layer and another layer is performed, it is possible to implement parallel processing of the neural network computation, thereby improving processing efficiency.

In another selectable method, the machine learning apparatus may each calculate output neuron data of all neurons in each layer of a neural network in a predetermined sequence after using a plurality of machine learning units. In the above process, the immediately preceding machine learning unit can transmit and store the output neuron data of all neurons of the layer through the transmission circuit to the shared memory, so the next machine learning unit extracts the output neuron data of all the neurons of the layer to the next layer. Can be used as input neuron data. It can be understood that the above application is applied to a situation where the amount of computation of a neural network in each layer is not large, for example, a neural network computation with a small number of neurons in each layer.

Referring to FIG. 44, the machine learning unit 0 of FIG. 43 will be described in detail as an example. In one scheme, the machine learning unit 15 includes a transmission interface 141, a shared data reception interface 142, at least one calculation unit 151, and a controller unit 152 connected to the calculation unit 151. The calculation unit 151 includes one mast processing circuit 151a and a plurality of slave processing circuits 151b, and the calculation unit 151 receives the transmission interface 141 and shared data. Connected to the transmission circuit 12 through an interface 142,

The controller unit 152 transmits the data operation signal and the output neuron data to the transmission circuit 12 through the transmission interface 141, and the transmission circuit ( 12) receives the input neuron data and the weight obtained from the shared memory 13 and then transmits the input neuron data and the weight to the mast processing circuit 151a and/or the slave processing circuit 151b. Is used to

The mast processing circuit 151a transmits the input neuron data and/or weights to the plurality of slave processing circuits 151b, respectively, and the plurality of slave processing circuits 151b are intermediate by the neuron data and the weight. After obtaining a plurality of intermediate results by performing operations in parallel, the plurality of intermediate results are transmitted to the mast processing circuit 151a. The mast processing circuit 151a is also used to subsequently process the plurality of intermediate results to obtain calculation results. Here, the subsequent processing may include an activation operation. Specifically, the controller unit 152 may obtain a calculation command, analyze the calculation command to obtain a plurality of calculation commands, and transmit the plurality of calculation commands to the mast processing circuit. In this embodiment, when a machine learning unit includes a plurality of computation units, it can be understood that each computation unit can share the transmission interface and the shared data reception interface.

For example, in one selectable technical solution, the mast processing circuit may further include one controller unit, and the controller unit may include a master instruction processing unit, and specifically, an operation instruction is a micro instruction. Is used to decode. In another selectable technical solution, the slave processing circuit may further include another controller unit, and the other controller unit includes a slave command processing unit, and is specifically used to receive and process micro-instructions. The micro-instruction may be a next level instruction of the instruction. The micro-command may be obtained by dividing or decoding the command, and may be further decoded as a control signal of each member, each unit or each processing circuit. For example, the multiplication micro instruction is the next level instruction of the convolution instruction.

For example, the structure of the machine learning unit will be described in detail with respect to the neural network operation procedure of the machine learning unit. Please refer to steps S5101-S5106 below.

S5101: One IO instruction is stored in advance in the first address of the instruction storage unit of the controller unit,

S5102: The controller unit reads the IO command from the first address of the command storage unit and then acquires a neural network operation command corresponding to the machine learning unit from the off-chip memory through an off-chip interface according to a control signal decoded from the IO command. Alternatively, a neural network calculation instruction corresponding to the machine learning unit is obtained from a shared memory through a transmission circuit, and the obtained calculation instruction is stored in an instruction storage unit.

S5103: The controller unit reads the next IO instruction from the instruction storage unit and then reads all the data blocks required for the operation unit from the shared memory through the transmission circuit according to the data operation signal decoded from the IO instruction, and the data block is allocated. It includes input neuron data and weights of the neurons of the corresponding layer to be performed, and may further include an interpolation table used for calculating a fast activation function, a constant table for setting parameters of a computing device, offset data, etc., and the data operation signal is the The data block contains the source address in shared memory.

S5104: The controller unit reads the next CONFIG (set) command from the command storage unit, and then sets various constants necessary for the neural network calculation of the corresponding layer by the control signal decoded from the CONFIG command. For example, the arithmetic unit sets the value of its inner register according to the constant required for the activation function.

S5105: The controller unit reads the next COMPUTE (calculation) instruction from the instruction storage unit, and according to the control signal (i.e., operation instruction) decoded from the COMPUTE instruction, the operation unit receives the input neuron data and weights of the corresponding layer neurons distributed. And an operation command is transmitted to the mast processing circuit, and the mast processing circuit determines the input neuron data of the distributed neurons in the layer as broadcast data and the weights as distribution data, and distributes one distribution data to a plurality of data blocks. , At least one data block among a plurality of data blocks, broadcast data, and at least one operation instruction among a plurality of operation instructions are transmitted to the slave processing circuit, and an intermediate result through a multiplication processing circuit, an accumulation processing circuit, etc. Is obtained, and neuron data output by the neurons of the corresponding layer distributed according to the intermediate result and the activation processing circuit or the like from the mast processing circuit can be obtained.

S5106: The controller unit reads the next IO command from the command storage unit, and according to the data operation signal decoded from the IO command, transmits and stores the output neuron data to the shared memory through a transmission circuit to store the next layer partial neuron. The input neuron data is obtained, and the data operation signal includes the target address of the output neuron data in a shared memory.

S5105 will be described below as an example. For example, a fully connected operation in a neural network operation may be a process of a neural network network of a certain layer, y=f(wx+b), where x is an input neuron matrix, w is a weight matrix, b is an offset scalar, f is the activation function. Specifically, it may be any one of the sigmoid function, tanh, relu, and softmax function. Here, assuming that there is a binary tree relationship (tree relationship) between the mast processing circuit and the slave processing circuit, and the operation unit has one mast processing circuit and eight slave processing circuits, the implementation method of S5105 is as follows. . The controller unit acquires the input neuron matrix x, the weight matrix w, and the fully connected operation instruction from the shared memory, and transmits the input neuron matrix x, the weight matrix w, and the fully connected operation instruction to the mast processing circuit, and the mast processing circuit receives the input The neuron matrix x is determined as broadcast data, the weight matrix w is determined as the distribution data, the weight matrix w is divided into 8 child matrices, and then the 8 child matrices are distributed to 8 slave processing circuits through the tree module, and the input neuron The matrix x is broadcast to 8 slave processing circuits, and the slave processing circuit performs multiplication and accumulation operations of 8 child matrices and input neuron matrix x in parallel to obtain 8 intermediate results, and then mast processing 8 intermediate results. It transmits to the circuit, and the mast processing circuit obtains an operation result of wx by arranging the eight intermediate results, and performs an activation operation after performing an operation of offset b on the operation result to obtain a final result y.

Since each of the machine learning units can perform parallel calculations on each of the allocated neurons in any one layer, the output neuron data of all neurons in each layer and the input neuron data required for all neurons in the next layer can be stored in the shared memory. It can be understood that the weights can be recycled, or the weights of a new layer of neural networks can be obtained from shared memory.

Note that each machine learning unit may include one computing unit and may include a plurality of computing units. The structure of each operation unit may be the same or different. Here, the structure of each operation unit is embodied in the relationship between the mast processing circuit and each slave processing circuit, and includes, but is not limited to, a relationship of a tree type, an H type, and a systolic array type. In the technical scheme according to the present application, the operation unit may be configured in a one-master-multiple-slave structure, and data may be divided by a calculation instruction for forward operation. Accordingly, since a parallel operation can be performed even on a portion with a large amount of calculation through the plurality of slave processing circuits, it is possible to improve the operation speed and save the operation time, thereby reducing power consumption.

Next, description will be made back to the data processing apparatus according to the present embodiment shown in FIG. 43 above. Here, the unicast read operation is a read operation in the unicast method, and the corresponding data operation signal may be a unicast read command or a unicast read request, and the data operation signal corresponding to the broadcast operation is a broadcast command or multicast. It may be a command, a broadcast request, or a multicast request. Illustratively, the unicast read command is a read command in the unicast method, and may be a read command for input neuron data and weights of the source address of the shared memory transmitted from a machine learning unit, and the input neuron data and Weights should be returned to the machine learning unit. The input neuron data and weight may be input neuron data and weight required for the distributed neurons in the process of the machine learning unit calculating neurons distributed to a layer according to a calculation instruction. Similarly, a unicast read request is a read request in a unicast manner; The broadcast command is a read command for input neuron data and weights of a source address of a shared memory, transmitted from a machine learning unit, and must return the input neuron data and weights to all machine learning units in the machine learning device. The input neuron data may be input neuron data required for all neurons in a layer, that is, all output neuron data of a previous layer, and the weight is a reusable weight such as a convolution kernel, and a multicast instruction and The difference between the broadcast instruction is that the data return object of the multicast instruction is not all machine learning units in the machine learning apparatus, but a plurality of machine learning units corresponding to a marker field in the multicast instruction. On the other hand, in general, the difference between a command and a request is that the overhead according to the command execution is relatively large, but a lot of information is included in the command, and the overhead according to the request is relatively small, but the information included in the request is small. will be.

In general, when a machine learning unit receives data returned by a unicast read operation and a broadcast operation, at least two corresponding data interfaces are required, each of which returns for a unicast read data operation signal in the transmitting circuit. Receives unicast read data, and receives broadcast and/or multicast data returned for broadcast and/or multicast data operation signals in a transmission circuit. In this embodiment, as shown in Fig. 1, there is only one receiving interface of the machine learning unit 0, which is a shared data receiving interface. For example, interface c0 receives the unicast read data that the transmitting circuit returns for the unicast read data operation signal, and the broadcast and/or multicast data that the transmitting circuit returns for the broadcast and/or multicast data operation signal. Cast data can be received.

After the transmitting circuit extracts the required input neuron data and weights from the shared memory, if a cache exists, it temporarily stores it in the cache, and then returns the data corresponding to the data operation signal from which the transmitting circuit requested the data, i.e. the data related to the data. It can be understood that the object (machine learning unit) can be determined and the data can be transmitted to the shared data receiving interface. In a unicast read operation, the shared data receiving interface is a shared data receiving interface of one machine learning unit corresponding to a return target of the data, and in a broadcast operation, the shared data receiving interface is a return target of the data A plurality of shared data reception interfaces of a plurality of machine learning units corresponding to.

Therefore, in the data processing apparatus according to the present embodiment, when at least one machine learning unit performs a unicast read operation and a broadcast operation, the machine learning unit shares one data receiving interface in the machine learning unit. It is possible to efficiently reduce the number of interfaces that return a, save hardware sources, and reduce hardware area and power consumption.

Hereinafter, the transmission interface in the machine learning unit will be described in detail. Referring to FIG. 45, on the basis of FIG. 43, the transmission interface 141 may include a unicast read signal transmission interface 1411 and a broadcast signal transmission interface 1412, and the machine learning unit 15 ) Are respectively connected to the transmission circuit 12 through the unicast read signal transmission interface 1411 and the shared data reception interface 142 to implement a unicast read operation, and the broadcast signal transmission interface 1412 and Each is connected to the transmission circuit 12 through the shared data reception interface 142 to implement a broadcast operation. In the case of MLU0, the unicast read signal transmission interface corresponds to the interface a0, the broadcast signal transmission interface corresponds to the interface b0, and the shared data reception interface corresponds to the interface c0. Here, interface a0 transmits a unicast read data operation signal to the transmission circuit, interface b0 transmits a broadcast and/or multicast data operation signal to the transmission circuit, and interface c0 transmits a unicast read data operation signal from the transmission circuit. It is possible to receive the unicast read data that is returned for, and the broadcast and/or multicast data that the transmitting circuit returns for the broadcast and/or multicast data operation signal. Accordingly, the present embodiment simplifies processing logic because different types of data operation signals are transmitted through the unicast read signal transmission interface and the broadcast signal transmission interface, respectively.

In one embodiment, corresponding to the unicast read operation and the broadcast operation, referring to FIG. 45, the transmission circuit 12 in the data processing apparatus includes a second transmission interface 120 and the second transmission interface 120. A read/write processing circuit 121 connected to ), and an arbitration circuit 122 connected to the read/write processing circuit 121 may be included. The read/write processing circuit 121 receives the data operation signal transmitted by the at least one machine learning unit 15 through the transmission interface 141 and the second transmission interface 120 to operate the data. A signal is transmitted to the arbitration circuit 122, and the data obtained by the arbitration circuit 122 from the shared memory 13 is transmitted through the second transmission interface 120 and the shared data reception interface 142. It is used to return to the machine learning unit corresponding to the data operation signal. The arbitration circuit 122 arbitrates the data operation signal received from the read/write processing circuit 121 according to a pre-set arbitration rule, and transfers the data in the shared memory 13 according to the data operation signal successfully arbitrated. It is used to operate.

Specifically, the read/write processing circuit 121 may process a unicast read signal, and may process a broadcast signal and/or a multicast signal. In one embodiment, the read/write processing circuit 121 may include a unicast read processing circuit, and the unicast read processing circuit may process a unicast read signal, and may process a broadcast signal and/or a multicast signal. It can also process cast signals. Here, when the unicast readout processing circuit processes a broadcast signal and/or a multicast signal, at least one machine learning unit transmits a broadcast through the broadcast signal transmission interface and the second transmission interface and/or Or receiving a multicast signal and transmitting the broadcast and/or multicast signal to the arbitration circuit, and the data obtained by the arbitration circuit from the shared memory through the second transmission interface and the shared data reception interface in advance. It transmits to a plurality of machine learning units corresponding to the broadcast and/or multicast signals according to a set order. The preset order is an order of returning data to the plurality of machine learning units, which can be sorted according to the priority of each machine learning unit, and sorted according to a number order or other order of the plurality of machine learning units. May be.

Alternatively, the read/write processing circuit 121 may include a unicast read processing circuit and a broadcast processing circuit, and the unicast read processing circuit processes a unicast read signal, and the broadcast processing circuit Processes a broadcast signal and/or a multicast signal.

Here, the unicast read processing circuit may receive a unicast read signal transmitted by at least one machine learning unit through a unicast read signal transmission interface and a second transmission interface, and the unicast read signal is transmitted to the arbitration circuit. And the arbitration circuit transmits the data obtained from the shared memory to the machine learning unit corresponding to the unicast read signal through the second transmission interface and the shared data reception interface. The broadcast read processing circuit may receive a broadcast and/or multicast signal transmitted by at least one machine learning unit through the broadcast signal transmission interface and the second transmission interface, and the broadcast and/or Or transmitting a multicast signal to the arbitration circuit, and transmitting the data obtained by the arbitration circuit from the shared memory to a plurality of broadcast and/or multicast signals through the second transmission interface and the shared data reception interface. It is used to transmit to the machine learning unit.

Here, the pre-set arbitration rule causes the arbitration circuit to determine the priority levels of the plurality of data operation signals according to a predetermined rule, so that the arbitration circuit determines an object to be operated according to the priority of each data operation signal. That is, a data operation signal having a high priority is selected and used as a data operation signal that has successfully mediated. For example, a priority of a data operation signal having a high transmission rate may be set to a high priority, and a priority of a data operation signal having a low transmission rate may be set to a low priority. For example, the pre-set arbitration rule may be a Round-Robin Scheduling arbitration rule, a Maximum Carrier-to-Interference Rate scheduling rule, a Proportional Fair rule, and the like. Meanwhile, the arbitration circuit may use as an auxiliary arbitration rule whether a data path (from an interface to an interface) between the machine learning unit and the read/write processing circuit is in an idle state. That is, when the data path corresponding to the data operation signal for which the mediation is successful is in an idle state.

Specifically, the unicast read processing circuit is connected to the plurality of machine learning units through the second transmission interface to process the unicast read operation of the plurality of machine learning units and transmits a plurality of unicast read commands to the unicast read processing circuit. It can be cached in the cast read command cache queue, and the unicast read command can be analyzed and the corresponding unicast read command is obtained and stored in the unicast read request cache queue in the unicast read processing circuit for arbitration through an arbitration circuit. On the other hand, unicast read requests can be cached in the unicast read request cache queue without performing analysis. Similarly, the broadcast processing circuitry may also be connected to a plurality of machine learning units via a second transmission interface, and may include a broadcast and/or multicast instruction cache queue and a broadcast and/or multicast request cache queue. It can be, but it is not repeated here. In one selectable approach, the read/write processing circuit may include one unicast read processing circuit and one broadcast processing circuit.

Accordingly, in the present embodiment, the unicast read operation can be processed through the unicast read processing circuit, and the broadcast operation can be processed through the broadcast processing circuit. Each operation can be processed, simplifying the processing logic.

In one selectable scheme, referring to FIG. 45, on the basis of the data processing apparatus of FIG. 43, the second transmission interface may be subdivided into interfaces that process different types of data operations. Specifically, the second transmission interface 120 includes at least one group or more unicast read signal reception interfaces and unicast read data transmission interfaces connected to the unicast read processing circuit, and at least one group or more connected to the broadcast processing circuit. It may include a broadcast signal reception interface and a broadcast data transmission interface, the unicast read signal reception interface is connected to the unicast read signal transmission interface of the machine learning unit, and the broadcast signal reception interface is the machine learning The unit is connected to a broadcast signal transmission interface, and the unicast read data transmission interface and the broadcast data transmission interface of the transmission circuit are connected to the shared data reception interface of the machine learning unit. This embodiment simplifies processing logic because different types of data operations can be processed through each interface in the second transmission interface.

In one embodiment, referring to FIG. 45, the read/write processing circuit may be divided into a plurality of processing circuit groups, one machine learning unit corresponds to one processing circuit group, and the processing circuit group includes at least one processing circuit group. And a unicast read processing circuit and one broadcast processing circuit. For example, MLU0 corresponds to a unicast read processing circuit 0 and a broadcast processing circuit 0, and MLUn corresponds to a unicast read processing circuit n and a broadcast processing circuit n. Similarly, the second transmission interface 0 has one processing circuit group and one group of interfaces each connected to one machine learning unit, and one-to-one connection of the machine learning unit and the unicast read processing circuit, and broadcast with the machine learning unit. It is configured to implement a one-to-one connection of the processing circuit.

For example, in the MLU0 and the unicast read processing circuit 0, the interface d0 in the second transmission interface is a unicast read signal receiving interface, and is respectively connected to the unicast read signal transmission interface a0 and the unicast read processing circuit 0 of the MLU0. It is connected to receive the unicast read signal transmitted from MLU0 and then transmits it to the unicast read processing circuit 0 for processing. Interface e0 in the second transmission interface is a unicast read data transmission interface, which is connected to the shared data reception interface c0 and unicast read processing circuit 0 of MLU0, respectively, to the unicast read signal transmitted from the unicast read processing circuit 0. It can be used to receive the corresponding input neuron data and weights and transmit them to interface c0 in MLU0. In the MLU0 and the broadcast processing circuit 0, the interface f0 in the second transmission interface is a broadcast signal reception interface, which is connected to the broadcast signal transmission interface b0 and the broadcast processing circuit 0 of the MLU0, respectively, and broadcasts transmitted from the MLU0 and / Or, after receiving the multicast signal, it can be used for processing by transmitting to the broadcast processing circuit 0 The interface g0 in the second transmission interface is a broadcast data transmission interface, and is connected to the shared data reception interface ci and the broadcast processing circuit 0 of a plurality of MLUs, and the broadcast and/or multi-channel transmitted from the After receiving the input neuron data and weight corresponding to the cast signal, it can be used to transmit to the shared data receiving interface ci in the plurality of MLUs.

Accordingly, this embodiment provides a one-to-one connection of a machine learning unit and a unicast read processing circuit; And it is possible to implement customized one-to-one data operation processing through a one-to-one connection between the machine learning unit and the broadcast processing circuit, thereby reducing the complexity of the access logic of the data operation and reducing collisions, thereby improving processing efficiency.

Referring to FIG. 46, in one selectable scheme, on the basis of the data processing apparatus shown in FIG. 3, the number of interfaces in the transmission circuit can be reduced. Specifically, the read/write processing circuit 121 may include a broadcast processing circuit and a plurality of unicast read processing circuits, and the plurality of unicast read processing circuits and the plurality of machine learning units are connected one-to-one And, the broadcast processing circuit and the plurality of machine learning units are connected one-to-many. Exemplarily, MLU0 corresponds to the unicast read processing circuit 0 and the broadcast processing circuit, and MLUn corresponds to the unicast read processing circuit n and the broadcast processing circuit. Similarly, the second transmission interface has one unicast read processing circuit and a group of interfaces each connected to one machine learning unit, thereby implementing a one-to-one connection between the machine learning unit and the unicast read processing circuit. Further, in the second transmission interface, one broadcast processing circuit and a group of interfaces each connected to a plurality of machine learning units are further present, thereby implementing a many-to-one connection between the machine learning unit and the broadcast processing circuit. Specifically, the second transmission interface may include a group of broadcast interfaces connected to a broadcast processing circuit, and the broadcast interface may include a broadcast signal reception interface and a broadcast data transmission interface, and the plurality of The machine learning unit of is connected to the broadcast processing circuit through the group of broadcast interfaces.

For example, in the plurality of MLUs and the broadcast processing circuit, the interface dn+1 in the second transmission interface is a broadcast signal receiving interface, and receives broadcast and/or multicast signals transmitted from the plurality of MLUs. Then, it can be used for processing by transmitting it to the broadcast processing circuit. Interface en+1 in the second transmission interface is a broadcast data transmission interface, and after receiving input neuron data and weights corresponding to the broadcast and/or multicast signal transmitted from the broadcast processing circuit, a plurality of MLUs It can be used to transmit to the shared data receiving interface within.

As can be seen from this, a plurality of machine learning units may share one broadcast processing circuit and simultaneously share a group of broadcast signal reception interfaces and broadcast data transmission interfaces. Accordingly, the data processing apparatus according to the present exemplary embodiment can reduce the number of returned data interfaces among the machine learning units and interfaces in the transmission circuit, thereby further saving hardware resources and reducing hardware area and power consumption.

Referring to FIG. 47, in one selectable scheme, the number of interfaces in the transmission circuit on the basis of FIG. 46 is further reduced. The second transmission interface 120 includes a plurality of groups of unicast read signal reception interfaces and shared data transmission interfaces connected one-to-one with the plurality of unicast read processing circuits, and a broadcast signal reception interface connected to the broadcast processing circuit. And the shared data transmission interface is also connected to the broadcast processing circuit, the unicast read signal reception interface is connected to the unicast read signal transmission interface of the machine learning unit, and the broadcast signal reception interface It is connected to the broadcast signal transmission interface of the machine learning unit, and the shared data transmission interface is connected to the shared data reception interface of the machine learning unit.

Exemplarily, in the unicast read processing circuit 0, the second transmission interface includes a group of unicast read signal reception interfaces d0 and shared data transmission interfaces e0 connected to the unicast read processing circuit 0 in one-to-one, and The cast read signal reception interface d0 is connected to the unicast read signal transmission interface a0 in MLU0, and the shared data transmission interface e0 is connected to the shared data reception interface c0 in MLU0. In the unicast read processing circuit n, the second transmission interface comprises a group of unicast read signal receiving interfaces dn and shared data transmission interfaces en connected to the unicast read processing circuit n in a one-to-one, unicast read signal The reception interface dn is connected to the unicast read signal transmission interface an in the MLUn, the shared data transmission interface en is connected to the shared data reception interface cn in the MLUn, and the second transmission interface receives broadcast signals connected to the broadcast processing circuit. An interface dn+1 is further included, and the broadcast signal reception interface dn+1 is connected to a broadcast signal transmission interface (MLUi is connected to interface bi) of each MLU. On the other hand, the most important part to note is that all of the shared data transmission interfaces ei among the transmission circuits are all connected to the broadcast processing circuit, and input neuron data and weights corresponding to the broadcast and/or multicast signals transmitted from the broadcast processing circuit. May be received and transmitted to the shared data receiving interface ci in the plurality of MLUs. As can be seen from this, each unicast read processing circuit i in the transmission circuit shared the shared data transmission interface ei with the broadcast processing circuit, respectively, and the shared data reception interface ci in the MLUi and the shared data transmission interface ei in the transmission circuit. The data path configured with may transmit unicast read data, broadcast and/or multicast data between the MLUi and the transmission circuit.

As can be seen from this, since each of the plurality of unicast read processing circuits shares the data transmission interface with the broadcast processing circuit, the data processing apparatus according to the present embodiment reduces the number of interfaces of the transmission circuit, thereby further saving hardware resources. This reduces the hardware area and power consumption.

As information technology is constantly evolving and improving day by day, the demands of people for data access and data processing are increasing, and the demands for processors that process and access some data are also becoming increasingly stringent. Taking a general-purpose processor as an example, a multi-core processor composed of a plurality of general-purpose processor cores (eg, CPU cores) has become a major trend due to its strong parallel computing power.

However, as machine learning algorithms are constantly developing, more and more machine learning chips are created. Since such machine learning chips need to frequently access or process data being stored and stored through various methods such as unicast reading and broadcast, a plurality of transmission interfaces corresponding thereto must be installed. This increased the area of the machine learning chip.

Therefore, simplifying the transmission interface of the machine learning chip in order to reduce the area of the machine learning chip is a technical problem to be urgently solved by those skilled in the art.

In order to solve the above problem, the present application provides the following technical solutions.

With reference to FIG. 49, the machine learning unit will be described in detail by taking the machine learning unit 0 in FIG. 48 as an example. In one scheme, the machine learning unit 15 includes at least one transmission interface 141, at least one reception interface 142, at least one calculation unit 151, and a controller unit connected to the calculation unit 151 ( 152), wherein the calculation unit 151 includes one mast processing circuit 151a and a plurality of slave processing circuits 151b, and the calculation unit 151 includes the at least one transmission interface ( 141) and connected to the transmission circuit 12 through at least one receiving interface 142,

The controller unit 152 transmits the data operation signal and the output neuron data to the transmission circuit 12 through the at least one transmission interface 141, and through the at least one reception interface 142 After the transmission circuit 12 receives the input neuron data and the weight obtained from the shared memory 13, the input neuron data and the weight are transferred to the mast processing circuit 151a and/or the slave processing circuit ( 151b),

The mast processing circuit 151a transmits the input neuron data and/or weights to the plurality of slave processing circuits 151b, respectively, and the plurality of slave processing circuits 151b are intermediate by the neuron data and the weight. After obtaining a plurality of intermediate results by performing operations in parallel, a plurality of intermediate results are transmitted to the mast processing circuit 151a, and the mast processing circuit 151a also performs subsequent processing on the plurality of intermediate results to obtain a calculation result. Is used to get Here, the subsequent processing may include an activation operation. Specifically, the controller unit 152 may obtain a calculation command, analyze the calculation command to obtain a plurality of calculation commands, and transmit the plurality of calculation commands to the mast processing circuit.

In the present embodiment, when a machine learning unit includes a plurality of operation units, it may be understood that each operation unit may share the at least one transmission interface and the at least one reception interface.

For example, in one selectable technical solution, the mast processing circuit may include one controller unit, and the controller unit may include a master instruction processing unit, and specifically, an operation instruction as a micro instruction. It is used for decoding. In another selectable technical solution, the slave processing circuit may include another controller unit, and the other controller unit includes a slave command processing unit, and is specifically used to receive and process micro-instructions. The micro-instruction may be a next level instruction of the instruction. The micro-command may be obtained by dividing or decoding the command, and may be further decoded into a control signal of each member, each unit or each processing circuit. For example, the multiplication micro instruction is the next level instruction of the convolution instruction.

For example, the structure of the machine learning unit will be described in detail with respect to the neural network operation procedure of the machine learning unit. Please refer to steps S6101-S6106 below.

S6101: Save one IO instruction in advance in the first address of the instruction storage unit of the controller unit,

S6102: The controller unit reads the IO command from the first address of the command storage unit and then obtains a neural network operation command corresponding to the machine learning unit from the off-chip memory through an off-chip interface according to a control signal decoded from the IO command. Alternatively, a neural network calculation instruction corresponding to the machine learning unit is obtained from a shared memory through a transmission circuit, and the obtained calculation instruction is stored in an instruction storage unit.

S6103: The controller unit reads the next IO instruction from the instruction storage unit, and then reads all the data blocks required for the operation unit from the shared memory through the transmission circuit according to the data operation signal decoded from the IO instruction, and the data block is distributed. Including input neuron data and weights of the neurons of the corresponding layer to be performed, an interpolation table used for a fast activation function calculation, a constant table for setting parameters of a computing device, offset data, and the like, the data operation signal The data block contains the source address in shared memory.

S6104: The controller unit reads the next CONFIG (set) command from the command storage unit, and then sets various constants necessary for the neural network calculation of the corresponding layer by the control signal decoded from the CONFIG command. For example, the arithmetic unit sets the value of its inner register according to the constant required for the activation function.

S6105: The controller unit reads the next COMPUTE (computation) instruction from the instruction storage unit, and according to the control signal (i.e., operation instruction) decoded from the COMPUTE instruction, the operation unit reads the input neuron data and weight of the corresponding layer neuron. And an operation command is transmitted to the mast processing circuit, and the mast processing circuit determines the input neuron data of the distributed neurons in the layer as broadcast data and the weights as distribution data, and distributes one distribution data to a plurality of data blocks. , At least one data block among a plurality of data blocks, broadcast data, and at least one operation instruction among a plurality of operation instructions are transmitted to the slave processing circuit, and an intermediate result through a multiplication processing circuit, an accumulation processing circuit, etc. Is obtained, and neuron data output by the neurons of the corresponding layer distributed according to the intermediate result and the activation processing circuit or the like from the mast processing circuit can be obtained.

S6106: The controller unit reads the next IO instruction from the instruction storage unit, and transmits and stores the output neuron data to the shared memory through a transmission circuit according to the data operation signal decoded from the IO instruction, and stores the next layer partial neuron. To obtain input neuron data, the data operation signal includes the target address of the neuron data of the output in shared memory.

An exemplary description will be given of S6105 below. For example, a fully connected operation in a neural network operation may be a process of a neural network network of a certain layer, y=f(wx+b), where x is an input neuron matrix, w is a weight matrix, b is an offset scalar, f is the activation function. Specifically, it may be any one of the sigmoid function, tanh, relu, and softmax function. Here, assuming that there is a binary tree relationship (tree relationship) between the mast processing circuit and the slave processing circuit, and the operation unit has one mast processing circuit and eight slave processing circuits, the implementation method of S5105 is as follows. . The controller unit acquires the input neuron matrix x, the weight matrix w, and the fully connected operation instruction from the shared memory, and transmits the input neuron matrix x, the weight matrix w, and the fully connected operation instruction to the mast processing circuit, and the mast processing circuit receives the input The neuron matrix x is determined as broadcast data, the weight matrix w is determined as the distribution data, the weight matrix w is divided into 8 child matrices, and then the 8 child matrices are distributed to 8 slave processing circuits through the tree module, and the input neuron The matrix x is broadcast to 8 slave processing circuits, and the slave processing circuit performs multiplication and accumulation operations of 8 child matrices and input neuron matrix x in parallel to obtain 8 intermediate results, and then mast processing 8 intermediate results. It transmits to the circuit, and the mast processing circuit obtains an operation result of wx by arranging the eight intermediate results, and performs an activation operation after performing an operation of offset b on the operation result to obtain a final result y.

Since each of the machine learning units can perform parallel calculations on each of the allocated neurons in any one layer, the output neuron data of all neurons in each layer and the input neuron data required for all neurons in the next layer can be stored in the shared memory. It can be understood that the weights can be recycled, or the weights of a new layer of neural networks can be obtained from shared memory.

Note that each machine learning unit may include one computing unit and may include a plurality of computing units. The structure of each operation unit may be the same or different. Here, the structure of each operation unit is embodied in the relationship between the mast processing circuit and each slave processing circuit, and includes, but is not limited to, a relationship of a tree type, an H type, and a systolic array type. In the technical scheme according to the present application, the operation unit may be configured in a one-master-multiple-slave structure, and data may be divided by a calculation instruction for forward operation. Accordingly, since a parallel operation can be performed even on a portion with a large amount of calculation through the plurality of slave processing circuits, it is possible to improve the operation speed and save the operation time, thereby reducing power consumption.

Next, description will be made back to the data processing apparatus according to the present embodiment shown in FIG. 43 above. Here, the data operation signal corresponding to the unicast read operation may be a unicast read command and a unicast read request, and the data operation signal corresponding to the unicast write operation may be a unicast write command and a unicast write request, The data operation signal corresponding to the broadcast operation may be a broadcast command, a multicast command, a broadcast request, or a multicast request. Illustratively, the unicast read instruction is an instruction to read input neuron data and weights of the source address of the shared memory, transmitted from a machine learning unit. The input neuron data and weight may be returned to the machine learning unit. The input neuron data and weights are input neuron data and weights required for the distributed neurons in the process of calculating neurons distributed to one layer by the machine learning unit according to a calculation instruction, and the unicast recording command is This is a write command that writes output neuron data transmitted from the learning unit, obtained through neural network calculation, to a target address of the shared memory. Since the output neuron data in the previous layer neural network can be used as input neuron data required for the next layer neural network, it is recorded in the shared memory so that each machine learning unit obtains the necessary input neuron data from the shared memory. The broadcast command is a read command for input neuron data and weights of a source address of a shared memory, transmitted from a machine learning unit, and must return the input neuron data and weights to all machine learning units of the machine learning device. The input neuron data may be input neuron data required for all neurons in one layer, that is, all output neuron data in a previous layer. The weight is a reusable weight such as, for example, a convolutional kernel, and the data return object of the multicast instruction is not all machine learning units in the machine learning device, but a plurality of marker fields in the multicast instruction. Is a machine learning unit. On the other hand, in general, the difference between a command and a request is that the overhead according to the command execution is relatively large, but a lot of information is included in the command, and the overhead according to the request is relatively small, but the information included in the request is small. will be.

In general, when a machine learning unit transmits a unicast read signal, a unicast record signal and a broadcast and/or multicast signal, at least three corresponding data signal transmission interfaces are required. These are each used to transmit a unicast read signal to a transmission circuit, a unicast write signal to a transmission circuit, and to transmit a broadcast and/or multicast signal to a transmission circuit. In this embodiment, when at least one machine learning unit performs at least two types of data operations among a unicast read operation, a unicast write operation, and a broadcast operation, one transmission interface of the machine learning unit is shared. Referring to FIG. 48, at least one transmission interface 141 of the machine learning unit 0 may include two data operation signal transmission interfaces, interface a0 and interface b0. In one embodiment, interface a0 may be a unicast read signal transmission interface, and interface b0 may be a signal transmission interface shared by a unicast record signal and a broadcast and/or multicast signal. In one embodiment, interface a0 may be a unicast write signal transmission interface, and interface b0 may be a signal transmission interface shared by a unicast read signal and a broadcast and/or multicast signal. In one embodiment, interface a0 may be a broadcast and/or multicast signal transmission interface, and interface b0 may be a signal transmission interface shared by a unicast read signal and a unicast write signal. Meanwhile, in one selectable method, when at least one machine learning unit performs a unicast read operation, a unicast write operation, and a broadcast operation, one of the machine learning units may share a transmission interface. That is, the transmission interface may transmit a unicast read signal, a unicast record signal, and a broadcast and/or multicast signal.

Accordingly, in the data processing apparatus according to the present embodiment, when at least one machine learning unit performs at least two data operations among a unicast read operation, a unicast write operation, and a broadcast operation, one of the machine learning units is transmitted. Since interfaces can be shared, the number of data operation signal transmission interfaces among machine learning units can be efficiently reduced, and hardware sources can be saved, thereby reducing hardware area and power consumption.

In one selectable scheme, corresponding to the unicast read operation, unicast write operation and broadcast operation, referring to Fig. 3, on the basis of Fig. 48, the transmission circuit 12 in the data processing apparatus, A second transmission interface 120, a read/write processing circuit 121 connected to the second transmission interface 120, and an arbitration circuit 122 connected to the read/write processing circuit 121 can do. The read/write processing circuit 121 receives a data operation signal transmitted by the at least one machine learning unit 15 through the at least one transmission interface 141 and the second transmission interface 120 The data operation signal is transmitted to the arbitration circuit 122, and the data obtained by the arbitration circuit 122 from the shared memory 13 are transmitted to the second transmission interface 120 and the at least one reception interface 142. ) Is used to return to the machine learning unit corresponding to the data operation signal. The arbitration circuit 122 arbitrates the data operation signal received from the read/write processing circuit 121 according to a pre-set arbitration rule, and transfers the data in the shared memory 13 according to the data operation signal successfully arbitrated. It is used to operate.

Specifically, the read/write processing circuit 121 may process a unicast read signal, and may process a broadcast signal and/or a multicast signal. In one embodiment, the read/write processing circuit 121 may include a unicast read processing circuit and a unicast write processing circuit, and the unicast read processing circuit may process a unicast read signal, Broadcast signals and/or multicast signals may be processed. Here, for example, sharing one transmission interface of the machine learning unit in a unicast recording operation and a broadcast operation performed by the machine learning unit, the at least one transmission interface is a unicast read signal transmission interface and a shared signal transmission Includes an interface. Broadcast and/or multicast signals transmitted by at least one machine learning unit through the shared signal transmission interface and the second transmission interface when the unicast read processing circuit processes a broadcast signal and/or a multicast signal And transmits the broadcast and/or multicast signal to the arbitration circuit, and the arbitration circuit presets the data acquired from the shared memory through the second transmission interface and the at least one reception interface In one order, the broadcast and/or multicast signals are transmitted to a plurality of machine learning units. The pre-set order is an order of returning data to the plurality of machine learning units, which can be sorted according to the priority of each machine learning unit, and sorted according to the number order or other order of the plurality of machine learning units. You may.

Alternatively, the read/write processing circuit 121 may include a unicast read processing circuit, a unicast write processing circuit, and a broadcast processing circuit, wherein the unicast read processing circuit processes a unicast read signal and , The unicast recording processing circuit processes a unicast recording signal, and the broadcast processing circuit processes a broadcast signal and/or a multicast signal.

In the same manner, in a unicast recording operation and a broadcast operation performed by the machine learning unit, a transmission interface of one of the machine learning units is shared as an example. Here, the unicast read processing circuit receives the unicast read signal transmitted by the at least one machine learning unit through the unicast read signal transmission interface and the second transmission interface and transmits the unicast read signal to the arbitration circuit, , The arbitration circuit may be used to transmit the data obtained from the shared memory to a machine learning unit corresponding to the unicast read signal through the second transmission interface and the at least one reception interface. The unicast recording processing circuit receives a unicast recording signal transmitted by at least one machine learning unit through a shared signal transmission interface and a second transmission interface and transmits the unicast recording signal to the arbitration circuit, and the unicast It can be used to write unicast write data corresponding to a write signal to the shared memory. In addition, the broadcast recording processing circuit receives the broadcast and/or multicast signal transmitted by at least one machine learning unit through the shared signal transmission interface and the second transmission interface, and the broadcast and/or multicast A plurality of machines that transmit a signal to the arbitration circuit and transmit the data obtained by the arbitration circuit from the shared memory to the broadcast and/or multicast signal through the second transmission interface and the at least one receiving interface It can be used to transmit to the learning unit. Note that in general, the unicast record signal may include unicast record data, and after transmitting the unicast record signal, the unicast record data may be transmitted using the same data path.

Here, the pre-set arbitration rule causes the arbitration circuit to determine the priority levels of the plurality of data operation signals according to a predetermined rule, so that the arbitration circuit determines an object to be operated according to the priority of each data operation signal. That is, a data operation signal having a high priority is selected and used as a data operation signal that has successfully mediated. For example, a priority of a data operation signal having a high transmission rate may be set to a high priority, and a priority of a data operation signal having a low transmission rate may be set to a low priority. For example, the pre-set arbitration rule may be a Round-Robin Scheduling arbitration rule, a Maximum Carrier-to-Interference Rate scheduling rule, a Proportional Fair rule, and the like. Meanwhile, the arbitration circuit may use as an auxiliary arbitration rule whether the data path (from interface to interface) between the machine learning unit and the read/write processing circuit is idle. That is, the data path corresponding to the data operation signal for which the mediation was successful is in an idle state.

It is connected to the machine learning unit to process unicast read operations of multiple machine learning units, and can cache multiple unicast read commands in the unicast read command cache queue in the unicast read processing circuit, and analyze the unicast read commands A corresponding unicast read command can be obtained and stored in the unicast read request cache queue in the unicast read processing circuit to mediate through the arbitration circuit. On the other hand, unicast read requests can be cached in the unicast read request cache queue without performing an analysis operation. Similarly, the broadcast processing circuitry may also be connected to a plurality of machine learning units via a second transmission interface, and may include a broadcast and/or multicast instruction cache queue and a broadcast and/or multicast request cache queue. I can. Likewise, the unicast write processing circuit may also be connected to a plurality of machine learning units through the second transmission interface, and may include a unicast write command cache queue and a unicast write request cache queue, which will not be described herein again. In one selectable approach, the read/write processing circuit may include one unicast read processing circuit, one unicast write processing circuit, and one broadcast processing circuit.

Accordingly, in the present embodiment, a unicast read operation can be processed through a unicast read processing circuit, a unicast write operation can be processed through a unicast write processing circuit, and a broadcast operation is performed through the broadcast processing circuit. As it can be processed, different types of data operations can be processed through different processing circuits, thereby simplifying processing logic.

Alternatively, referring to FIG. 50, when at least one machine learning unit performs a unicast recording operation or a broadcast operation, one transmission interface in the machine learning unit is shared. That is, the at least one transmission interface 141 may include a shared signal transmission interface shared by a unicast write operation and a broadcast operation, and a unicast read signal transmission interface. For example, in the case of MLU0, interface a0 is a unicast read signal transmission interface, and interface b0 is a shared signal transmission interface, so that unicast recording signals, broadcast and/or multicast signals can be transmitted, and interface c0 is a unicast read signal transmission interface. It is a cast read data reception interface, and interface d0 is a broadcast and/or multicast data reception interface. For convenience of explanation, in the following embodiments, when at least one machine learning unit all performs a unicast recording operation or a broadcast operation, a case in which one transmission interface in the machine learning unit is shared is described as an example. Of course, the following embodiments may be applied to a method of sharing other signal transmission interfaces.

In one selectable scheme, referring to FIG. 50, the read/write processing circuit may be divided into a plurality of processing circuit groups, and one machine learning unit corresponds to one processing circuit group, and the processing circuit group Includes one unicast read processing circuit, one unicast write processing circuit, and one broadcast processing circuit. Exemplarily, MLU0 corresponds to a unicast read processing circuit 0, a unicast write processing circuit 0 and a broadcast processing circuit 0, and MLUn is a unicast read processing circuit n, a unicast write processing circuit n, and a broadcast processing circuit n. Corresponds to Similarly, one processing circuit group and one group of interfaces each connected to one machine learning unit exist in the second transmission interface, and one-to-one connection of the machine learning unit and the unicast read processing circuit, and the machine learning unit and unicast write processing. It implements a one-to-one connection of circuits, and one-to-one connection of a machine learning unit and a broadcast processing circuit.

Specifically, referring to FIG. 50, the second transmission interface 120 includes a plurality of interface groups, the one processing circuit group corresponds to one interface group, and the one interface group is A unicast read signal reception interface and a unicast read data transmission interface connected to a cast read processing circuit, a unicast read signal reception interface connected to the unicast write processing circuit, a broadcast signal reception interface and a broadcast connected to the broadcast processing circuit Includes a data transmission interface.

For example, in MLU0, an interface group corresponding to the corresponding processing circuit group includes an interface e0, an interface f0, an interface g0, an interface h0, and an interface i0. For example, in the MLU0 and the unicast read processing circuit 0, the interface e0 in the second transmission interface is a unicast read signal reception interface, which is connected to the unicast read signal transmission interface a0 and the unicast read processing circuit 0 of MLU0, respectively. After receiving the unicast read signal transmitted from MLU0, it can be used for processing by transmitting it to the unicast read processing circuit 0. Interface f0 in the second transmission interface is a unicast read data transmission interface, the unicast read signal transmitted from the unicast read processing circuit 0, respectively connected to the unicast read data receiving interface c0 and the unicast read processing circuit 0 of the MLU0 After receiving the input neuron data and the weight corresponding to the MLU0 can be used to transmit to the interface c0. In the MLU0 and unicast recording processing circuit 0, the interface g0 in the second transmission interface is a unicast recording signal interface, which is connected to the shared signal transmission interface b0 and the unicast recording processing circuit 0 of MLU0, respectively, and transmitted from MLU0. It can be used to receive the recording signal and transmit it to the unicast recording processing circuit 0 for processing. In the MLU0 and the broadcast processing circuit 0, the interface h0 in the second transmission interface is a broadcast signal reception interface, which is connected to the shared signal transmission interface b0 and the broadcast processing circuit 0 of the MLU0, respectively, and broadcasts transmitted from the MLU0 and/ Alternatively, after receiving the multicast signal, it may be used for processing by transmitting it to the broadcast processing circuit 0. Interface i0 in the second transmission interface is a broadcast data transmission interface, which is connected to the broadcast data reception interface di and the broadcast processing circuit 0 of a plurality of MLUs, and the broadcast and/or multicast transmitted from the broadcast processing circuit 0. After receiving the input neuron data and the weight corresponding to the signal, it may be used to transmit it to the broadcast data reception interface di in the plurality of MLUs.

Accordingly, this embodiment provides a one-to-one connection of a machine learning unit and a unicast read processing circuit; One-to-one connection of the machine learning unit and the unicast recording processing circuit; And it is possible to implement customized one-to-one data operation processing through a one-to-one connection between the machine learning unit and the broadcast processing circuit, thereby reducing the complexity of the access logic of the data operation and reducing collisions, thereby improving processing efficiency.

In one selectable scheme, referring to FIG. 51, the number of interfaces in the transmission circuit is reduced based on FIG. 50. The unicast recording processing circuit and the broadcast processing circuit in the one processing circuit group share one shared signal reception interface in the corresponding interface group, and the shared signal reception interface corresponding to the processing circuit group is the processing circuit group. Is connected to the shared signal transmission interface of the machine learning unit corresponding to, and the unicast read signal receiving interface in the processing circuit group is connected to the unicast read signal transmission interface of the machine learning unit corresponding to the processing circuit group. 4, in the corresponding processing circuit group of MLU0, the unicast recording processing circuit and the broadcast processing circuit share one shared signal reception interface g0, and the shared signal reception interface g0 transmits a shared signal in MLU0. Receives a unicast record signal, broadcast and/or multicast signal transmitted from interface b0 and transmits a shared signal, and transmits a unicast record signal, broadcast and/or multicast signal to the unicast record processing circuit 0 And transmission to the broadcast processing circuit 0 for processing. As can be seen from this, in the transmission circuit, the unicast recording processing circuit i and the broadcast processing circuit i share a shared signal reception interface gi, and consist of a shared signal transmission interface bi in the MLUi and a shared signal reception interface ei in the transmission circuit. The data path may transmit unicast recording signals, broadcast and/or multicast signals between the MLUi and the transmission circuit.

As can be seen from this, since the unicast recording processing circuit and the broadcast processing circuit in one processing circuit group share a signal reception interface, the data processing apparatus according to the present embodiment includes a data operation signal transmission interface and a transmission circuit among machine learning units. By reducing the number of interfaces inside, additional hardware resources are saved, thus reducing the hardware area and power consumption.

In one embodiment, the shared signal receiving interface corresponding to the processing circuit group is connected to a unicast recording processing circuit and a broadcast processing circuit in the processing circuit group, respectively, and data operation transmitted from the shared signal transmission interface of the machine learning unit A signal is received, the data operation signal is divided into two identical data operation signals, and transmitted to the unicast recording processing circuit and the broadcast processing circuit, respectively. Referring to FIG. 51, taking the shared signal receiving interface g0 as an example, this is a unicast recording processing circuit by dividing a received data operation signal (unicast read signal, broadcast and/or multicast signal) into two identical data operation signals. 0 and broadcast processing circuit 0 are sent to process. For example, the shared signal reception interface may be connected to the unicast recording processing circuit 0 and the broadcast processing circuit 0 through a hardware circuit, so as to divide a data operation signal of one branch into two identical data operation signals. The data operation signal may be a high or low level signal.

Each processing circuit may analyze the data operation signal to determine the type of the data operation signal. For example, in the case of a unicast recording signal, the unicast recording processing circuit performs processing, and the broadcast processing circuit does not perform processing. For example, in the case of a broadcast and/or multicast signal, the broadcast processing circuit Performs processing, and the unicast recording processing circuit does not perform processing. Specifically, each processing circuit determines the type of the operation signal through the operation code of the data operation signal. For example, "write" means that the data operation signal is a unicast record signal, and "cast" is the data operation signal. Is a broadcast and/or multicast signal, and the operation signal type may be determined based on the number of machine learning units (data return targets) indicated in a marker field. For example, 0 return targets mean that the data operation signal is a unicast record signal, one return target means that the data operation signal is a unicast read signal, and multiple (less than n+1) return targets are It indicates that the data operation signal is a multicast signal, and n+1 return targets indicate that the data operation signal is a broadcast signal.

In one selectable scheme, referring to FIG. 52, the number of interfaces in the machine learning unit is further reduced on the basis of FIG. 51. When the machine learning unit performs a unicast read operation and a broadcast operation, it shares one data receiving interface on the machine learning unit. That is, the data returned by the unicast read processing circuit and the broadcast processing circuit in the processing circuit group share one shared data receiving interface in the machine learning unit. As compared with FIG. 51, taking MLU0 as an example, at least one receiving interface 142 includes one interface c0 and does not include the previous interface c0 and interface d0. The interface c0 of FIG. 52 is connected to the interface f0 of the processing circuit to receive the unicast read data returned by the unicast read processing circuit 0, and is connected to a plurality of interfaces ii of the processing circuit to receive a plurality of broadcast processing circuits i. Broadcast and/or multicast data returned by may be received.

Accordingly, in the data processing apparatus according to the present embodiment, when at least one machine learning unit performs a unicast read operation and a broadcast operation, it shares one data reception interface in the machine learning unit. By further reducing the number of interfaces for data to be processed, hardware resources are saved, thereby reducing hardware area and power consumption.

Further, referring to FIG. 53, the number of interfaces of the transmission circuit is further reduced on the basis of FIG. 52. The unicast read processing circuit and the broadcast processing circuit in the one processing circuit group share one shared data transmission interface in the corresponding interface group, and the shared data transmission interface corresponding to the processing circuit group is the processing circuit group. The corresponding machine learning unit is connected to the shared data receiving interface. As compared with FIG. 52, in the processing circuit group corresponding to MLU0, the unicast read processing circuit and the broadcast processing circuit share one shared data transmission interface i0, and the interface i0 in FIG. 53 is a unicast processing circuit. It is connected to the read processing circuit 0 and can receive the unicast read data returned by the unicast read processing circuit 0, and is connected to a plurality of broadcast processing circuits i in the processing circuit and the broadcast returned by the plurality of broadcast processing circuits i. Cast and/or multicast data may be received.

Therefore, in the data processing apparatus according to the present embodiment, since the unicast read processing circuit and the broadcast processing circuit in one processing circuit group share one shared data transmission interface in the corresponding interface group, the machine learning unit returns In addition to further reducing the number of data interfaces, hardware resources are saved, thereby reducing hardware area and power consumption.

In one selectable method, referring to FIG. 54, on the basis of FIG. 53, at least one or more transmission interfaces of the machine learning unit, because at least one computing unit of the non-shared data return interface exists in the machine learning unit, At least one or more independent data receiving interfaces may be further included. The independent data receiving interface is connected to one operation unit in the machine learning unit, the second transmission interface further comprises an independent data transmission interface connected to the independent data receiving interface, and the operation unit is the independent A connection with a processing circuit group corresponding to the machine learning unit is implemented through the in-data reception interface and the independent data transmission interface. For example, referring to FIG. 54, in the case of MLU0, a plurality of calculation units are included, at least one of them is connected to interface j0, and other calculation units are connected to interface c0, respectively. That is, interface c0 is a shared data reception interface shared by the other operation unit, and interface j0 is an independent data reception interface. Accordingly, the second transmission interface 120 further includes an independent data transmission interface h0 connected to the interface j0. In FIG. 54, the independent data transmission interface h0 is connected to the unicast read processing circuit 0 and a plurality of broadcast processing circuits i to receive unicast read data, broadcast and/or multicast data, and receive independent data. It is transmitted to the operation unit of the non-shared data return interface through the interface j0.

In one selectable scheme, referring to FIG. 55, on the basis of FIG. 53, each machine learning unit may share one broadcast processing circuit in the processing circuit. The shared broadcast processing circuit may be connected to each shared signal receiving interface gi and each shared data transmitting interface ii. Therefore, in the data processing apparatus according to the present embodiment, each machine learning unit can share one broadcast processing circuit in the processing circuit, thereby reducing the number of broadcast processing circuits, simplifying the transmission circuit, and reducing the area of the hardware. And reduced power consumption.

As information technology is constantly evolving and improving day by day, the demands of people for data access and data processing are increasing, and the demands for processors that process and access some data are also becoming increasingly stringent. Taking a general-purpose processor as an example, a multi-core processor composed of a plurality of general-purpose processor cores (eg, CPU cores) has become a major trend due to its strong parallel computing power.

However, as the current machine learning algorithms are constantly evolving, more and more machine learning chips are created. However, these machine learning chips have the same problem when accessing or processing data stored in shared storage. It reduces data processing efficiency during learning.

Therefore, simplifying the data access logic of machine learning chips is an urgent technical challenge for the current technician.

In order to solve the above problem, the present application provides the following technical solutions.

First, a data processing device used in this application is introduced. Referring to FIG. 56, a data processing device that can be implemented through a combination of hardware or software is provided. The data processing device is used to process machine learning data. As shown in Fig. 56, the data processing device includes a machine learning device 11, a transmission circuit 12, and a shared memory 13, and the machine learning device 11 includes a first transmission interface 14. The transmission circuit 12 is connected to the transmission circuit 12 through the transmission circuit 12 is connected to the shared memory 13.

The transmission circuit 12 obtains input data required for the machine learning device 11 from the shared memory 13 according to a data operation signal sent from the machine learning device, and then returns the input data to the machine learning device 11. Here, the data operation signal indicates an operation method for data in the shared memory 13.

Alternatively, the machine learning apparatus 11 obtains output data by performing a machine learning operation according to input data. Alternatively, the machine learning device 11 transmits and stores the output data to the shared memory 13 through the transmission circuit 12. Specifically, when the machine learning apparatus 11 performs a neural network operation, the machine learning apparatus 11 obtains output neuron data by performing an artificial neural network operation according to the input neuron data and weights, and also obtains the output neuron data. It is used as input neuron data and is transmitted to the shared memory 13 through the transmission circuit 12 and stored. Note that the machine learning apparatus 11, the transmission circuit 12, the shared memory 13, and the first transmission interface 14 may all be implemented in a manner using a hardware circuit. For example, the transmission circuit 12 may be a broadcast bus, and the shared memory 13 may be a non-volatile and/or volatile memory, and may include a random access memory (RAM), a cache memory, etc. Including, but not limited to. The first transmission interface 14 may correspond to one or more data I/O (in/out, read-in-lead-out) interfaces or I/O pins.

Alternatively, the machine learning device 11 may include one first transmission interface 14 and may include a plurality of first transmission interfaces. The first transmission interface 14 may be a transmission interface or a reception interface. When the first transmission interface 14 is a transmission interface, the machine learning apparatus 11 may transmit a data operation signal or data to the transmission circuit 12 connected to the transmission interface. When the first transmission interface 14 is a reception interface, the machine learning apparatus 11 may receive data returned by the transmission circuit 12.

Here, the data operation signal indicates an operation method for data in the shared memory 13. In one selectable scheme, specifically, the data operation signal may indicate a read operation for data in the shared memory 13 and may indicate a write operation for data in the shared memory 13. Therefore, when the data operation signal transmitted from the machine learning device 11 is a read operation, the transmission circuit 12 finds and reads data corresponding to the corresponding address from the shared memory 13, and then reads at least one or more data. It returns to the machine learning device 11. When the data operation signal transmitted from the machine learning device 11 is a write operation, the transfer circuit 12 writes the output write data of the machine learning device 11 to the shared memory 13.

The input data is data to be input to the machine learning device 11 during a machine learning operation. The abnormal data may be initial data previously stored in the shared memory 13, may be an intermediate result or a final result output when the machine learning device 11 performs a machine learning operation, and is recorded in the shared memory 13 It could be data.

Alternatively, the input data may include input neuron data and/or weight, and the input neuron data and weight may include data required to be input to the machine learning apparatus when performing artificial neural network operation. Accordingly, the output data may include output neuron data, and the output neuron data is an intermediate result or a final result of a machine learning device when an artificial neural network operation is performed.

Note that the data processing device used in the present application may be at least one or more of the following structure types. That is, the machine learning device 11 is connected to one transmission circuit 12 through a plurality of first transmission interfaces 14 and then connected to one shared memory 13 through the one transmission circuit 12. The data can be obtained. Alternatively, the machine learning device 11 is also connected to a plurality of transmission circuits 12 via a plurality of first transmission interfaces 14 and then to one shared memory 13 via these transmission circuits 12. To obtain the data. Alternatively, the machine learning device 11 is also connected to one transmission circuit 12 via a plurality of first transmission interfaces 14 and then a plurality of shared memories 13 via the one transmission circuit 12. It can also be connected to to obtain the data.

Alternatively, when the machine learning device 11 performs an artificial neural network operation, the machine learning device 11 can calculate the neuron data output by the neural network of each layer regardless of whether it is a forward or reverse operation for a multilayer neural network operation. I can. Specifically, a set of operations required for a series of artificial neural network operations, such as multiplication operation, summation operation, convolution operation, and activation operation, for a plurality of input neuron data and weights corresponding to the neural network input terminals of each layer can be performed to obtain the operation result. After obtaining the output neuron data of the current layer through the artificial neural network operation, the machine learning apparatus 11 performs the artificial neural network operation again by using the output neuron data as input neuron data of the next layer neural network. Before doing this, first, through the transmission circuit 12, the output neuron data of the current layer can be recorded and stored in the shared memory 13, through which the machine learning device 11 can read the data in real time. The next layer of artificial neural network operation can be conveniently performed.

The data processing device for performing the machine learning operation according to the above embodiment includes a machine learning device, a transmission circuit connected through a first transmission interface of the machine learning device, and a shared memory connected to the transmission circuit. Here, the transmission circuit acquires input data necessary for the machine learning apparatus from the shared memory by the data operation signal transmitted from the machine learning apparatus, and then returns the input data to the machine learning apparatus. In the process of manipulating the data, in consideration of the situation in which a lot of data is shared when performing a machine learning operation, the data processing device used in the present application installs a corresponding transmission circuit to allow the machine learning device to read data from the shared memory. And implement to record. Conventional CPUs have the following problems in the process of directly accessing data in memory. In other words, when the CPU performs parallel operations, the parallel data access logic is complicated, and blockage and deadlock easily occur. Compared to the conventional CPU, the data processing device used in the present application simplifies the logic for the machine learning device to access data in the shared memory and improves the data access efficiency, thereby further improving the machine learning operation speed of the machine learning device.

56A is a block diagram of a machine learning apparatus according to an embodiment of the present application. Based on the above embodiment, referring to FIG. 56A, the machine learning apparatus 11 includes at least one or more calculation units 151, and at least one machine including a controller unit 152 connected to the calculation unit 151. It includes a learning unit (15). The operation unit 151 includes one mast processing circuit 151a and a plurality of slave processing circuits 151b, and the operation unit 151 is connected to the transmission circuit 12 through the first transmission interface 14. .

Here, the controller unit 152 transmits a data operation signal and output data to the transmission circuit 12 through a transmission interface in the first transmission interface 14, and through a reception interface in the first transmission interface 14. The transmission circuit 12 receives the input data obtained from the shared memory 13, and also transmits the input data to the mast processing circuit 151a and/or the slave processing circuit 151b. The mast processing circuit 151a distributes the input data to a plurality of slave processing circuits 151b. The plurality of slave processing circuits 151b perform intermediate operations in parallel according to the data transmitted from the mast processing circuit 151a to obtain a plurality of intermediate results, and transmit the plurality of intermediate results to the mast processing circuit 151a. The mast processing circuit 151a also performs subsequent processing on a plurality of intermediate results to obtain calculation results.

Alternatively, the machine learning device 11 may include one machine learning unit 15, and the machine learning device 11 may be applied when performing an artificial neural network operation. When the number of neurons included in the related neural network structure is small, the entire neural network may be computed at once using one machine learning unit 15. The detailed operation process is as follows. The machine learning unit 15 obtains output neuron data corresponding to each layer neuron by performing an artificial neural network operation according to the input neuron data and weight corresponding to each layer neuron of the neural network, and the operation of the entire neural network is completed to obtain a final operation result. Until then, the output neuron data can be used as new input neuron data and the next layer neural network operation can be performed again. In this process, the machine learning device 11 may transmit and store the output neuron data or the final operation result obtained by the machine learning unit 15 for each layer through the transmission circuit 12 to the shared memory 13. .

Alternatively, the machine learning device 11 may include a plurality of machine learning units 15, and the machine learning device 11 includes the number of neurons included in a related neural network structure when performing an artificial neural network operation. Can be applied in many cases. For example, the operation of a multilayer neural network will be described by taking one layer neural network operation among forward operations as an example. When the number of neurons in the layer is large, the machine learning device 11 can calculate the output neuron data of the partial neurons of the higher network in parallel by using a plurality of machine learning units 15 among them according to a selectable calculation method. have. For example, if there are 4 machine learning units 15 in one machine learning device 11 and 100 neurons in the first-layer neural network, the machine learning device 11 will have 25 of them per machine learning unit 15 Each neuron can be allocated to be processed and configured to output corresponding output neuron data. In this way, parallel calculations are performed on the neural network of one layer and another. This calculation method can improve processing efficiency because it enables parallel processing of neural network calculations.

Alternatively, the controller unit 152 in the machine learning unit 15 may include one instruction storage unit 152a and one instruction processing unit 152b. Alternatively, the controller unit 152 may also include a plurality of instruction storage units 152a and a plurality of instruction processing units 152b.

Here, the instruction storage unit 152a, when the machine learning unit 15 performs the machine learning operation, all relevant operation instructions and a corresponding data read/write operation instruction when the data read/write operation is to be performed. Used to store. Here, the instruction processing unit 152b is used to process all instructions in the instruction storage unit 152a, and may specifically include the following operations. That is, the operation instruction in the instruction storage unit 152a is sent to the operation unit 151 so that the operation unit 151 performs a corresponding operation operation according to the operation instruction, and the data read/write operation in the instruction storage unit 152a After analyzing the command to obtain a data operation signal, the data operation signal is transmitted to the first transmission interface 14 so that the first transmission interface 14 reads and writes data from the shared memory 13 through the data operation signal. You can do it.

Alternatively, the operation unit 151 in the machine learning unit 15 may include one mast processing circuit 151a and one slave processing circuit 151b. Alternatively, the calculation unit 151 may also include one mast processing circuit 151a and a plurality of slave processing circuits 151b. This structural design is suitable for handling large data volumes. In particular, it is suitable for use when a large amount of parallel operations must be performed in a machine learning operation process. Therefore, this operation structure according to the present application can reduce power consumption by increasing the operation speed and saving the operation time.

Note that each slave processing circuit 151b in the above structure can directly perform a parallel operation according to the input data transmitted from the mast processing circuit 151a. Alternatively, each slave processing circuit 151b may also directly perform a parallel operation according to the input data transmitted from the controller unit 152.

When there is one mast processing circuit 151a and a plurality of slave processing circuits 151b in each of the calculation units 151, the mast processing circuit 151a and a plurality of slave processing circuits 151b in the calculation unit 151 The structure of) can be the same or different. Specifically, a structure of a mast processing circuit 151a and a plurality of slave processing circuits 151b, which may include one of H-type, systolic array type, and tree-type structure, may be included.

The machine learning apparatus according to the above embodiment includes at least one machine learning unit, and each machine learning unit includes at least one calculation unit and a controller unit connected to the calculation unit. At the same time, the arithmetic unit includes one mast processing circuit and a plurality of slave processing circuits, and is connected to the transmission circuit through a first transmission interface. The controller unit in the machine learning apparatus transmits a data operation signal and output data to a transmission circuit through a transmission interface in a first transmission interface, and input data obtained by the transmission circuit from a shared memory through a reception interface in the first transmission interface. After receiving, the input data can be transmitted to the mast processing circuit and/or the slave processing circuit. Since the machine learning device includes a mast processing circuit and a plurality of slave processing circuits, the mast processing circuit distributes the acquired data to a plurality of slave processing circuits at the same time so that the plurality of slave processing circuits perform parallel calculations, and the intermediate calculation results are obtained. The machine learning operation can be implemented by returning to the mast processing circuit and allowing the mast processing circuit to calculate an intermediate result. Compared to a traditional method in which only one processing circuit among processors used for machine learning operations operates on data, the machine learning apparatus according to the present application has a high data operation and data operation speed.

57 is a configuration diagram of a transmission circuit according to an embodiment of the present application. Referring to FIG. 57, the transmission circuit 12 includes a second transmission interface 120, at least one read/write processing circuit 121 connected to the second transmission interface 120, and a read/write processing circuit 121. ), the at least one machine learning unit 15 is connected to the second transmission interface 120 through the first transmission interface 14, and at least one machine learning unit 15 And implement the connection between the transmission circuit 12.

The read/write processing circuit 121 receives the data operation signal transmitted by the at least one machine learning unit 15 through the first transmission interface 14 and the second transmission interface 120 and then transmits the data operation signal. It is used to transmit to the arbitration circuit 122 and to transmit data read from the shared memory 13 to at least one machine learning unit 15 through the second transmission interface 120. The arbitration circuit 122 arbitrates the data operation signals received from at least one read/write processing circuit 121 according to a pre-set arbitration rule, and operates the data in the shared memory 13 according to the data operation signal successfully arbitrated. It is used to order.

Alternatively, the transmission circuit 12 may include a plurality of second transmission interfaces 120, and the second transmission interface 120 may be a transmission interface or a reception interface. When the second transmission interface 120 is a transmission interface, the transmission circuit 12 transmits data to the machine learning unit 15 connected to the transmission interface, and when the second transmission interface 120 is a reception interface, transmission The circuit 12 may receive a data operation signal and/or data transmitted by the machine learning unit 15 to the receiving interface. Alternatively, the transmitting interface of the second transmitting interface 120 is connected to a receiving interface in the first transmitting interface 14, and the receiving interface of the second transmitting interface 120 is a transmitting interface in the first transmitting interface 14. Is connected to

Alternatively, referring to FIG. 57A, the transmission circuit 12 may include a plurality of read/write processing circuits 121, and the input terminals of the plurality of read/write processing circuits 121 2 It may be connected in correspondence with the transmission interface 120. Alternatively, referring to FIG. 2B, the transmission circuit 12 may include only one read/write processing circuit 121, and the input terminal of the one read/write processing circuit 121 is 2 Can be connected to the transmission interface 120 in a one-to-many (一對多). That is, one read/write processing circuit 121 is connected to a plurality of second transmission interfaces 120.

Alternatively, when the plurality of read/write processing circuits 121 are connected in a one-to-one correspondence with the plurality of second transmission interfaces 120, each read/write processing circuit 121 is 2 Data may be transmitted to one machine learning unit 15 through the transmission interface 120, or data may be transmitted to a plurality of machine learning units 15 through a second transmission interface 120 connected thereto. have. When the one read/write processing circuit 121 is connected to a plurality of second transmission interfaces 120 in a one-to-many manner, the read/write processing circuit 121 includes a plurality of second transmission interfaces 120 connected thereto. The data may be transmitted to the plurality of machine learning units 15 through or may be transmitted to one machine learning unit 15 through one of the second transmission interfaces 120.

Alternatively, one arbitration circuit 122 is included in the structure of the transmission circuit 12, and an input terminal of the one arbitration circuit 122 may be connected to a plurality of read/write processing circuits 121. The output terminal of the arbitration circuit 122 may be connected to the shared memory 13 or may be connected to another memory device or a control device.

As can be seen from the above embodiment, the transmission circuit 12 used in the present application may include a plurality of read/write processing circuits 121. The types of the plurality of read/write processing circuits 121 may be the same or different. In the following embodiment, a data transmission method will be developed and described according to the type of the read/write processing circuit 121 and the type of the data signal received by the read/write processing circuit 121.

Specifically, the read/write processing circuit 121 may include one or more of processing circuits such as a unicast read processing circuit, a unicast write processing circuit, and a broadcast processing circuit. The data operation signal includes one or more of a unicast read request, a unicast write request, a unicast read command, a unicast write command, a multicast command, and a broadcast command.

Here, the unicast type processing circuit processes a unicast type signal. For example, the unicast read processing circuit of the above embodiment may process a corresponding unicast read request or a unicast read command, and the unicast write processing circuit may process a corresponding unicast write request or a unicast write command. I can. Therefore, multicast and broadcast type processing circuits process signals of multicast or broadcast type. For example, the broadcast processing circuit of the embodiment may process a corresponding multicast command or broadcast command.

Note that, when the data operation signal is an imperative signal such as a unicast read command, a unicast write command, a multicast command, or a broadcast command of the present embodiment, the read/write processing circuit 121 specifically analyzes the command type signal A request type signal is generated, and the request type signal is transmitted to the arbitration circuit 122. When the data operation signal is a request type signal such as a unicast read request or a unicast write request of the present embodiment, the read/write processing circuit 121 performs a temporary storage operation on the request type signal to store the request type signal. It transmits to the arbitration circuit 122.

Alternatively, when the data operation signal is a multicast command, if the multicast command includes identifiers of a plurality of target machine learning units to receive data, the read/write processing circuit 121 in the transmission circuit 12 When receiving one multicast command, the read/write processing circuit 121 identifies a plurality of target machine learning units according to the identifiers in the multicast command and returns to the finally identified target machine learning units. You can send the necessary data.

Alternatively, when the data operation signal is a broadcast command, the broadcast command may not include one identifier of the target machine learning unit receiving the data. However, when the read/write processing circuit 121 receives one broadcast command, the read/write processing circuit 121 transfers the data obtained by the arbitration circuit 122 from the shared memory 13 to the machine learning device 11 ) Can be transmitted to all the machine learning units 15 included in).

Alternatively, the arbitration rule set in advance allows the arbitration circuit 122 to determine the priority of a plurality of data operation signals according to a predetermined rule, so that the arbitration circuit 122 is based on the priority of each data operation signal. It is possible to determine the data operation signal for which the arbitration was successful. For example, if the transmission speed of the data operation signal transmitted by the 1# read/write processing circuit 121 is higher than the transmission speed of the data operation signal transmitted by the 2# read/write processing circuit 121, the arbitration circuit 122 ) Can set the priority of a data operation signal with a high transmission rate to a high priority, and can set the priority of a data operation signal with a low transmission rate to a low priority. Then, the arbitration circuit 122 selects a data operation signal of a high priority according to the priority and performs the next operation, but obtains data from the shared memory 13 according to the data operation signal.

In the above embodiment, the transmission circuit includes a second transmission interface, at least one read/write processing circuit connected to the second transmission interface, and an arbitration circuit connected to the read/write processing circuit, and receives data read from the shared memory. 2 Transmits to at least one machine learning unit through a transmission interface. Here, the read/write processing circuit receives the data operation signal transmitted by the at least one machine learning unit through the first transmission interface and the second transmission interface and transmits the data operation signal to the arbitration circuit. Therefore, the arbitration circuit arbitrates the data operation signals received from the at least one read/write processing circuit according to a pre-set arbitration rule, and operates the data in the shared memory by the data operation signal successfully arbitrated. In the transmission circuit, a plurality of read/write processing circuits are connected to the machine learning device through a plurality of second transmission interfaces, and mediation is performed by an arbitration circuit, so that data can be effectively transmitted, so that the machine learning device can transmit a plurality of data operation signals. It is possible to prevent data collisions and clogging that occur when simultaneously transmitting data. On the other hand, the transmission circuit of the present embodiment can process various types of commands or requests, and thus the scope of application of the data processing device is greatly expanded.

In one embodiment, the data processing apparatus of the present embodiment may be divided into at least one or more clusters, each cluster including a plurality of machine learning units 15, one transmission circuit 12, and at least one shared memory 13 do. When a plurality of clusters exist, referring to FIG. 58, the transfer circuit 12 includes a first type of direct memory access controller DMA123 connected to the arbitration circuit 122 in the cluster to which it belongs and the shared memory 13 in the cluster. And/or a second type DMA124 connected to the arbitration circuit 122 in the cluster to which it belongs and the shared memory 13 in the other cluster.

The first type of DMA123 controls data interaction between the arbitration circuit 122 in the cluster and the shared memory 13 in the cluster. The second type of DMA124 controls data interaction between the arbitration circuit 122 in the cluster and the shared memory 13 in another cluster, and controls the data interaction between the arbitration circuit 122 and the off-chip memory in the cluster. .

Alternatively, DMA123 of the first type and DMA124 of the second type mainly control the arbitration circuit 122 to be connected to at least one or more shared memories 13, and rapidly retrieve data from the connected at least one shared memory 13 It plays the role of reading or writing it properly.

When a first type of DMA123 or a second type of DMA124 is present in the transmission circuit, referring to FIG. 59, the transmission circuit 12 used in the present application is a first selective transmission circuit 125 connected to the first type of DMA123. , May further include a second selective transfer circuit 126 connected to a second type of DMA. Here, the first selection transfer circuit 125 is selectively connected to the shared memory 13 in the cluster to which it belongs. The second selective transfer circuit 126 is selectively connected to the shared memory 13 and off-chip memory in the cluster to which it belongs and other clusters.

Alternatively, the first selection transmission circuit 125 and the second selection transmission circuit 126 may be a type of circuit such as a crossbar switch, a diverter switch, etc., and an on/off current or an on/off current It may be a circuit that controls the connection between each circuit by setting the /off signal. This embodiment does not limit this.

Alternatively, referring to FIG. 60, when the transfer circuit 12 writes data to the shared memory 13 or the shared memory 13 returns the read data to the transfer circuit 12, the transfer circuit 12 ) First waits for processing by temporarily storing the data to be recorded or the data to be returned. Accordingly, the transmission circuit 12 used in the present application due to this demand further includes an arbitration circuit 122 and a caching circuit 127 connected to the shared memory 13. The caching circuit 127 is configured to temporarily store the data obtained by the arbitration circuit 122 from the shared memory 13 and temporarily store the data written by the arbitration circuit 122 to the shared memory 13.

Alternatively, the caching circuit 127 provides a buffer for data exchange, and the caching circuit may be a random access memory (RAM), which belongs to the prior art and is not described herein again.

In the data processing apparatus used in the present application, the data transmission bandwidth between each circuit may be different. Alternatively, the transmission bandwidth between the transmission circuit 12 and the shared memory 13 is greater than the transmission bandwidth between the transmission circuit 12 and the machine learning unit 15.

For example, in one machine learning device 11, N (N is an integer greater than or equal to 1) number of machine learning units 15, one transmission circuit 12, and one shared memory 13 Including, and assuming that the bandwidth from the transmission circuit 12 to each machine learning unit 15 is M, the bandwidth from the broadcast processing circuit in the transmission circuit 12 to the shared memory 13 is M*N Can be set to The advantage of this design is that collisions can be avoided even in extreme situations. For example, when a plurality of machine learning units 15 simultaneously transmit a broadcast command to the transmission circuit 12, the arbitration circuit 122 in the transmission circuit 12 sequentially sends the command to the shared memory 13 When transmitting, there is sufficient bandwidth so no collision occurs. Meanwhile, the arbitration circuit 122 in the transmission circuit 12 selects and processes one broadcast command having a high priority according to a preset arbitration rule, and then waits for the return of data from the shared memory 13 to be One broadcast command can continue to be processed. This design speeds up data processing time and allows efficient use of the bandwidth to transmit data. Note that in the actual circuit design, the bandwidth between the transmission circuit 12 and the shared memory 13 may be 2 times, 4 times, 6 times the bandwidth between the transmission circuit 12 and each machine learning unit 15, etc. In addition, a condition larger than the bandwidth between the transmission circuit 12 and each machine learning unit 15 may be satisfied. This embodiment does not limit this.

Currently, as artificial neural networks continue to develop, more and more multi-architecture machine learning chips are gradually being released. When such a machine learning chip accesses or processes data in a shared memory, a large amount of data is required, and the demand for data processing speed is high. Therefore, in the process of accessing or operating data, the number of hardware is often increased to increase the data processing speed by increasing the bandwidth of data transmission, thus satisfying the characteristics of machine learning chips that require high data processing speed.

However, when the machine learning chip accesses or operates data using the above method, hardware overhead is large and hardware redundancy may be caused.

In order to solve the above problem, the present application provides the following technical solutions.

The data processing apparatus according to the embodiment of the present application may be implemented through software, hardware, or a combination thereof. The data processing device may be a part or the whole shown in FIG. 43. The data processing device processes machine learning data, and includes a machine learning device 11, a transfer circuit 12 and a shared memory 13, and the transfer circuit 12 includes a plurality of read/write processing circuits 121 And one arbitration circuit 122, and the arbitration circuit 122 arbitrates on the data operation signals transmitted from the plurality of machine learning units 15, and then from the shared memory 13 by the data operation signal successfully arbitrated. Input data necessary for the machine learning device 11 is acquired. The read/write processing circuit 121 determines a target machine learning unit or a target operation unit among a plurality of machine learning units according to the address information included in the arbitration successful data operation signal or the type of the data operation signal, and sets the input data to target machine learning. Return to the unit or target operation unit. The machine learning device 11 includes a plurality of machine learning units 15, each machine learning unit 15 includes at least one computational unit 151, and the plurality of machine learning units includes a first transmission interface. It is connected to the transmission circuit 12 through 14, and the transmission circuit 12 is connected to the shared memory 13.

Alternatively, the machine learning apparatus 11 may obtain output data by performing a machine learning operation based on input data. Alternatively, the machine learning device 11 may transmit and store output data to the shared memory 13 through the transmission circuit 12. Specifically, when the machine learning apparatus 11 performs a neural network operation, the machine learning apparatus 11 obtains output neuron data by performing an artificial neural network operation based on input neuron data and weights, and new output neuron data. As input neuron data, it is transmitted to the shared memory 13 through the transmission circuit 12 and stored.

Note that the machine learning unit, the transmission circuit, the shared memory, and each type of interface can all be implemented in a manner by a hardware circuit. Illustratively, the transmission circuit may be a broadcast bus. The shared memory may be non-volatile and/or volatile memory, and includes, but is not limited to, random access memory (RAM), high-speed cache memory, and the like. Various interfaces may correspond to one or a plurality of data I/O (in/out, read-in, read-out) interfaces or I/O pins.

Referring to FIG. 61, in one selectable method, the machine learning device 11 may include a plurality of machine learning units 15. In the calculation of a multilayer neural network, the calculation of a single layer neural network during forward operation will be described as an example. In an embodiment, the machine learning device may concurrently calculate output neuron data of all neurons in a corresponding layer among a neural network through a plurality of machine learning units (MLUs). For example, if the machine learning device includes 4 machine learning units and the neural network of the corresponding layer has 100 neurons, each machine learning unit can be allocated to process 25 neurons, and a corresponding operation instruction It can be implemented by setting In the above process, each machine learning unit obtains input neuron data and weights respectively corresponding to the 25 neurons of the layer distributed from the shared memory through the transmission circuit, and output neuron data of the 25 neurons of the layer distributed. Is calculated, and output neuron data of 25 neurons of the layer distributed through the transmission circuit can be transmitted and stored in the shared memory. When each of the machine learning units processes data of a plurality of neurons in a corresponding layer, it can be understood that they can be processed through parallel calculation. In this way, since the computation of the neural network of one layer and another layer is performed in parallel, it is possible to implement parallel processing of the neural network computation, thereby improving processing efficiency.

When a plurality of machine learning units 15 simultaneously transmit data operation signals to the transmission circuit 12 via the first transmission interface 14, the read/write processing circuit 121 via the first transmission interface 14 Data operation signals can be transmitted. The read/write processing circuit 121 may be a single read/write processing circuit, or may be a plurality of read/write processing circuits. When the read/write processing circuit 121 is a plurality of read/write processing circuits, one machine learning unit 15 can correspond to one read/write processing circuit, and one machine learning unit 15 It can also correspond to a read/write processing circuit. When the read/write processing circuit 121 transmits the data operation signal to the arbitration circuit 122, the arbitration circuit 122 arbitrates a plurality of data operation signals and operates the data from the shared memory 13 according to the data operation signal successfully arbitrated. A machine learning unit corresponding to the signal acquires necessary input neuron data and weights. The read/write processing circuit 121 determines a target machine learning unit or a target operation unit according to the address information included in the data operation signal or the type of the data operation signal, and calculates the input neuron data and the weight as the target machine learning unit or target operation. Return to unit.

Exemplarily, the machine learning apparatus includes four machine learning units of machine learning unit 0, machine learning unit 1, machine learning unit 2, and machine learning unit 3, which are read/write processing circuit 0, read/write processing circuit 1, the read/write processing circuit 2 and the read/write processing circuit 3 correspond to four read/write processing circuits, respectively. Here, the machine learning unit 0, the machine learning unit 1, the machine learning unit 2, and the machine learning unit 3 are read/write processing circuit 0, read/write processing circuit 1, and read/write processing circuit through the first transmission interface 14. 2 and the read/write processing circuit 3 transmit a data operation signal. Specifically, the read/write processing circuit 0 operates the data operation signal 0, the read/write processing circuit 1 sends the data operation signal 1, the read/write processing circuit 2 sends the data operation signal 2, and the read/write processing circuit 3 operates the data. Signal 3 can be transmitted. Read/write processing circuit 0, read/write processing circuit 1, read/write processing circuit 2 and read/write processing circuit 3 mediate data operation signal 0, data operation signal 1, data operation signal 2, and data operation signal 3, respectively. It is sent to the circuit 122 to proceed with arbitration, and the arbitration circuit 122 arbitrates a plurality of data operation signals to determine the data operation signal 2 as a successful data operation signal, and according to the data operation signal 2, the shared memory 13 ) To obtain input neuron data and weights. The read/write processing circuit 2 sets the machine learning unit 1 and the machine learning unit 2 to the target machine by address information included in the data operation signal 2, for example, address information including the addresses of the machine learning unit 1 and the machine learning unit 2. After determining as a learning unit, input neuron data and weights are returned to the machine learning unit 1 and the machine learning unit 2 according to the data operation signal 2.

In another selectable method, the machine learning apparatus may calculate output neuron data of all neurons in each layer of the neural network according to a predetermined order of precedence and subsequent steps using a plurality of machine learning units. In the above process, the immediately preceding machine learning unit can transmit and store the output neuron data of all neurons in the layer through the transmission circuit to the shared memory, so the next machine learning unit extracts the output neuron data of all neurons in the layer It can be calculated using the layer's input neuron data. It can be understood that the above application is applied to the calculation of a neural network with a small number of neurons in each layer, for example, when the amount of computation of each layer neural network is not large.

The machine learning unit 15 will be described in detail with reference to FIG. 62. In one approach, the machine learning unit 15 includes at least one or more computation units 151 and a controller unit 152 connected to the computation unit 151. The operation unit 151 includes one mast processing circuit 151a and a plurality of slave processing circuits 151b, and the operation unit 151 is connected to the transmission circuit 12 through the first transmission interface 14. .

The controller unit 152 transmits a data operation signal and output neuron data to the transmission circuit 12 through the first transmission interface 14, and the transmission circuit 12 through the first transmission interface 14 is a shared memory After receiving the input neuron data and the weight obtained from (13), the input neuron data and the weight are transmitted to the mast processing circuit 151a and/or the slave processing circuit 151b.

The mast processing circuit 151a distributes input neuron data and weights to a plurality of slave processing circuits 151b. The plurality of slave processing circuits 151b obtain a plurality of intermediate results by performing intermediate operations in parallel according to neuron data and weights, and then transmit the plurality of intermediate results to the mast processing circuit 151a. The mast processing circuit 151a also performs subsequent processing including an activation operation on a plurality of intermediate results to obtain a calculation result. Specifically, the controller unit 152 further obtains a calculation command, analyzes the calculation command, obtains a plurality of calculation commands, and transmits the plurality of calculation commands to the mast processing circuit.

In this embodiment, it can be understood that the machine learning unit includes a plurality of computation units, and each computation unit can transmit or receive data through the first transmission interface.

For example, in one selectable technical solution, the mast processing circuit may further include one controller unit, and the controller unit may include a master instruction processing unit, and specifically, an operation instruction is a micro instruction. To decode. In another selectable technical solution, the slave processing circuit may include another controller unit, and the other controller unit includes a slave command processing unit, and specifically receives and processes micro instructions. The micro-instruction may be a next level instruction of the instruction. The micro-command may be obtained by dividing or decoding the command, and may be further decoded as a control signal of each member, each unit, or each processing circuit. For example, the multiplication micro instruction is the next level instruction of the convolution instruction.

A data processing device for processing machine learning data according to the present embodiment includes a machine learning device, a transmission circuit, and a shared memory. The transmission circuit includes a plurality of read/write processing circuits and one arbitration circuit. The machine learning apparatus includes a plurality of machine learning units, each machine learning unit includes at least one computational unit, and the plurality of machine learning units are connected to a transmission circuit through a first transmission interface, and the transmission circuit is shared. Connected to memory. In this embodiment, the data processing apparatus mediates data operation signals transmitted by a plurality of machine learning units through the mediation circuit, and obtains input neuron data and weights required for the machine learning apparatus from the shared memory based on the mediation result. Therefore, when a data processing device operates data, a plurality of machine learning units perform data operations on the shared memory through one transmission circuit, and mediate a plurality of data operation signals through an arbitration circuit to reduce hardware overhead. It prevents blocking of the data operation signal.

In one embodiment, with continued reference to FIG. 61, the read/write processing circuit includes any one of a unicast read processing circuit and a broadcast processing circuit, and the data operation signal is a unicast read request, a unicast read command. , Multicast command, or broadcast command. Here, the unicast type processing circuit processes a unicast type signal, and the broadcast type processing circuit processes a multicast or broadcast type signal.

Illustratively, the unicast read instruction is an instruction to read input neuron data and weights of the source address of the shared memory, transmitted from a machine learning unit. Input neuron data and weights may be returned to the machine learning unit through a unicast read command, and the input neuron data and weights are calculated by the machine learning unit in the process of calculating neurons distributed to one layer according to a calculation instruction. It may be input neuron data and weight required for the distributed neurons. The broadcast instruction is an instruction for reading input neuron data and weights corresponding to a source address from a shared memory transmitted from a machine learning unit. The input neuron data and weights can be returned to all machine learning units in the machine learning device through a broadcast command, and the input neuron data is input neuron data required for all neurons in a certain layer, that is, all outputs of the previous layer. It may be neuron data, and the weight is a reusable weight such as a convolution kernel. The difference between the multicast command and the broadcast command is that the data of the multicast command is returned not all machine learning units in the machine learning device, but a plurality of machine learning units corresponding to a marker field in the multicast command. will be. On the other hand, in general, the difference between a command and a request is that the overhead according to the command execution is relatively large, but a lot of information is included in the command, and the overhead according to the request is relatively small, but the information included in the request is small. will be.

In this embodiment, the data processing apparatus mediates the data operation signals transmitted from the plurality of machine learning units through the mediation circuit, and acquires input neuron data and weights required for the machine learning apparatus from the shared memory based on the mediation result. Therefore, when a data processing apparatus operates data, a plurality of machine learning units manipulate data on a shared memory through a single transmission circuit, thereby reducing hardware overhead and preventing redundancy. A detailed process of determining the priority of data operation signals transmitted from the plurality of read/write processing circuits by the arbitration module through the following embodiments will be described in detail.

In one embodiment, the arbitration circuit 122 specifically determines the priority of the data operation signals transmitted from the plurality of read/write processing circuits 121, and arbitrates the data operation signal having the highest priority. Use as a signal.

Here, the arbitration circuit 122 determines the priority of the plurality of data operation signals according to a preset rule, so that the arbitration circuit 122 determines the object that needs to be operated according to the priority of each data operation signal, that is, data that has successfully arbitrated. Confirm the operation signal. The transmission time of each data operation signal may be used as an arbitration criterion, and transmission rate information included in the data operation signal may be used as an arbitration criterion. For example, when the time when the read/write processing circuit 1 transmits the data operation signal is T time and the time when the read/write processing circuit 2 transmits the data operation signal is T+1, the data operation signal is transmitted. If time is used as the arbitration criterion, it can be seen that the data operation signal transmitted from the read/write processing circuit 1 is a high-priority data operation signal, that is, a data operation signal that has successfully arbitrated. The arbitration circuit 122 acquires data from the shared memory 13 by the data operation signal transmitted from the read/write processing circuit 1 in which the arbitration was successful according to the arbitration result.

The data processing apparatus according to the present embodiment determines the priority of the data operation signals transmitted from the plurality of read/write processing circuits through the arbitration circuit, and uses the data operation signal having the highest priority as the arbitration successful data operation signal. . When a plurality of data operation signals are simultaneously received, one data operation signal that can be executed is determined through the arbitration circuit, thereby preventing data clogging that occurs when performing the plurality of data operation signals simultaneously. Furthermore, a plurality of machine learning units can perform data operations on a shared memory through a single transmission circuit, thereby reducing hardware overhead and preventing hardware redundancy.

In one embodiment, the arbitration circuit 122, specifically, when the priority of the data operation signals transmitted from the plurality of read/write processing circuits 121 are the same, the types of the plurality of data operation signals and the preset execution conditions According to the method, the data operation signal for which arbitration was successful is determined.

Here, based on the above embodiment, when the priority of the data operation signals transmitted from the plurality of read/write processing circuits 121 is the same, the arbitration circuit 122 performs the types and preset execution of the plurality of data operation signals. According to the conditions, it is possible to determine the successful data operation signal.

Here, the pre-set execution condition may be to determine whether an arbitration result is determined by measuring whether the data transmission channel corresponding to the data operation signal is idle. When the data transmission channel is in an idle state, a data operation signal corresponding to the data transmission channel is mediated with a data operation signal that has successfully mediated. In addition, the arbitration result may be determined based on transmission time information included in the data operation signal. Exemplarily, the arbitration circuit 122 receives four data operation signals of a data operation signal 0, a data operation signal 1, a data operation signal 2, and a data operation signal 3, wherein the data operation signal 1 and the data operation signal 2 are The priority is the same, data operation signal 1 is a unicast read command, data operation signal 2 is a broadcast command, and machine learning unit 1 is determined as a target machine learning unit according to address information included in data operation signal 1 , Determine machine learning unit 0, machine learning unit 1, machine learning unit 2, and machine learning unit 3 as target machine learning units according to the type of data motion signal 2, and machine learning unit 0, machine learning unit 1, and machine learning unit When the data channel of 2 is in an idle state and the data channel of the machine learning unit 3 is in a busy state, the arbitration circuit 122, the data operation signal 1 is a unicast read command and the data operation signal 2 is It is a broadcast command and determines the data operation signal 1 as the arbitration successful data operation signal according to the busy state of the data channel of the machine learning unit 3.

Alternatively, when the data operation signal is a unicast type signal, the execution condition is that the channel of the machine learning unit transmitting the unicast type signal is in an idle state, or a unicast type signal It includes that the channel of the computation unit in the transmitted machine learning unit is in an idle state.

Alternatively, when the data operation signal is a multicast type signal, the execution condition is that the channel of the machine learning unit that transmitted the multicast type signal is in an idle state and the target machine designated by the multicast type signal The channel of the learning unit is in the idle state, or the channel of the computation unit in the machine learning unit that transmitted the multicast type signal is in the idle state, and the channel of the target computation unit specified by the multicast type signal is Includes being in an idle state.

Alternatively, when the data operation signal is a broadcast type signal, the execution condition is that the channel of the machine learning unit that transmitted the broadcast type signal is in an idle state and the channels of the other machine learning units are idle. Including being in the (idle) state, or that the channel of the computational unit in the machine learning unit that transmitted the broadcast type signal is in the idle state, and the channel of the computational unit in the other machine learning units is in the idle state. do.

In the data processing apparatus according to the present embodiment, when the priority of the data operation signals transmitted from the plurality of read/write processing circuits is the same, the arbitration circuit arbitrates according to the types of the plurality of data operation signals and the execution conditions set in advance. The successful data operation signal can be confirmed. In the present embodiment, when the priority of the data operation signals is the same, the data operation signal that has successfully arbitrated is determined according to the type of the data operation signal and a pre-set execution condition. Furthermore, it prevents data clogging caused by simultaneous execution of multiple data operation signals, and allows multiple machine learning units to operate data on shared memory through one transmission circuit, reducing hardware overhead and reducing hardware Redundancy was prevented.

In one embodiment, referring to FIG. 63, the transmission circuit 12 further includes a second transmission interface 120, each interface in the second transmission interface 120 and each interface in the first transmission interface 14 Are connected correspondingly, and one machine learning unit 15 and one read/write processing circuit 121 are correspondingly connected.

Here, the first transmission interface 14 transmits a data operation signal to the corresponding read/write processing circuit 121 through the second transmission interface 120. The transmission circuit 12 returns input neuron data and weights required for the machine learning device returned through the second transmission interface 120 to the first transmission interface 14, and then the target through the first transmission interface 14 again. It can return to a machine learning unit or a target operation unit. The first transmission interface 14 may include one interface and may include a plurality of interfaces. The second transmission interface 120 may include one interface or may include a plurality of interfaces. Exemplarily, when the first transmission interface 14 includes one transmission interface 141 and one data reception interface 142, the second transmission interface 120 includes one transmission interface 141 and one It includes a second receiving interface 1201 and a second return interface 1202 corresponding to the return interface 142 of.

Alternatively, referring to FIG. 64, a plurality of calculation units 151 in one machine learning unit 15 share one transmission interface 141 in the first transmission interface 14, but each calculation unit Corresponds to one data receiving interface 142.

Here, when one machine learning unit 15 includes a plurality of calculation units 151, the plurality of calculation units 151 may share one transmission interface 141 of the first transmission interface 14. have. A plurality of calculation units 151 in one machine learning unit 15 transmits a data operation signal to the transmission circuit 12 through a shared transmission interface 141, and the transmission circuit 12 is a target calculation unit. The input neuron data and weights obtained through the data receiving interface 142 corresponding to 151 are returned to the target operation unit.

Therefore, in the data processing apparatus according to the present embodiment, a plurality of operation units in one machine learning unit share one transmission interface in the first transmission interface, but each operation unit corresponds to one data reception interface. . Accordingly, it is possible to efficiently reduce the number of data operation signal transmission interfaces in the machine learning unit, save hardware resources, and reduce hardware area and power consumption.

In one embodiment, referring to FIG. 65, a plurality of operation units 151 in one machine learning unit 15 are each one transmission interface 141 and one data reception interface 142 in the first transmission interface. ) Corresponds to.

Here, as shown in FIG. 65, one operation unit 151 corresponds to one transmission interface 141 and one data reception interface 142, respectively. The operation unit 151 transmits a data operation signal to the transmission circuit 12 through a transmission interface 141 corresponding thereto, and the transmission circuit 12 transmits an input neuron obtained through the corresponding data reception interface 142. Data and weights are returned to the corresponding target operation unit 151. Exemplarily, the operation unit 1 corresponds to the transmission interface 1 and the data reception interface 1, and the operation unit 2 corresponds to the transmission interface 2 and the data reception interface 2. The operation unit 1 transmits a data operation signal to the transmission circuit 12 via the transmission interface 1, and the transmission circuit 12 determines the operation unit 1 and the operation unit 2 as target operation units by the data operation signal. In this case, the transmission circuit returns the input neuron data and weights acquired through the data reception interface 1 and the data reception interface 2 to the calculation unit 1 and the calculation unit 2.

Thus, in the data processing apparatus according to the present embodiment, a plurality of operation units in one machine learning unit correspond to one transmission interface and one data reception interface in the first transmission interface, respectively. Since the plurality of operation units correspond to the transmission interface and the data reception interface in the first transmission interface, the control logic can be effectively simplified during the data transmission process.

In one embodiment, referring to FIG. 66, a plurality of machine learning units 15 share one signal receiving interface 81201 and one data return interface 81202 in the second transmission interface 120.

Here, the plurality of machine learning units 15 may share one signal reception interface 81201 and one data return interface 81202 in the second transmission interface 120. Illustratively, when the read/write processing circuit 121 is a broadcast read processing circuit, data operation signals transmitted from a plurality of machine learning units are transmitted to the broadcast read processing circuit through one signal reception interface 81201 The broadcast readout processing circuit acquires the input neuron data and weight according to the data operation signal, and returns the input neuron data and weight to the target machine learning unit through the data return interface 81202 according to the address information in the data operation signal. do.

In the data processing apparatus according to this embodiment, a plurality of machine learning units share one signal receiving interface and one data return interface in the second transmission interface. In this embodiment, since the data processing apparatus shares one signal reception interface and one data return interface in the second transmission interface, hardware overhead is further reduced to prevent hardware redundancy.

In one embodiment, with continued reference to FIG. 66, the read/write processing circuit 121 further includes a signal queue. The signal queue stores the data operation signals transmitted from each machine learning unit 15. When receiving the data operation signal, the read/write processing circuit 121 determines whether there is a residual space in the signal queue, but if "Yes", the data operation signal is cached as a signal queue, and if "No", Block the data operation signal.

Here, the signal queue may store data operation signals transmitted from each machine learning unit 15, may be installed outside the read/write processing circuit 121, and installed in the read/write processing circuit 121 It could be. When the data operation signal is received, the read/write processing circuit 121 may secure a storage space of the signal queue by sending a memory inquiry command to the signal queue. If the size of the storage space of the signal queue can store the data operation signal, the data operation signal is cached in the signal queue, and if the storage space of the signal queue cannot store the data operation signal, the data operation signal is blocked. .

In the data processing apparatus according to this embodiment, the read/write processing circuit further includes a signal queue. The signal queue stores data operation signals transmitted from each machine learning unit. When the data operation signal is received, the read/write processing circuit determines whether there is any remaining space in the signal queue, but if "Yes", the data operation signal is cached in the signal queue, and if "No", the data operation signal Block. In this embodiment, the read/write processing circuit, when receiving a plurality of data operation signals, caches the data operation signals in the signal queue or blocks the data operation signals to transmit the data operation signals one by one to the arbitration circuit to be processed. Blocking of data operation signals can be prevented. Furthermore, since a plurality of machine learning units perform data operations on shared memory through one transmission circuit, hardware overhead is reduced and hardware redundancy is prevented.

Alternatively, when the read/write processing circuit 121 is a broadcast processing circuit, the signal queue includes a command queue and a request queue. The command queue caches the imperative signal received by the broadcast processing circuit, and the request queue caches the requested signal obtained by analyzing the imperative signal.

Here, when the read/write processing circuit 121 is a broadcast processing circuit, the signal queue may include a command queue and a request queue. This stores the imperative signal transmitted from each machine learning unit 15 in the command queue, analyzes the imperative signal through a broadcast processing circuit to obtain a request signal, and then stores the acquired request signal in the request queue. do. Here, the command queue caches the command type signal received by the broadcast processing circuit. The request queue caches the requested signal obtained by analyzing the imperative signal.

In the data processing apparatus according to the present embodiment, when the read/write processing circuit is a broadcast processing circuit, the signal queue includes a command queue and a request queue, and the command queue caches an imperative signal received by the broadcast processing circuit. . The request queue caches the requested signal obtained by analyzing the imperative signal. In this embodiment, since the command-type signal and the request-type signal are stored in the command queue and the request queue, respectively, the command-type signal and the request-type signal are sent to the arbitration circuit one by one to be processed. Accordingly, blockage of the data operation signal is further prevented, and a plurality of machine learning units perform data operations on the shared memory through a single transmission circuit, thereby reducing hardware overhead and preventing hardware redundancy.

In each of the above embodiments, the same or similar parts may refer to each other, and contents not described in detail in some embodiments may refer to the same or similar contents of other embodiments.

It should be noted that, in describing the present application, terms such as "first", "second", and the like are used only to describe the purpose, and should not be understood as indicating or implying relative importance. In addition, in describing the present application, unless otherwise specified, the meaning of “plural” means two or two or more.

A process or method described in a flow chart or elsewhere herein is to be understood as including modules, segments, or portions of code of implementable instructions for implementing one or more specific logic functions or steps of the process. In addition, the scope of the preferred embodiment of the present application includes separate implementations, may not follow the order of presentation or discussion, and includes performing functions in an almost simultaneous manner or an opposite order by related functions. This will be understood by those of ordinary skill in the art to which the embodiment of the present application belongs.

It will be understood that each part of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiment, the plurality of steps or methods may be implemented by software or firmware stored in a memory and executed by a suitable command execution system. For example, when implemented in hardware, as in the other embodiment, a discrete logic circuit with a logic gate circuit for implementing the logic function of a data signal in the present field, and a suitable combination logic gate circuit. A dedicated integrated circuit, programmable gate array (PGA), field programmable gate array (FPGA), or any one of known techniques, or a combination thereof may be used.

Those of ordinary skill in the art to which the present application belongs can implement all or some of the steps of the method of the above-described embodiment by commanding related hardware with a program, and the program is stored in a computer-readable storage medium. It will be appreciated that one or a combination of the steps of the method embodiment is included when the program is executed.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, each unit may be a single physical entity, and two or at least two or more units may be integrated into one module. The above-described integrated module may be implemented using a form of hardware or may be implemented using a form of a software function module. The integrated module may be implemented in the form of a software function module, and may be stored in a computer-readable storage medium when sold or used as an independent product.

The above-described storage medium may be a read-only storage device, a disk, or a CD.

In the description of the present specification, the description of the reference terms “one embodiment”, “some embodiments”, “examples”, “specific examples” or “some examples” refers to specific features described by combining the embodiments or examples, It means that a structure, material or feature is included in at least one embodiment or illustration of this application. In the present specification, implied expressions for the terms described above do not necessarily refer to the same embodiment or example. And, the specific features, structures, materials or features described may be combined in any suitable manner in any one or more embodiments or examples.

Although the embodiments of the present application have already been presented and described, the embodiments are illustrative and should not be understood as limiting the present application. It will be understood that those of ordinary skill in the art to which this application belongs can change, modify, substitute, and modify the above embodiments within the scope of the present application.

Claims (15)

Receiving a data operation signal transmitted from an internal or external device,-The data operation signal includes an operation domain and an operation code, the operation code includes a first type flag bit, and the operation domain is a second type A flag bit, wherein the first type flag bit indicates whether the data operation signal is an I/O command, and the second type flag bit indicates whether the data operation signal is broadcast or multicast in the I/O command. Indicates whether it is an order; And
And obtaining necessary input data by performing a corresponding operation on operation-planned data in a memory according to the data operation signal. The method according to claim 1,
And the operation domain further includes a data reception flag bit, and the data reception flag bit represents an apparatus or processing circuit that receives the input data. The method according to claim 2,
Wherein the number of data reception flag bits indicates the number of devices or processing circuits capable of interacting with the memory. The method according to any one of claims 1 to 3,
The operation domain further includes information on operation schedule data, and the operation schedule data information includes a source address in the memory, an operation schedule data length, and a data return address after data operation. The step of obtaining necessary input data by performing a corresponding operation on the operation scheduled data in the memory according to the data operation signal,
Starting to read the memory from the source address to obtain input data satisfying the length of the data;
Determining an apparatus or processing circuit for receiving input data by the data reception flag bit; And
And returning the input data to a storage space corresponding to the data return address in the device or processing circuit by the data return address. The method of claim 4,
The apparatus comprises at least one machine learning unit, and each machine learning unit comprises a mast processing circuit and a plurality of slave processing circuits. The method of claim 5,
The operation domain further includes a jump sub-operation domain, the jump sub-operation domain includes a jump width and a jump data length that operates after each jump, and the data length is satisfied by starting the memory read from the source address. The step of obtaining the input data to be,
Reading the memory from the source address and acquiring first jump data based on a jump data length after the current jump;
Acquiring the last address of the jump data and jumping from the last address to a target jump address by the jump width;
And acquiring second jump data according to the jump data length after jumping from the target jump address until the length of jump data obtained after each jump satisfies the data length. The method of claim 6,
The jump sub-operation domain includes a stride operation domain and/or a segment operation domain, the stride operation domain represents a jump width of the data operation signal each time, and the segment operation domain is a preset division of the data operation signal. Data processing method, characterized in that indicating the size. The method of claim 7,
And the operation domain further includes a function flag bit indicating a processing operation to be performed on the read data. The method of claim 8,
If the value of the first type flag bit is I/O, determining the data operation signal as an I/O command; And
And if the value of the second type flag bit is 1, determining the data operation signal as a broadcast or multicast command among the I/O commands. The method of claim 9,
Receiving the data operation signal transmitted from the internal or external device,
Analyzing the data operation signal to obtain information on the type flag bit of the data operation signal and operation schedule data;
And performing the analyzed data operation signal by a command queue indicating an execution order of the data operation signal. The method of claim 10,
Before performing the analyzed data operation signal by the command queue, the method,
Determining a dependence relationship between the analyzed data operation signals adjacent to each other to obtain a determination result, the dependence relationship is an s-th data operation signal and an s-1th data operation signal before the s-th data operation signal Indicates whether there is an association -; And
If the determination result is that there is a dependency relationship between the s-th data operation signal and the s-1 th data operation signal, the s-th data operation signal is cached, and the s-1 th data operation signal And extracting the s-th data operation signal after performing the data processing method. The method of claim 11,
The step of determining the dependence relationship between the analyzed data operation signals adjacent to each other,
A first storage address section for extracting data necessary for the s-th data operation signal based on the s-th data operation signal; And obtaining 0th storage address intervals for extracting data necessary for the s-1th data operation signal according to the s-1th data operation signal.
Determining that the s-th data operation signal and the s-1 th data operation signal have a dependency relationship if there is an overlapped region between the first storage address section and the 0-th storage address section; And
If there is no overlapped region between the first storage address section and the 0th storage address section, determining that the s-th data operation signal and the s-1 th data operation signal have no dependency relationship. Data processing method characterized in that. A data processing apparatus comprising a processor and a memory in which a computer program is stored,
A data processing apparatus comprising the steps of the method according to any one of claims 1 to 12 when the processor executes the computer program. A board card comprising the data processing device according to claim 13. An electronic device comprising the board card according to claim 14.

Images