CN110135557B - Neural network topology architecture of image processing system - Google Patents

Abstract

The invention discloses an image processing system architecture, which comprises a pixel array, a synaptic array and a multi-stage neuron array group, wherein the 1 st-stage neuron array group only comprises one 1 st-stage neuron array, the last-stage neuron array group only comprises one last-stage neuron array, each intermediate-stage neuron array group comprises at least one neuron array, and at least one intermediate-stage neuron array and at least one non-adjacent-stage neuron array are connected through synapses in the synaptic array. The invention can simulate the neural network topology architecture of the image processing system of the complex neural network.

Description

A Neural Network Topology Architecture of Image Processing System

technical field

The invention relates to the technical field of image processing, in particular to a neural network topology architecture of an image processing system.

Background technique

With the development of CMOS integrated circuit technology, electronic imaging products are more and more widely used in daily life and industrial production. Correspondingly, image processing technology can be used as an automatic image processing technology, which plays an important role in key fields such as intelligent monitoring, aerospace, intelligent driving, rapid identification, and accurate capture. The development of artificial intelligence algorithms has also accelerated the breadth of image processing. In today's applications, in image processing systems, neural networks can be used as a means of image processing. As shown in Figure 1, the neural network-based image processing system includes a pixel array and a neural network. The role of the pixel (PD) array is to convert the optical signal into an electrical signal, and transfer this electrical signal to the input of the neural network as the initial image electrical signal. A neural network consists of multiple stages, including arrays of synapses and arrays of neurons. Preferably, the pixel array, the synapse array and the neuron array each occupy a chip layer in physical structure, the pixel array is on the upper layer, the synapse array is on the middle layer, and the neuron array is on the lower layer, thus forming a 3D three-dimensional stacked structure , reducing the physical space of the image processing system architecture.

In the existing brain-like neural network structure, the neural network can be simply divided into neurons and synapses. Neurons are used as signal processing units, and synapses are used as signal transmission units. The usual neural network production method is shown in Figure 2. The neurons are divided into several levels, and each level is connected to the adjacent neurons through synapses. This connection method is the simplest logical architecture method, and its layout is based on a simple single-layer stack. Although it can simulate the effect of a neural network, it is still different from the actual neural network. Neural networks in reality are often more complex and cannot be realized simply through connections between adjacent levels of neurons. Therefore, a more complex and diverse neural network topology is needed to simulate various complex neural networks.

Contents of the invention

The main purpose of the present invention is to overcome the defects of the prior art, and provide a neural network topology structure capable of simulating a complex neural network image processing system.

To achieve the above object, the present invention provides a neural network topology of an image processing system, including a pixel array, a synapse array, and an N-level neuron array group, wherein the first-level neuron array group only includes one first-level neuron array, the Nth-level neuron array group includes only one Nth-level neuron array, the i-level neuron array group includes at least one i-th level neuron array, and at least one j-th level neuron array and at least one j-th level neuron array The +k-level neuron array is connected through synapses in the synapse array, wherein N is a positive integer greater than or equal to 4, i is a positive integer greater than 1 and less than N, and j is greater than or equal to 1 and less than N-1 is a positive integer, k is a positive integer greater than or equal to 2 and less than N-1, and j+k is less than or equal to N.

Preferably, the first-level neuron array is connected to at least one non-Nth-level subsequent neuron array.

Preferably, each second-level neuron array is connected to the first-level neuron array through a synapse in the synapse array.

Preferably, the neuron array of the Nth level is connected to at least one neuron array of the previous level that is not the first level.

Preferably, each N-1th level neuron array is connected to the Nth level neuron array through a synapse in the synapse array.

Preferably, each i-th level neuron array in the i-level neuron array group is connected to at least one l-th level neuron array through a synapse in the synapse array, where l is greater than or equal to 1 and less than or equal to i A positive integer of -1.

Preferably, each i-th level neuron array in the i-th level neuron array group is connected to at least one i-1th level neuron array through a synapse in the synapse array.

Preferably, each i-th level neuron array in the i-th level neuron array group is connected to at least one m-th level neuron array through a synapse in the synapse array, m is greater than or equal to i+1 and less than or equal to A positive integer of N.

Preferably, said array of neurons comprises a plurality of neurons, each said neuron comprising a pre-neuron and a post-neuron;

For the neurons of the pth neuron array and the neurons of the qth neuron array connected by a synapse, wherein the pre-neurons of the neurons of the pth neuron array receive sampling signals, and The synaptic electrical signal is transmitted to the rear neuron of the neuron of the qth level neuron array through the synapse, and the rear neuron of the neuron of the qth level neuron array correspondingly outputs a sampling signal to the front of the same neuron A neuron, where p is a positive integer greater than or equal to 1 and less than N-1, and q is a positive integer greater than p and less than N.

Compared with the prior art, the entire image processing system of the present invention has significant advantages in speed and area due to the reduction of additional connections between the pixel array and the neural network, and the image processing does not go through additional analog and digital signals. Transformation, image data can be processed in parallel in real time. In addition, since the neural network in the present invention has a topological structure, it can better meet the complex structure of the actual neural network. Compared with the fixedness of traditional algorithms, the accuracy of image processing using the neural network with a topological structure is higher.

Description of drawings

FIG. 1 is a schematic diagram of the basic structure of an image processing system in the prior art;

FIG. 2 is a schematic diagram of a neural network topology of an image processing system in the prior art;

FIG. 3 is a schematic diagram of a multi-level structure of a neural network topology of an image processing system according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a neural network topology of an image processing system according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of two-stage neuron signal transmission according to an embodiment of the present invention;

FIG. 6 is an internal block diagram of a single neuron of a neural network according to an embodiment of the present invention.

Detailed ways

In order to make the content of the present invention clearer and easier to understand, the content of the present invention will be further described below in conjunction with the accompanying drawings. Of course, the present invention is not limited to this specific embodiment, and general replacements known to those skilled in the art are also covered within the protection scope of the present invention.

The present invention will be described in further detail below in conjunction with accompanying drawings 2-6 and specific embodiments. It should be noted that the drawings are all in a very simplified form, using imprecise scales, and are only used to facilitate and clearly achieve the purpose of assisting in describing the present embodiment.

The neural network topology of the image processing system of the present invention includes a pixel array, a synapse array and a group of neuron arrays. The function of the pixel array is to convert the light signal into an image signal, and transfer the image signal to the neuron array group. The neuron array group is composed of multiple levels, and multiple topological connection structures are used between the neuron array groups. Each level of neuron array group includes at least one neuron array. The synapse array connects two arrays of neurons at different levels. Each neuron array is composed of multiple neuron circuits, and each neuron circuit is used to process received image signals and output corresponding processing and feedback signals. The synaptic array is composed of multiple synapses, and each synapse is a two-terminal structure that connects two neurons. The function of the synapse is to transmit the image signal from one neuron to another neuron, and is also responsible for the corresponding feedback. transmission of signals.

Please refer to Figure 3. The neural network is hierarchical, and there is a multi-level structure. Both the first-level neuron array group and the last-level (Nth level) neuron array group contain only one neuron array A ₁ and A _n , and the rest The neuron array group at an intermediate level may contain one or more neuron arrays, and multiple neuron arrays at each level constitute the neuron array group at that level. It should be noted that in the present invention, it is not necessary for each neuron array in the adjacent two-level neuron array group to have a corresponding connection relationship. In other words, there is at least one neuron array of a certain level that can communicate with other levels across multiple levels. (not the upper or lower level) neuron arrays are connected, but in the present invention, the neuron arrays of the first level and the last level cannot be directly connected across levels. In addition, when the neuron array groups of two adjacent levels are connected, it is not limited that each neuron array of the current level must be connected with each neuron array of the adjacent level. It should be noted that the definition of the number of neuron arrays in the present invention is that if the neuron array is located at the i-th level, then the maximum number of neuron arrays between the i-th level neuron array and the first-level neuron array There are i different levels of neuron arrays on the long connection link.

Next, the neural network architecture is further explained. In this implementation, the first-level neuron array is connected to at least one neuron array in the subsequent level (not the last level). Preferably, each neuron array A ₂₁ -A _2k2 at the second level must be connected to the first level neuron array A ₁ .

The last level of neuron arrays is connected to at least one neuron array in the previous level (not level 1). Preferably, each neuron array A _(n-1)1 -A _(n-1)kn-1 at the penultimate level must be connected to the neuron array A _n at the last level.

The neuron array at the intermediate level is at least respectively connected to one neuron array at the front level and one neuron array at the next level. In particular, in the connection relationship with the neuron array of the previous level, each neuron array of the current level is connected with at least one neuron array of the previous level. In particular, each neuron array of the current level is connected to at least one neuron array of the previous level. For example, assuming that an intermediate neuron array is located at the fifth level, then in the connection to the previous level, each fifth-level neuron array is connected to any one or more neuron arrays of the fourth level, and the fifth-level neuron array The neuron array can also be selected for connection in any neuron array from the first level to the third level. For the connection to the next level, each neuron array is connected to at least one neuron array at the next level. For example, the neuron array at the fifth level may be selectively connected to at least one neuron array at any of the sixth level to the last level. Of course, there is no connection relationship between neuron arrays at the same level.

Please refer to FIG. 4 , which shows a neural network topology of an image processing system according to an embodiment of the present invention. There are five levels of neuron arrays in this embodiment.

Both the first and fifth levels have only one neuron array. The second, third, and fourth levels in the middle have m, n, k neuron arrays.

The first-level neuron array A ₁ is respectively connected to the second-level neuron arrays A ₂₁ , A ₂₂ to A _2m , which means that each neuron array located in the second level must have a connection relationship with the first-level neuron array topology rules.

The fifth-level array A ₅ is respectively connected to the fourth-level A ₄₁ , A ₄₂ to A _4k , which conforms to the topological structure rule that every neuron array in the penultimate level must have a connection relationship with the last-level neuron array .

The neuron arrays of the second level to the fourth level ensure that each array is connected to at least one neuron array of the previous level, and also guarantees that there is at least one connection with the neuron array of the back level, which meets at least the previous level and the back level respectively. The rule that a neuron array at the first level has a connection relationship also conforms to the rule that at least one connection with the neuron array at the previous level needs to be connected to the neuron array at the previous level.

It can be seen that A ₂₂ does not have a connection relationship with every neuron array in the third level, and A _2m directly skips the third level and only has a connection relationship with the neuron array in the fourth level.

For connected two-level neuron arrays, the most basic connection relationship is neuron array-synapse array-neuron array. When two neuron arrays located at two levels respectively need to have a connection relationship, there needs to be a synapse array between the neuron arrays, and each synapse in the synapse array connects two of the two neuron arrays respectively. Neurons establish a connection relationship. As shown in Figure 5, each neuron includes a pre-neuron and a post-neuron. In any two connected stages of the neural network, any pre-neuron of one stage is connected with any post-neuron of the other stage through synapses. Neurons are composed of components manufactured by standard CMOS technology, which can simulate the transmission and analysis capabilities of neurons. Synapses are composed of non-volatile memory devices whose electrical parameters change with external electrical signals. Fig. 6 is a schematic diagram of any two-level neuron signal transmission. For any two-level neuron array, it is divided into a front-level neuron array and a rear-level neuron array, where the front-level neuron receives the sampling signal and transmits synaptic electrical signals to the connected synapses; Post-neurons collect synaptic signals transmitted by connected synapses, and output electrical signals (ie, sampled signals) to pre-neurons in the same neuron. The exception is that the rear neurons of the neurons in the first-level neuron array do not participate in the work, and the sampling signals of the front neurons are directly provided by the pixel array, and the front neurons of the neurons in the last-level neuron array do not participate in the work, and the rear neurons do not participate in the work. The neuron output signal is directly used as the final output signal of image processing.

It should be noted that the image processing system of the present invention can be trained. Training is a key step for the image processing system to generate image processing capabilities. After the training is completed, it has image processing capabilities and can be used normally. Image processing systems can be trained multiple times, or the system can be reset and trained again. In the present invention, the neuron array changes the synaptic weights of the synapses during training. For two connected neurons, assuming that a pre-neuron of the previous stage is connected to a post-neuron of the post-stage through a synapse, the post-neuron of the post-stage judges whether to update the synapse according to the synaptic electrical signal it receives weight, and when judging to update the synapse weight, the output feedback signal is fed back to the pre-neuron of the previous stage through the synapse, and jointly changes the synaptic weight of the synapse with the pre-neuron.

The signals that each neuron obtains from the outside world include synaptic electrical signals and feedback signals, and the signals transmitted to the outside world also include synaptic electrical signals and feedback signals. Please refer to Fig. 5 and Fig. 6, each pre-neuron comprises a sampling module, a feedback processing module and a first output control module; the sampling module is used to receive a sampling signal, and the feedback processing module is used to receive a feedback signal; the first output control module uses Yu is responsible for coordinating the transmission of signals, making the signals work in a reasonable sequence, and finally outputting synaptic electrical signals. Each post-neuron includes a judgment module, a feedback output module and a second output control module, the judgment module is used to receive the outstanding electrical signal and judge whether to update the synaptic weight, the feedback output module is used to generate the feedback signal, and the second output control module is used to It is responsible for coordinating the transmission of signals, making the signals work in a reasonable timing, and finally outputting sampling signals to the pre-neurons of the same neuron and outputting feedback signals to the pre-neurons of the previous stage.

In this embodiment, the electrical structure of the synapse is a two-terminal device, and its electrical characteristics are a resistive variable device whose resistance value can be changed by an external electrical signal. Usually, a new type of non-volatile memory with multi-value resistance change capability can be used, such as resistive memory RRAM, PCRAM, etc., or a new type of non-volatile memory with single-value resistance change capability in parallel structure, such as MRAM , FeRAM, RRAM, PCRAM, etc., to achieve. Without loss of generality, other electrical devices having such properties with a smaller area can also be used to realize the synapse. The synaptic weight is related to the resistance of the synapse, that is, the change of the synaptic resistance corresponds to the change of the weight. The smaller the resistance value, the larger the weight; the larger the resistance value, the smaller the weight. Moreover, only when the electrical signal applied to both ends exceeds a certain threshold can the resistance value change, that is, the weight change. In this embodiment, a pre-neuron in the neuron array of one level and a post-neuron in the neuron array of the other level control the external electrical signal to change the synapse connecting the two levels. The resistance value, so as to achieve the change of weight.

In summary, the entire image processing system of the present invention has significant advantages in speed and area due to the reduction of additional connections between the pixel array and the neural network, and the image processing no longer undergoes additional conversion between analog and digital signals. Image data can be processed in parallel in real time. In addition, since the neural network in the present invention has a topological structure, it can better meet the complex structure of the actual neural network. Compared with the fixedness of traditional algorithms, the accuracy of image processing using the neural network with a topological structure is higher. And because the neural network has a multi-level neuron structure, each training can update the connection weights between each neuron through a reasonable signal trigger mechanism. After many times of training, the neural network can finally be trained into a system with specific image processing capabilities.

Although the present invention has been disclosed above with preferred embodiments, the various embodiments described are only examples for convenience of description, and are not intended to limit the present invention. Those skilled in the art can Some changes and modifications are made, and the scope of protection claimed by the present invention should be based on the claims.

Claims (9)

1. A neural network topology architecture of an image processing system, comprising a pixel array, a synaptic array, and an N-level neuronal array group, wherein the 1-level neuronal array group comprises only one 1-level neuronal array, the N-level neuronal array group comprises only one N-level neuronal array, the i-level neuronal array group comprises at least one i-level neuronal array, and the at least one j-level neuronal array and the at least one j+k-level neuronal array are connected by synapses in the synaptic array, wherein N is a positive integer greater than or equal to 4, i is a positive integer greater than 1 and less than N, j is a positive integer greater than or equal to 1 and less than N-1, k is a positive integer greater than or equal to 2 and less than N-1, and j+k is less than or equal to N.

2. The neural network topology of an image processing system of claim 1, wherein a level 1 array of neurons is connected to at least one non-N-th level of a subsequent array of neurons.

3. The neural network topology of an image processing system of claim 2, wherein each level 2 neuron array is connected to the level 1 neuron array by a synapse in the synapse array.

4. The neural network topology of an image processing system of claim 1, wherein the N-th level of neuron array is connected to at least one non-level 1 preceding level of neuron array.

5. The neural network topology of an image processing system of claim 4, wherein each of said N-1 th level of neuron arrays is connected by a synapse of said synapse array and said N-th level of neuron array.

6. The neural network topology of an image processing system of claim 1, wherein each of a group of i-th level of neuron arrays is connected to at least one of the i-th level of neuron arrays by a synapse in the synaptic array, where l is a positive integer greater than or equal to 1 and less than or equal to i-1.

7. The neural network topology of an image processing system of claim 6, wherein each of a group of i-th level neuron arrays is connected to at least one i-1-th level neuron array by a synapse in said synapse array.

8. The neural network topology of an image processing system of claim 6, wherein each of a group of i-th-level neuron arrays is connected to at least one m-th-level neuron array by a synapse in said synapse array, m being a positive integer greater than or equal to i+1 and less than or equal to N.

9. The neural network topology of an image processing system of claim 1, wherein said array of neurons comprises a plurality of neurons, each of said neurons comprising a pre-neuron and a post-neuron;

for the neurons of the p-th level neuron array and the neurons of the q-th level neuron array which are connected through a synapse, wherein the front neurons of the p-th level neuron array receive sampling signals and transmit the synaptic electric shock signals to the rear neurons of the q-th level neuron array through the synapse, the rear neurons of the q-th level neuron array correspondingly output the sampling signals to the front neurons of the same neurons, wherein p is a positive integer which is greater than or equal to 1 and less than N-1, and q is a positive integer which is greater than p and less than N.

Images