TWI730452B - Stereo artificial neural network system - Google Patents

Abstract

The present invention discloses a stereo artificial neural network system comprising an input layer, at least one hidden layer and an output layer. The abovementioned input layer includes at least one input layer node. Each abovementioned hidden layer includes at least one hidden layer node, and each input layer node is connected to one of the hidden layer nodes. The abovementioned output layer includes at least one output layer node, and each output layer node is connected to any one of the hidden layer nodes. Furthermore, each of the hidden layer nodes is connected to each of the hidden layer nodes included in any of the other hidden layers. Thereby, the system may use a small number of nodes to form a more powerful connection, and provide advanced processing performance and speed.

Description

Three-dimensional neural network system

The present invention provides a three-dimensional neural network system, in particular a three-dimensional neural network system in which each hidden layer node of each hidden layer can be connected to each hidden layer node of any other hidden layer.

Artificial Neural Network (ANN) in the field of machine learning and cognitive science is a mathematical model or computational model that imitates the structure and function of biological neural networks (such as the central nervous system of animals). Estimate or approximate. Neural-like networks are calculated by connecting a large number of artificial neurons. In most cases, neural-like networks can change the internal structure on the basis of external information. It is an adaptive system.

Modern neural network is a non-linear statistical data modeling tool. Neural network is usually optimized through a learning method based on mathematical statistics. A large number of statistical methods can be obtained through statistical standard mathematical methods. Function, to express the local structural space; on the other hand, in the field of artificial perception of artificial intelligence, the application of mathematical statistics can also be used to determine the problem of artificial perception. This method is compared with traditional logical reasoning calculations. It has more advantages.

Like other machine learning methods, neural networks have been used to solve a variety of problems, such as machine vision and speech recognition. These problems are difficult to be solved by traditional rule-based programming. The solution; however, in the current neural network-like architecture, each hidden layer is not likely to be connected to the nodes of other hidden layers, which partially reduces the overall connection mode of the neural network.

Accordingly, in order to improve the prior art mentioned, in the conventional artificial neural network system (ANN) architecture, each hidden layer may not be connected to the nodes of other hidden layers; this case proposes A three-dimensional neural network system includes at least one input layer node; at least one hidden layer, each hidden layer includes at least one hidden layer node, and each input layer node is connected to one of the hidden layer nodes; and an output layer , Including at least one output layer node, and each output layer node is connected to one of the hidden layer nodes.

Furthermore, each hidden layer node is connected to each hidden layer node included in any other hidden layer.

Furthermore, the input layer to the output layer have a signal transfer vector, and the signal transfer vector is based on the order of the at least one hidden layer as the signal transfer direction.

The above brief description of the present invention aims to provide a basic description of several aspects and technical features of the present invention. The brief description of the invention is not a detailed description of the invention. Therefore, its purpose is not to specifically enumerate the key or important elements of the invention, nor to define the scope of the invention. It merely presents several concepts of the invention in a concise manner.

1: Three-dimensional neural network system

10: Input layer

10a: Input layer node

20: hidden layer

21: The first hidden layer

21a: The first hidden layer node A

21b: Node B of the first hidden layer

22: second hidden layer

22a: The second hidden layer node A

22b: second hidden layer node B

23: The third hidden layer

23a: third hidden layer node A

23b: third hidden layer node B

24: Fourth hidden layer

24a: fourth hidden layer node A

24b: fourth hidden layer node B

30: output layer

30a: Output layer node

The first figure is a schematic diagram of a three-dimensional neural network architecture according to a preferred embodiment of the present invention.

The second figure is an expanded view of the three-dimensional neural network architecture of the preferred embodiment of the present invention.

In order to understand the technical features and practical effects of the present invention, and implement it in accordance with the content of the specification, the preferred embodiment shown in the figure is further described in detail as follows:

In order to improve the prior art mentioned, in the conventional artificial neural network system (ANN) architecture, each hidden layer may not be connected to the nodes of other hidden layers; the present invention proposes a The three-dimensional neural network system is a three-dimensional neural network system in which the hidden layer node of each hidden layer can be connected to each hidden layer node of any other hidden layer. Accordingly, the system can use relatively fewer nodes than the conventional human neural network to form a more powerful connection mode to provide advanced processing performance and speed.

First of all, please refer to the first and second figures. The first figure is a schematic diagram of the three-dimensional neural network architecture of the preferred embodiment of the present invention, and the second figure is the three-dimensional neural network architecture of the preferred embodiment of the present invention. Schematic diagram of the expansion. As shown in the first and second figures, the three-dimensional neural network system 1 of this embodiment includes an input layer 10, the input layer 10 includes at least one input layer node 10a; at least one hidden layer 20, each hidden layer (21, 22, 23, 24) includes at least one hidden layer node (21a, 21b, 22a, 22b, 23a, 23b, 24a, 24b), and each input layer node 10a is connected to one of the hidden layer nodes (21a, 21b, 22a, 22b, 23a, 23b, 24a, 24b) connection; and an output layer 30 including at least one output layer node 30a, and each output layer node 30a is connected to one of the hidden layer nodes (21a , 21b, 22a, 22b, 23a, 23b, 24a, 24b) connection.

Furthermore, each hidden layer node (21a, 21b, 22a, 22b, 23a, 23b, 24a, 24b) is connected to each hidden layer node included in any other hidden layer. Taking the first figure (and the second figure) as an example, the number of layers of the at least one hidden layer 20 is four, which in sequence are the first hidden layer 21, the second hidden layer 22, the third hidden layer 23, and The fourth hidden layer 24; and each hidden layer (21, 22, 23, 24) contains two hidden layer nodes. In other words, the first hidden layer 21 includes a first hidden layer Node A (21a) and a first hidden layer node B (21b); the second hidden layer 22 includes a second hidden layer node A (22a) and a second hidden layer node B (22b); the third hidden layer Layer 23 includes a third hidden layer node A (23a) and a third hidden layer node B (23ba); the fourth hidden layer includes a fourth hidden layer node A (24a) and a fourth hidden layer node B ( 24b).

It can be clearly seen from the first figure (and the second figure) that the first hidden layer node A (21a) can be connected to the first hidden layer node B (2ba) contained in the first hidden layer 21 itself, but also With all nodes in other hidden layers, including the second hidden layer nodes A, B (22a, 22b), the third hidden layer nodes A, B (23a, 23b), and the fourth hidden layer nodes A, B (24a, 24b) )connection.

In the same way, the second hidden layer node A (22a) can be connected to the second hidden layer node B (22b) contained in the second hidden layer itself, and can also be connected to all nodes of other hidden layers, including the first hidden layer. The nodes A and B (21a, 21b), the third hidden layer nodes A and B (23a, 23b), and the fourth hidden layer nodes A and B (24a, 24b) are connected. The third hidden layer node A (23a) can be connected to the third hidden layer node B (23b) contained in the third hidden layer itself, and can also be connected to all nodes in other hidden layers, including the first hidden layer node A, B (21a, 21b), the second hidden layer nodes A, B (22a, 22b), and the fourth hidden layer nodes A, B (24a, 24b) are connected. The fourth hidden layer node A (24a) can be connected to the fourth hidden layer node B (24b) contained in the fourth hidden layer itself, and can also be connected to all nodes in other hidden layers, including the first hidden layer node A, B (21a, 21b), the second hidden layer nodes A, B (22a, 22b), and the third hidden layer nodes A, B (23a, 23b) are connected. It is worth noting that in other possible embodiments, the number of hidden layers in the three-dimensional neural network system and the number of hidden layer nodes included in each hidden layer can be adjusted according to requirements, and the present invention should not be based on this. limit.

In addition, each node from the input layer 10 to the output layer 30 has a signal transfer vector, and the signal transfer vector is based on the order of the at least one hidden layer 20 as the signal transfer direction.

The following will further explain the three-dimensional neural network system of the present invention.

，其中D為輸入向量x=[x ₀ ,x ₁ ,...,x _D-1]；M ^(L)為輸出向量f(x)，且f(x)的矩陣表示為

，其中

，且n=0,1,...,M ⁽⁰⁾-1；偏置向量b ^(L)表示為[

,

,...,

]，權重向量

表示為由第i層中的第m個節點至第l層中的第n個節點的權重矩陣；

，且l=1,2,...,L-1，n=1,2,...,M ^(l)-1，且

；其中，G和h皆屬於一激勵函數(Activation Function)。 First of all, suppose a three-dimensional neural network system has a hidden layer with L number of layers, then its function can be expressed as

, Where D is the input vector x =[ x ₀ ,x ₁ , ... ,x _{D -1} ]; M ^{( L )} is the output vector f ( x ), and the matrix of f ( x ) is expressed as

,among them

, And n =0 , 1 , ... ,M ⁽⁰⁾ -1; the offset vector b ^{( L ) is} expressed as [

,

, ... ,

], weight vector

Expressed as a weight matrix from the mth node in the i-th layer to the nth node in the l-th layer;

, And l =1 , 2 , ... ,L -1, n =1 , 2 , ... ,M ^{( l )} -1, and

; Among them, G and h belong to an activation function (Activation Function).

In conclusion, the activation function h ( x ) can form at least one hidden layer, and it is defined as sigmoid ( x )=1/(1+ e ^{- x} ).

，且l=1,2,...,L-1，n=1,2,...,M-1，

，以及

。 Wherein, the signal transmission direction from the input layer to the output layer is based on the order of at least one hidden layer as the signal transmission direction. Taking the first figure as an example, the three-dimensional neural network system has four hidden layers, and The number of hidden layer nodes in each hidden layer is two, then the node can be expressed as

, And l =1 , 2 , ... ,L -1, n =1 , 2 , ... ,M -1,

,as well as

.

In addition, the three-dimensional neural network system in the first figure can be further expanded into the neural network system in the second figure. As shown in the second figure, the signal transfer vectors of the hidden layer nodes of the same hidden layer are downward at the same time; however, in other possible embodiments, the signal transfer direction can be modified or adjusted according to the actual situation. The present invention does not This should be the limit.

The above-mentioned three-dimensional neural network system can be applied to artificial intelligence related fields.

However, the above are only the preferred embodiments of the present invention, and should not be used to limit the scope of implementation of the present invention, that is, simple changes and modifications made in accordance with the scope of the patent application and the description of the present invention still belong to the present invention. Covered in the scope.

1　…　3D neural network system 10　…　input layer 10a　…　input layer node 21　…　The first hidden layer 22　…　Second hidden layer 23　…　The third hidden layer 24　…　The fourth hidden layer 30　…　 output layer 30a　…　output layer node

Claims (5)

A three-dimensional neural network system, including: an input layer including at least one input layer node; at least one hidden layer, each hidden layer includes at least one hidden layer node, and each input layer node is connected to one of the hidden layer nodes Connection; and an output layer, including at least one output layer node, and each output layer node is connected to one of the hidden layer nodes; wherein each hidden layer node is connected to every hidden layer node included in any other hidden layer Connected and each hidden layer node is connected across layers; among them, the three-dimensional neural network system with the number of hidden layers of L layer satisfies the following conditions: the function system is

, D is an input vector x =[ x ₀ ,x ₁ , ... ,x _{D -1} ], M ^{( L )} is an output vector f ( x ) = G ( b ^{( L )} +

,

, And n =0 , 1 , ... , M ⁽⁰⁾ -1, b ^{( L )} is the offset vector, expressed as [

,

, ... ,

],

Is the weight vector expressed as a weight matrix from the m- th node in the i-th layer to the n- th node in the l-th layer,

, L =1 , 2 , ... ,L -1, n =1 , 2 , ... ,M ^{( l )} -1,

; Among them, G and h both belong to an excitation function. The three-dimensional neural network system according to claim 1, wherein the input layer to the output layer has a signal transfer vector. The three-dimensional neural network system of claim 2, wherein the signal transmission vector is based on the order of the at least one hidden layer as the signal transmission direction. The three-dimensional neural network system according to claim 1, wherein the number of layers of the at least one hidden layer is four layers, which are sequentially the first hidden layer, the second hidden layer, the third hidden layer, and the fourth hidden layer. The three-dimensional neural network system according to claim 4, wherein the number of hidden layer nodes included in each hidden layer is two.

Images