WO2020042832A1

WO2020042832A1 - Method and device for automatically adding/removing neural network nodes, and storage medium

Info

Publication number: WO2020042832A1
Application number: PCT/CN2019/097181
Authority: WO
Inventors: 洪国强; 肖龙源; 蔡振华; 李稀敏; 刘晓葳; 谭玉坤
Original assignee: 厦门快商通信息技术有限公司
Priority date: 2018-08-29
Filing date: 2019-07-23
Publication date: 2020-03-05
Also published as: CN109508781A

Abstract

Disclosed are a method and device for automatically adding/removing neural network nodes, and a storage medium. The method is such that a neural network structure is designed as required, data is used to train a neural network model, thus allowing the neural network model to converge or the number of iterations of a neural network to exceed a certain threshold; and addition/removal operations are performed with respect to neurons on a per layer basis. The present invention, by means of automatically adding/removing neural network nodes, increases the learning capacity of the neurons on the one hand, and reduces the amount of useless computation for the neurons on the other hand.

Description

Method, device and storage medium for self-increment and decrease of neural network nodes

Technical field

The present invention relates to the field of neural networks, and in particular, to a method, device, and storage medium for self-incrementing and decreasing of neural network nodes.

Background technique

Neural networks (Neural Networks, abbreviated as NNs), or Connection Models, are mathematical models of algorithms that imitate the behavioral characteristics of animal neural networks and perform distributed parallel information processing. This kind of network depends on the complexity of the system and adjusts the interconnection relationship between a large number of internal nodes to achieve the purpose of processing information.

A neural network consists of many neurons. Each neuron is also called a unit or a node. Neurons are connected together to form a network. Generally, neurons form multiple layers and are divided into three types of layers: input layer, hidden layer, and output layer, where the first layer is the input layer, and the first layer can contain one or more neurons; the output layer can contain one Or multiple neurons; the hidden layer can be one or more layers. Neurons in each layer will feed their output to the inputs of the neurons in the next hidden layer.

After the training set of the neural network is determined, the number of nodes in the input layer and the number of nodes in the output layer are determined accordingly. A very important and difficult problem encountered first is how to optimize the number of nodes in the hidden layer. Experiments show that once the number of hidden nodes in the neural network is too small, the network cannot have the necessary learning and information processing capabilities. Conversely, if the number of hidden nodes in the neural network is too large, it will not only greatly increase the complexity of the network structure (this is especially important for hardware-implemented networks), the neural network is more likely to fall into the local minimum during the learning process, and it will Makes the learning speed of the neural network very slow. The selection of the number of nodes in the hidden layer has been highly valued.

Summary of the Invention

The purpose of the present invention is to overcome the existing technical problems, and propose a method, a device and a storage medium for self-incrementing and decreasing of a neural network node, so as to improve the learning ability of the neuron and reduce the useless calculation amount of the neuron.

In order to achieve the above object, the present invention provides a self-incrementing and subtracting method of a neural network node, which is applied to the training of a neural network model. The original neural network model is labeled as Mo, and the neural network model Mo is performed layer by layer. Self-increasing or decreasing operation processing; the self-increasing or decreasing method includes the following steps:

Step 1: Determine whether there are deductible neurons in the current layer. If yes, go to step 2; otherwise, go to step 4;

In step two, the neurons in the current layer are subjected to a subtraction operation, and the neural network model after the subtraction operation processing is marked as Mnew, and the weights and biases of the associated neurons are iteratively updated. The element refers to the neuron of the next layer associated with the deleted neuron; after a certain number of iterations or convergence, iteratively trains the neural network model Mnew after the addition operation process;

Step 3: Compare the error function value of the loss function value of the neural network model Mnew after the iterative training with the loss function value of the neural network model Mo before the subtraction operation processing is less than the first set value a1, and then set the neural network model as the neural network. Model Mnew, return to step one; otherwise, restore the neural network model to neural network model Mo, and proceed to step four;

Step four: add the neurons of the current layer to the operation layer, mark the neural network model processed by the operation as Mnew, add a neuron to the current layer, and iterate the weights and biases of the newly added neurons. Update; after a certain number of iterations or convergence, the iterative training neural network model Mnew is then iteratively trained;

Step 5: Compare the error function of the neural network model Mnew after the iterative training with the loss function value of the neural network model Mo before the increase operation process is less than the second set value a2, then accept the neural network model Mnew and return Step four, otherwise return to step one to enter the neuron increase / decrease operation processing until all neuron layers are processed.

Preferably, in step three, when the neural network model Mnew converges or after a certain iteration, the error ratio between the loss function value of the neural network model Mnew and the loss function value of the neural network model Mo before the subtraction operation processing is less than the first set value, then Set the neural network model as the neural network model Mnew; set the loss value of the neural network model Mnew as Lossnew, the loss value of the neural network model Mo as Losso, and the first set value a1, when (Lossnew-Losso) / Losso <a1 Accept the neural network model Mnew. When (Lossnew-Losso) / Losso≥a1, restore the neural network model to the neural network model Mo.

Preferably, in step 5, after the training is completed, the error ratio between the loss function value of the neural network model Mnew and the loss function value of the neural network model Mo before the increase operation processing is less than the second set value, then the neural network model Mnew is accepted; The second set value a2, when (Lossnew-Losso) / Losso <a2, the neural network model Mnew that has been processed by the increment operation, and when (Lossnew-Losso) / Losso≥a2, the neural network model is restored to the neural network model Mo.

Further, according to an embodiment of the present invention, judging whether a neuron that can be subtracted exists in the current layer is specifically:

Let the current layer be the i-th layer and the current node be the j-th node of the i-th layer, denoted as Xij, and judge

Whether it is less than the set threshold, if

Less than the set threshold, the node Xij can be deleted, where Winijt is the input weight vector, t = (1 ～ n), that is: Winij = {Winij1, Winij2 ... Winijn, bij}; Woutijp is the output weight vector p = (1 ～ m), that is: Woutij = {Woutij1, Woutij2 ... Woutijm}, and bij is an offset value.

Further, the present invention iteratively updates the weights and biases of the associated neurons after performing a subtraction operation on the neurons in the current layer, specifically:

Let the current layer be the i-th layer, the current node is the j-th node of the i-th layer, the input of the deleted neuron is Xij, the input weight vector corresponding to the deleted neuron is Wij, and the output corresponding to the deleted neuron Foutij = F (Xij * Wij);

Let the bias of the associated neuron corresponding to the (i + 1) layer be b (i + 1) j, and the corresponding input weight vector of the associated neuron be W (i + 1) j, then After the iterative update, the offset update value is: b` (i + 1) j = b (i + 1) j + foutij * W (i + 1) j.

Further, in another embodiment of the present invention, determining whether a neuron that can be subtracted exists in the current layer is specifically:

The current layer is the i-th layer, and the next layer is the (i + 1) -th layer, where j ≠ j ’,

Take the j-th neuron Xij of the i-th layer and the j'-th neuron Xij 'of the i-th layer,

Compare if and only if the (i + 1) th layer of neurons connected to Xij and Xij 'are the same,

θ represents the angle between the two input weights. When θ is less than the set threshold (Ф), it is judged that there is a subtractable node in the current layer;

Among them, Ф is the set value; the input weight vector of Xij is Winij = {Winij1, Winij2 ... Winijn, bij}, the output weight vector of Xij is Woutij = {Woutij1, Woutij2 ... Woutijm}, and bij is the offset value;

The input weight vector of Xij 'is Winij' = {Winij'1, Winij'2 ... Winij'n, bij '}, and the output weight vector of Xij' is Woutij '= {Woutij'1, Woutij'2 ... Woutij'm}, bij 'is the offset value.

Further, in the self-incrementing and subtracting method of the neural network node described above, the neuron in the current layer is subjected to a subtraction operation, specifically, deleting both the node Xij and the node Xij ′, and adding a new neuron Xij ”;

When Winij · Winij '> 0, Winij "= (Winij + Winij') / 2, Woutij" = Woutij + Woutij ';

When Winij · Winij '<0, Winij "= (-Winij + Winij') / 2, Woutij" =-Woutij + Woutij '.

Corresponding to the method, the present invention also provides a self-incrementing and decreasing device of a neural network node, which is applied to the training of a neural network model, which marks the original neural network model as Mo, and layers the neural layer by layer. The network model Mo performs self-increasing and decreasing operations; the self-increasing and decreasing device includes:

A model analysis module is used to determine whether there are neurons that can be subtracted in the current layer, and if so, the subtraction operation processing module is executed, otherwise the increment operation processing module is executed;

The subtraction operation processing module is configured to perform subtraction processing on the neurons in the current layer, mark the neural network model processed by the subtraction operation as Mnew, and perform iterative update on the weights and offsets of the associated neurons. The associated neuron refers to the neuron in the next layer associated with the deleted neuron; after a certain number of iterations or convergence, the neural network model Mnew after the addition operation is subjected to iterative training;

The first error checking module is used to compare the error ratio between the loss function value of the neural network model Mnew after iterative training and the loss function value of the neural network model Mo before the subtraction operation processing is less than the first set value a1, and the neural network The network model is set to the neural network model Mnew, and the model analysis module is returned; otherwise, the neural network model is restored to the neural network model Mo, and an operation processing module is added;

The augmented operation processing module is used to augment the neurons in the current layer, and mark the neural network model processed by the augmented operation as Mnew, add a neuron to the current layer, and add the weight and Iteratively update the bias; after a certain number of iterations or convergence, iteratively train the neural network model Mnew after the increase operation is processed;

The second error checking module is used to compare the error function value of the loss function value of the neural network model Mnew after the iterative training with the loss function value of the neural network model Mo before the increase operation processing is less than the second set value a2, then the neural network is accepted. The network model Mnew is returned to the increase operation processing module, otherwise it returns to the model analysis module to enter the increase and decrease operation processing of the neurons in the next layer, and ends until all the neuron layers are processed.

Further, the present invention also provides a computer-readable storage medium that stores computer instructions, and when the computer instructions are executed by a processor, the self-increment of the neural network node according to any one of the foregoing is implemented. Subtraction method

Compared with the prior art, the method, device and storage medium for self-incrementing and decreasing of a neural network node of the present invention can not only improve the learning ability of a neuron, but also calculate

Calculate the value of a neuron. When its value is less than a set threshold, it indicates that the value of the node is not large and can be deleted; or determine whether the contributions of Xij and Xij 'in the current layer are the same, and delete Xij and Xij ', replaced with Xij ", can reduce the amount of useless computation of neurons.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings described herein are used to provide a further understanding of the invention and constitute a part of the invention. The exemplary embodiments of the invention and the description thereof are used to explain the invention, and do not constitute an improper limitation on the invention. In the drawings:

FIG. 1 is a schematic flowchart of a self-incrementing / decreasing method of a neural network node according to Embodiment 1 of the present invention; FIG.

FIG. 2 is a schematic flowchart of a self-incrementing / decreasing method of a neural network node according to Embodiment 2 of the present invention.

detailed description

In order to make the technical problems, technical solutions and beneficial effects to be more clearly understood by the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

In the present invention, a neural network is composed of many neurons, and each neuron is also called a unit or a node, and the neurons are connected together to form a network. Generally, neurons form multiple layers and are divided into three types of layers: input layer, hidden layer, and output layer, where the first layer is the input layer, and the first layer can contain one or more neurons; the output layer can contain one Or multiple neurons; the hidden layer can be one or more layers, and each layer of neurons will feed their output to the input of the neuron in the next hidden layer.

In order to optimize the number of nodes in the hidden layer, the present invention provides a method for self-incrementing and decreasing of neural network nodes, which is applied to the training of a neural network model. The original neural network model is labeled as Mo and the neural network is layer-by-layer. The model Mo performs self-increasing and decreasing operations; the self-increasing and decreasing method includes the following steps:

Among them, how to determine whether an increase or decrease operation is required, a threshold value judgment method or a contribution degree judgment method may be adopted, and specific implementation methods are described as follows:

Example 1 (accumulating or decreasing neurons according to the threshold method)

An embodiment of the present invention provides a method for increasing or decreasing a neural network node, which is applied to training a neural network model. The method for increasing or decreasing a neural network node includes the following steps, as shown in FIG. 1:

Step S11: design the neural network structure according to the requirements, and train the neural network model using the data to make the neural network model converge or the number of iterations of the neural network exceeds a certain threshold;

In step S12, the neuron is increased or decreased layer by layer;

Mark the current neural network model as Mo to determine whether there are deductible neurons in the current layer. If there are deductible neurons in the current layer, go to step S13, otherwise, go to step S15;

Among them, the current layer is the i-th layer, and the current node is the j-th node of the i-th layer, denoted as Xij.

Whether it is less than the set threshold value. In the embodiment of the present invention, the set threshold value is preferably less than 0.01. It should be noted that the set threshold value can be adjusted according to the model and requirements.

If

Less than the set threshold, the node Xij can be deleted. The expression

Calculate the value of a neuron. When its value is less than a set threshold, it indicates that the value of the node is not large, and can be deleted. When its value is greater than or equal to a set threshold, it indicates that the value of the node is large Cannot be deleted;

Among them, Winijt is the input weight vector, t = (1 ～ n), that is: Winij = {Winij1, Winij2 ... Winijn, bij}; Woutijp is the output weight vector, p = (1 ～ m), that is: Woutij = {Woutij1, Woutij2 ... Woutijm}, and bij is the offset value.

Step S13: Perform a subtraction operation on the neurons in the current layer, delete the node Xij, and mark the neural network model after the subtraction operation as Mnew, where the current layer is the i-th layer;

Step S14: Perform iterative training. When the neural network model Mnew converges or the value of the loss function of the neural network model Mnew after a certain iteration is compared with the loss function of the neural network model Mo before the increase or decrease operation processing is within an acceptable range, return to step S12; otherwise, restore the neural network model to the neural network model Mo, and proceed to step S15;

For the i layer, the jth neuron has its weight expressed as Wij = {Wij1, Wij2, .. Wijn, bij}, n is the number of inputs, and bij is the bias value; its output weight vector is Wijout = {Wijout1, Wijout2, .. Wijoutm}, where m is the number of i + 1 layers that use this neuron as input.

Since subtracting some neurons from the i-th layer will affect the input of the i + 1 layer, when the subtraction operation is met, the following optimizations are performed to reduce the impact of the subtraction operation; that is, the neurons in the current layer are replaced in this embodiment. After the element performs the subtraction operation, it further iteratively updates the weights and biases of the associated neurons, specifically:

The smaller the loss function value of the neural network model, the better its model performance. In step S14, when the neural network model Mnew converges or after a certain iteration, the value of the loss function of the neural network model Mnew is compared with the neural network before the increase or decrease operation processing. The loss function value of the model Mo is in an acceptable range, specifically, the error ratio of the loss function value of the neural network model Mnew after the neural network model Mnew converges or after a certain iteration is compared with the loss function value of the neural network model Mo before the subtraction operation processing. Less than the first set value, the neural network model is set to the neural network model Mnew, the loss value of the neural network model Mnew is Lossnew, the loss value of the neural network model Mo is Losso, and the first set value is a1, when (Lossnew -Losso) / Losso <a1, accept the neural network model Mnew, and when (Lossnew-Losso) / Losso≥a1, restore the neural network model to the neural network model Mo, where the first set value a1 can be determined according to the model and Adjustment needs to be made. In the embodiment of the present invention, a1 = 0 is preferable. It should be noted that a1 may also be greater than 0.

In step S15, the neurons in the current layer are processed by adding operations, and the neural network model processed by the adding operations is marked as Mnew. A neuron is added to the current layer and randomly initialized. The current layer is the i-th layer. The added neuron is the j-th neuron in the i-th layer, labeled Xij; proceed to step S16;

Step S16, using the data to iteratively update only the newly-added neurons Xij Winij and Winoutij, and after a certain number of iterations or convergence, iteratively train the neural network model Mnew after the addition operation is processed; compare the neural network after training The loss function value of the model Mnew and the loss function value of the neural network model Mo. If there is a good change in the value, accept the neural network model Mnew and return to step S15, otherwise return to step S12 to enter the next layer (i.e. (1 layer) The increase and decrease of neurons is processed until all the neuron layers are processed.

The smaller the loss function value of the neural network model, the better its model performance. In step S16, after training, compare the loss function value of the neural network model Mnew with the loss function value of the neural network model Mo. If the value is good Changes in the neural network model Mnew, specifically: comparing the loss function value of the neural network model Mnew after the training with the loss function value of the neural network model Mo before the increase operation is smaller than the second set value, then Accept the neural network model Mnew and set the second set value to a2. When (Lossnew-Losso) / Losso <a2, accept the neural network model Mnew that is processed by the augmented operation. When (Lossnew-Losso) / Losso≥a2, set The neural network model is reduced to a neural network model Mo, where the second set value a2 can be adjusted according to the model and needs. In the embodiment of the present invention, preferably, a2 = 0, of course, a2 can also be greater than zero.

Example 2 (increase or decrease neurons based on contribution)

An embodiment of the present invention provides a method for increasing or decreasing a neural network node, which is applied to training a neural network model. The method for increasing or decreasing a neural network node includes the following steps, as shown in FIG. 2:

Step S21: design the neural network structure according to the requirements, and train the neural network model using the data to make the neural network model converge or the number of iterations of the neural network exceeds a certain threshold;

In step S22, the neuron is increased or decreased layer by layer;

Mark the current neural network model as Mo, and determine whether there are deductible neurons in the current layer. If there are deductible neurons in the current layer, go to step S23, otherwise, go to step S25;

θ represents the angle between the two input weights. When θ is less than the set threshold (Ф) (θ <threshold (Ф)), the contribution of Xij and Xij 'is considered to be the same. It can be judged that there is a subtractable node in the current layer. In the embodiment of the present invention, preferably, Φ <1 °, it should be noted that, Φ to is adjusted according to a specific model and requirements.

Step S23: Perform a subtraction operation on the neurons in the current layer, and mark the neural network model after the subtraction operation as Mnew, where the current layer is the i-th layer;

Subtract the neurons in the current layer, remove Xij and Xij 'at the same time, replace them with a new neuron Xij ", specifically delete both nodes Xij and Xij', and add a new neuron Xij ",

When Winij · Winij '> 0, Winij "= (Winij + Winij') / 2, Woutij" = Woutij + Woutij '; When Winij · Winij' <0, Winij "= (-Winij + Winij ') / 2, Woutij "=-Woutij + Woutij ';

In step S24, iterative training is performed. When the value of the loss function of the neural network model Mnew after convergence or after a certain iteration is compared with the loss function value of the neural network model Mo before the increase or decrease operation processing is within an acceptable range, return to step S22; otherwise, restore the neural network model to the neural network model Mo, and proceed to step S25;

The smaller the loss function value of the neural network model is, the better its model performance is. In step S24, when the neural network model Mnew converges or after a certain iteration, the value of the loss function of the neural network model Mnew and the neural network before the increase and decrease operations are processed. The loss function value of the model Mo is in an acceptable range, specifically, the error ratio of the loss function value of the neural network model Mnew after the neural network model Mnew converges or after a certain iteration is compared with the loss function value of the neural network model Mo before the subtraction operation processing. Less than the first set value, the neural network model is set to the neural network model Mnew, the loss value of the neural network model Mnew is Lossnew, the loss value of the neural network model Mo is Losso, and the first set value is a1, when (Lossnew -Losso) / Losso <a1, accept the neural network model Mnew, and when (Lossnew-Losso) / Losso≥a1, restore the neural network model to the neural network model Mo, where the first set value a1 can be determined according to the model and Adjustment needs to be made. In the embodiment of the present invention, preferably a1 ≦ 0. It should be noted that a1 may be greater than 0.

In step S25, the neurons in the current layer are processed by adding operations, and the neural network model processed by the adding operations is marked as Mnew. A neuron is added to the current layer and initialized randomly, and the current layer is the i-th layer. The added neuron is the j-th neuron in the i-th layer, labeled Xij; proceed to step S26;

In step S26, the data is used to iteratively update only the newly-added neurons Xij Winij and Winoutij. After a certain number of iterations or convergence, the neural network model Mnew after the addition operation is iteratively trained; the neural network is compared after the training is completed The loss function value of the model Mnew and the loss function value of the neural network model Mo. If there is a good change in the value, accept the neural network model Mnew and return to step S25, otherwise return to step S22 to enter the next layer (i.e. (1 layer) The increase and decrease of neurons is processed until all the neuron layers are processed.

The smaller the loss function value of the neural network model, the better its model performance. In step S26, after training, compare the loss function value of the neural network model Mnew with the loss function value of the neural network model Mo. If the value is good Changes in the neural network model Mnew, specifically: comparing the loss function value of the neural network model Mnew after the training with the loss function value of the neural network model Mo before the increase operation is smaller than the second set value, then Accept the neural network model Mnew and set the second set value to a2. When (Lossnew-Losso) / Losso <a2, accept the neural network model Mnew that is processed by the augmented operation. When (Lossnew-Losso) / Losso≥a2, set The neural network model is reduced to a neural network model Mo, where the second set value a2 can be adjusted according to the model and needs. In the embodiment of the present invention, preferably, a2 ≤ 0, of course, a2 can also be greater than 0.

Example 3 (device example corresponding to the method)

In the embodiment of the present invention, a self-incrementing / decreasing device for a neural network node is also provided, which is applied to the training of a neural network model. The original neural network model is labeled as Mo, and the neural network model Mo is performed layer by layer. Self-increasing or decreasing operation processing; the self-increasing or reducing device includes:

The self-increasing or decreasing device includes at least one processor and a memory communicatively connected to the at least one processor, the memory stores instructions executable by the at least one processor, and the instructions are processed by the at least one processor. The processor executes, so that the at least one processor executes a self-incrementing / decreasing method of a neural network node. The self-incrementing / decreasing method of the neural network node is the same as that in Embodiment 1 or 2. The present invention is not described herein again.

Embodiment 4 (storage medium corresponding to the method)

In the embodiment of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium stores computer instructions, and the computer instructions are executed by a processor to perform the above-mentioned method for increasing or decreasing a neural network node. The method for self-incrementing and decrementing of network nodes is the same as that in Embodiment 1 or 2. The present invention will not repeat them here.

It should be noted that Embodiment 1 and Embodiment 2 of the present invention are two methods for judging whether or not there are subtractable neurons in the current layer. Of course, the two can also be combined under the technical inspiration of the embodiment of the present invention.

In the description of the specification of the present invention, the above description describes the preferred embodiments of the present invention, but it should be understood that the present invention is not limited to the above embodiments and should not be regarded as excluding other embodiments. Those of ordinary skill in the art can understand that various changes, modifications, substitutions, and alterations can be made to these embodiments without departing from the principle and spirit of the present invention, and the scope of the present invention is defined by the claims and their equivalents.

Claims

A self-incrementing and decreasing method of a neural network node, which is applied to the training of a neural network model, is characterized in that the original neural network model is marked as Mo, and the neural network model Mo is processed by self-incrementing and decreasing operation layer by layer; The self-increase / decrease method includes the following steps:

Step 1: Determine whether there are deductible neurons in the current layer. If yes, go to step 2; otherwise, go to step 4;

In step two, the neurons in the current layer are subjected to a subtraction operation, and the neural network model after the subtraction operation processing is marked as Mnew, and the weights and biases of the associated neurons are iteratively updated. The element refers to the neuron of the next layer associated with the deleted neuron; after a certain number of iterations or convergence, iteratively trains the neural network model Mnew after the addition operation process;

Step 3: Compare the error function value of the loss function value of the neural network model Mnew after the iterative training with the loss function value of the neural network model Mo before the subtraction operation processing is less than the first set value a1, and then set the neural network model as the neural network. Model Mnew, return to step one; otherwise, restore the neural network model to neural network model Mo, and proceed to step four;

Step four: add the neurons of the current layer to the operation layer, mark the neural network model processed by the operation as Mnew, add a neuron to the current layer, and iterate the weights and biases of the newly added neurons. Update; after a certain number of iterations or convergence, the iterative training neural network model Mnew is then iteratively trained;

Step 5: Compare the error function of the neural network model Mnew after the iterative training with the loss function value of the neural network model Mo before the increase operation process is less than the second set value a2, then accept the neural network model Mnew and return Step four, otherwise return to step one to enter the neuron increase / decrease operation processing until all neuron layers are processed.
The method for increasing or decreasing a neural network node according to claim 1, wherein:

Determining whether there are neurons that can be subtracted in the current layer is as follows:

Let the current layer be the i-th layer and the current node be the j-th node of the i-th layer, denoted as Xij, and judge
Whether it is less than the set threshold, if
Less than the set threshold, the node Xij can be deleted, where Winijt is the input weight vector, t = (1 ～ n), that is: Winij = {Winij1, Winij2 ... Winijn, bij}; Woutijp is the output weight vector p = (1 ～ m), that is: Woutij = {Woutij1, Woutij2 ... Woutijm}, and bij is an offset value.
The method for increasing or decreasing a neural network node according to claim 1, wherein:

After performing the subtraction operation on the neurons in the current layer, the weights and biases of the associated neurons are updated iteratively, as follows:

Let the current layer be the i-th layer, the current node is the j-th node of the i-th layer, the input of the deleted neuron is Xij, the input weight vector corresponding to the deleted neuron is Wij, and the output corresponding to the deleted neuron Foutij = F (Xij * Wij);

Let the bias of the associated neuron corresponding to the (i + 1) layer be b (i + 1) j, and the corresponding input weight vector of the associated neuron be W (i + 1) j, then After the iterative update, the offset update value is: b` (i + 1) j = b (i + 1) j + foutij * W (i + 1) j.
The method for increasing or decreasing a neural network node according to claim 1, wherein:

Determining whether there are neurons that can be subtracted in the current layer is as follows:

Let the current layer be the ith layer and the next layer be the (i + 1) th layer, where j ≠ j ’,

Take the j-th neuron Xij of the i-th layer and the j'-th neuron Xij 'of the i-th layer,

Compare if and only if the (i + 1) th layer of neurons connected to Xij and Xij 'are the same,
θ represents the angle between the two input weights. When θ is less than the set threshold (Ф), it is judged that there is a subtractable node in the current layer;

Among them, Ф is the set value; the input weight vector of Xij is Winij = {Winij1, Winij2 ... Winijn, bij}, the output weight vector of Xij is Woutij = {Woutij1, Woutij2 ... Woutijm}, and bij is the offset value;

The input weight vector of Xij 'is Winij' = {Winij'1, Winij'2 ... Winij'n, bij '}, and the output weight vector of Xij' is Woutij '= {Woutij'1, Woutij'2 ... Woutij'm}, bij 'is the offset value.
The method for automatically increasing or decreasing a neural network node according to claim 4, wherein:

Subtract the neurons in the current layer, specifically deleting both the node Xij and the node Xij ’and adding a new neuron Xij",

When Winij · Winij '> 0, Winij "= (Winij + Winij') / 2, Woutij" = Woutij + Woutij ';

When Winij · Winij '<0, Winij "= (-Winij + Winij') / 2, Woutij" =-Woutij + Woutij '.
The method for increasing or decreasing a neural network node according to claim 1, wherein:

In step three, when the neural network model Mnew converges or the error ratio between the loss function value of the neural network model Mnew and the loss function value of the neural network model Mo before the subtraction operation is less than the first set value, the neural network is changed. The model is set to the neural network model Mnew; the loss value of the neural network model Mnew is Lossnew, the loss value of the neural network model Mo is Losso, and the first set value is a1. When (Lossnew-Losso) / Losso <a1, the neural network is accepted. The network model Mnew restores the neural network model to the neural network model Mo when (Lossnew-Losso) / Losso≥a1.
The method for increasing or decreasing a neural network node according to claim 1, wherein:

In step 5, after the training, the error ratio between the loss function value of the neural network model Mnew and the loss function value of the neural network model Mo before the increase operation process is less than the second set value, then the neural network model Mnew is accepted; the second set The fixed value a2, when (Lossnew-Losso) / Losso <a2, the neural network model Mnew that has been processed by the increment operation, and when (Lossnew-Losso) / Losso≥a2, the neural network model is restored to the neural network model Mo.
A self-incrementing and decreasing device for a neural network node, which is applied to the training of a neural network model, is characterized in that the original neural network model is marked as Mo, and the neural network model Mo is processed by self-incrementing and decreasing operation layer by layer; The self-increasing and decreasing device includes:

A model analysis module is used to determine whether there are neurons that can be subtracted in the current layer, and if so, the subtraction operation processing module is executed, otherwise the increment operation processing module is executed;

The subtraction operation processing module is configured to perform subtraction processing on the neurons in the current layer, mark the neural network model processed by the subtraction operation as Mnew, and perform iterative update on the weights and offsets of the associated neurons. The associated neuron refers to the neuron in the next layer associated with the deleted neuron; after a certain number of iterations or convergence, the neural network model Mnew after the addition operation is subjected to iterative training;

The first error checking module is used to compare the error ratio between the loss function value of the neural network model Mnew after iterative training and the loss function value of the neural network model Mo before the subtraction operation processing is less than the first set value a1, and the neural network The network model is set to the neural network model Mnew, and the model analysis module is returned; otherwise, the neural network model is restored to the neural network model Mo, and an operation processing module is added;

The augmented operation processing module is used to augment the neurons in the current layer, and mark the neural network model processed by the augmented operation as Mnew, add a neuron to the current layer, and add the weight and Iteratively update the bias; after a certain number of iterations or convergence, iteratively train the neural network model Mnew after the increase operation is processed;

The second error checking module is used to compare the error function value of the loss function value of the neural network model Mnew after the iterative training with the loss function value of the neural network model Mo before the increase operation processing is less than the second set value a2, then the neural network is accepted. The network model Mnew is returned to the increase operation processing module, otherwise it returns to the model analysis module to enter the increase and decrease operation processing of the neurons in the next layer, and ends until all the neuron layers are processed.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the self-implementation of the neural network node according to any one of claims 1-7 is performed. Increase or decrease method.