TW201636905A - Neural network and method of neural network training - Google Patents

Abstract

A neural network includes a plurality of inputs for receiving input signals, and synapses connected to the inputs and having corrective weights. The network additionally includes distributors. Each distributor is connected to one of the inputs for receiving the respective input signal and selects one or more corrective weights in correlation with the input value. The network also includes neurons. Each neuron has an output connected with at least one of the inputs via one synapse and generates a neuron sum by summing corrective weights selected from each synapse connected to the respective neuron. Furthermore, the network includes a weight correction calculator that receives a desired output signal, determines a deviation of the neuron sum from the desired output signal value, and modifies respective corrective weights using the determined deviation. Adding up the modified corrective weights to determine the neuron sum minimizes the subject deviation for training the neural network.

Description

Neural network and neural network training method [Reciprocal Reference of Related Applications]

This application is based on the U.S. Provisional Application No. 61/949,210 filed on March 6, 2014, and the U.S. Provisional Application No. 62/106,389, issued on January 22, 2015, March 2015. The PCT application filed on the 6th, PCT/US2015/19236, the US provisional application number No. 62/173,163, issued on June 9, 2015, and the US invention application number number issued on September 23, 2015 The benefit of 14/862,337, the entire contents of which are hereby incorporated by reference.

The disclosure relates to an artificial neural network and a method of training an artificial neural network.

In machine learning, the term "neural network" generally refers to a software and/or computer architecture, that is, the overall design or structure of a computer system or a microprocessor, including the hardware and software needed to perform it. The artificial neural network is a series of statistical learning algorithms driven by a biological neural network, also known as the central nervous system of animals (especially the brain). Artificial neural networks are primarily used to estimate or approximate estimates of functions that may depend on a large number of inputs that are not known in general. This type of neural network has been used for a wide variety of tasks that are difficult to solve using general rule-based programming, including computer vision and speech recognition.

Artificial neural networks are generally presented as "neuron" systems that can calculate values from inputs and are capable of machine learning and pattern recognition due to their adaptive nature. Each neuron is often connected to several inputs by a synapse with a weight of the nerve knot.

Neural network systems are not programmed as typical software and hardware, but are trained. Such training systems are typically achieved by analysis of a sufficient number of representative instances and by selection of statistics or algorithms for neural node weights such that a given set of input images is an output image corresponding to a given set. A common criticism of classical neural networks is that they often require considerable time and other resources for their training.

A variety of artificial neural networks are described in U.S. Patent Nos. 4,979,124, 5,479,575, 5,493,688, 5,566,273, 5,682,503, 5,870,729, 7,577,631, and 7,814,038.

The network includes a plurality of network inputs such that each input is assembled to receive an input signal having an input value. The neural network also includes a plurality of neural nodes, wherein each neural node is coupled to one of the plurality of inputs and includes a plurality of corrective weights, wherein each of the correction weights is defined by a weight value. The neural network additionally includes a set of dispensers. Each of the dispensers is operatively coupled to one of the plurality of inputs for receiving respective input signals and is configured to select one or more correction weights from the plurality of correction weights associated with the input values. The neural network also includes a set of neurons. Each neuron has at least one output and is coupled to at least one of the plurality of inputs via one of the plurality of neural nodes and is configured to select from each of the neural nodes connected to the respective neuron The The weight values of the correction weights are summed and thereby a sum of neurons is generated. Furthermore, the neural network system includes a weight correction calculator that is configured to receive a desired output signal having a value, determine a deviation of the sum of the neurons from the expected output signal value, and use the determined Deviation to modify the individual correction weight values. The modified correction weight values are aggregated to determine that the neuron sum is minimized by the deviation of the neuron sum from the expected output signal value to provide training for the neural network. Such a neural network can be implemented and implemented as a soft body and/or a hardware.

The determination of the deviation of the sum of the neurons from the desired output signal may include dividing the desired output signal value by the sum of the neurons, thereby generating a coefficient of variation. Moreover, the modification of the respective correction weights can include the respective correction weights used to generate the sum of the neurons multiplied by the deviation coefficients.

The deviation of the sum of the neurons from the desired output signal can be a mathematical difference therebetween. In such a case, the respective modified correction weights may be generated by including the mathematical differences to the respective correction weights used to generate the sum of the neurons. The distribution of the mathematical difference to each of the correction weights is intended to converge the sum of the individual neurons on the desired signal value.

The addition of the mathematical difference may also include dividing the determined difference equally between the respective correction weights used to generate the sum of the neurons.

The dispenser can be additionally configured to assign a plurality of impact coefficients to the plurality of correction weights such that each of the influence coefficients is assigned to one of the plurality of correction weights at a predetermined ratio to generate the The sum of neurons.

Each individual plurality of influence coefficients can be defined by an influence distribution function. The plurality of input values can be received into a range of values, which are divided into intervals according to an interval distribution function, so that the respective input values are received in a respective interval, and each correction The weights correspond to one of the intervals. In addition, each of the dispensers can use the input values received by the input signals to select the respective intervals. In addition, each of the plurality of distributors may assign the respective plurality of influence coefficients to at least one of the correction weights corresponding to the selected respective intervals and to at least one of the intervals corresponding to the respective intervals adjacent to the selection. Weights.

Each neuron system can be assembled to total the product of the correction weights for all of the neural nodes connected thereto to a specified influence coefficient.

The predetermined ratio of the influence coefficients can be defined according to a statistical distribution, such as: using a Gaussian function.

The weight correction calculator can be configured to apply a portion of the determined difference to each of the correction weights used to generate the sum of the neurons based on the ratio established by the respective influence factors.

Each correction weight can be additionally defined by a set of indices. The indexing system can include: an input index configured to identify a correction weight corresponding to the input; an interval index set to specify a selection interval for the respective correction weight; and, a neuron index It is set to specify the correction weight corresponding to the neuron.

Each of the correction weights can in turn be defined by an access index that is set to settle the number of times the respective correction weight is accessed by the input signal during training of the neural network.

A method of training such a neural network is also disclosed.

The above features and advantages, and other features and advantages of the present invention will be apparent from the following detailed description of the embodiments and the preferred embodiments And it is obvious.

10‧‧‧Neural Network

12‧‧‧ Input device

14‧‧‧ nerve knot

16‧‧‧ nerve knot weight

18‧‧‧ neurons

20‧‧‧Adder

22‧‧‧Activity function device

24‧‧‧ neuron output

26‧‧‧ Weight Correction Calculator

28‧‧‧ Training pairing

28-1‧‧‧ Input image

28-2‧‧‧ Expected output image

100‧‧‧ Progressive neural network (p network)

102‧‧‧ Input

104‧‧‧Input signal

106‧‧‧ Input image

108‧‧‧ nerve knot weight

110‧‧‧weight correction block

112‧‧‧correction weight

114‧‧‧Distributor

116‧‧‧ neurons

117‧‧‧ Output

118‧‧‧ nerve knot

119‧‧‧neuronal unit

120‧‧‧The sum of neurons

122‧‧‧ Weight Correction Calculator

124‧‧‧ Expected output signal

126‧‧‧ Output image

128‧‧‧ Deviation

134‧‧‧Impact coefficient

136‧‧‧Impact distribution function

138‧‧‧value range

140‧‧‧ interval distribution function

200‧‧‧ method

202, 204, 206, 208, 210, 212, 214‧‧ box

Figure 1 is an illustration of a classical artificial neural network of the prior art.

2 is an illustration of a "progressive neural network" (p-network) having a plurality of neural nodes, a set of dispensers, and a plurality of correction weights associated with each of the neural nodes.

3A is an illustration of a portion of the p-network shown in FIG. 2 having a plurality of neural nodes and a neural node weight positioned upstream of each of the dispensers.

3B is an illustration of a portion of the p-network shown in FIG. 2 having a plurality of neural nodes and a set of neural node weights positioned downstream of respective plurality of correction weights.

3C is an illustration of a portion of the p-network shown in FIG. 2 having a plurality of neural nodes and a neural node weight positioned upstream of each of the dispensers and positioned at a respective plurality of correction weights. A group of nerve nodes in the downstream are weighted.

4A is an illustration of a portion of the p-network shown in FIG. 2 with a single dispenser for all of the neural inputs of a given input and a neural node weight positioned upstream of each dispenser.

4B is an illustration of a portion of the p-network shown in FIG. 2 having a single distributor for all of the neural inputs of a given input and a set positioned downstream of the respective plurality of correction weights. The nerve knot is weighted.

Figure 4C is an illustration of a portion of the p-network shown in Figure 2 with a single dispenser for all of the neural inputs of a given input, with a neural node weight positioned upstream of each dispenser And a set of neural node weights positioned downstream of the respective plurality of correction weights.

FIG. 5 is an explanatory diagram in which the range of input signal values in the p network shown in FIG. 2 is divided into individual sections.

Fig. 6A is an explanatory diagram of an embodiment of a distribution of influence coefficient values for correction weights in the p network shown in Fig. 2.

Fig. 6B is an explanatory diagram of another embodiment of the distribution of the influence coefficient values for the correction weights in the p network shown in Fig. 2.

Fig. 6C is an explanatory diagram of still another embodiment of the distribution of the influence coefficient values of the correction weights in the p network shown in Fig. 2.

7 is an explanatory diagram for an input image of the p network shown in FIG. 2, and a corresponding table representing the image in the form of a digital code and representing the same image as a group of respective intervals. Another corresponding table.

Figure 8 is an illustration of one embodiment of the p-network shown in Figure 2, the p-network being trained for identification of two distinct images, wherein the p-network is assembled to recognize that it includes individual images. An image of some features.

Figure 9 is an illustration of one embodiment of the p-network shown in Figure 2 with an example of the distribution of neural node weights surrounding a "central" neuron.

Figure 10 is an illustration of one embodiment of the p-network shown in Figure 2 depicting a uniform distribution of training deviations between correction weights.

Figure 11 is an illustration of one embodiment of the p-network shown in Figure 2, which utilizes the modification of the correction weights during p-network training.

Figure 12 is an illustration of one embodiment of the p-network shown in Figure 2, wherein the basic algorithm produces a primary set of output neuron sums, and wherein the generated set is used to generate The sum of several "winners" that retain or increase the value and the contribution of the remaining sums are invalid.

Figure 13 is an illustration of one embodiment of the p-network shown in Figure 2, which identifies a complex image of elements having multiple images.

Figure 14 is an explanatory diagram of a model for the object-oriented programming of the p-network shown in Figure 2 using Unified Modeling Language (UML).

Fig. 15 is an explanatory diagram showing a general formation sequence of the p network shown in Fig. 2.

Fig. 16 is an explanatory diagram showing representative analysis and preparation of data for formation of the p network shown in Fig. 2.

Figure 17 is an illustration of a representative input setup that allows for the interaction of the p-network and input data shown in Figure 2 during training and p-network applications.

Fig. 18 is an explanatory diagram for the establishment of a representative of a neuron unit of the p network shown in Fig. 2.

Figure 19 is an explanatory diagram showing the establishment of a representative representation of each nerve node connected to a neuronal unit.

Fig. 20 is an explanatory diagram of training the p network shown in Fig. 2.

Figure 21 is an explanatory diagram of neuron unit training in the p network shown in Figure 2.

Fig. 22 is an explanatory diagram showing the extension of the sum of neurons during the training of the p network shown in Fig. 2.

Figure 23 is a flow diagram of a method for training the neural network shown in Figures 2-22.

As shown in FIG. 1, a classical artificial neural network 10 system typically includes an input device 12, a neural node 14 having a nerve node weight 16, a neuron 18 (which includes an adder 20 and an actuation function device 22), and a nerve. The meta output 24 and the weight correction calculator 26 are provided. Each neuron 18 is connected to two or more input devices 12 through a neural node 14. Nerve node weight 16 The system is typically represented by electrical resistance, electrical conductivity, voltage, electrical charge, magnetic properties, or other parameters.

The supervised training system of the classical neural network 10 is summarized as an application based on a set of training pairs 28. Each training pair 28 is typically comprised of an input image 28-1 and a desired output image 28-2 (also referred to as a supervisory signal). The training system of the classical neural network 10 is typically provided as follows. An input image in the form of a set of input signals (I ₁ -I _m ) enters the input device 12 and is transferred to its neural node weight 16 having an initial weight (W ₁ ). The value of the input signal is modified by the weights, typically by multiplying or dividing the value of each signal (I ₁ -I _m ) by the respective weight. From these neural node weights 16, the modified input signals are each transferred to a respective neuron 18. Each neuron 18 receives a set of signals from a group of neural nodes 14 associated with the associated neuron 18. The adder 20 included in the neuron 18 sums all of the input signals modified by the weights and received by the associated neuron. The actuation function device 22 receives the sum of the respective synthesized neurons and modifies the sum according to a mathematical function, thus forming the respective output images as a set of neuron output signals (ΣF ₁ ... ΣF _n ).

The neuron output image system defined by the neuron output signals (ΣF ₁ ... ΣF _n ) and the predetermined desired output image (O ₁ -O _n ) are compared by a weight correction calculator 26. Based on a predetermined difference between the neuron output the resulting image with a desired output image ΣF _n O _n for changing these ganglia right signal correction system 16 using a predetermined weight programming algorithms formed. After the correction system is made for all of the neural node weights 16, the set of input signals (I ₁ -I _m ) are again introduced to the neural network 10 and a new calibration system is created. And said circulation system until it is determined based on the difference between the neuron output image obtained ΣF _n O _n and a desired output image is smaller than a predetermined error is repeated. A cycle of network training for all individual images is typically identified as a "training period." In summary, the magnitude of the error is reduced by each training period. However, depending on the number of individual input signals (I ₁ -I _m ) and the number of inputs and outputs, the training system of classical neural network 10 may require a significant number of training periods, which may be similar in some cases. Hundreds of thousands of them.

A variety of classical neural networks exist, including the Hopfield network, the Restricted Boltzmann Machine, the radial basis function network, and the recurrent neural network. The specific tasks of classification and clustering require a specific type of neural network, Self-Organizing Map, which uses only the input image as the network input training information, and the desired output image system corresponding to an input image. A single winning neuron based on an output signal having a maximum value is formed directly during the training process.

As noted above, one of the main concerns with existing classical neural networks (such as neural network 10) is that their successful training system requires considerable time duration. Some additional concerns about classical networks can be a significant expense for computing resources, which in turn will drive the need for powerful computers. Another concern is that without the full recalibration of the network, the size of the network cannot be increased, and for a tendency like "network 瘫痪" and "freezing at a local minimum", It makes it impossible to predict whether a particular neural network will be able to train with a given set of images in a given order. In addition, there may be limitations regarding the particular order of images introduced during training, where changing the order in which training images are introduced may result in network freezes and the inability to perform additional training on an already trained network.

Referring to the remaining figures, wherein the same reference numerals refer to the same components, and FIG. 2 shows a progressive type referred to as "progressive network" or "p-net" 100 hereinafter. Schematic diagram of a neural network. The p-network 100 includes a plurality or set of inputs 102 of the p-network. Each input 102 is assembled to receive an input signal 104, wherein the input signals are represented as I ₁ , I ₂ ... I _{m in} FIG. Each of the input signals I ₁ , I ₂ ... I _m represents a value of some features of an input image 106, such as size, frequency, phase, signal polarization angle, or correlation with a different portion of the input image 106. Each of the input signals 104 has an input value, wherein the plurality of input signals 104 collectively describe the input image 106.

Each input value may be within a range of values between -∞ and +∞ and may be set to a digital and/or analog form. The range of input values can depend on a set of training images. In the simplest case, the range of input values can be the difference between the minimum and maximum values of the input signals for all training images. For practical reasons, the range of input values can be limited by excluding the input values that are considered too high. For example, such a limitation of the range of input values can be achieved via known statistical methods for the reduction of variance, such as importance sampling. Another example of limiting the range of input values is the assignment of all signals assigned to a particular minimum value and all of its signals exceeding a predetermined maximum level to a particular maximum value for all signals below a predetermined minimum level.

The p-network 100 also includes a plurality of or a set of neural nodes 118. Each neural node 118 is coupled to one of a plurality of inputs 102, including a plurality of correction weights 112, and may also include a neural node weight 108, as shown in FIG. Each correction weight 112 is defined by a respective weight value 112. The p-network 100 also includes a set of allocators 114. Each of the distributors 114 is operatively coupled to one of the plurality of inputs 102 for receiving respective input signals 104. In addition, each of the dispensers 114 is configured to select one or more correction weights from a plurality of correction weights 112 associated with the input values.

The p-network 100 additionally includes a set of neurons 116. Each neuron 116 has at least one output 117 and is via at least one neural node 118 and at least a plurality of inputs 102 One is the connection. Each neuron 116 is assembled to sum or total the weight values of the correction weights 112 selected from the respective neural nodes 118 that are connected to the respective neuron 116 and thereby generate a sum of neurons 120, or Σn. A separate dispenser 114 can be used for each nerve node 118 of a given input 102, as shown in Figures 3A, 3B, and 3C, or a single dispenser system can be used for all of the nerve nodes 118, such as 4A, 4B, and 4C. During the formation or setup of the p-network 100, all of the correction weights 112 are assigned initial values that may change during the training process of the p-network. The initial values of the correction weights 112 can be specified as in the classical neural network 10, for example, the weights can be randomly selected, calculated with the help of a predetermined mathematical function, from a predetermined template. Choice, and so on.

The p-network 100 also includes a weight correction calculator 122. The weight correction calculator 122 is configured to receive a desired (i.e., predetermined) output signal 124 having a signal value and representing a portion of an output image 126. The weight correction calculator 122 is also assembled to determine a deviation 128 between the sum of the neuron sum 120 and the desired output signal 124, also known as the training error, and the determined deviation weight 128 is used to modify the respective correction weight value. Thereafter, the modified correction weight values are totaled to determine that the sum of neurons 120 minimizes the deviation of the associated neuron sum from the value of the desired output signal 124, and the result is valid for training the p-network 100.

To be analogous to the classical network 10 discussed with respect to FIG. 1, the deviation 128 can also be described as a training error between the determined sum of neuron 120 and the value of the desired output signal 124. Compared with the classical neural network 10 discussed with respect to FIG. 1, in the p network 100, the input value of the input signal 104 is changed only during the general network setting, and the training in the p network. The period is unchanged. Rather than changing the input value, the training of the p-network 100 is provided by changing the value 112 of the correction weight 112. In addition, although each neuron 116 includes a total of merit Yes, wherein the neuron system sums the correction weight values, the neuron 116 does not require an actuation function and is actually characterized by no actuation function, such as: caused by the classical neural network 10 Provided by the dynamic function device 22.

In the classical neural network 10, the weight correction during training is achieved by changing the neural node weight 16, and in the p network 100, the corresponding weight correction is provided by changing the correction weight value 112. As shown in Figure 2. The respective correction weights 112 can be included in a weight correction block 110 that is positioned over all or some of the neural nodes 118. In neural network computer simulations, individual neural node weights and correction weights can be represented by a digital device (such as a memory unit) and/or by an analog device. In neural network software simulations, the value of the correction weight 112 can be provided via an appropriately programmed algorithm, while in hardware simulation, known methods for memory control can be used.

In p-network 100, the deviation 128 of neuron sum 120 from desired output signal 124 can be expressed as a mathematical difference therebetween. Moreover, the generation of the individually modified correction weights 112 may include the division of the calculated differences to the respective correction weights used to generate the sum of neurons 120. In such an embodiment, the separately modified correction weights 112 will allow the neuron sum 120 to be converged to the desired output signal value within a few periods, in some cases only a single period is required. To quickly train the p network 100. In a particular case, the sharing of the mathematical differences between the correction weights 112 used to generate the sum of neurons 120 may include dividing the determined differences equally among the various corrections used to generate the respective sum of neurons 120. Between weights.

In a different embodiment, the determination of the deviation 128 of the sum of neurons 120 from the desired output signal value can include the desired output signal value divided by the sum of the neurons, thereby generating a coefficient of variation. In this particular case, the modification of the individually modified correction weights 112 includes the respective correction weights used to generate the sum of neurons 120 multiplied by the deviation coefficients. Each of the dispensers 114 can be additionally configured to assign a plurality of influence coefficients 134 to the plurality of correction weights 112. In the present embodiment, each of the influence coefficients 134 may be assigned to one of the plurality of correction weights 112 at a predetermined ratio to produce a respective sum of neurons 120. To correspond to each respective correction weight 112, each influence coefficient 134 can be assigned a "C _i,d,n " designation as shown in the drawing.

Each of the plurality of influence coefficients 134 corresponding to a particular neural node 118 is defined by a respective influence distribution function 136. The influence distribution function 136 may be the same for all impact coefficients 134 or only for a plurality of influence coefficients 134 corresponding to one particular neural node 118. Each of the plurality of input values can be received a range of values 138 that is divided into intervals or sub-divisions "d" according to an interval distribution function 140 such that each input value is received in a respective interval "d" And each correction weight corresponds to one of the intervals. Each of the distributors 114 can use the respective received input values to select respective intervals "d", and assign the respective plurality of influence coefficients 134 to their respective respective intervals "d". The weight 112 is corrected and to at least one correction weight corresponding to an interval adjacent to the selected respective interval, such as: W _{i, d+1, n} or W _{i, d-1, n} . In another non-limiting example, the predetermined ratio of the influence coefficients 134 can be defined according to a statistical distribution.

Generating the sum of neurons 120 may include initially assigning respective influence coefficients 134 to respective correction weights 112 based on the input values 102 and then multiplying the associated influence coefficients by the values of the respective applied correction weights 112. Next, via the individual neurons 116, the individual weights of the correction weights 112 for all of the neural nodes 118 connected thereto are summed with the specified influence coefficients 134.

The weight correction calculator 122 can be assembled to apply the respective influence coefficients 134 to generate respective modified correction weights 112. In particular, the weight correction calculator 122 can apply a portion of the calculated mathematical difference between the neuron sum 120 and the desired output signal 124 to generate the nerve based on the ratio established by the respective influence coefficient 134. The respective correction weights 112 of the sum totals 120. Moreover, the mathematical differences divided between the correction weights 112 used to generate the sum of neurons 120 may be further divided by the respective influence coefficients 134. Subsequently, the result of dividing the sum of neurons 120 by the respective influence factor 134 can be added to the correction weight 112 to cause the sum of neurons 120 to converge on the desired output signal value.

Typically, the formation of p-network 100 will occur before the training of the p-network begins. However, in a different embodiment, if during the training, the p-network 100 receives an input signal 104 for which the initial correction weight is not present, an appropriate correction weight 112 can be generated. In this case, the particular distributor 114 will determine the appropriate interval "d" for the particular input signal 104, and a group of correction weights 112 having the initial value will be for the given input 102, the established interval "d", and All individual neurons 116 are produced. Additionally, a corresponding impact factor 134 can be assigned to each newly generated correction weight 112.

Each correction weight 112 is defined by a set of indices that are set to identify a respective location of the respective correction weights above the p-network 100. The set of indices may explicitly include an input index "i" that is set to identify a correction weight 112 corresponding to a particular input 102; an interval index "d" that is set to specify the above for each correction weight And a neuron index "n" that is set to specify a correction weight 112 corresponding to a particular neuron 116 by the designation "W _i,d,n ". Thus, each correction weight 112 corresponding to a particular input 102 is assigned the particular index "i" to the subscript symbol to indicate the location of the location. Similarly, the respective correction weights "W" corresponding to a particular neuron 116 and a respective neural node 118 are assigned the particular index "n" and "d" to the subscript symbol to indicate above the p network 100. The location of the correction weight. The set of indices may also include an access index "a" that is set to settle the number of times the respective correction weights 112 were accessed by the input signal 104 during the training of the p-network 100. In other words, whenever a particular interval "d" and a respective correction weight 112 are selected for training from a plurality of correction weights associated with the input value, the access index "a" is incremented to count the input signal. . The access index "a" can be used to further specify or define the current state of each correction weight by employing a designation "W _i,d,n,a ". Each of the indices "i", "d", "n", and "a" may be a value in the range of 0 to +∞.

The various possibilities for dividing the range of the input signal 104 into the intervals d ₀ , d ₁ ... d _m are shown in FIG. The particular interval distribution may be uniform or linear, for example, it may be achieved by specifying that all intervals "d" are of the same size. All input signals 104 having their respective input signal values below a predetermined minimum level can be considered to have a zero value, and all input signals 104 having their respective input signal values greater than a predetermined highest level can be specified. To the highest level, as shown in Figure 5. The particular interval distribution may also be non-uniform or non-linear, such as: symmetrical, asymmetrical, or infinite. When the range of the input signals 104 is deemed to be unrealistically large, a non-linear distribution of the interval "d" may be useful, and a portion of the range may include input signals that are considered to be the most critical, such as : at the beginning, middle, or end of the range. This particular interval distribution can also be described by a random function. All of the foregoing examples are non-limiting in nature, as other variations of the interval distribution are also possible.

The number of intervals "d" within the selection range of the input signal 104 can be increased to optimize the p-network 100. The optimization of the p-network 100 can be done, for example, by training It is desirable to increase the complexity of the image 106. For example, a larger number of intervals may be required for a multi-color image compared to a monochrome image, and a larger number of intervals may be required for a complex decoration than for a simple image. Compared to the image described by the contour, it may be necessary to increase the number of intervals for accurate identification of images with complex color gradients, as for a larger overall number of training images. If there is a high amount of noise, high fluctuations in the training image, and excessive consumption of computing resources, it may also be necessary to reduce the number of intervals "d".

Depending on the task or type of information handled by the p-network 100, for example: visual or textual material, data from sensors of various natures, different numbers of intervals and their distribution patterns may be specified. For the interval "d" of each input signal value, the corresponding correction weight of the given neural node having the index "d" can be specified. Therefore, a certain interval "d" will include all of the correction weights 112 having an index "i" for a given input, an index "d" for a given interval; and, for all values of the index "n" from 0 to n . In the process of training the p-network 100, the allocator 114 defines the respective input signal values and thus causes the associated input signal 104 to be associated with the corresponding interval "d". For example, if there are 10 equal intervals "d" in the range of input signals from 0 to 100, the input signal having a value between 30 and 40 is related to interval 3, namely: "d" = 3.

For all of the correction weights 112 of the respective neural nodes 118 to which the given input 102 is connected, the distributor 114 can specify the value of the influence coefficient 134 based on its interval "d" for the particular input signal. The distributor 114 may also specify the value of the influence coefficient 134 based on a predetermined distribution of the values of the influence coefficients 134 (shown in Figure 6) such as a sine wave, a normal state, a logarithmic distribution curve, or a random distribution function. In many cases, the sum or integral of the influence factor 134 or C _i,d,n for a particular input signal 102 for each neural node 118 will have a value of 1 (one).

Σ _Synapse C _i,d,n =1 or ∫ _Synapse C _i,d,n =1 [1] In the simplest case, the correction weight 112 closest to the input signal value can be assigned 1 (one) One value to the influence factor 134 (C _i,d,n ), and the correction weight for the other interval can receive a value of 0 (zero).

Compared to the classical neural network 10, the p-network 100 is designed to reduce the duration of use during training of the p-network with other resources. Although some of the elements disclosed herein as part of the p-network 100 are named by certain names or identifying symbols that are familiar to those familiar with classical neural networks, the specific names are used for simplicity and may be different. It is used by the counterparts of the classical neural network. For example, the neural node weight 16 that controls the magnitude of the input signal (I ₁ -I _m ) is established during the general setup process of the classical neural network 10 and changes during training of the classical network. On the other hand, the training of the p-network 100 is achieved by changing the correction weight 112, while the neural node weights 108 are unchanged during training. Moreover, as discussed above, each of the neurons 116 includes a totaling or adding component, and does not include an actuation function device 22 that is typical for the classical neural network 10.

In summary, p-network 100 is trained by training individual neuron units 119, which include a respective neuron 116 and all connected neural nodes 118, including the particular neuron All of the individual neural nodes 118 to which the associated neuron is connected are corrected for weight 112. Thus, the training system of the p-network 100 includes changing the correction weights 112 that contribute to the respective neurons 116. The change to the correction weight 112 is based on a group training algorithm that includes a method 200 disclosed in detail below. In the revealed calculus In the method, the training error (i.e., the deviation 128) is determined for each neuron, based on which correction values are determined and assigned to each of the weights 112 used to determine the sum obtained by the respective individual neurons 116. By. The introduction of such correction values during training is intended to reduce the deviation 128 for the associated neuron 116 to zero. During training with additional images, new errors in the images that were used earlier may occur again. In order to eliminate such additional errors, the error of all training images for the entire p-network 100 can be calculated after completion of a training period, and if the errors are greater than a predetermined value, one or more additional training periods The system can be performed until the error becomes less than a target or a predetermined value.

Figure 23 depicts a method 200 of training the p-network 100 as described above with respect to Figures 2-22. The method 200 begins at block 202, where the method includes receiving, via input 102, an input signal 104 having an input value. Following block 202, the method proceeds to block 204. At block 204, the method includes passing the input signal 104 to the dispenser 114 that is operatively coupled to the input 102. At block 202 or block 204, the method can include defining, by the set of indices, individual correction weights 112. As described above with respect to the structure of p-network 100, the set of indices can include an input index "i" that is set to identify a correction weight 112 corresponding to the input 102. The set of indices may further include an interval index "d" that is set to specify a selection interval for the respective correction weights 112; and a neuron index "n" that is set to specify that corresponds to the particular neuron 116 The correction weight 112 is "W _i,d,n ". The set of indices may additionally include an access index "a" that is set to settle the number of times the respective correction weights 112 were accessed by the input signal 104 during training of the p-network 100. Therefore, the current state of each correction weight can be named "W _i,d,n,a ".

After block 204, the method proceeds to block 206 where The method includes selecting one or more correction weights 112 via the dispenser 114 from which the plurality of correction weights are located above the neural node 118 connected to the associated input 102 and the input value is associated. As noted above, each of the correction weights 112 is defined by its respective weight value. At block 206, the method can additionally include assigning a plurality of influence coefficients 134 to the plurality of correction weights 112 via the distributor 114. At block 206, the method may further include assigning each of the influence coefficients 134 to one of the plurality of correction weights 112 at a predetermined ratio to generate a sum of neurons 120. Moreover, at block 206, the method can include summing, by the neuron 116, a product of the correction weight 112 for all of the neural nodes 118 connected thereto and a specified influence coefficient 134. Moreover, at block 206, the method can include, via the weight correction calculator 122, applying a portion of the determined difference to the sum used to generate the neuron based on the ratio established by the respective influence coefficient 134 Each of the correction weights 120 of 120.

As described above with respect to the structure of p-network 100, a plurality of influence coefficients 134 may be defined by an influence distribution function 136. In such a case, the method can additionally include receiving the input value into a range of values 138 that is divided into intervals "d" according to an interval distribution function 140 such that the input values are received in each Within the interval, each correction weight 112 corresponds to one of the intervals. Moreover, the method can include, via the distributor 114, using the received input value to select the respective interval "d" and assigning the plurality of influence coefficients 134 to their respective intervals corresponding to the selection. The correction weight 112 of d" and to its at least one correction weight corresponding to an interval adjacent to the respective interval "d" of the selection. As described above with respect to the structure of the p-network 100, the correction weights 112 corresponding to an interval adjacent to the selected respective interval "d" can be identified as, for example, W _{i, d+1, n} or W _{i , d-1, n} .

Following block 206, the method proceeds to block 208. In the box 208. The method includes summing the weight values of the correction weights 112 selected by the particular neuron 116 connected to the input 102 via the neural node 118 to produce the sum of the neurons 120. As described above with respect to the structure of p-network 100, each neuron 116 includes at least one output 117. After block 208, the method continues to block 210, wherein the method includes receiving, via the weight correction calculator 122, a desired output signal 124 having the signal value. Following block 210, the method proceeds to block 212 where the method includes determining, via the weight correction calculator 122, a deviation 128 between the sum of the neuron sum 120 and the desired output signal 124.

As disclosed above in the description of p-network 100, the determination of the deviation 128 of the neuron sum 120 from the desired output signal value can include determining a mathematical difference therebetween. Moreover, the modification of the respective correction weights 112 can include allocating the mathematical differences to respective correction weights used to generate the sum of the neurons 120. Alternatively, the mathematical difference distribution may include dividing the determined difference equally between the respective correction weights 112 used to generate the neuron sum 120. In still another different embodiment, the determining of the deviation 128 can further include dividing the value of the desired output signal 124 by the sum of the neurons 120, thereby generating the coefficient of variation. Moreover, in such a case, the modification of the respective correction weights 112 can include multiplying the respective correction weights 112 used to generate the sum of the neurons 120 by the resulting deviation coefficients.

After block 212, the method continues to block 214. At block 214, the method includes modifying the respective correction weight values using the determined deviation 128 via the weight correction calculator 122. The modified correction weight values can then be aggregated or totaled and then used to determine a new sum of neurons 120. The modified modified weight value of the total is used to minimize the deviation of the sum of the neuron 120 from the expected output signal 124. The p network 100 is trained. Following block 214, method 200 can include returning to block 202 to effect an additional training period until the deviation of neuron sum 120 from the desired output signal 124 is sufficiently minimized. In other words, an additional training period can be implemented to cause the neuron sum 120 to converge above the desired output signal 124 to be within a predetermined deviation or error value, such that the p-network 100 can be considered trained and ready For the operation of new images.

In summary, the input images 106 must be prepared for training by the p-network 100. The preparation of the p-network 100 for training summarizes the formation of a set of training images, including the input images 106 and, in most cases, the desired output image 126 corresponding to the associated input image. The input image 106 (shown in Figure 2) defined by the input signals I ₁ , I ₂ ... I _m for the training of the p-network 100 is selected according to the task that the p-network is designated for disposal, for example: Identification of human images or other objects, identification of certain activities, clustering or data classification, analysis of statistical data, pattern identification, prediction, or control of certain processes. Therefore, the input image 106 can be rendered in any format suitable for introduction to a computer, for example, using the format jpeg, gif, pptx, in the form of a table, a chart, a graph and a graph, various file formats, or a set of symbols. .

The preparation for the training of the p-network 100 may further include: the selected input image 106 is for its unified conversion, in order to facilitate processing of the associated image by the p-network 100, for example, all images Convert to a format that has the same number of signals, or the same number of pixels in the case of an image. The color image can be presented, for example, as a combination of three primary colors. The image conversion system may also include modification of features, such as shifting an image spatially, changing visual features of the image (such as resolution, brightness, contrast, color, angle of view, perspective, focus, and focus), and adding symbols , label, or annotation.

After the selection of the number of intervals, a particular input image can be converted to An input image of the interval format, that is, the actual signal value can be recorded as the number of intervals to which the respective signals to which it belongs. This program can be implemented during each training session for a given image. However, the image system can also be formed as a set of intervals at a time. For example, in Figure 7, the initial image is presented as an image, and in the table "Image in digital format", the same image is presented in the form of a digit code, and in the table "in the format of the interval" Among the images, the image is presented as a set of intervals, and a single interval is assigned to the digit code for each of the 10 values.

The above-described structure of the p-network 100 and the training algorithm or method 200 as described allow for continuous or repeated training of the p-network, so that it is not necessary to form a complete set of training input images 106 at the beginning of the training process. . It is possible to form a training image of a relatively small initial group, and the starting group can be expanded as necessary. The input image 106 can be divided into unique categories, such as: a group of images of a person, a group of cats, or a set of photos of a car, such that each category corresponds to a single output image, the name of the person Or a specific label. The desired output image 126 is a field or table representing a digit (where each point corresponds to a particular value from -∞ to +∞) or an analog value. It is contemplated that the various points of the output image 126 may correspond to the output of one of the neurons of the p-network 100. The desired output image 126 can be encoded with an image, a table, a text, a formula, a set of symbols (such as a bar code), or a digital or analog code of the sound.

In the simplest case, each input image 106 can correspond to an output image that encodes the associated input image. One of the points of the output image can be assigned a maximum possible value, for example: 100%, and all other points can be assigned a minimum possible value, for example: zero. In this case, after the training, the probability identification of various images in the form of a percentage of their similarity to the training image will be enabled. Figure 8 shows its training for How the identification of the two images (a square and a circle) of the p-network 100 can identify an instance of an image that contains features, each of which is expressed as a percentage and the sum does not have to equal 100. %. This pattern recognition process, which defines the percentage of similarity between different images for training, can be used to classify specific images.

To improve accuracy and eliminate errors, the coding system can use a set of several neuron outputs rather than one output (see below). In the simplest case, the output image can be prepared before training. However, it is also possible to have an output image formed by the p-network 100 during training.

Among the p networks 100, there is also the possibility of inverting the input and output images. In other words, the input image 106 can be in the form of a field or table of digits or analog values, where each point corresponds to an input to the p network, and the output image can be used in any format that is suitable for introduction to a computer. Rendering, for example: using the format jpeg, gif, pptx, in the form of tables, charts, graphs and graphics, various file formats, or a set of symbols. The resulting p-network 100 system is quite suitable for archival systems, as well as for the search of images, musical terms, equations, or data sets.

Following the preparation of the input image 106, typically, the p-network 100 must be formed and/or an existing p-network parameter must be set for handling the intended task. The formation of the p-network 100 may include the following designations: The size of the ̇p network 100, as defined by the number of inputs and outputs; 神经 the neural node weights for all inputs 108; ̇ the number of correction weights 112; The distribution of the weighting influence coefficient (C _i,d,n ) for the different values of the input signal 104; and the desired accuracy of the training. The number of inputs is determined based on the size of the input image 106. For example, several pixels may be used for the image, and the number of selections of the output may depend on the size of the desired output image 126. In some cases, the number of choices of outputs may depend on the number of classifications of the training images.

The value of individual nerve knot weights 108 can range from -∞ to +∞. A value of neural node weight 108 less than 0 (zero) may mean signal amplification, which may be used to enhance the effect of a signal from a particular input or from a particular image, for example, for containing a large number of different individuals or objects. The face in the photo is more effectively identified. On the other hand, a value of the neural node weight 108 greater than 0 (zero) can be used to mean signal attenuation, which can be used to reduce the number of calculations required and to increase the operating speed of the p-network 100. In summary, the greater the value of the neural node weight, the more the signal is transmitted to the corresponding neuron. If all of the neural node weights 108 corresponding to all inputs are equal and all neurons and all inputs are equally connected, the neural network will become generic and will be most effective for general tasks, such as when there is an image The nature of the system is rarely known in advance. However, such a structure will generally increase the number of calculations required during training and operation.

Figure 9 shows an embodiment of a p-network 100 in which the relationship between an input and a respective neuron is reduced according to a statistical normal distribution. The uneven distribution of the neural node weights 108 can cause the entire input signal to be passed to a target or "central" neuron for a given input, thus assigning a value of zero to the associated neural node weight. In addition, the uneven distribution of neural node weights can cause other neurons to receive reduced input signal values, such as using normal, log-normal, sinusoidal, or other distributions. The value of the neural node weight 108 for the neuron 116 that receives the reduced input signal value may increase as its distance from the "central" neuron increases. In this case, the number of calculations can be reduced and the p The operation of the network can be accelerated. Such a network of known fully connected and non-fully connected neural networks can be very effective for image analysis with strong internal patterns, such as continuous pictures of faces or movie films.

Figure 9 shows an embodiment of a p-network 100 that is valid for identification of local patterns. In order to improve the identification of the general pattern, a strong connection system in which the value of the neural node weight 108 is small or 10-20% can be distributed over the entire p network in a certain deterministic manner (such as in the form of a grid) or in a random manner. In the middle of the road 100. The actual formation of the p-network 100 intended to handle a particular task is performed using, for example, a program written in an object-oriented programming language that produces the main components of the p-network, such as: neural knots, nerves Weights, allocators, correction weights, neurons, and so on, like soft objects. Such a program can specify the relationship between the object being pointed out and the algorithm that clearly states its action. In particular, the neural node and correction weights can be formed at the beginning of the formation of the p-network 100, along with setting its initial value. The p-network 100 can be fully formed before the start of its training, and modified or appended in a later picture if necessary, for example, when the information capacity of the network becomes exhausted or if A fatal mistake occurred. The completion of the p network 100 may also be as training continues.

If the p-network 100 is pre-formed, the number of selection correction weights on a particular neural node can be equal to the number of intervals within the range of the input signal. In addition, the correction weights may be generated after the formation of the p-network 100 as a signal in response to the occurrence of individual intervals. Similar to the classical neural network 10, the selection of parameters and settings for the p-network 100 provides a series of targeted experiments. Such experiments may include: (1) the formation of a p-network with the same neural node weights 108 at all inputs, and (2) an evaluation of the input signal values for the selected image and an initial selection of the number of intervals. For example, for binary (monochrome) image recognition, there are only 2 intervals. The system may be sufficient; up to 256 intervals can be used for qualitative identification of 8-bit images; complex estimates of statistical dependence may require many or even hundreds of intervals; for large databases In other words, the number of intervals can be thousands.

During the training of the p-network 100, the value of the input signal can be rounded down as it is distributed between specific intervals. Therefore, the accuracy of the input signal that is greater than the width of the range divided by the number of intervals may not be required. For example, if the input value range is set for 100 units and the number of intervals is 10, an accuracy better than ±5 would not be required. Such experiments further comprising: selecting a uniform distribution of the (3) in a section of the entire range of input signal values and the weight for correcting the influence coefficient C _{i, d, n} the distribution system may be the easiest for which correspond to The correction weight of the interval of the specific input signal is set equal to 1, and the correction weight influence for all remaining correction weights can be set to 0 (zero). The experiments may additionally include: (4) training the p-network 100 with one, more, or all of the prepared training images having a predetermined accuracy.

The training time for the p-network 100 with predetermined accuracy can be established by experimentation. If the accuracy and training time of the p-network 100 is satisfactory, the selected settings can be maintained or changed, while the search for a more efficient variant continues. If the required accuracy is not achieved, the impact of the particular modification may be evaluated for the purpose of optimization, which may be carried out one by one or in groups. The evaluation of the modification may include: changing (increasing or decreasing) the number of intervals; changing the pattern of the distribution of the correction weight influence coefficient (C _{i, d, n} ), testing the variation having an uneven distribution of intervals, such as: Use a normal, power, log, or log-normal distribution; and, change the value of the neural node weight 108, for example, its transition to an uneven distribution.

If the training time required for an accurate result is considered excessive, the training system with an increased number of intervals can evaluate its effect on training time. If the result is The training time is reduced and the increase in the number of intervals is repeatable until the desired training time is obtained without compromising the required accuracy. If the training time is increased rather than reduced as the number of intervals is increased, the additional training can be performed with a reduced number of intervals. If the reduced number of intervals results in reduced training time, the number of intervals can be further reduced until the desired training time is obtained.

The formation of the settings of the p-network 100 can be determined via experiments with predetermined training time training and training accuracy. The parameter system can be improved via experimental changes similar to those described above. The actual implementation of various p-networks has shown that the procedure for setting the selection is usually straightforward and time consuming.

As the beginning of the actual method of training system 200 shown in FIG. 23 are part of, network 100 is the P input image signal I _1, I ₂ ... I _n the web is fed to the input device 102, from there, the signals are It is transmitted to the nerve knot 118, passes through the nerve knot weight 108 and enters the dispenser (or a group of dispensers) 114. Based on the input signal value, the allocator 114 sets the number of intervals "d" for which the predetermined input signal 104 corresponds, and assigns all of the weight correction blocks 110 for all of the neural nodes 118 to which the respective input 102 is connected. The correction weight of the correction weight 112 affects the coefficient C _i,d,n . For example, if the interval "d" can be set to 3 for the first input, and for the weights W _{1, 3, n} , C _{1, 3, n} = 1 is set to 1, and for i ≠ 1 and d ≠ All other weights of 3, C _{i, d, n} can be set to 0 (zero).

For each neuron 116, it is identified as "n" in the following relationship, and the neuron output sums Σ1, Σ2...Σn by correcting weights 112 for each of the neural nodes 118 that contribute to a particular neuron (in The following relationship is identified as W _i,d,n ) multiplied by a corresponding correction weight influence coefficient C _i,d,n and is formed by adding all the obtained values: Σ _n = Σ _i,d,n The multiplication of W _i,d,n × C _i,d,n [2]W _i,d,n ×C _i,d,n can be performed by various devices, for example, by the dispenser 114, with storage The weight of the device or directly by the neuron 116. The sums are transferred to the weight correction calculator 122 via the neuron output 117. The desired output signals O ₁ , O ₂ . . . O _n describing the desired output image 126 are also fed to the calculator 122.

As discussed above, weight correction calculator 122 based neuron output by the sum Σ1, Σ2 ... Σn and the expected output signals O _1, O ₂ ... O _n of the comparison a calculation means for calculating a modified value for the correction weight. Figure 11 is a graph showing the contribution weights W _i,d,1 contributed to the neuron output sum Σ1, multiplied by the corresponding correction weight influence coefficients C _{i,d, 1} , and the product is subsequently derived from the nerve The sum of the meta-outputs Σ1 is added: Σ1=W _1,0,1 × C _{1 , 0 , 1} .+W _1,1,1 × C _{1 , 1 , 1} .+W _1,2,1 × C _{1 , 2 , 1} .+...[3] As the training begins, that is, during the first period, the correction weight W _i,d,1 does not correspond to the input image 106 for training, therefore, the neuron output sum Σ1 is not Equal to the corresponding desired output image 126. Based on the initial correction weight W _i,d,1 , the weight correction system calculates the correction value Δ ₁ , which is used to change all the correction weights (W _i,d,1 ) contributing to the neuron output sum Σ1. . The p-network 100 is a variety of options or variations that allow the formation and utilization of collective correction signals for all of the correction weights W _{i, d, n} that contribute to a particular neuron 116.

The following are two exemplary, non-limiting variations on the formation and utilization of such collective correction signals. Variant 1 - Based on the difference between the expected output signal and the sum of the outputs obtained, the formation and utilization of the correction signal is as follows:

计算 The calculation of the equal correction value Δ _{n for} all the correction weights contributing to the neuron “n” is based on the following equation: Δ _n =(O _n - Σ _n )/S [4], where: O _n - corresponds to the nerve The expected output signal of the meta output sum Σn; S- the number of neural nodes connected to the neuron "n".

修改 All correction weights W _i,d,n contributed to the neuron "n" are modified according to the following formula: W _{i , d , n} modified = W _{i , d , n} + Δ _n /C _i,d,n [ 5] , variant 2 - based on the ratio of the expected output signal to the sum of the outputs obtained, the formation and utilization of the correction signal is as follows:

˙ contribution to neuron "n" all the weight of the correction weight is equal to the correction value [Delta] _n of the system is calculated according to the following _{formula: Δ n = On / Σ n} [6],

修改 All correction weights W _i,d,n contributed to the neuron "n" are modified according to the following formula: W _{i , d , n ,} modified = W _{i , d , n ,} × Δ _n [7] , according to any The modification of the available correction correction weights W _i,d,n is intended to reduce the training error for each neuron 116 by converging its output sum Σ _{n to} the value of the desired output signal. In this way, the training error for a given image can be reduced until it becomes equal to or close to zero.

An example of the modification of the correction weights W _i,d,n during training is shown in FIG. The values of the correction weights W _i,d,n are set before the start of training to have the weight value set to a form of random weight distribution from 0 ± 10% of the correction weight range, and reach the final after training. distributed. The calculations described for the collective signal are directed to all of the neurons 116 in the p-network 100. The training program described for one training image can be repeated for other training images. Such a program may result in the occurrence of training errors for some previously trained images, since some of the correction weights W _{i, d, n} may be involved in several images. Therefore, the training system with another image may partially disturb the distribution of the correction weights W _{i, d, n} formed for the previous image. However, due to the fact that each neural node 118 includes a set of correction weights W _i,d,n , the training system with new images that may increase the training error does not delete the p network 100 that was previously trained for it. These images. Moreover, more neural nodes 118 contribute to each neuron 116 and the number of correction weights W _i,d,n at each neural node is greater, and the training effect on a particular image is more important for the training of other images. less.

Each training period usually ends with a substantial convergence of the total training error and/or local training error for all training images. Errors can be evaluated using known statistical methods such as, for example, Mean Squared Error (MSE), Mean Absolute Error (MAE), or Standard Error Mean (SAE). If the total error or some local errors are too high, an additional training period can be performed until the error is reduced to less than a predetermined error value. The image recognition process described earlier by the percentage of similarity defined between the different images used for training (shown in Figure 8) is itself a classification process along the images of the previously defined categories.

For clustering, that is, dividing the image into natural categories or groups that were not previously specified, the basic training algorithm of method 200 can be modified by Self-Organizing Maps (SOM). The desired output image 126 corresponding to a given input image can be formed directly during the training of the p-network 100 based on the winning neurons having one of the maximum values of the output neuron sum 120. Figure 22 is a diagram showing how the use of the basic algorithm of the method 200 can produce a primary group of output neuron sums, wherein the group is further Convergence so that several larger sums retain their value, or increase, while all other sums are considered equal to zero. The output neuron sum of this transition group can be accepted as the desired output image 126.

Formed as described above, the desired output image 126 of the set includes clusters or groups. As such, the desired output image 126 of the set is a cluster of images that are linearly inseparable, which is different from the classical network 10. Figure 13 is a diagram showing how the described manner can contribute to clustering a composite hypothetical image "cat-car" in which different features of the image are assigned to different clusters - cats and cars. Establishing a set of desired output images 126 as described may be used, for example, to establish different classifications, statistical analyses, image selections, based on criteria formed by clustering. In addition, the desired output image 126 produced by the p-network 100 can be used as an input image of another or additional p-network that can also be formed in the manner described for the associated p-network 100. Thus, it is contemplated that the output image 126 is intended for use in a subsequent layer of a multi-layer p-network.

The classical neural network 10 training system is outlined as being provided via a supervised training method based on an initial preparation of an input image and a desired output image. This generalization method is also used for the training of the p-network 100. However, the improved training speed of the p-network 100 is also considered to be trained by an external trainer. The task of the external trainer can be performed, for example, by an individual or by a computer program. Acting as an external trainer, the individual can be involved in performing a physical task or operating in a gaming environment. The p-network 100 receives input signals in the form of data relating to a particular situation and changes thereto. A signal reflecting the trainer's motion can be introduced as the desired output image 126 and allows the p-network 100 to be trained according to the basic algorithm. In this way, the modeling of the various processes can be generated immediately by the p-network 100.

For example, the p-network 100 can be trained to drive a vehicle by receiving information about the road conditions and the driver's actions. By modeling a large number of key scenarios, the same p-network 100 can be trained by a number of different drivers and accumulate more driving skills than would normally be the case for a single driver. The p-network 100 is capable of assessing a particular road condition within 0.1 seconds or more quickly and accumulating substantial "driving experience" that enhances traffic safety in a variety of situations. The p network 100 can also be trained to work with a computer (eg, a chess machine). The ability of the p-network 100 to easily move from the training mode to the recognition mode (and vice versa) takes into account the implementation of a "learn from error" mode when the p-network 100 is trained by an external trainer. . In this case, the partially trained p-network 100 can generate its own actions, for example, to control a technical process. The trainer can control the actions of the p-network 100 and correct their actions when necessary. Therefore, an additional training system for the p-network 100 can be provided.

The information capacity of the p-network 100 is enormous, but not unlimited. Due to the set scale of the p-network 100 (such as: input, output, and number of intervals), and due to the increase in the number of images used to train the p-network, the number of training errors after a certain number of images is The size may also increase. When the increase in error is detected, the number and/or size of the errors can be reduced by increasing the size of the p-network 100, since the p-network allows for the enhancement of neurons between training sessions. The number of 116 and/or the number of signal intervals "d" across the p network or in its components. The extension of the p-network 100 can be achieved by adding new neurons 116, adding new inputs 102 and neural nodes 118, changing the distribution of the weighting influence coefficients C _{i, d, n} , and dividing the existing interval "d". provide.

In most cases, the p-network 100 will be trained to ensure that it is capable of recognizing images, patterns, and correlations inherent to the image or to a group of images. In the simplest case, the identification process repeats the initial steps of the training process according to its basic algorithm, which is disclosed as part of method 200. In particular: ̇ Direct recognition begins with the formatting of the image, according to the same rules that it uses to format the image for training; ̇ the image is transmitted to the input of the trained p-network 100, and the splitter is assigned Corresponding to the correction weights W _i,d,n of the input signal values set during the training, and the neurons generate the sum of the individual neurons, as shown in FIG. 8; The resulting output sum is completely dependent on one of the images used by the p-network 100 to be trained, and the actual identification of the object; and if the output image 126 is partially compliant with the p network 100 being trained for several The image, the result is displayed as a percentage of the matching ratio for different images. Figure 13 shows that during the identification of a composite image based on a combination of images of a cat and a vehicle, the output image 126 represents a given image combination and indicates its contribution to each initial of the combination. The percentage of the image.

For example, if several images of a particular person are used for training, the identified image may be 90% corresponding to the first image, 60% corresponding to the second image, and 35% corresponding to the third image. Images. It is possible that the identified image has a certain probability and corresponds to an image of another person or even an animal, which means that there is some similarity between the images. However, the probability of this similarity is likely to be lower. Based on these probabilities, the reliability of the identification can be determined, for example, based on Bayes' theorem.

With p-network 100, it is also possible to implement multi-stage identification, which combines the advantages of algorithms and neural network identification methods. The multi-stage identification system can include: The initial identification of an image by a pre-trained network, by using not all but only 1%-10% of the input, is referred to herein as "basic input." The input to this portion can be distributed uniformly, randomly, or distributed within the p-network 100 by any other distribution function. For example, the identification of a person in a photo that includes a plurality of other objects; and ̇ select the item or object portion that provides the most useful information for further detailed identification. The identification may be provided based on the structure of the particular object that is pre-set in the memory, as in the calculation method, or based on a gradient of color, brightness, and/or depth of the image. For example, in the identification of portraits, the following identification zones can be selected: eyes, mouth corners, nose shape, and certain characteristics, such as tattoos, license plate numbers, or house number numbers can also be selected in a similar manner. Identification; and 详细 If necessary, a detailed identification of the selected image is also possible.

The formation of a computer simulation of the p-network 100 and its training can be provided based on the above description by using any programming language. For example, an object-oriented programming can be used, wherein the neural node weights 108, the correction weights 112, the allocator 114, and the neurons 116 represent programming objects or object categories, and the relationship is established via a link or message. Between object categories, and interactive algorithms are set between objects and between object categories.

The software simulation formation and training system of the p network 100 may include the following:

1. Preparation for the formation and training of the p-network 100, in particular: ̇ According to a given task, the conversion of the grouped training input images into a digital form; the analysis of the digital image caused by the ,, including its use in Selection of parameters of the trained input signal, such as: frequency, size, phase, or coordinates; and ̇ setting a range for the training signal, a number of intervals within the range, and correcting the weight influence coefficient C _{i ,} The distribution of _{d, n} .

2. The formation of the software simulation of the p network, including:

The formation of a set of inputs for the p-network 100. For example, the number of inputs can be equal to the number of signals in the training input image;

a group of neurons formed, wherein each neuron represents an additive device;

a group of nerve nodes having a weight of a neural node, wherein each nerve node is connected to a p-network input and a neuron;

Forming a weight correction block in each neural node, wherein the weight correction block includes a divider and a correction weight, and wherein each of the correction weights has the following characteristics: ○ correction weight input index (i); ○ correction weight nerve Meta index (n); ○ correction weight interval index (d); and ○ correction weight initial value (W _{i, d, n} ).

̇ Specify a correlation between the interval and the correction weight.

3. Train each neuron with an input image, including:

̇Specifying the correction weight influence coefficient C _i,d,n , including: ○ determining a section corresponding to the input signal of the training input image received by each input; and ○ specifying the correction weight of all correction weights for all the neural nodes The magnitude of the influence coefficient C _i,d,n .

Multiplying the correction weights W _{i , d , n} of all the neural node weights that contribute to the neuron by the corresponding correction weight influence coefficients C _i,d,n and then adding them to calculate for each neuron “n” The sum of the neuron outputs (Σ _n ): Σ _n = Σ _{i, d, n} W _{i, d, n} × C _{i, d, n}

_{T n = O n -Σ n:} ˙ sum of the neuron output Σ _n calculates the training error or deviation (T _n) are subtracted from the desired output signal O _n via the corresponding

˙ via the number of training error divided by its connection to the neuron "n" of the ganglia of the "S" is calculated for which contribute to neuronal "n" of all the correction weight is equal to the correction value (Δ _n): Δ _n = T _n /S

̇ By dividing the correction value Δ _n by the corresponding correction weight influence coefficient C _i,d,n and then adding to each correction weight, modifying all the correction weights W _i,d,n contributing to the respective neurons : W _{i , d , n} modified = W _{i , n , d} + Δ _n /C _{i , d , n} .

Another method for calculating the equal correction value (Δ _n ) and modifying the correction weights W _{i,d,n for} all of the correction weights contributing to the neuron "n" may include the following:

除 Dividing the signal O _n of the desired output image by a neuron output sum Σ _n : Δ _n =O _n / Σ _n

˙ the like by the correction weight is multiplied by the correction value Δ _n, the right to modify its contribution to the correction of the neuron weight _{W i, d, n: W} i, d, n modify _{= W i, d, n ×} Δ n

4. Using all of the training images to train the p-network 100, including: 重复 repeating the above-described process for all selected training images that are included in a training session; Determining one or more errors for the particular training period, comparing the errors to a predetermined acceptable error level, and repeating the training period until the training errors become less than the predetermined acceptable error level .

A practical example of a software simulation of a p-network 100 using object-oriented programming is described below and shown in Figures 14-21.

A "Neuron Unit" object class formation system can include: forming a group of objects of the "nerve knot"category; the sacral neuron 116 proposing a variable, wherein the addition system is implemented during training; and proposed a variable calculator 122, wherein the desired value of the neuron 120 is stored and the sum of the correction value Δ is calculated based _n the implementation of the training process period.

The category "neuron unit" provides training for the p-network 100 and may include: the formation of the ̇ neuron sum 120; ̇ setting the desired sum; calculating the ̇ correction value Δ _n ; and 相 adding the calculated correction value Δ _n To the correction weight W _i,n,d .

The formation of the object category "Synapse" may include: a set of correction weights W _{i, n, d} ; and ̇ indicate an indicator of its input to the neural node 118.

The category "nerve knot" can perform the following functions: 初始化 initialization of the weights W _{i, n, d} ; 乘 multiply the weights W _{i, n, d} by the coefficients C _{i, d, n} ; and ̇ weights W _{i, n,} Correction of _d .

The formation of the object category "InputSignal" may include: 索引 an index of the group of neural nodes 118 connected to a given input 102; 包括 it includes the variable of the value of the input signal 104; ̇ the smallest possible The value of the maximum input signal; the number of the interval "d"; and the length of the interval.

The category "input signal" provides the following functions: 形成 The formation of the structure of the p-network 100 includes: ○ the addition and removal of a link between an input 102 and a neural node 118; and ○ setting for a particular The number "d" of the intervals of the neural nodes 118 of 102 is entered.

̇ setting the parameters of the minimum and maximum input signals 104; ̇ contributing to the operation of the p network 100: ○ setting an input signal 104; and ○ setting the correction weight influence coefficient C _{i, d, n} .

The formation of the object category "pNet" (PNet) includes a set of object categories: ̇ neuron units; and ̇ input signals.

The category "p network" provides the following functions: ̇ setting the number of objects in the "input signal" category; 数目 setting the number of objects in the "neuron unit" category; and ̇ object "neuron unit" and "input signal" Group request for features.

During the training process, the cycles can be formed, wherein: the sum of the neuron outputs with ̇ equal to zero is formed before the start of the cycle; 所有 all the neural nodes contributing to the established "neuron unit" are examined. For each neural node 118: ○ based on the input signal 104, the dispenser forms a set of correction weight influence coefficients C _i,d,n ; ○ the ownership weight W _{i,n,d of} the neural node 118 is examined, and for each Weight: ■ The value of weight W _i,n,d is multiplied by the corresponding correction weight influence coefficient C _i,d,n ;■ The multiplication result is added to the sum of the formed neuron outputs; ̇correction value Δ _n Calculation; ̇ correction value Δ _n is divided by the correction weight influence coefficient C _i,d,n , ie, Δ _n /C _i,d,n ; and ̇ all the neural nodes 118 contributing to the established "neuron unit" are examined . For each neural node 118, the weight of the associated neural node W _{i, n, d} is examined, and for each weight, its value is modified to the corresponding correction value Δ _n .

The aforementioned possibility of additional training of the p-network 100 allows for a combination of training and image recognition, so that the training process can be accelerated and its accuracy improved. When the p-network 100 is trained through a sequence of sequentially changed images, such as through continuous pictures of films that are slightly different from each other, the additional training system may include: 训练 training with the first image; The recognition of the image and the percentage of similarity between the new image and the image used for the initial training of the network. If the identification error is less than its predetermined value, additional training is not required; and 附加 if the identification error exceeds a predetermined value, additional training is provided.

The training system of the p-network 100 by the basic training algorithm described above is effectively used to solve the problem of image recognition, but the loss or fallacy of the data attributed to the overlapping images is not excluded. Therefore, the use of the p-network 100 for memory purposes is possible, but may not be completely reliable. This embodiment describes the training of the p-network 100, which provides protection against loss or corruption of information. Another limitation can be introduced into the basic network training algorithm, which requires that each correction weight W _i,n,d can be trained only once. After the first training period, the values of the weights W _{i, n, d} are maintained at a fixed or fixed value. This can be achieved by inputting another access index "a" for each correction weight, which represents access to the associated correction weight W _i,n,d during the training process. The number of the above index.

As noted above, each correction weight may be named after W _i,n,d,a , where "a" is the number of accesses to the associated weight during the training process. In the simplest case, for an unmodified (ie, unfixed) weight, a=0, and for a weight that has been modified or fixed by the described basic algorithm, a=1. Moreover, when the basic algorithm is applied, the correction weights W _{i, n, d, a} with a fixed value a = 1 can be excluded from being corrected to the weights made thereto. In this case, the formulas [5], [61, and [7] can be transformed as follows:

The above limitations may be applied in part to the correction of the previously trained correction weights W _{i, n, d, a} , but only to the weights to which the most important images are applied. For example, within a group of portrait training for a single person, a particular image can be declared as the primary and assigned a priority. After the training of the image of the priority level, all the correction weights W _i,n,d,a which are changed during the training process can be fixed, that is, wherein the index a=1, so the weight is designated as W _{i, n, d, 1} , and other images of the same person can be maintained and changeable. The priority level may include other images, such as those that are used as encryption keys and/or contain key numerical data.

The change to the correction weight W _i,n,d,a may also not be completely prohibited, but is limited by the increase of the index "a". That is, each subsequent use of the weights W _{i, n, d, a} can be used to reduce its ability to change. The more often a particular correction weight W _i,n,d,a is used, the smaller the weight change with each access, and thus the stored image changes less and suffers during subsequent training of the image. Fallacy. For example, if a=0, any change in weight W _i,n,d,a is possible; when a=1, the probability of change for the weight can be reduced to ±50% of the weight value With a = 2, the probability of change can be reduced to ± 25% of the weight value.

After a predetermined number of accesses have been reached, as indicated by the index "a", for example, when a = 5, further changes in the weights W _{i, n, d, a} may be prohibited. This approach provides a combination of high intelligence and information security within a single p-network 100. Using a network error calculation mechanism, the level of allowable error can be set such that information having a loss within a predetermined accuracy range can be stored, wherein the accuracy range is specified according to a particular task. In other words, for the p-network 100 operating on the visual image, the error can be set at a level that cannot be captured by the naked eye, which will provide a significant "factor" increase in storage capacity. The above can enable the establishment of highly efficient storage of visual information, such as movies.

The ability to selectively erase computer memory is useful for the continued high-level operation of p-network 100. The selective removal of such a memory can be made by removing certain images without losing or damaging the remaining stored information. The removal system can provide the following: ̇ recognition of all correction weights W _{i, n, d, a} involved in image formation, for example, by introducing images into the network or by compiling for each image The use of the series of correction weights; 降低 the reduction of the index "a" for the individual correction weights W _{i, n, d, a} ; and ̇ when the index "a" is reduced to zero, with zero or with it close to the The correction weights W _i,n,d,a are replaced by a random value in the middle of the range of possible values of the weights.

An appropriate level of index "a" and a series of reductions can be selected experimentally to identify strong patterns that are hidden in the images of the sequence. For example, for every 100 images introduced to the p-network 100 during training, there may be a decrease count of index "a" until the "a" reaches zero. In this case, the value of "a" can be increased corresponding to the introduction of a new image. The competition between the increase and decrease of "a" can lead to a situation in which the random variation is gradually removed from the memory, and it has been used multiple times and the confirmed correction weights W _{i,n,d, a} series can be stored. When the p-network 100 is trained through a large number of images having similar attributes such as the same object or similar environment, the frequently used correction weights W _{i, n, d, a} are constantly confirming their values and the information systems in such areas Become very stable. Furthermore, the random noise system will gradually disappear. In other words, the p-network 100 with a decreasing index at index "a" can function as an effective noise filter.

The described embodiment of the training of the p-network 100 in the absence of loss of information allows for the establishment of a p-network memory with high capacity and reliability. This kind of memory system can be used as a high-capacity high-speed computer memory, which provides higher speed than "cache memory", but will not improve the computer as the typical "cache memory" system. Cost and complexity. According to the published information, in general, when a movie is recorded using a neural network, the memory system can be compressed by tens or hundreds of times without significant loss of recording quality. In other words, a neural network can operate as an extremely efficient archiving program. Combining this capability of the neural network with the high-speed training capabilities of the p-network 100 allows for the creation of high-speed data transmission systems, memory with high storage capacity, and high-speed decryption program multimedia files, ie, transcoders ( Codex).

Due to the fact that in the p-network 100, the data is stored as a set of correction weights W _i,n,d,a , which is a type of code record, via an existing method and without an equivalent It is not possible to decode or unauthorized access to the p network in the case of a network and a key. Therefore, the p-network 100 provides a considerable degree of data protection. In addition, unlike conventional computer memory, the damage to the individual storage elements of the p-network 100 is negligible, since other components greatly compensate for the lost functionality. In the image recognition process, the intrinsic pattern of the image used is actually not distorted by damage to one or more components. The above can significantly improve the reliability of the computer and allow the use of certain memory blocks, which would normally be considered defective. Moreover, this type of memory system is less susceptible to hacking attacks due to the lack of a permanent address for key bytes within the p-network 100, such that it is not subject to various computer viruses for such systems. The impact of the attack.

The aforementioned image recognition process by determining the percentage of similarity between different images for training can also be applied as an image classification process according to a previously defined category, as described above. For clusters, the image is divided into non-predefined natural categories or groups, and the basic training process can be modified. This embodiment can include:

准备 Preparation of a set of input images for training, not including prepared output images;

Forming and training the network, by the formation of the sum of the neuron outputs, as it is based on the basic algorithm;

The choice of the output image resulting from the output of the largest output sum, ie the winner output, or a group of winner outputs, which may be organized similar to the Kohonen network;

The establishment of a desired output image, wherein the winner output or the winner output of the group receives the maximum value. At the same time: ○ The number of selected winners can be predetermined, for example, in the range of 1 to 10, or the winner output can be selected according to the rule "not less than N% of the sum of the largest neurons", where "N " can be, for example, within 90-100%; and ○ All other output systems can be set equal to zero.

̇ According to the training of the basic algorithm, by using the established desired output image, Figure 13;

重复 Repeat all procedures for other images that have their own images for different winners or groups of winners.

The desired output image of the set formed in the manner described above can be used to describe a cluster or group into which the plurality of input images can be naturally separated. The desired output image of the group can be used to generate different classifications, such as: selection of images for use in statistical analysis based on established criteria. The above can also be used for the aforementioned inversion of the input and output images. In other words, the desired output image can be used as an input image for another (ie, additional) network, and the output of the additional network can be any image that is rendered for any form of computer input.

In the p-network 100, after a single cycle of training by the above algorithm, the desired output image system can be produced with a small output sum change, which may slow the training process and may also reduce its accuracy. . In order to improve the training of the p-network 100, the initial changes in the points can be artificially increased or extended such that the size changes of the points will cover the entire range of possible output values, for example: -50 to +50, as in Figure 21 Shown. This type of extension of the initial change in the point may be linear or non-linear.

A situation may develop where the maximum value of an output is an outlier or error, such as a representation of noise. This situation can be manifested by the phenomenon that it is a maximum value surrounded by many small signals. When the winner's output is selected, the small signal value can be ignored, and the winner of the largest signal surrounded by other large signals is selected as the winner. For this purpose, known statistical techniques for variability reduction can be used, such as: importance sampling. This approach allows for the removal of noise and maintains a substantially useful pattern. Winning group establishment line A cluster of inseparable images (ie, images of more than one cluster), as shown in Figure 13. The above can provide a significant improvement in accuracy and reduce the number of cluster errors.

During the training of the p-network 100, the typical error corrected is:

The error correction may also be an algorithm described above by training with an external trainer.

The hardware implementation of the p-network 100 can provide digital and/or analog microchips. A representative p-network 100 microchip system can be used for information storage and processing. The microchip system of the p-network 100 can be based on a variety of variable resistors, field effect transistors, memristors, capacitors, switching elements, voltage generators, non-linear photovoltaic cells, and the like. A variable resistor can be used as the nerve knot weight 108 and/or the correction weight 112. A plurality of such resistors can be connected in parallel, in series, and in series and in parallel. If the respective resistors are connected in parallel, the signal can be encoded by the current value, which can then be an analogy sum that facilitates the automation of the current. In order to obtain a positive or negative signal, two sets of resistors (excitation and suppression) can be provided on each nerve node. In this hardware configuration, the suppressed signal can be subtracted from the excited signal.

Each of the correction weights 112 can be implemented as a device (memory resistor) like a memory resistor. As is known to those skilled in the art, a memory resistor has a variable resistor that is controlled by a current in the circuit or by a potential or charge. Appropriate memory resistor functionality can be achieved via an actual memory resistor device, as well as software or its physical simulation. In operation of the p-network 100 at low voltage potentials, the memory resistor operates as a simple resistor. During the training mode, the resistance of the memory resistor can be varied, for example, by a strong voltage pulse. The change in the value of the memory resistor (the increase or decrease in resistance) may depend on the polarity of the voltage, and the magnitude of the change in the value may depend on the magnitude of the voltage pulse.

The detailed description and the drawings are to support and describe the disclosure, and the scope of the disclosure is defined only by the scope of the claims. While some of the best modes and embodiments of the claimed subject matter have been described in detail, various alternative designs and embodiments exist for the disclosure of the scope of the appended claims. Furthermore, the embodiments shown in the figures or the features of the various embodiments described in the specification are not necessarily construed as being independent of the embodiments. Rather, it is possible that features described in one of the examples of one embodiment may be combined with one or more other desired features from other embodiments, such that they are not described in words or Other embodiments of the drawings. Accordingly, the other embodiments are within the scope of the scope of the appended claims.

100‧‧‧ Progressive neural network (p network)

102‧‧‧ Input

104‧‧‧Input signal

106‧‧‧ Input image

110‧‧‧weight correction block

112‧‧‧correction weight

114‧‧‧Distributor

116‧‧‧ neurons

117‧‧‧ Output

118‧‧‧ nerve knot

119‧‧‧neuronal unit

120‧‧‧The sum of neurons

122‧‧‧ Weight Correction Calculator

124‧‧‧ Expected output signal

126‧‧‧ Output image

128‧‧‧ Deviation

Claims (16)

A neural network comprising: a plurality of inputs of the neural network, each input being assembled to receive an input signal having an input value; a plurality of neural nodes, wherein each neural node is coupled to one of the plurality of inputs And including a plurality of correction weights, wherein each correction weight is defined by a weight value; a set of distributors, wherein each of the distributors is operatively coupled to one of the plurality of inputs for receiving respective input signals and Assembling to select one or more correction weights from the plurality of correction weights associated with the input value; a set of neurons, wherein each neuron has at least one output and via the one of the plurality of neural nodes At least one of the plurality of inputs is a connection, and wherein each neuron is assembled to sum up the weight values of the correction weights selected from the respective neural nodes connected to the respective neuron and thereby generate a neural a summation; and a weight correction calculator configured to receive a desired output signal having a value, determine the sum of the neurons and the desired output a deviation of the values, and using the determined deviation to modify the respective correction weight values, such that the modified correction weight values are aggregated to determine that the sum of the neurons is the sum of the neurons and the desired output signal value This deviation is minimized, thereby training the neural network. The neural network of claim 1, wherein: the determination of the deviation of the sum of the neurons from the desired output signal comprises dividing the desired output signal value by the sum of the neurons to generate a deviation coefficient; and The modification of the individual correction weight values includes the schools used to generate the sum of the neurons. The positive weight is multiplied by the deviation coefficient. The neural network of claim 1, wherein the deviation of the sum of the neurons from the desired output signal is a mathematical difference therebetween, and wherein the respective modified correction weights include the mathematical difference It is distributed to the respective correction weights used to generate the sum of the neurons. The neural network of claim 3, wherein the mathematical difference is divided by equally dividing the determined difference between respective correction weights used to generate the sum of the neurons. The neural network of claim 3, wherein each of the dispensers is additionally configured to assign a plurality of influence coefficients to the respective plurality of correction weights such that the respective influence coefficients are at a predetermined ratio. Assigning to one of the plurality of correction weights to generate the sum of the respective neurons; each neuron is assembled to have the correction weight for all of the neural nodes connected thereto and one of the specified influence coefficients The product is summed; and the weight correction calculator is assembled to apply a portion of the determined difference to the respective correction weights used to generate the sum of the neurons based on the ratio established by the respective influence factors. For example, in the neural network of claim 5, wherein: each of the plurality of influence coefficients is defined by an influence distribution function; the plurality of input values are received into a range of values, which is based on an interval distribution function Divided into intervals, such that each input value is received in a respective interval, and each correction weight corresponds to one of the intervals; Each of the dispensers uses the separately received input values to select the respective intervals, and assigns the respective plurality of influence coefficients to the correction weights corresponding to the selected respective intervals and to correspond to a proximity At least one of the intervals of the selected respective intervals is corrected for the weight. The neural network of claim 6, wherein each of the correction weights is additionally defined by a set of indexes, the set of indexes comprising: an input index configured to identify a correction weight corresponding to the input; An index, which is set to specify a selection interval for the respective correction weights; and a neuron index that is set to specify a correction weight corresponding to the neuron. The neural network of claim 7, wherein each of the correction weights is further defined by an access index, the access index being set to settle the respective correction weights during the training of the neural network by the input The number of times the signal was accessed. A method of training a neural network, comprising: receiving an input signal having an input value via an input to the neural network; transmitting the input signal to a dispenser operatively coupled to the input; The allocator selects one or more correction weights from a plurality of correction weights associated with the input value, wherein each of the correction weights is defined by a weight value and is positioned above a neural node connected to the input; And summing the weighted values of the selected correction weights to generate a sum of neurons via a neural node coupled to the input through the neural node and having at least one output; receiving a one via a weight correction calculator a desired output signal of the value; determining, by the weight correction calculator, a deviation of the sum of the neurons from the expected output signal value; Using the weight correction calculator, the determined deviation is used to modify the respective correction weight values, so that the modified correction weight values are aggregated to determine the sum of the neurons to cause the sum of the neurons and the desired output signal This deviation of values is minimized, thereby training the neural network. The method of claim 9, wherein: determining the deviation of the sum of the neuron and the expected output signal value comprises dividing the desired output signal value by the sum of the neurons to generate a deviation coefficient; and Modifying the respective correction weights includes multiplying each of the correction weights used to generate the sum of the neurons by the deviation coefficient. The method of claim 9, wherein the determining the deviation of the sum of the neurons from the expected output signal value comprises determining a mathematical difference therebetween, and wherein modifying the respective correction weights includes The mathematical difference is spread to the respective correction weights used to generate the sum of the neurons. The method of claim 11, wherein the distribution of the mathematical difference comprises dividing the determined difference equally between respective correction weights used to generate the sum of the neurons. The method of claim 9, further comprising: assigning, by the allocator, a plurality of influence coefficients to the plurality of correction weights, and assigning each influence coefficient to the plurality of correction weights at a predetermined ratio One of generating a sum of the neurons; via the neuron, summing the correction weights for all of the neural nodes connected thereto to one of the specified influence coefficients; and via the weight correction calculator, A portion of the determined difference is applied to each of the corrected weights used to generate the sum of the neurons based on the ratio established by the respective influencing factors. The method of claim 13, wherein the plurality of influence coefficients are defined by an influence distribution function; the method further comprises: receiving the input value into a range of values, which is divided according to an interval distribution function In the interval, the input value is received in a respective interval, and each correction weight corresponds to one of the intervals; and the received input value is used to select the respective And an interval, and assigning the plurality of influence coefficients to the correction weights corresponding to the selected respective sections and to at least one correction weight corresponding to a section adjacent to the selected respective sections. The method of claim 14, further comprising: additionally defining, by a set of indexes, respective correction weights, wherein the set of indexes comprises: an input index configured to identify a correction weight corresponding to the input; An interval index, which is set to specify a selection interval for the respective correction weights; and a neuron index that is set to specify a correction weight corresponding to the neuron. The method of claim 15, further comprising: additionally defining, by an access index, each of the correction weights, the access index being set to settle the respective correction weights during the training of the neural network by the input The number of times the signal was accessed.