JPH0484356A - Centroid outputting device by neural network - Google Patents

Abstract

PURPOSE:To make learning be done at a high learning efficiency by normalizing the value of input data within a prescribed input range within each unit becomes the most sensible and, at the same time, restoring the output value of a neural network to its original value. CONSTITUTION:Input normalizing devices 21-2n which map numerals corresponding to each coordinates in a prescribed input range of each unit forming a neural network 1 by using an appropriate function upon receiving the numerals are provided before each input unit. In addition, an output restoring device 3 which maps the output value of an output unit from the output range of the output unit by using a function appropriated to the numeral range of the value upon receiving the output value is provided after the output unit. Therefore, learning is made at a high learning efficiency and the reliability is improved.

Description

[Detailed description of the invention] [Table of contents] Overview Industrial field of application Prior art (Fig. 11.12) Problems to be solved by the invention Means for solving the problem (Fig. 1) Effect (Fig. 1. 2.3) Embodiment (Figures 4 to 10) Effects of the invention [Summary] At least, each input unit provided corresponding to each given coordinate on the number line, and one output unit. With regard to the center of gravity output device using a neural network, which inputs numerical values corresponding to each coordinate and outputs the center of gravity on a number line, it is possible to perform learning with high learning efficiency, and this makes it possible to achieve reliable The purpose is to provide a center-of-gravity output device for a neural network, in front of each input unit, upon receiving a numerical value corresponding to each coordinate, the value is appropriately set within a predetermined input range of each unit forming the neural network. An input normalization device that maps with a function is provided, and when an output value output by the unit is received after the output unit, the value is mapped from within the output range of the unit to the numerical range using an appropriate function. This configuration includes an output restoration device.

[Industrial application field]

The present invention relates to a center of gravity output device using a neural network, and in particular, at least each input unit provided as one corresponding to each given coordinate on a number line;
The present invention relates to a center-of-gravity output device using a neural network that inputs numerical values corresponding to each coordinate and outputs the center of gravity on a number line using a neural network consisting of output units.

In recent years, pattern recognition devices and adaptive filters using neural networks and the like have been used.

Furthermore, in fields where fuzzy models have traditionally been applied, such as plant control, fuzzy models are now being realized using neuromodels. Fuzzy models often use the center of gravity to calculate the consequent. In order to realize a fuzzy model entirely or partially using a neural model, a neural network that outputs elements that determine the center of gravity is required. The present invention provides a center of gravity output device using a neural network that approximately outputs such a center of gravity.

(Prior Art) Conventional computers with built-in programs have had to specify operations for all cases.

On the other hand, neural networks and the like can learn by presenting typical learning patterns several times, and can provide appropriate solutions even for patterns that are not presented.

In a data processing device with a neural network configuration, an output signal (output pattern) from a network that is output in response to the presentation of an input signal (input pattern) prepared for learning, without creating an explicit program, is The weight value of each internal connection of the network is determined according to a predetermined learning algorithm in order to match the teacher signal (teacher pattern). Once the weight values are determined through the learning process, a 'soft' data processing function is realized in which the hierarchical network outputs a suitable output signal even if an unexpected input signal is input. That will happen.

Typical structures and learning methods of neural networks include hierarchical networks and pack propagation methods, hierarchical networks and virtual impedance methods, etc.

FIG. 11 shows a center of gravity output device in a neural network according to a conventional example.

As shown in the figure, this device is a centroid output device using a neural network, which takes a value of a random number at a given coordinate and maps the random number to the input/output range of the neural network using an appropriate linear function. A center-of-gravity learning device 93 supplies training data obtained as learning data to the neural network to learn the neural network and outputs an approximate center of gravity, and a center-of-gravity learning device 93 that trains the neural network to output the center of gravity approximately. It has a neural network control unit 92 that performs control to determine the number of states, and a neural network 91 that is composed of neurons and that converts input data based on the determined internal state and outputs the corresponding data. .

Note that each unit (neuron) has multiple input lines and one output line, as shown in Fig. 11, and each input value is multiplied and added by the weight associated with each input line. This is to perform function conversion processing such as non-linear threshold processing on the output signal and output the corresponding signal from the output line.

Here, as shown in FIG. 12, the i-th basic unit 110 includes an accumulation section 110a and a dark value processing section 110b.

Let Ih be the input value output from each unit (h) in the previous stage, respectively Whi, the weight added to each input line, let θ5 be the threshold value associated with the unit, and let the sum of the input numerical values be is set to S, and the output value of the unit is set to O.
When i, the accumulator 110a of the unit calculates Si-ΣIhWh.

Further, in the threshold processing unit 110b, for example, Or =
f(S t)・1/(1+exp(−S x 10
01)) will be determined and output.

[Problem to be solved by the invention]

By the way, in the conventional center of gravity output device, the value of data input to the neural network is not limited at all, and any value can be input.

However, the range to which each unit is sensitive (sensi-t
ive; a range where a small difference in input value results in a relatively large difference in output value) is a place where the rate of change of the function where threshold processing is performed is large, and is near an inflection point or a staircase in the case of a step-like function. is within the range of Even if some input values belonging to outside the range are input to the unit,
Since there is no large difference in output value, it is not possible to achieve a sufficient learning effect, and the learning efficiency is poor. Therefore, even if the center of gravity is determined using a neural network trained using such a method, there is a possibility that a reliable center of gravity cannot be obtained.

SUMMARY OF THE INVENTION Therefore, it is an object of the present invention to provide a reliable center-of-gravity output device for a neural network that enables learning to be performed with high learning efficiency.

(Means for Solving the Problems) In order to solve the above-mentioned technical problems, the present invention, as shown in FIG. In a center of gravity output device using a neural network that outputs the center of gravity on a number line by inputting numerical values corresponding to each coordinate using a neural network 1 consisting of an input unit and one output unit, in front of each input unit, An input normalization device 21゜2□...s2n is provided which, upon receiving the numerical value corresponding to each coordinate, maps the value into a predetermined input range of each unit forming the neural network using an appropriate function; After the output unit, an output restoring device 3 is provided which, upon receiving the output value output by the unit, maps the value within the output range of the unit to the coordinate value range using an appropriate function. .

(Operation) Numerical values as learning data associated with each coordinate value are input to an input normalization device provided in front of each input unit associated with each coordinate value.

When the normalization device 25 receives the value, it maps the value within a predetermined input range of each neuron using an appropriate function.

Here, the "appropriate function" is, for example, the linear function described in claim 2 shown in FIG.

Furthermore, a "linear function" refers to a function expressed by a linear equation of variables.

Furthermore, the "predetermined input range of each unit" refers to the range that includes the part where the rate of change of the function used in the threshold processing performed in each unit is the largest, such as the vicinity of the inflection point and the step part of the step-like function. This is the most sensitive range of the unit, where a small difference in the input data will result in a large difference in the output data.

Therefore, learning efficiency is high.

The input data mapped to the predetermined input range is input to the input unit associated with each coordinate, subjected to dark value processing, and the data is output from the output line and input to the next unit.

In this way, the values output from the neural network are input to the output restoring device, and are restored to coordinate values in the original numerical range by an appropriate function, thereby obtaining the center of gravity.

Here, as the "appropriate function", for example, a linear function such as that used in the input normalization device is used.

In addition, the "center of gravity X" means, for example, each coordinate Xi on the number line.
For the numerical value (considered as a type of mass) mt associated with , we say: X=Σmi Xl 72m4.

This is because in fuzzy inference, an inference value is generally calculated by finding the center of gravity of the sum of functions of reduced membership functions for the same output variable according to the above equation.

Incidentally, as described in claim 3, a hierarchical neural network as shown in FIG. 3 may be employed.

〔Example〕

Next, examples of the present invention will be described.

FIG. 4 shows an overall block diagram including a center of gravity output device using a neural network according to this embodiment.

As shown in the figure, this system includes a fuzzy inference unit 40 consisting of a neural network that performs fuzzy inference,
A corresponding value output unit 50 for each coordinate that distributes a numerical value corresponding to each given coordinate on the number line, a neural network 11 that outputs the center of gravity value on the number line, and a neural network 11 that outputs the weight value of the neural network. Neural network control unit 2 that controls setting and changing of the internal state of the threshold value
0, and a center-of-gravity learning device 30 that generates teaching data for the neural network 11 so that the neural network 11 outputs a correct center-of-gravity value.

Here, a "neural network" is formed by a large number of basic units, notes called neurons), and internal connections with weight values corresponding to data representing the internal state, and each unit has data representing the internal state. Input data is converted into output data by performing threshold processing using a threshold value corresponding to .

In addition, when the neural network 11 receives a numerical value corresponding to each coordinate before each input unit, it appropriately sets a numerical range that includes the numerical value within a predetermined input range of each unit forming the neural network. Input normalization device 121.12□... that maps with a linear function
, 12n are provided, and an output restoration device 13 is provided which maps the output value of the neural network from within the output range of the neural network to the range of coordinate values using a linear function as an appropriate function.

Furthermore, as shown in FIG.
, a constant storage unit 33 that stores externally input coordinate values, input/output ranges, and the number of training data; a random number generation unit 34 that generates random numbers; and a constant storage unit 33 that stores coordinate values and input/output values held in the constant storage unit 33. A constant storage unit 33 that stores constants such as range and number of training data, and a random number generation unit 34 that generates random numbers.
and a teacher data generation section 32 that generates teacher data used for learning based on the constants stored in the constant storage section 33 and the random numbers generated from the generation section 34.

Further, as shown in FIG. 5, the neural network control section 20 includes a learning data storage section 22 that stores the teaching data generated from the teaching data generating section 32 as learning data, and a an internal state storage section 23 that stores internal state data to be followed by the neural network 11, such as weights associated with connection lines, thresholds used by each unit for threshold processing, learning constants, moments, etc.; When there is an instruction from the control unit 31, the learning data stored in the learning data storage unit 22 is input to the neural network 11, and the output value representing the center of gravity output from the neural network 11 and the center of gravity are input. and a learning control section 21 that controls changes in the internal state stored in the internal state storage section 23 by comparing it with teacher data representing the internal state.

Further, as shown in FIG. 6, the neural network 11 according to this embodiment has five input units 101 provided corresponding to each given coordinate on the number line.
An input layer 101 consisting of 1 to 1015 and a single intermediate layer 1
02 and one output unit 1031.
Consists of 03.

Note that the middle layer 102 includes 10 units 102□ to 102
□. has.

Next, the control operation of the center of gravity learning device of this embodiment will be explained.

As shown in the flowchart of FIG. 7, in step SJI, the control unit 31 of the center of gravity learning device 30 reads coordinate values, input/output range, and number of teacher data from the outside, and reads the constant storage unit 3.
Save to 3.

In step SJ2, the control section 31 controls the teacher data generation section 3.
2 to generate training data.

The teacher data generation section 32 reads constant data from the constant storage section 33 in step SJ3.

In step SJ4, the teacher data generator 32 extracts a random number from the random number generator, and in step SJ5, the teacher data generator 32 generates teacher data from the read constant and random number.

In step SJ6, the teacher data generation section 32 transfers the teacher data to the learning data storage section 22 of the neural network control section 20 and stores it therein.

In step SJ7, the control section 31 of the learning device 30 issues a learning instruction command to the learning control section 21 of the neural network control section 20.

In this example, given coordinates on the number line are (1,
2, 3, 4, 5), and the input range of the numerical value corresponding to each coordinate (corresponding to "mass") is 0≦y≦10,
The input/output range of the unit is 0≦t≦1, but the coordinates and values at the coordinates are mapped to the range of 0, 2≦t≦0.8, which is the predetermined input range of the unit. Therefore, as a linear function of the normalizer 12° (i=1.2...), 1nput=0.6y/10+0.2
...■, and as an appropriate linear function used by the output restoration device 13, c=4(output-0,2)10.6+1...
・We will use ■.

Learning data generated by the center of gravity learning device (0≦y≦10) 9
A part of 0 is 0.2≦t≦0.8 by the normalization device.
An example of what is mapped to the range is shown in FIG. here. p
The five values arranged in each column of attern (an) are given the coordinates (1, 2° 3.4.5
) is the input value as normalized teaching data corresponding to , and the value following the arrow in each column is the output value as normalized teaching data.

The input value of this teacher data is input to 5 units in the input layer and 1 unit in the middle layer.
A hierarchical neural network with 0 units and 1 unit in the output layer has a learning constant of 5.0, a moment of 0.4, and a convergence condition such that the difference between the output value of the network and the output value of the training data is 0.01 or less. When set and input to the internal state storage unit 23, learning is 2853
The test was completed in 88 times, and the weights and threshold values were as shown in FIG.

In the figure, the five values indicated by weightcl><Q> are applied to the 0th (starting from 0th) unit 102□ of the intermediate layer 102 and each of the five input units 10 of the input layer 101.
It is associated with the input line connecting 1□ to 1015, and indicates the weight obtained by learning.
<2> The 10 values indicated by <0> are the 0th values of the output layer 103.
unit 103□ (only one neuron exists in the output layer) and each of the ten units 1021 to 102 of the intermediate layer 102. This shows the weight associated with the line that connects o.

Further, the value of threshold (1) (0) indicates the threshold value used in the threshold processing performed in the 0th unit 102□ of the intermediate layer 102, and
hold (2) (0) indicates the threshold value performed in the O-th output unit 103□ of the output layer 103.

Figure 10 shows the value of the center of gravity output by the center of gravity output device using the neural network obtained in the example and the actual center of gravity. .12249, and the average error value is 0.12249.
It is 036018.

In addition, although the case where the number of output units is one was explained in the above example, it is not limited to this case.

Further, in the above example, the functions of the input normalization device and the output restoration device are linear functions, but the function is not limited to this case, and may be nonlinear functions.

(Effects of the Invention) As explained above, in the present invention, the value of the input data is normalized to a predetermined input range in which the neuron in each unit is most sensitive, and the output value of the neural network is restored to the original value. I try to do that.

Therefore, differences in input data are linked to differences in output data, making it possible to perform learning with high learning efficiency.
This makes it possible to provide a reliable center of gravity output device for a neural network.

[Brief explanation of the drawing]

Figure 1 is a block diagram of the principle of the invention, Figures 2.3 are block diagrams showing embodiments of the invention, Figure 4 is an overall system diagram according to the embodiment, and Figure 5 is a center of gravity output device according to the embodiment. Overall block diagram, FIG. 6 is a block diagram showing the neural network according to the embodiment, FIG. 7 is the control flow of the center of gravity learning device according to the embodiment, and FIG. 8 is an example of training data generated by the center of gravity learning device according to the embodiment. , FIG. 9 is a diagram showing an example of learned network weights and threshold values according to the embodiment, FIG. 10 is a diagram showing experimental results according to the embodiment, and FIG. 11 is a center of gravity output device according to the conventional example. A block diagram showing the 12th
The figure is a configuration diagram of the basic unit. 1.10.11...Neural network 21 +
22r~2n. 12□, 12□, ~12n1 521, 522. ~52n...Input normalization device 3.1
3.53...Output restoration device

Claims (4)

[Claims] (1) A neural network (1) consisting of at least each input unit provided corresponding to each given coordinate on the number line and one output unit inputs a numerical value corresponding to each coordinate. In a center-of-gravity output device using a neural network that outputs the center of gravity on a number line, when a numerical value corresponding to each coordinate is received before each input unit, the value is transferred to a predetermined input range of each unit forming the neural network. Input normalization device (2_1, 2_2..., 2_n) that maps with an appropriate function within
and, after the output unit, an output restoring device (3) that, upon receiving the output value outputted by the unit, maps the value within the output range of the unit to the range of the coordinate values using an appropriate function. A center of gravity output device using a neural network, characterized in that it is provided with a. (2) A neural network (1) consisting of at least each input unit provided corresponding to each given coordinate on the number line and one output unit inputs a numerical value corresponding to each coordinate. In a center-of-gravity output device using a neural network that outputs the center of gravity on a number line, when a numerical value corresponding to each coordinate is received before each input unit, the value is transferred to a predetermined input range of each unit forming the neural network. In addition to providing an input normalization device (52_1, 52_2..., 52_n) that maps with an appropriate linear function in A center-of-gravity output device using a neural network, characterized in that an output restoring device (53) is provided for mapping the value to the range of the coordinate values using an appropriate linear function. (3) Input layer (101) consisting of each input unit provided as one corresponding to each given coordinate on the number line
A neural network (10) consisting of an intermediate layer (102) with 0 or more stages and an output layer (103) consisting of one output unit inputs numerical values corresponding to each coordinate and calculates the result on the number line. In a center of gravity output device using a neural network that outputs a center of gravity of Normalization device (52_1, 52_
2, ..., 52_n), and after the output unit receives the output value output by the unit, it maps the value within the output range of the output unit to the range of the coordinate values using an appropriate linear function. A center-of-gravity output device using a neural network, characterized in that it is provided with an output restoration device (53). (4) In the center-of-gravity output device using a neural network according to claim 1, claim 2, or claim 3, a value based on a random number is taken as a value at a given coordinate, and the random number is applied to an input/output range of the neural network using an appropriate linear function. The training data mapped to the image is supplied as learning data,
A center of gravity output device using a neural network, comprising a center of gravity learning device that trains a neural network to approximately output a center of gravity. (5) In the center of gravity learning device provided in the center of gravity output device using a neural network according to claim 4, there is provided a random number generation unit that generates a value of random numbers as a value at given coordinates, and coordinate values and input ranges corresponding to each input. and a constant storage section that stores constants such as the number of required training data, and a random number value at the given coordinates based on the constants stored in the constant storage section and the random numbers generated in the random number generation section. and a teacher data generating section that generates teacher data that maps random numbers to the input/output range of the neural network using an appropriate linear function, supplies the teacher data as learning data, and outputs the center of gravity approximately. A center-of-gravity output device using a neural network, characterized in that it includes a center-of-gravity learning device for learning a neural network.