JPH03105662A - Self-learning processing system for learning device - Google Patents

Abstract

PURPOSE:To constitute the system so that the learning device itself can generate a teacher pattern by providing an evaluating part for evaluating input and output patterns conforming to the purpose or not, based on a rule determined in advance, with regard to the correspondence of an output pattern to which a noise is added and an input pattern at that time. CONSTITUTION:In an input part 1, an input pattern (a) is generated from external information, and in a learning device 2, the input pattern (a) is processed, and to its result, a noise is added from a noise part 3 and it becomes an output pattern (b). In an input/output pattern corresponding part 5, by coordinating the input pattern (a) and the output pattern (b), an input/output pattern (c) is generated, and in an evaluating part 6, the input/output pattern (c) is evaluated, and a pattern held in a teacher pattern table part 7 and an unnecessary pattern are selected. At the time of learning, the input/output pattern held in the teacher pattern table part 7 is learned as teacher pattern (d) by the learning device 2. In such a way, it is unnecessary to prepare the teacher pattern in advance.

Description

[Detailed Description of the Invention] [Summary] A learning processing device having a learning device that processes an inputted input pattern based on a presented teacher pattern, in which self-learning is performed. Regarding the self-learning processing method, the purpose is to make the learning device automatically extract a purposeful input/output pattern, and the output pattern with noise added to the output from the learning device and the Regarding correspondence with input patterns, we have an evaluation section that evaluates whether or not it is a purposeful input/output pattern based on predetermined rules, and we take care to maintain purposeful human output patterns as teacher patterns. [Industrial Application Field] The present invention is directed to a learning processing device having a learning device that processes an input pattern input based on a presented teacher pattern. Concerning learning processing methods. recent years. Devices (neural networks, etc.) that learn by presenting teacher patterns can be used for pattern recognition such as alphabet font recognition and image recognition, adaptive filters, etc. It has come to be applied to various types of robot control. [Prior Art] The conventional teacher pattern in a learning device that requires a teacher pattern is as follows. Learning is performed using teacher patterns prepared in advance by humans. [Problem to be solved by the invention] However, in the case of actual applicability, patterns that deal with time series. A pattern in which the teacher pattern itself changes.
Since there are patterns for unpredictable states, it is difficult to determine the type and amount of patterns to prepare as teacher patterns. Therefore, it takes a lot of time to create a teacher pattern. In order to put the learning device into practical use in actual abduction, an algorithm is required in which the learning device itself disciplines the teacher pattern. The present invention is.
The purpose is to enable the learning device to automatically extract purposeful input/output patterns. [Means for solving the problem] Figure 1 shows a diagram explaining the principle of the present invention. In the figure. ■ is an input section that adjusts the input pattern (a) (generally multiple bits of information) based on stimuli detected from the external environment. ■ is a learning device that incorporates the teacher pattern (d). Process the input pattern (a) based on the teacher pattern (b). Processing is performed to generate an output in accordance with the teacher pattern (b). ■ is the noise part, which is expected to add random noise to the output of the learning device ■ to obtain an output pattern that is more purposeful than the output of the learning device ■ itself. ．． ■
is the output part. For example, it performs operations such as driving a motor (not shown). 3 is an input/output pattern correspondence unit provided in the present invention, which takes correspondence between an input pattern (a) and its corresponding output pattern (b), and outputs an input/output pattern (c). ■ is an evaluation section created in the present invention. Many input/output patterns (c)
For each, the previous input/output pattern and the current person's output pattern are retained, and based on the comparison of both patterns. Evaluate whether the current input/output pattern is appropriate for the learning device [2], and selectively extract the appropriate input/output pattern. ■ is the teacher pattern table section. After temporarily accumulating a large number of input/output patterns (c) obtained by the input/output pattern correspondence section (■), the above-mentioned evaluation section (■) is allowed to refer to the input/output patterns (c), and the purposeful input/output patterns selectively extracted as described above are created. Keep it. And, send the relevant human output pattern to the learning device [2]. This will be supplied as the teacher pattern (d) for the next try. [Function] The input section generates an input pattern (al) from external information. This input pattern (a) is sent to the learning device ■ and the input/output pattern correspondence section ■. In the learning device [2], the input pattern (a) is processed. For example, noise is added from the noise section (2) to the result, resulting in an output pattern (c).
The output section ■ controls the external output according to output pattern (b). On the other hand, in the input/output pattern correspondence section■. The input pattern (a) and the output pattern (b) are associated with each other, and the input/output pattern (c) is determined. The evaluation section (■) evaluates the input/output pattern (c) and selects patterns to be retained in the teacher pattern table section [7] and unnecessary patterns. When learning. The input/output pattern held in the teacher pattern table section [7] is made to be learned by the learning device [2] as a teacher pattern (b). [Example] This will be explained using a specific example. Figure 2 shows an example of a robot, assuming a robot ■ with visual sensor input @ and a target @. We will explain an example in which a robot ■ learns the behavior of approaching a target @. The robot has a self-learning function using the self-learning processing method of the present invention. The robot ■ has 11 visual sensor inputs.For example, the visual sensor that captured the target is "l", and the others are "0".Therefore, the visual sensor input in Figure 2 is ro0000111000J. ．This visual sensor input ■ is sent as external information to the input unit ■.The input unit ■ generates an input pattern (a) from the sent visual sensor input @.This input pattern (a) is sent to the learning device ■ and the input/output pattern correspondence section ■.The learning device ■ processes the input pattern (a).The noise created by the noise section ■ is added to the result of the learning device ■.The final output pattern (b). In the noise part ■, create noise using uniform random numbers. Of course, noise such as Gaussian distribution may also be used. In the output part ■, the angle in the traveling direction from the output pattern (b) is 0. (Fig. 2) is calculated and sent to the motor as an external output.As a result, the robot moves (Fig. 2).This movement of III is called a step.On the other hand, in the input/output pattern correspondence section [F] .The output pattern (b) is matched with the input pattern (b) and the input/output pattern (c) is controlled.This input/output pattern (c) is temporarily stored in the teacher pattern table section [7].In the evaluation section ■ , the input/output pattern is evaluated according to the evaluation method described later.The purposeful human output pattern selected by the evaluation section is held in the teacher pattern table section [7].It is omitted from illustration in Fig. 2. In order to limit the movement range of the robot, we assume that there is a wall around the robot, and one try ends when it reaches the wall.We also set the maximum movement time during which the robot can move freely. When the time is reached, it is also regarded as an I try.In other words, as a condition for learning.(1) Either the target is captured or
Repeat the above until either (ii) the robot hits the surrounding wall or (ii) the maximum travel time is reached. One try is until the condition is met. After completing one try. Learning is performed for the next trial using the input/output pattern held in the teacher pattern table section [7] as the teacher pattern (b). After learning, for example, place the robot and target in another position and repeat the above process. Figure 3 is an explanatory diagram explaining the evaluation method. In the figure. A series of behavioral trajectories [stock] for one try, a series of input/output patterns [stock] ■ [phase] for one try, and their evaluation values [phase] are shown. In Figure 3, past input [phase]
is the previous visual sensor input. Current human power @ is the current visual sensor human power. Output [stock] is the final output pattern. The evaluation value [phase] is the result of evaluation by the evaluation section. In the evaluation department,
The previous input/output pattern (f) (Figure 3) is stored, and the following (
A) Evaluate the input/output pattern according to the conditions in (B). Evaluation is performed using O and X (called reinforcement signals). An input/output pattern that satisfies the following conditions (A) and (B) is evaluated as ○. The input/output patterns evaluated as 0 in the evaluation results are held in the teacher pattern table section ■. The evaluation value [phase] is "1" if the condition (^) is satisfied. If condition CB) is satisfied, “
If neither is satisfied, set it as ``0''. (^) Input/output pattern ((f in block A in Figure 3)
)) There is currently some kind of input to human power [7]. When the current human power @ in the input/output pattern ((e) in block A in Figure 3) reacts more than the current human power in the input/output pattern ((f) in block A in Figure 3), for example, the block in Figure 3 In A, human output pattern A (f
There is one "1" in the current human power @ of ).As there are three l'l's in the current human power @ of input/output pattern A (e), it is better than the current human power ■ of input/output pattern A (f). "When the number of 1's increases, this means that the target has been captured larger. In other words, it has gotten closer. (B) Input/output pattern (past input [phase] like (0) in block B of Figure 3) There is some input to the input/output pattern ((') in block B in Figure 3), and there is currently no input to human power [7].Input/output pattern ((e) in block B in Figure 3) When there is some input to the current human power @, for example, in block B of Figure 3, there is one "1" in the past input [phase] of the human output pattern B (f),
The current human power ■ in input/output pattern B (f) is all "0". r1 to the current human power [7] of input/output pattern B (e),
When there is one. This means that it loses sight of the target for a moment and then returns to its original position. The above evaluation is performed while the behavior trajectory [phase] shown in Figure 3 is obtained. Figure 4 shows the progress in which the robot learns to catch the target based on the present invention. Figure 4 is. Behavior trajectory in unlearned state 0. Behavior trajectory after learning on the 1st try. Behavior trajectory after learning on the 2nd try. Behavior trajectory after learning on the 0th and 3rd trials [phase]. From Figure 4, as the learning progresses, the robot approaches the target faster. It can be seen that in the end, the user was able to take the route closest to the shortest distance. Figure 5 shows the trajectory @ of one try action from the unlearned state, the input/output pattern @ @ and the evaluation value [phase] at that time. 1} Although only 10 types of teacher patterns for lie eyes have been created, we have been able to obtain a reasonable number of teacher patterns. [Effects of the invention] As explained above. According to the invention. The device creates a teacher pattern corresponding to the execution. It is possible to learn using the teacher pattern. Therefore. There is no need to prepare a teacher pattern in advance, which saves a lot of time and effort. Humans only need to monitor the operation of the device; there is no need to know the detailed internal structure.

Note that this time we learned a pattern that deals with time series, but the teacher pattern itself changes. It goes without saying that it is also possible to learn patterns for unpredictable situations. of course. It is also possible to prepare a known pattern. In addition, in the example given this time, a learning example was described for 11 visual sensors, but it is also possible to attach various other sensors.

[Brief explanation of drawings]

Figure 1 is an explanatory diagram of the principle of the present invention, and Figure 2 is an example of a robot.
Figures 3 to 5 are explanatory diagrams. In the figure, ■Second input section. ■: Learning device, ■: Noise section, ■: Output section, ■: Input/output pattern correspondence section, ■: Evaluation section, ■: Teacher pattern table section, (a): Input pattern, Φ): Output pattern. (c): Input/output pattern, @: Teacher pattern.

Claims (1)

[Claims] Input unit [1] that creates an input pattern (a) from external information
], a learning device [2], a learning device [2], which processes an input pattern (a) based on a presented teacher pattern;
2], a learning processing device comprising a noise unit [3] that creates noise to be added to the processing result, and an output unit [4] that receives the output pattern (b) that is the final result,
An input/output pattern correspondence unit [5] that matches the input pattern (a) and the output pattern (b) to create an input/output pattern (c), an evaluation unit [6] that evaluates the input/output pattern (c), and an evaluation unit It is equipped with a teacher pattern table section [7] that stores the input/output patterns sent from [6], and the evaluation section evaluates the correspondence of the input/output patterns (c).
The input/output pattern is held in the teacher pattern table section [7] according to the evaluation criteria, and the held input/output pattern is supplied as the teacher pattern (b) to the learning device [2]. A self-learning processing method for learning devices.