JPH056348A - Self-learning processing system for adaptive data processor - Google Patents

Abstract

PURPOSE:To efficiently self-learn with respect to the self-learning processing system of an adaptive data processor for self-learning so that a data converting function in an adaptive data processor is matched with a data processing purpose. CONSTITUTION:Trial-and-error processing is executed by superposing a noise to each output signal from the adaptive data processor 10 and applying the noise-superposed output signal to data to be processed, an I/O signal matched with a data processing purpose is selected as teacher data out of plural I/O signals obtained by the trial-and-error processing and the self-learning of the data converting function in the processor 10 is executed in accordance with the teacher data. The trial-and-error processing for obtaining the teacher data and learning processing can be executed in parallel by independently preparing a processor 20 having the same data converting function as that of the processor as a learning processor, so that the trial-and-error processing of the processor 10 is not stopped.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention self-learns the data conversion function of an adaptive data processing device having a configuration for executing data processing according to a data conversion function whose setting can be changed. The present invention relates to a self-learning processing method of an adaptive data processing device for moving, and more particularly to a self-learning processing method of an adaptive data processing device that enables efficient self-learning of a data conversion function.

For example, like a neural network,
There is an adaptive data processing device that executes data processing according to a configurable data conversion function. In such an adaptive data processing device, the teacher data related to the input / output signals that realize the data processing purpose is obtained, and the data conversion function is learned and set so that the input / output signal relationships of the teacher data are realized. It is required to continue. The learning process of the data conversion function needs to be executed efficiently.

[0003]

2. Description of the Related Art Obtaining teacher data required for learning processing of a data conversion function of an adaptive data processing device is
For example, if the input / output signal relationship for data processing is explicit, such as calculating the logical product of the input values, or if it can be modeled without being explicit, it is relatively easy. When the purpose of data processing becomes complicated, it becomes extremely difficult. From now on, the practical use of the adaptive data processing device is limited.

[0004]

In view of such a reality, the applicant of the present invention filed a Japanese Patent Application No. 1-328 filed on Dec. 20, 1989.
In 401 (Invention title "External evaluation standard learning method"), etc., the input / output signal relationship of the adaptive data processing device is obtained by performing trial and error operation in consideration of noise generation, and the input / output signal is acquired. By adopting a configuration in which teacher data is obtained by selecting one of the relationships that suits the data processing purpose according to the external evaluation criteria, and the data conversion function is reset according to this teacher data, this series of processing is performed. A self-learning processing method has been disclosed in which the data conversion function of the adaptive data processing device is set to one that is suitable for the data processing purpose by repeating the processing.

That is, as shown in FIG. 8, noise is superimposed on the output signal of the adaptive data processing device 1 which is output corresponding to the input signal obtained from the data processing target (environment) and is output to the data processing target. The teacher data is obtained by judging the quality of the input / output signal pair obtained by this series of processing according to the evaluation standard (external evaluation standard) from outside the system, and the input / output signal of this teacher data is adopted. By repeating the process of setting the data conversion function of the adaptive data processing device 1 so that the relationship is realized,
The invention has been disclosed in which the data conversion function of the adaptive data processing device 1 is self-learned.

Here, the reason why noise is superposed on the output signal of the adaptive data processing device 1 is explained by taking an example in which the adaptive data processing device 1 is composed of a neural network. Since the input and output signals of the network itself are learned, the weight values that define the data conversion function of the neural network remain the same as in the initial state forever, and it does not grow to execute the desired data conversion function. This is because, by superimposing noise, a trial-and-error operation is caused to obtain teacher data that meets the purpose of data processing. Further, as the external evaluation standard, as disclosed in Japanese Patent Application No. 1-328401, it is possible to adopt a configuration in which it is self-generated inside the system by using an adaptive data processing device or the like.
In the figure, 2 is an input mechanism for supplying an input signal to the adaptive data processing device 1, 3 is a noise generator for generating noise,
4 is a noise superimposing device that superimposes the noise generated by the noise generator 3 on the output signal of the adaptive data processing device 1, and 5 is an output that gives the output signal of the adaptive data processing device 1 on which noise is superposed to the data processing target. Mechanism, 6 is a short-term memory device for storing input / output signal pairs collected by trial and error operation,
7 selects teacher data from the input / output signal pair stored in the short-term storage device 15 according to an external evaluation standard, controls the short-term storage device 15 to store the teacher data, and selects the selected teacher data. The learning control device executes a learning process of a data conversion function of the adaptive data processing device 1 using data.

Further, in order to improve the present invention, the present applicant has filed in Japanese Patent Application No. 2-19228 filed on January 31, 1990 (the title of the invention "stabilization method of self-learning system"). , A quality level value of an input / output signal pair of the adaptive data processing device 1 is calculated, and when the input / output signal pair serves a data processing purpose according to the quality level value,
The present invention has disclosed the invention in which the operation of the system is stabilized by suppressing the level value of the noise superimposed on the output signal so that a desired data conversion function can be quickly obtained.

Certainly, the inventions disclosed by the applicant of the present application make the adaptive data processing device 1 complicated as the adaptive data processing device 1 can be used as a control device for driving control of a robot. It becomes possible to construct as a data processing device for realizing various data processing purposes.

However, in these inventions, the three phases of the execution phase / evaluation phase / learning phase are repeated to perform learning of the data conversion function of the adaptive data processing device 1, so that data conversion is performed. There was a problem that learning of functions could not be performed efficiently. That is, first, in the execution phase, an input signal obtained from the data processing target is given to the adaptive data processing device 1, and noise is superimposed on the output signal from the adaptive data processing device 1 output in response to the input signal. Input / output signal pairs are obtained by giving them to the data processing target, and then, in the evaluation phase, the teacher data is selected by judging the quality of these input / output signal pairs according to the prescribed evaluation criteria, and subsequently, In the learning phase, by repeating learning of the data conversion function of the adaptive data processing device 1 using this selected teacher data and upgrading the data conversion function, the adaptive data Since the configuration is such that the learning of the data conversion function of the processing device 1 is performed, the execution phase and the learning phase cannot proceed at the same time, which is why応型 data processing apparatus 1
Since it is necessary to stop the trial-and-error operation of, there is a problem that the learning of the data conversion function cannot be executed efficiently. Therefore, there is a problem in that these inventions cannot be used for data processing targets that need to keep the system in the running state at all times.

The present invention has been made in view of the above circumstances, and is a self-learning of a new adaptive data processing device that enables the self-learning process of the data conversion function of the adaptive data processing device to be efficiently executed. The purpose is to provide a processing method.

[0011]

FIG. 1 illustrates the principle configuration of the present invention. In the figure, 10 is an adaptive data processing device that calculates and outputs an output signal corresponding to an input signal according to a data conversion function whose setting can be changed. A device for executing data processing for processing purposes, 11 is an input mechanism for giving an input signal obtained from a data processing target (environment) to the adaptive data processing device 10, and 12 is a noise generator , 13 is a noise superimposing device, which superimposes the noise generated by the noise generator 12 on the output signal of the adaptive data processing device 10, and 14 is an output mechanism of the noise superimposing device 13. An output signal is given to a data processing target, and 15 is a short-term storage device, which is an input / output signal pair collected by trial and error operation of the adaptive data processing device 10. Those that store, 16
Is a learning control device, and controls so that the short-term memory device 15 stores the teacher data by selecting the teacher data from the input / output signal pair stored in the short-term memory device 15 according to a prescribed evaluation criterion. To do.

These adaptive data processing device 10, noise generator 12, noise superimposing device 13, and short-term storage device 15 are provided.
The learning control device 16 is configured on the execution processor 100.

Reference numeral 20 is an adaptive data processing device for learning, which has the same data conversion function as the adaptive data processing device 10, and 21 is a teacher data storage device, which is a teacher selected by the learning control device 16. What stores data, 2
Reference numeral 2 denotes a learning processing device, and when an input signal of teacher data is input to the learning adaptive data processing device 20, an output signal output from the learning adaptive data processing device 20 forms a pair of teacher data. To match the signal,
The data conversion function of the learning adaptive data processing device 20 is learned.

These learning adaptive data processing devices 2
0, the teacher data storage device 21 and the learning processing device 22 are
It will be configured on the learning processor 200. Thirty
Reference numeral 0 denotes an interface device, which is an execution processor 10
The interface processing between 0 and the learning processor 200 is executed.

[0015]

In the present invention, the input signal indicating the performance status of the data processing purpose is applied to the adaptive data processing device 10. In response to this input signal, the adaptive data processing device 10 calculates and outputs an output signal according to the set data conversion function, and the noise superimposing device 13 outputs the output signal of the adaptive data processing device 10. The noise generated by the noise generator 12 is superimposed, the output signal on which the noise is superimposed is received, the data processing purpose is performed, and the next input signal is input to the adaptive data processing device 10 according to the execution. What is done is repeated. When this trial and error operation is executed, the short-term storage device 15 stores the input / output signal pair of the input signal of the adaptive data processing device 10 and the output signal of the noise superimposing device 13, and the learning control device 16 When an input / output signal pair is stored in the short-term storage device 15, according to the evaluation value of the input / output signal pair given by an operator or the like,
Control is performed so that the short-term storage device 15 stores the teacher data by leaving the input / output signal pair having the high evaluation value as the teacher data and canceling the input / output signal pair having the low evaluation value. To do. Then, when the prescribed condition is satisfied, the execution processor 100 follows the inter-processor communication and selects the selected short-term storage device 15.
Of the teacher data is transferred to the teacher data storage device 21.

When the teacher data is transferred, the learning processor 200 executes the data conversion function obtained by the previous learning cycle of the learning processing device 22 to be described, according to the interprocessor communication, to the adaptive data processing device 10.
Set to. Then, the learning processing device 22 reads the teacher data from the teacher data storage device 21 and inputs the input signal of the teacher data to the learning adaptive data processing device 20. The data conversion function of the learning adaptive data processing device 20 is learned so that the output signal to be output matches the output signal forming a pair of teacher data.

As described above, according to the present invention, the learning processing device 22 uses the teacher data obtained in the previous operation cycle while the adaptive data processing device 10 is performing the trial and error operation. Since the learning of the data conversion function used in the next operation cycle is executed, the adaptive data processing device 10 does not need to stop the trial and error operation for learning the data conversion function. As a result, the self-learning process of the data conversion function of the adaptive data processing device 10 can be efficiently executed.

[0018]

The present invention will be described in detail below with reference to examples. FIG. 2 shows the adaptive data processing device 10 described with reference to FIG.
Neural network 3 suitable for use as
An example of 0 is illustrated. This neural network 30
Is an input unit 3 for receiving and distributing an input signal indicating a performance state of a data processing target, as shown in this figure.
An input layer composed of a plurality of 1's, and one or more inputs from the input layer, which are positioned after the input layer, and a weight value to be multiplied with respect to this input, and sum of products is calculated. The intermediate layer, which is composed of a plurality of basic units 32 that obtains the final output by converting the sum of products value by a predetermined defining function, and is positioned after the intermediate layer,
A basic unit that receives a plurality of inputs from the hidden layer and a weight value to be multiplied with respect to the inputs to obtain a product sum, and obtains a final output by converting the product sum value by a prescribed function. And an output layer composed of 32. Here, since the neural network 30 in this figure assumes that there is only one type of output signal given to the data processing target, the basic unit 3 of the output layer is shown.
Although the number of two is disclosed as one, when there are a plurality of types, the basic units 32 corresponding to the number of types are provided. Further, the intermediate layer does not need to be configured in one stage as in this figure, and may be configured in one or more stages with one or more basic units 32 as one stage. Good.

The neural network 30 changes its data conversion function according to the weight value assigned to each unit. This weight value is well known when the teacher data is given, and the back propagation method is well known. It will be learned by etc. When the adaptive data processing device 10 is implemented by this neural network 30, the learning adaptive data processing device 20 described in FIG. 1 must also be implemented by this neural network 30. However, the learning of the neural network 30 takes considerably longer than the execution. From this, it is preferable that the learning processor 200 that develops the learning adaptive data processing device 20 is configured by a dedicated processor in order to enable high-speed calculation. On the other hand, the execution processor 100 that deploys the adaptive data processing device 10 does not require so much calculation capacity, and therefore, it is preferable that the execution processor 100 also serves as a processor that controls the system.

FIG. 3 shows an embodiment of a processing flow executed by the execution processor 100 for expanding the adaptive data processing apparatus 10, and FIG. 4 executes the learning processor 200 for expanding the learning adaptive data processing apparatus 20. 4 illustrates an example of a processing flow. Next, the self-learning process of the present invention will be described in detail according to this process flow.

As shown in the processing flow of FIG. 3, the execution processor 100 first executes the system initialization in step 1, and performs data conversion of the adaptive data processing device 10 developed in its own processor. Set the function to the initial state. That is, when the neural network 30 is used as the adaptive data processing device 10, for example, a random initial value is set as the weight value assigned to each unit. Next, in step 2, an input signal indicating the performance state of the data processing purpose is given to the adaptive data processing device 10, and noise is superimposed on the output signal output in response to this input signal to generate the noise. By applying the superimposed output signal of
The adaptive data processing device 10 is operated by trial and error.

For example, the adaptive data processing device 10 is used as a control device for inputting detection values of a plurality of visual sensors included in a robot and controlling a rotation angle of a drive motor included in the robot according to the detected values. Is used as a control device for controlling to move from the start position to the target position, the detection value of the visual sensor of the robot is given to the adaptive data processing device 10 and the output output in response to this input signal. Noise is superimposed on the signal and the output signal on which the noise is superimposed is applied to the drive motor of the robot, whereby the adaptive data processing device 10 is operated by trial and error. According to this trial-and-error operation, the robot will move by trial-and-error in units of 1 second from the start position, as shown in FIG.

Next, in step 3, the execution processor 100 inputs / outputs the input signal to the adaptive data processing device 10 during this trial and error operation and the output signal on which the noise to be given to the data processing target is superimposed. The signal pair is stored in the short-term storage device 15, and in step 4, the evaluation value of the input / output signal pair given by the operator is obtained. For example, in the case of the robot control example of FIG. 5, when the operator performs the trial-and-error operation in units of one second toward the target position, it is “good”, and when the operator is far from the target position, “good” is displayed. Since "No" is specified, the evaluation value of "Good / No" is acquired. Here, as disclosed in Japanese Patent Application No. 1-328401, this evaluation value is executed by an execution processor 1 using an adaptive data processing device or the like.
When adopting a configuration in which self-generation is performed inside 00, the self-generated evaluation value is acquired. Then, in the following step 5, it is judged whether the obtained evaluation value is high or low, and when it is high, that is, when the input / output signal pair meets the data processing purpose, the step is determined. Proceed to 6 and short-term memory device 1
One of the storage pointers indicating the storage destination of No. 5 is updated. On the other hand, if it is determined in step 5 that the acquired evaluation value is low, that is, if it is determined that the pair of input / output signals does not meet the purpose of data processing, the memory pointer should not be updated and the overwrite instruction should be given. Processing is performed so as to return to step 2. By this update processing of the storage pointer, the pair of input / output signals that meet the data processing purpose can be stored in the short-term storage device 15.
Will be stored in.

In this way, when the input / output signal pairs that meet the data processing purpose are stored in the short-term storage device 15, the execution processor 100, in the following step 7, inputs / outputs stored in the short-term storage device 15. When it is determined whether the number of signal pairs has reached the specified number, and when it is determined that the number has not reached the specified number, the process directly returns to step 2 and waits until the specified number is reached. When it is determined in step 7 that the prescribed number is reached, step 8
To the learning processor 200, the input / output signal pair stored in the short-term storage device 15 is determined as teacher data.
Data is transferred to the teacher data storage device 21. At this time, the input / output signal pair stored in the short-term storage device 15 is cleared, and the storage pointer for instructing the storage destination of the short-term storage device 15 is reset to the initial state.

Subsequently, in step 9, the execution processor 100 updates the learning processor 200 with a data conversion function obtained by learning in order to update the data conversion function of the adaptive data processing device 10 to an upgraded version. Issue a transfer request. That is, when the neural network 30 is used as the adaptive data processing device 10, the transfer request of the weight value between the units obtained by learning is issued. As will be described later, in response to the issuance of this transfer request, the data conversion function learned from the learning processor 200 will be transferred. Therefore, in the subsequent step 10, the data transfer of the data conversion function is terminated. After waiting, when the end of the data transfer is confirmed, finally, in step 11, the learning processing device 22 of the learning processor 100 executes learning of the data conversion function using the transferred teacher data. After the instruction, the process returns to step 2.

On the other hand, the learning processor 200, as shown in the processing flow of FIG.
The system is initialized to set the data conversion function of the learning adaptive data processing device 20 developed in the processor to the same initial state as the initial state of the data conversion function of the adaptive data processing device 10. Next, Step 21
Then, it waits for a command sent from the execution processor 100. When a command is sent while waiting for this command, it is determined in step S22 whether the sent command is a command for instructing transfer of the data conversion function or a command for learning processing. to decide.

If it is determined in step 22 that the command is an instruction to transfer the data conversion function, the process proceeds to step 23, and the learning processing device that has started learning in response to the instruction of the previous learning process. The learning of 22 is cut off at a good break, and the data conversion function obtained by the learning up to that point is transferred to the execution processor 100. On the other hand, when determining that the command is a command for instructing the learning process, the process proceeds to step 24 to instruct the learning processing device 22 to learn the data conversion function using the transferred teacher data. In response to this instruction, the learning processing device 22 reads the teacher data from the teacher data storage device 21 and, when inputting the input signal of the teacher data to the learning adaptive data processing device 20, learn learning adaptive data. The data conversion function of the learning adaptive data processing device 20 is learned so that the output signal output from the processing device 20 matches the output signal forming a pair of teacher data. That is, for example, when the neural network 30 is used as the learning adaptive data processing device 20, the weight value between each unit is learned according to the well-known back propagation method, and the learning is terminated. Occasionally, when the weight value between all units becomes a value with the same number of learnings, it is cut off.
Then, the learning processor 200 finally performs step 2
In step 5, the execution processor 200 is notified of the end report, and the process is returned to step 21.

According to the processing executed by the execution processor 100 and the learning processor 200 described above, as shown in the timing chart of FIG.
When 0 is executing the trial-and-error operation of the (n + 1) -th trial, the learning processor 200 uses the teacher data obtained by the trial-and-error operation of the execution processor 100 to perform the data conversion function. Perform learning processing. Then, the execution processor 100 executes the trial and error operation of the next (n + 2) trial by using the data conversion function learned by the teacher data of the n trial, and the learning processor 100:
The learning process of the data conversion function is executed using the teacher data obtained by the trial and error operation of the (n + 1) th trial of the execution processor 100.

Thus, by using the present invention,
The adaptive data processing device 10 can execute the trial and error operation without stopping. As described above, the data conversion function used in the trial-and-error operation of the n-th trial of the adaptive data processing device 10 is (n-
2) Although it was learned by the teacher data obtained in the trial eye and not by the teacher data obtained in the (n-1) trial eye, the self-learning processing method of the present invention is used. There is no problem because it is a trial-and-error operation.

FIG. 7 illustrates another embodiment of the processing flow executed by the execution processor 100. In the embodiment of FIG. 3, the teaching data unit used for learning is disclosed as being composed of a plurality of teaching data. However, in this embodiment, every time one teaching data is obtained, the teaching data is used for learning. This is an embodiment constituted by treating as one teacher data unit and starting learning of the data conversion function.

Although the illustrated embodiment has been described, the present invention is not limited to this. For example, the learning adaptive data processing device 20 has been disclosed as having the same data conversion function as the adaptive data processing device 10. However, the same meaning does not require that it is completely the same. There is no such requirement, and the same is required within the range of accuracy.

[0032]

As described above, according to the present invention,
By performing trial-and-error operation, the data conversion function of the adaptive data processing device is configured to execute data processing according to the data conversion function whose setting can be changed. When I go,
Since the trial and error operation does not have to be stopped, the data conversion function of the adaptive data processing device can be efficiently self-learned.

[Brief description of drawings]

FIG. 1 is a principle configuration diagram of the present invention.

FIG. 2 is an explanatory diagram of a neural network.

FIG. 3 is an example of a processing flow executed by an execution processor.

FIG. 4 is an example of a processing flow executed by a learning processor.

FIG. 5 is an explanatory diagram of robot control.

FIG. 6 is a timing chart of processing of the present invention.

FIG. 7 is another example of the processing flow executed by the execution processor.

FIG. 8 is an explanatory diagram of the invention previously filed.

[Explanation of symbols]

 10 Adaptive Data Processing Device 11 Input Mechanism 12 Noise Generator 13 Noise Superimposer 14 Output Mechanism 15 Short-term Memory Device 16 Learning Control Device 20 Learning Adaptive Data Processing Device 21 Teacher Data Storage Device 22 Learning Processing Device 100 Execution Processor 200 Learning Processor 300 interface device

 ─────────────────────────────────────────────────── ─── Continuation of front page (72) Inventor Atsuko Asakawa 1015 Kamiodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture, Fujitsu Limited (72) Inventor Shigemi Nagata 1015, Kamedotachu, Nakahara-ku, Kawasaki City, Kanagawa Prefecture, Fujitsu Limited

Claims (1)

Claim: What is claimed is: 1. An adaptive data processing device (10) for calculating and outputting an output signal corresponding to an input signal according to a data conversion function capable of changing a setting, and the adaptive data processing device ( 1
0) an output signal of the noise superimposing device for superimposing a noise (13), the input and output signal pair of the input signal of the adaptive data processing device (10) and the output signal of the noise superimposing device (13). Of the adaptive data processing device (10), a pair of input / output signals suitable for the data processing purpose of the adaptive data processing device (10) is selected as the teacher data, and the input / output signal relationship of the selected teacher data is realized to realize the input / output signal relationship. A self-learning processing method of an adaptive data processing device for learning the data conversion function of a data processing device (10), the learning adaptive type having the same data conversion function as the adaptive data processing device (10). A data processing device (20) and the learning adaptive data processing device when the teacher data is obtained.
(20) using the learning processing device (22) for learning the data conversion function of the adaptive data processing device (10) that realizes the input / output signal relationship of the teacher data, When data is obtained, by setting the data conversion function obtained by the learning process of the learning processing device (22) in the adaptive data processing device (10), the adaptive data processing device (10) The self-learning processing method of the adaptive data processing device, characterized in that the processing is performed so as to have a data conversion function that realizes the above data processing purpose.