JP2523150B2 - Robot control system - Google Patents

Description

The present invention relates to a learning method for setting weights in a robot control method in which a robot that behaves based on multiple sensor information is controlled by a hierarchical network having weights. Robot control using a hierarchical network For the purpose of improving the practicality of the method, a robot learning simulator with the same configuration as the hierarchical network control robot is constructed on the computer, and it is further configured with a motion mimicking means for simulating the behavior of this robot learning simulator. Using this robot learning simulator, the weight value of the hierarchical network required by the hierarchical network control robot is obtained, and the effectiveness of the obtained weight value is verified by the action mimicking means. Value of hierarchical network control robot floor And it constitutes a to set the value of the network weights.

[Industrial applications]

The present invention relates to a learning method in a robot control method for controlling a robot that behaves based on a plurality of sensor information by a hierarchical network having weights.

In recent years, with the progress of artificial intelligence (AI), factory automation (FA), and office automation (OA), there is an increasing demand for an intelligent "soft" system that is easy for humans to use and can coexist with humans. In order to meet this expectation, systems to which expert systems and the like have been applied and robots with pattern recognition functions are beginning to be provided, but they are still far from being "soft" systems. From now on, there is a demand for the development of a "soft" robot control method that adaptively controls the behavior of the robot according to changes in usage and environment.
It is also desired to develop a technique for enhancing the practicality of such a robot control system.

[Conventional technology]

The conventional robot control method is realized by a sequential processing computer (Neumann type computer).
Therefore, in the control method of a robot that behaves based on multiple sensor information, when a human considers in advance which sensor will bring what kind of information, when the input pattern from which sensor comes in, This has been realized by a configuration in which what output pattern is generated and what action should be taken is strictly described in the form of a program.

[Problems to be solved by the invention]

Therefore, the robot is only acting on the basis of this program, and naturally cannot cope with situations not described in the program. If it breaks, there is a problem that it is not possible to take appropriate actions. Furthermore, when the robot has a large number of sensors, it is extremely difficult or impossible to create the program itself.

In order to solve these problems, the applicant has proposed a “soft” robot control method based on a hierarchical network structure (filing date: December 28, 1987, invention title “robot control method”). As described in detail in the present specification, in the robot control method based on this hierarchical network structure, it is necessary to learn and verify the weight value of the hierarchical network.

However, learning and verifying weights on an actual robot requires a long development period to work on a robot with a bad development environment, and the programs and hardware necessary for learning and verification of results are installed. There is a problem that it must be done and that it is affected by variations in the characteristics of components such as robot sensors and motors, and the execution speed of the control processor on the robot that executes calculations is low Therefore, it is not always a good idea because the weights of the hierarchical network of robots cannot be learned quickly.

The present invention has been made in view of such circumstances,
Fast and accurate learning of the weight value of the hierarchical network in the robot control method by the hierarchical network structure,
By providing a robot learning method that can be verified,
The purpose is to improve the practicality of a robot control system with a hierarchical network structure.

[Means for solving problems]

 FIG. 1 is an explanatory view of the principle of the present invention.

In the figure, 1 is a sensor means, which is composed of a plurality of sensors for fetching external information surrounding the robot, 2 is an action pattern generation means, which generates an action pattern signal to define the action pattern of the robot, 3 is a robot control means, which is composed of a hierarchical network having weights, and controls the action pattern generating means 2 which generates a control signal according to the detection pattern of the sensor means 1 according to this hierarchical network, 5 is a basic control operation The storage means is for storing control signal information to be generated by the robot control means 3 for one or a plurality of specific detection patterns of the sensor means 1 selected in advance. The sensor means 1, the action pattern generation means 2, the robot control means 3, and the basic control operation storage means 5 constitute a hierarchical network control robot 10 for executing actual control of the robot. Reference numeral 20 denotes a robot learning simulator configured on a computer, for simulating the behavior of the pseudo-hierarchical network control robot 10 'and the pseudo-hierarchical network control robot 10' having the same configuration as the hierarchical network control robot 10. No. 7 of the action imitation means.

[Work]

In the present invention, the robot learning simulator 20 learns the weights of the hierarchical network so that the basic control operation of the basic control operation storage means 5 can be realized by using the pseudo hierarchical network control robot 10 ', and the obtained weights are obtained. Is verified by the action mimicking means 7, and the verified weight is transferred to the hierarchical network control robot 10 to set the weight of the hierarchical network control robot 10 and verify by actually operating the hierarchical network control robot 10. If it becomes necessary to learn one or more basic control actions by doing so, the basic control actions are transferred from the hierarchical network control robot 10 to the robot learning simulator 20 and the basic control actions retained previously are retained. Learning with or with additional learning and weighting the results as a hierarchical network. Executing a set of weights of the hierarchical network control robot 10 by transferring the to control the robot 10.

As described above, according to the present invention, the robot learning simulator 20 can learn the basic control operation for setting the weight, which requires a great deal of processing time. By using the circuit, high-speed weight learning is possible without increasing the hardware of the hierarchical network control robot 10. In addition, since the weights learned can be verified without making the actual robot act, it is expected that the development efficiency will be improved.

〔Example〕

 Hereinafter, the present invention will be described in detail according to examples.

FIG. 2 is a detailed configuration diagram of the robot control means 3 described in FIG. As shown in this figure, the robot control means 3 of the present invention is configured by a hierarchical network including a kind of node called a unit and an internal connection having a weight. This configuration is basically a processing method called a back-propagation method (DERumelhart, GEHinton, and RJWilli) that has been proposed as a data processing method that gives an adaptive function to a computer.
ams, “Learning Internal Representations by Error P
ropagation, "PARALLEL DISTRIBUTED PROCESSING, Vol.1,
pp.318-364, The MIT Press, 1986).

Next, an outline of this back propagation method will be described.

In the back propagation method, a hierarchical network is composed of a kind of node called a unit and an internal connection with weight. FIG. 3 shows the internal structure of the unit and is called a basic unit 30. This basic unit 30
Is a multi-input, single-output system. In this unit,
It has an accumulator 31 that multiplies each input by a weight of each internal connection and sums the results of all multiplications, and a threshold processor 32 that performs threshold processing and outputs one output. . That is, the data processing in the basic unit 30 in the back propagation method consists of product-sum calculation of input and weight and threshold processing. Here, in the back propagation method, the sigmoid function represented by Expression (2) is used as a function of the threshold.

The calculation performed by the basic unit is shown below by mathematical expressions.

y _pi = 1 / (1 + exp (-x _pi + θ _i )) Equation (2) where h: h layer unit number i: i layer unit number p: input pattern number θ ₁ : i layer i unit Threshold value W _ih : h-weight of internal coupling between i layers x _pi : sum of inputs from each unit of layer i to unit i of layer y _ph : p output of layer y for pattern input y _pi : p In the i-layer output back-propagation method with respect to the pattern input, the basic unit 30 based on the sum of products and threshold processing has a layered network structure having an error feedback bag as shown in FIG. The adaptive data processing is executed by learning the desired data processing method (association between the input pattern and the output pattern) in the network by using an algorithm that adaptively and automatically adjusts by the feedback.

Here, from the equations (1) and (2), in the back propagation method, the adjustment of the weight and the threshold needs to be executed at the same time, but the setting of the weight and the threshold is a difficult task that interferes with each other. Become. In the conventional back-propagation method, a threshold is given to each unit. However, when the number of units is large, it is a complicated and troublesome task to give a threshold to each unit. It was a coveted one. Therefore, in the robot control means 3 of the present invention, as shown in FIG. 2, in addition to the input unit 3-h for the number of input signals, "1" is always input in the input layer, the threshold input unit for threshold adjustment. 3'-h is provided.

As described above, the reason why the threshold value input unit 3'-h is provided so that the threshold value of each unit 3-i in the intermediate layer can be set by the same process as setting the weight W _ih for the intermediate layer is as follows. You can think of it as such. That is, in each unit 3-i of the intermediate layer, 1 unit in the input layer
Since the value y _ph from the three extra units 3'-h is always "1", the value x _pi is ∴x ' _pi = x _pi + W _ih' (where h'is the number of the threshold input unit of the input layer). From this result, when substituting θ _i p −W _{ih ′ into} equation (2), y _pi = 1 / (1 + exp (−x _pi −W _{ih ′} )), which is the threshold θ _i shown in equation (2). Is substantially −
Corresponds to the setting change to _Wih . That is, the complexity of setting the threshold value of each unit in the intermediate layer is eliminated.

Next, the stored information stored in the basic control operation storage means 5 described with reference to FIG. 1 will be described according to the specific example shown in FIG. For example, when the sensor output is in the binarization mode and there are three input units 3-h as shown in FIG. 2, the number of input patterns to the robot control means 3 is eight as shown in FIG. . Therefore, if the action pattern of the robot is, for example, two states of the moving state and the stopped state, the action pattern of the robot of "1" or "0" is determined for each of these eight input patterns. . As described above, it is already determined what kind of action the robot should take with respect to the detection pattern of the sensor means 1. Therefore, some of such information is stored in the basic control operation storage means 5. Would be preselected and stored. It should be noted that it is possible to store all eight input patterns as shown in FIG. 4 and a small number of input patterns.

In order to deepen the understanding of the contents of the example of FIG. 4, a specific example of the action pattern of the robot shown in FIG. 5 will be described. FIG. 5 specifically considers a robot having three optical sensors shown in FIG. 5 (a) and one motor for performing the action of “turning to the right”. Then, it is assumed that the robot wants to take the action of following a light emitting source that revolves around the robot as shown in FIG. At this time, as the sensor input pattern of the robot, the combination shown in the input pattern column of FIG. 4 can be considered. For example, the input pattern of "0" in Fig. 4 is the state where all three optical sensors are OFF, and the input pattern of "4" is the state where the optical sensor is ON and the optical sensor is OFF, input of "6". The pattern is that the optical sensor is ON and the optical sensor is OFF. In order to make the robot follow the rotating light source, the output pattern (behavior) shown in the teacher signal column of FIG. 4 can be realized by network learning and training according to such input pattern. Good. For example, the input pattern of "0" in Fig. 4 is
Since both the optical sensors are in the OFF state, that is, no light is emitted from the light emitting source, the output pattern at this time represents "0", that is, "turn off the motor". The input pattern of "6" is the optical sensor,
Is ON and the optical sensor is OFF, that is, the state where the light source is captured (Fig. 5 (a)), so the control "turn off the motor" is indicated, and the input pattern of "4" is the optical sensor. Is ON and the optical sensor is OFF, that is, the light source captured by the robot has moved further (Fig. 5 (b)).
Therefore, it means "turn on the motor (turn to the right)." That is, if the teacher signal as shown in FIG. 4 is obtained, the action pattern of the robot as shown in FIG. 5 is preferably controlled.

In order to realize such preferable control, according to the present invention, the weights W _ih and W _ji of the internal connection of the hierarchical network of the robot control means 3 are learned in accordance with the storage information stored in the basic control operation storage means 5. It is a decision to make. That is, the operation mode of the robot control means 3 is set to the learning mode, the stored information is sequentially read from the basic control operation storage means 5, and the values of the weights W _ih and W _ji are determined by using the learning algorithm of the back propagation method. It is configured to automatically determine.

Next, a learning procedure for determining the values of the weights W _ih and W _ji will be described. Here, the explanation of this learning procedure is given by further generalizing the output layer of the robot control means 3 as shown in FIG. 6 having a plurality of output units 3-j.

First, the initial value of the weight of the inner join is determined. This initial value is determined by a random number of 1 or less. This is because if the weight values are all the same or are symmetric with respect to the unit, the back propagation method does not change the weight and learning does not proceed, so random numbers are used to avoid this. Then, the data of the input pattern stored in the basic control operation storage means 5 and the data of the desired output pattern paired with the input pattern are sequentially read.
For example, as shown in FIG. 4, if there are eight pieces of stored information, all of these pieces of information will be sequentially read.

Next, the control parameters related to the back propagation method are input. Here, the learning rate, which is the coefficient of the amount of change in weight per incremental iterative learning, and the learning speed for suppressing the oscillation at the time of convergence are used as control parameters. Then, according to the algorithm of the back propagation method, the weight of the inner coupling is incrementally learned so that the output value of the output layer unit 3-j matches the desired output, and finally, the output value of the output layer unit and the desired value are obtained. When the error from the output value becomes smaller than the predetermined value for all input patterns, the learning is finished.

A specific algorithm for updating the weight of the hierarchical network will be described below. The following equation is obtained by analogy with the equations (1) and (2). That is, y _pi = 1 / (1 + exp (−x _pi + θ _i )) Expression (4) y _pj = 1 / (1 + exp (−x _pj + θ _j )) Expression (6) where y _ph : output from the h-th unit of the h-layer (input layer) to the p-th pattern input value y _pi : p Output from the i-th unit of the i-th layer (here, the middle layer) for the p-th pattern input value y _pj : Output from the j-th unit of the j-th layer (here, the output layer) for the p-th pattern input value x _pi : i Sum of p-th pattern input to layer i-th unit x _pj : Sum of p-th pattern input to layer j-th unit W _ih : Weight W between h-layer h-th unit and i-th layer i-th unit _ji : Weight between the i-th unit of the i-th layer and the j-th unit of the j-th layer Next, the sum of squares E _p of the error between the teacher input vector and the output vector of the network is calculated as the error of the network from these values.

Where E _p : error vector for p-th pattern input E: sum of error vectors for all pattern inputs d _pj : teacher signal to j-th layer j-th unit for p-th pattern input Here, the relationship between error vector and output layer vector In order to obtain Eq. (7) is partially differentiated with respect to y _pj , Get. Further, in order to obtain the relationship between the error vector and the input to the j-th layer, when the error vector is partially differentiated by x _pj , Get. However, in the present embodiment, one input unit 3'-h is provided in the input layer and a value of "1" is always input to this unit 3'-h. It realizes automatic adjustment. Furthermore, in order to obtain the relationship between the error vector and the weight between the ij layers, the error vector is partially differentiated with W _ji , Get the solution represented by the sum of products of

Next, when the change of the error vector E _p with respect to the output y _pi of the i layer is _calculated , Get. Furthermore, when the change in the error vector E _p with respect to the change in the sum x _pi to the i-th layer input unit is calculated, Obtain the solution represented by the sum of products of. Furthermore, when the relation of the change of the error vector with respect to the change of the weight between the h-i layers is obtained, Obtain the solution represented by the sum of products of.

From these, the relationship between the error vector for all input patterns and the weight between the i-j layers is obtained as follows.

In addition, the error vector for all input patterns and h−i
The relationship with the weight between layers is calculated as follows.

Since equations (14) and (15) show the rate of change of the error vector with respect to the change of weight between layers, the weight is changed so that this value is always negative. E, which is the sum of squared errors, can be asymptotically set to zero. Therefore, in the present embodiment, the change amount ΔW _ji per weight is set as follows, and this operation is repeated based on the gradient method to converge E to zero.

However, ε: learning rate (the same function as the gradient rate of the gradient method) Further, in the present invention, the learning speed is applied to the equations (16) and (17) for the purpose of suppressing the oscillation at the time of convergence in the gradient method. , ΔW _ih and ΔW _ji are set as follows.

However, α: learning rate constant t: number of times The above weights W _ih and W _ji are the units 3-i and 3-
It is stored in the storage device corresponding to each j, and the contents are corrected by feedback during the learning described above. Then, the finally determined learning result weights W _ih and W _ji will be used in the actual processing described below. As this storage device, the basic control operation storage means 5 can be used, but it may be provided separately.

Next, control of the robot to be executed using the weights W _ih and W _ji of the internal connection of the hierarchical network determined in this way will be described.

When the learning of the hierarchical network is completed, the operation mode of the robot control means 3 is switched to the processing mode, that is, the mode for actually controlling the operation of the robot. In this processing mode, the input pattern composed of the detection signals of the sensors captured by the sensor means 1 composed of a plurality of sensors is input to the input layer of the hierarchical network of the robot control means 3. Then, the robot control means 3 obtains an output pattern by performing a product-sum operation and threshold processing on the input pattern using the weight value and the threshold determined in the learning mode, and outputs the obtained output pattern as a control signal. To the action pattern generating means 2. Then, the action pattern generation means 2 that has received this control signal will generate an action pattern signal corresponding to the control signal to define the action pattern of the robot. Therefore, by repeating such processing, the robot can take appropriate actions according to the detection patterns of the plurality of sensors.

The contents of this processing will be concretely examined according to the example of FIG. Now, the relationship between the robot and the light source is as shown in FIG. 5 (a), that is, the optical sensor of the robot is ON,
It is assumed that the optical sensor is OFF and the motor is OFF. At this time, if the light source moves to the right as shown in Fig. 5 (b),
The robot's optical sensor turns on and the optical sensor turns off. When this is input to the network shown in FIG. 2, the output pattern "1" representing the control "turn the motor ON (turn right)" is generated. Therefore, the robot turns right. This right turn is continued until two of the three optical sensors are turned on. In this case, as shown in Fig. 5 (c), when the optical sensor is in the ON state and the optical sensor is in the OFF state, the network outputs "0" indicating the control "turn off the motor". ， The robot stops. In this way, the robot inputs the sensor input signal pattern that is captured every moment into the network shown in FIG.
It is possible to follow the movement of the light emission source by controlling the motor according to the optimum output pattern that the network outputs every moment corresponding to the input pattern. In this example, the case where the light emitting source moves to the right has been described. However, even if the light emitting source moves to the left as shown in FIG. 5D, the robot can follow the same manner.

In this way, the robot is controlled to take appropriate actions. However, when the number of sensors and the number of control signal lines are large, the basic control operation storage means 5 cannot store all the data of the input pattern and the desired output pattern paired with the input pattern. Will be stored. From this, when the output pattern is obtained with the weight value and the threshold obtained by learning using this basic data, the desired robot behavior may not be obtained.
In such a case, in the present invention, the operation mode is switched to the learning mode, and the weight of the connection of the network at that stage is used as an initial value to set a pair of an input pattern and a desired output pattern at a time when a desired unobtainable time is obtained. Control of the robot by storing it in the basic control operation storage means 5 and automatically fine-adjusting the weight of the coupling so that the robot control means 3 can further combine and combine the input / output pattern pairs. The performance can be improved. Therefore,
Ultimately, optimal control of the robot will be realized.

FIG. 7 shows a network obtained as a result of learning a kind of parity check processing in the network by the back propagation method (automatic adjustment of weight and threshold). Here, Net1 shown in FIG. 7 (A) and Net2 shown in FIG. 7 (B) have the same characteristics for checking odd parity, that is, for three inputs that take "0" or "1". When the number of "1" is an odd number, the output layer outputs "1". In FIG. 7, the number next to the internal connection line is the weight, and the number in the circles representing the unit is the threshold value. Comparing Net1 and Net2, it can be seen that the weight and the threshold are completely different. This is because of the network redundancy, which makes these differences. As shown in FIG. 7, the output of the basic unit 30 is set to "1" and "0".
It is also possible to configure the network so that it only takes the following states.

Next, the redundancy of the number of units in the middle layer that constitutes the hierarchical network will be described. FIG. 8 is a network that realizes the exclusive OR function, and is an example of the case where the redundancy of the number of intermediate layer units is not provided. In the exclusive OR, as shown in FIG. 9, the identification of the input pattern by the straight line (linear identification) is impossible with one straight line, and at least two straight lines are required as shown in FIG. 9 (B). Therefore, for this identification, as shown in FIG. 9 (A), at least two intermediate layer units are required. However, in the present invention, in order to automatically adjust the threshold values of the intermediate layer unit and the output layer unit, the input layer is provided with the threshold value input unit 3'-h having "1" as an input signal at all times. In the network shown, there is no redundancy in the number of hidden units relative to the number to be identified. In this case, the two middle layer units 3-i have to perform automatic adjustment including the threshold value of the output layer unit 3-j, and the number of times of repeated learning until the exclusive OR characteristic is adapted increases. On the other hand, Fig. 10 shows an example of a network with redundancy. As shown in the figure, if there is redundancy in the number of middle layer units,
Since the surplus units can be dedicated to the adjustment of the threshold value of the unit 3-j in the output layer, the number of times of repeated learning can be greatly reduced as compared with the case where there is no redundant unit.

Next, a specific calculation example of the weight of the inner connection will be described.

11 and 13 show the internal connection weights between the units when learning of the odd parity check function is completed.
11 to 13 have different weights but the same function. This is because the learning rate and the learning speed parameter are different and the redundancy of the number of hidden layer units. Figure 12 (A)
(B) and (C) respectively show an input and an output when an arbitrary pattern is input after the learning shown in FIG. 11 is completed. Further, FIGS. 14 (A), (B) and (C) respectively show inputs and outputs when an arbitrary pattern is input after the learning shown in FIG. 13 is completed. In the robot control means 3, since the calculation is performed by an analog value, the output value is not completely 1 or 0. However, if a threshold value is used, 1 or 0 is output from the output value shown in FIGS. 12 and 14. Clearly, the decision is easy.

In addition, Fig. 15 shows the result of learning the even parity check function on the same network as that on which the odd parity check function was learned, and shows the weight of each inner connection. 16 (A), (B) and (C) are respectively
Figure 15 shows the input and output when an arbitrary pattern is input after the learning shown in Fig. 15.

Next, an embodiment of the present invention for controlling the action pattern of a robot that tracks a moving object or escapes from a moving object will be described. This robot is designed to follow a moving object, which is a light emitting source and an ultrasonic wave source, while avoiding an obstacle, or to escape from a moving object while avoiding an obstacle. It is controlled, and as a driving source for tracking a moving object or escaping from a moving object, to take two propulsion motors for taking forward and backward actions and for making left and right actions. In addition to being equipped with two steering motors, a piezoelectric buzzer corresponding to human speech is equipped.
The motor and the piezoelectric buzzer are turned on in the embodiment.
It is controlled in the binary mode of / OFF. That the robot from having a five driving source, number of units of the output layer of the robot control unit 3 of the hierarchical network that is provided for control of the robot becomes five, behavior patterns robot which There are two ⁵ become.

Figure 17 shows the layout of the sensors that this robot uses to track and escape. As shown in this figure, the robot 100 has two head light sensors 1 corresponding to both eyes on the head.
01R, 101L and two head ultrasonic sensors 10 corresponding to both ears
Equipped with 2R and 102L, the body has two body light sensors 103R and 103L corresponding to both eyes, one body ultrasonic sensor (or transmitter) 104 corresponding to the ears, and contact with obstacles. It is equipped with three touch sensors 105F, 105R, 105L provided with a central angle of 120 degrees in order to detect the fact. The robot 100 further detects the fact that the rotation reaches, for example, the limit of 180 ° when the head and the body rotate to the limit rotation angle by the rotation of the steering motor included in the robot 100.
It is equipped with 106R and 106L. This head light sensor 101
R, 101L and body light sensor 103R, 103L detect light emitted from a moving object that is a target object, and head ultrasonic sensor 102R, 102L
The body ultrasonic sensor 104 operates so as to detect ultrasonic waves emitted by a moving object that is a target. Here, two sets of sensors corresponding to both eyes of the human being, head light sensors 101R and 101L and body light sensors 103R and 103L, are provided because the wavelength of light emitted by a moving object is different between tracking and flight. Therefore, the sensitivity is adjusted to this wavelength.
This is because it is necessary to prepare various types of light detection sensors.
Further, the head and body of the robot are integrally rotated with respect to the base of the robot, and the wheels of the robot are directed toward the front of the head in accordance with this rotation.

As described above, since this robot has 12 sensors, the number of units in the input layer of the hierarchical network of the robot control means 3 provided for controlling this robot is
It becomes 12 (13 if the threshold unit is added). Further, in the embodiment, since these sensors are configured to output the binary mode output, there are 2 ¹² detection patterns.

With respect to these 2 ¹² detection patterns, the action taken by the robot to track a moving object is inherently determined, and as described above, the basic control action storage means 5 stores such information. Will be configured to be preselected and stored. Figure 18 shows an example. In this example, the drive patterns to be taken by the robot drive source for the 36 detection patterns of the robot sensor are stored in the basic control operation storage means 5. Here, "HEAD, EAR" in the figure is the head ultrasonic sensor 102R, L, "HEAD, EYE" is the head optical sensor 101R, L,
“BODY, E” is the body ultrasonic sensor 104, “BODY, EYE” is the body light sensor 103R, L, and “BODY, TOUCH” is the touch sensor 105.
F, R, L, "BODY, LIM" are limit switches 106R, L, "RIGHT" is a right turn steering motor, "LEFT" is a left turn steering motor, and "FWD" is a forward propulsion motor. "BWD" represents the reverse drive motor and "BUZZ" represents the piezoelectric buzzer.

To deepen the understanding of the example of stored information in Fig. 18,
Some of them will be specifically described. For example, the 18th
The sensor detection pattern on the fifth line in the figure indicates that the head optical sensor 101L corresponding to the left eye of the head is ON. In such a case, since the moving object is in the front left direction, as shown by the teacher signal, turning on the steering motor L and turning on the propulsion motor F causes the vehicle to turn left while moving forward to track. Can be executed. The sensor detection pattern on the eighth line indicates that the head light sensor 101L is ON and the touch sensor 105R provided on the right side of the robot is ON. In such a case, the right side surface of the robot is in contact with the obstacle, but the moving object is in the front left direction. Therefore, as shown by the teacher signal, the steering motor L is turned on and the propulsion motor F is turned on. By turning it ON, it will be possible to perform tracking while moving forward while turning to the left. Further, the sensor detection pattern on the 15th row indicates that the head light sensor 101L and the head light sensor 101R corresponding to the right eye of the head are ON. In such a case, since the moving object is present in front of the front, as shown by the teacher signal, turning on the propulsion motor F allows the vehicle to move forward and perform tracking.

Regarding the automatic determination of the internal connection weights W _ih and W _ji of the hierarchical network of the robot control means 3 by the learning algorithm of the back propagation method using the storage information of this basic control operation storage means 5, It is basically the same as the robot of the embodiment described in the figure. Figure 19 shows the calculation results of the weights W _ih and W _ji obtained by executing the learning algorithm of this back propagation method. This calculation result is executed by using the number of units in the intermediate layer of the hierarchical network of the robot control means 3 as 5, and using the information stored in the basic control operation storage means 5 as shown in FIG. . Then, similarly to the robot in the embodiment of FIG. 5, the control of the robot for tracking the moving object is executed using the weights W _ih and W _ji of the internal connection of the hierarchical network determined in this way. It will be.

Next, the contents of this robot control will be specifically described with reference to the explanatory diagram shown in FIG.

Now, it is assumed that the relationship between the robot and the moving object having the light emission source and the transmission source is as shown in FIG. 20 (a), that is, the head light sensor 101L corresponding to the left eye of the head is in the ON state. When this sensor input signal is input to the hierarchical network for which the weight has been obtained, the output pattern is "steering motor L
N, turn on the propulsion motor F (turn left, move forward) ", an output vector (0,1,1,0,0) representing the control is generated. Therefore, the robot turns left. At this time, if the moving object moves and the robot tracks, as shown in Fig. 20 (b), the sensor input signal of the robot is that the head light sensor 101L is ON, and the touch sensor 105 is
When R is ON, that is, the right side of the robot hits an obstacle and the moving object is visible to the left front. When this sensor output signal is input to the network, the output vector (0, 1, 1 indicating the control "turn on the steering motor L and turn on the propulsion motor F (turn left and move forward)" this time. , 0,0) is generated. Therefore, the robot moves forward while turning to the left. At this time, as shown in FIG. 20 (c), if the moving object moves and the robot tracks, the sensor input signal of the robot is the head light sensor 101R,
Both the head light sensor 101L and the head light sensor 101L are in the ON state, that is, the moving object is captured in front. In this case, when this sensor input signal is input to the network, an output vector (0,0,1,0,0) representing the control "Turn on the propulsion motor (move forward)" is generated, and the robot Figure 20 (d)
As shown in, move forward toward the moving object. In this way, the robot inputs the sensor input signal pattern that is captured moment by moment into the hierarchical network of the robot control means 3, and the hierarchical network responds to the input pattern according to an appropriate output pattern that is output momentarily. A moving object can be tracked by controlling the motor.

For the convenience of explanation, the explanation of FIG.
Although the same control information (fifth line, eighth line, fifteenth line) stored in the basic control operation storage means 5 shown in the figure is used to track a moving object, , Even if the sensor detects a detection pattern which is not stored in the basic control operation storage means 5, an output pattern for tracking a moving object is generated by the weight of the learned hierarchical network. A moving object can be tracked by controlling the motor. Of course, when it becomes impossible to track a moving object with the output of the hierarchical network, the weight of the hierarchical network is changed by learning as in the robot of the embodiment shown in FIG.

In the present invention, even if the structure of the hierarchical network of the robot control means 3 is the same, if the control information of different control contents is stored in the basic control operation storage means 5, the weights W _ih and W _ji are obtained according to the control information, and the Control of different control contents can be executed. FIG. 21 shows that the robot executes control to escape from a moving object.
6 is an example of a basic control operation stored in a basic control operation storage unit 5. The body optical sensors 103R and 103L are used for both eyes to control this escape. This 21st
FIG. 22 shows the calculation results of the weights W _ih and W _ji obtained using the storage information of the basic control operation storage means 5 in the figure. The structure of the hierarchical network is exactly the same as that of the tracking. The control of the robot that moves to escape from the moving object using this weight can be executed as it is.

Next, an embodiment of the present invention for _obtaining the weights W _ih , W _ji of the internal connection of the hierarchical network will be described. The weights W _ih and W _ji of the internal coupling are basically determined by the control processor mounted on the robot or the dedicated control hardware. However, as mentioned above, the weight
The backpropagation method for _calculating W _ih and W _ji is quite complicated. Therefore, a considerably long calculation execution time is required to obtain the weights W _ih and W _ji . Moreover, in order to enhance the qualification of the required weights W _ih and W _ji , the number of basic control operations stored in the basic control operation storage means 5 is increased, or the number of units in the middle layer of the hierarchical network is increased. This has to be done, which results in a much longer execution time. As a result, the robot control method with a hierarchical network structure may not be practical enough.

Therefore, in the present invention, a robot learning simulator consisting of a robot control system having a hierarchical network structure and a motion imitation means for simulating the action of a robot acting by this robot control system is constructed on a high-performance computer. We prepare a support system that is designed to obtain the weights of the hierarchical network by learning on a learning simulator, verify its effectiveness, and transfer the resulting weights to the actual robot processor. .

FIG. 23 shows a system configuration diagram of an embodiment of this support system. In the figure, the same parts as those shown in FIG. 1 are indicated by the same symbols, 1'is a pseudo sensor means which is configured on the computer corresponding to the sensor means 1, and 2'is an action pattern. Pseudo-behavior pattern generating means corresponding to the generating means 2 on the computer, 3'is pseudo robot controlling means corresponding to the robot control means 3 on the computer, and 5'is basic control It is a pseudo basic control operation storage means that is configured on a computer corresponding to the operation storage means 5. The pseudo sensor means 1 ', the pseudo behavior pattern generating means 2', the pseudo robot control means 3 ', and the pseudo basic control operation storage means 5'constitute the pseudo hierarchical network control robot 10' described in FIG. It will be. Reference numeral 6 denotes an operation mode determination means for setting the operation mode of the pseudo robot control means 3'to either the learning mode or the additional learning mode processing mode, and the pseudo basic control operation storage means 5'and the basic control operation storage. Transfer of the basic control operation information to and from the means 5 is set to any of transmission, reception, and no transfer, and the pseudo robot control means 3 '
By setting the transfer of the weight information between the robot control means 3 and the robot control means 3 to either transmission, reception, or no transfer,
The operation processing contents of the pseudo hierarchical network control robot 10 'are determined. Reference numeral 7 denotes the motion mimicking means described in FIG. 1, which simulates the behavior of the pseudo hierarchical network control robot 10 'on the basis of the behavior pattern defined by the behavior pattern signal generated by the pseudo behavior pattern generating means 2', Next stage pseudo sensor means 1 '
Generates a sensor signal that will be detected by the. And 2
Reference numeral 1 is a basic control operation information transfer line, which is a line used to transfer basic control operation information between the robot learning simulator 20 and the hierarchical network control robot 10, 2
Reference numeral 2 denotes a weight information transfer line, which is a line used to transfer the weight information of the hierarchical network between the robot learning simulator 20 and the hierarchical network control robot 10.

Next, the operation of the robot learning method of the present invention thus configured will be described in detail with reference to the flowcharts shown in FIGS. 24, 25 and 26.

The flowchart shown in Fig. 24 is the flowchart of the most basic operation, and the learning and verification of the weights of the internal connections of the hierarchical network are all performed by the robot learning simulator
It is a flowchart of the operation performed on the side. In this flowchart, the basic control operation information of the basic control operation storage means 5 is first transferred to the pseudo basic control operation storage means 5'by setting the operation mode determining means 6. Then, the operation mode determining means 6 sets the learning mode, and the high-speed computer of the robot learning simulator 20 uses the basic control operation information of the pseudo basic control operation storage means 5'to perform the pseudo robot by the back propagation method. The value of the weight of the hierarchical network of the control means 3'is obtained. continue. The value of the weight obtained by setting the processing mode by the operation mode determining means 6 and simulating the behavior of the pseudo hierarchical network control robot 10 'under the environment in which the behavior mimicking means 7 is assumed in advance. To verify. Specifically, the motion mimicking means 7 generates a pseudo sensor detection pattern which is input from the pseudo sensor means 1'to the pseudo robot control means 3 ', and is generated by the output of the pseudo robot control means 3'. However, the movement position of the pseudo hierarchical network control robot 10 'is simulated from the pseudo movement pattern signal of the pseudo movement pattern generating means 2', and the next pseudo sensor detection pattern at the movement position is generated. By repeating the process, it is verified whether the pseudo hierarchical network control robot 10 'can take an appropriate action with the obtained weight value. When it is determined that the value of the weight obtained by this verification is sufficiently qualified, a process of transferring the obtained weight to the hierarchical network control robot 10 and setting it by the setting of the operation mode determining means 6 is performed. In addition, if it is still insufficient, the basic control operation information of the pseudo basic control operation storing means 5'to be used is changed or added to learn to obtain a sufficiently qualified weight.

The flowchart shown in FIG. 25 is such that the hierarchical network control robot 10 having the weight value obtained by executing the flowchart shown in FIG. It is a flow chart of processing when there is. As shown in this flowchart, when there is a behavior that is not qualified, additional basic control motion information for more qualified behavior is automatically collected by the robot or collected by the observation device or the observer. The additional basic control operation information is transferred to the robot learning simulator 20, and the robot learning simulator 20 performs additional learning of the weight value to obtain a new weight value for verification. is there. As described above, this additional learning is performed by executing the back propagation method so that the newly added additional basic control motion information can be realized with the value of the weight obtained so far as the initial value. The learning method is to obtain new weight values. At this time, instead of the additional learning, the learning method of executing the backpropagation method from all the basic control operation information including the newly added basic control operation information to obtain a new weight value. It is also possible to take

In the flow chart shown in FIG. 26, in the flow chart shown in FIG. 25, the additional learning is executed by the robot learning simulator 20, while the hierarchical network control robot 10 itself performs the additional learning.
The method executed by using is disclosed.

As described above, the present invention is characterized in that the robot learning simulator 20 executes all or part of the learning of the weight value required by the hierarchical network control robot 10.

〔The invention's effect〕

As described above, according to the present invention, the learning of the basic control operation for setting the weight, which requires a great deal of processing time, can be performed by the robot learning simulator configured on the high speed computer. By using a high-speed dedicated circuit for the robot control procedure part, high-speed weight learning is possible without increasing the hardware of the hierarchical network control robot. In addition, since the weights learned can be verified without making the actual robot act, it is expected that the development efficiency will be improved.

From this, it is possible to make the "soft" robot control system having the hierarchical network structure proposed by the present applicant more practical.

[Brief description of drawings]

FIG. 1 is a diagram for explaining the principle of the present invention, FIG. 2 is a block diagram of an embodiment of robot control means, FIG. 3 is a block diagram of a basic unit, FIG. 4 is an example of information stored in a basic control operation storage means, and a fifth example. FIG. 6 is a concrete example of the action pattern of the robot, FIG. 6 is a block diagram of another embodiment of the robot control means, FIG. 7 is an example of a network having an odd parity check function, and FIG. 8 is the number of intermediate layers is redundant. Exclusive-OR network without asterisk, Fig. 9 is an explanatory diagram for explaining the roles of exclusive-OR and middle tier units, Fig. 10 is an exclusive-OR network in which the number of middle tiers is redundant, and Fig. 11 is a learning result. Fig. 12 is an explanatory view of the weight of internal coupling (I), Fig. 12 is a processing example according to the weight of Fig. 11, Fig. 13 is an explanatory diagram of the weight of internal coupling (II) showing the learning result, Fig. 14 Shows an example of processing according to the weights in Fig. 13, and Fig. 15 shows the weights of the inner joins (III) showing the learning results. Explanatory diagram, FIG. 16 is a processing example according to the weight of FIG. 15, FIG. 17 is an explanatory diagram of a sensor equipped in the mobile robot, and FIG. 18 is an example of information stored in the basic control operation storage means of the mobile robot (I ), FIG. 19 is an explanatory diagram of the weight (I) of the internal coupling showing the learning result of the mobile robot, FIG. 20 is an explanatory diagram when tracking a moving object, and FIG. 21 is a basic control operation of the mobile robot. An example of information stored in the storage means (II), Fig. 22 is an explanatory diagram of the weight of internal coupling (II) showing the learning result of a mobile robot, Fig. 23 is a system configuration diagram of a robot learning simulator, and Fig. 24 is a diagram of weight. 25 is a flowchart (I) for learning, FIG. 25 is a flowchart (II) for learning weights, and FIG. 26 is a flowchart (III) for learning weights. In the figure, 1 is a sensor means, 2 is an action pattern generating means, 3 is a robot control means, 3'-b is a threshold value input unit, 5 is a basic control operation storing means, 6 is an operation mode determining means, and 7 is an imitation of operation. Means, 10 is a hierarchical network control robot, 10 'is a pseudo hierarchical network control robot, 20 is a robot learning simulator, 30 is a basic unit, 31 is an accumulation unit, 32 is a threshold value processing unit, 10
Reference numeral 1 is a head light sensor, 102 is a head ultrasonic sensor, 103 is a body light sensor, and 105 is a touch sensor.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Nobuo Watanabe 1015 Kamiodanaka, Nakahara-ku, Kawasaki, Kanagawa Prefecture, Fujitsu Limited (72) Inventor Asahi Kawamura 1015, Kamedota, Nakahara-ku, Kawasaki, Kanagawa Prefecture, Fujitsu Limited ( 72) Inventor Hideki Yoshizawa 1015 Kamiodanaka, Nakahara-ku, Kawasaki-shi, Kanagawa, within Fujitsu Limited (72) Inventor Minoru Sekiguchi 1015 Kamedota, Nakahara-ku, Kawasaki, Kanagawa, within Fujitsu Limited (56) Reference JP 1 -173202 (JP, A) JP-A-1-177603 (JP, A) FUJITSU Vol. 39, No. 3 (10.06.1988) PP. 175-184 NIKKEI ELECTRONIC S No. 427 (10.08.1987) PP. 115-127 Neural Networks Vol. 1 (1988) PP. 251-265

Claims (1)

(57) [Claims] 1. A sensor means (1) comprising a plurality of sensors for taking in external environment information surrounding a robot, an action pattern generating means (2) for generating an action pattern signal defining an action pattern of the robot, and the sensor means. A robot control means (3) for sending a control signal to the action pattern generation means (2) according to the detection pattern of (1) and one or more specific ones of the sensor means (1) selected in advance. The robot control means (3) for the detection pattern
And a basic control operation storing means (5) for storing control signal information to be sent, and the robot control means (3) has one or more inputs and a weight to be multiplied with respect to the inputs. A basic unit (30) having an accumulator (31) for receiving a sum of products and obtaining a sum of products, and a threshold processor (32) for converting an output from the accumulator (31) by a threshold function to obtain a final output. As a unit unit, a plurality of the unit units (3-h) connected to the sensor means (1) as an input layer, and a plurality of the unit units (3-i) as an intermediate layer, or One or a plurality of the unit units (3-j) provided with a plurality of intermediate layers and connected to the action pattern generating means (2) are used as output layers, and the input layer and the frontmost intermediate layer are provided. Between the intermediate layers and between the intermediate layers at the final stage A hierarchical network is formed by forming an internal connection with the force layer, and the robot control means (3) implements learning control so that the stored information of the basic control operation storage means (5) can be realized. By doing so, the value of the weight of the hierarchical network provided by itself is set, and the output value of the output layer of the hierarchical network for which this weight value is set becomes the control signal to the action pattern generating means (2). Of the hierarchical network control robot (10), a similar hierarchical network control robot (10 ') configured on a computer with the same configuration as this hierarchical network control robot (10), and this pseudo hierarchical network control robot (10). Robot learning simulation consisting of motion mimicking means (7) for simulating the behavior of ′) on a computer (20) and the robot learning simulator (20) uses the pseudo-hierarchical network control robot (10 ′) as necessary to determine the weight required by the hierarchical network of the hierarchical network control robot (10). And verifying the validity of the obtained weight value by using the motion mimicking means (7), and the validity-verified weight value of the hierarchical network control robot (10). A robot control method characterized by being set as a weight value of the hierarchical network.