JP4160677B2 - Computer system with distributed learning function - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a computer system having a learning function corresponding to a control object and environment used for controlling a robot or the like.
[0002]
[Prior art]
In a computer system used for controlling a conventional robot or the like, there is a neurocomputer that operates based on a living mechanism based on a neuron and a connection network. A neuron is also called a nerve cell, and is a cell specialized to handle information processing exclusively in a living body. The neuron connected to each other is the neural network of the brain. The neurocomputer performs learning using a neural network model that models the principle of parallel learning information processing of a neural network.
[0003]
[Problems to be solved by the invention]
However, since the conventional neurocomputer as described above has only one CPU used for the learning process, it is difficult to expand the system when the number of control objects increases, and the learning process can be speeded up. There was no problem. In addition, neuronal connections in the neural network of the human brain include excitatory connections that stimulate the excitement of other cells, and conversely, inhibitory connections that try to suppress the excitement of other cells. On the other hand, in the conventional neurocomputer, when the input signal from the sensor corresponding to the neuron is on, it is positive for the output side CPU. There is a problem that complicated learning processing cannot be performed because only excitatory coupling that is an evaluation factor is used.
[0004]
The present invention has been made to solve the above-described problems, and can be adapted to a complicated environment by performing a complicated learning process, and the number of control objects is increased. In such a case, it is an object of the present invention to provide a computer system having a learning function that is easy to expand the system and can speed up the learning process.
[0005]
[Means for Solving the Problems]
In order to achieve the above object, the present invention is a computer system having a learning function according to an object to be controlled and an environment, and is composed of a unit group in which a plurality of units each having a CPU are connected, and each unit in the unit group is connected. The structure includes a horizontal parallel connection structure and a vertical connection structure, and among the units in the unit group, a unit of the highest hierarchy and a unit of the lowest hierarchy are included. Each unit except for the control object and environmental information (hereinafter referred to as first input information) acquired from the sensor connected to the own unit and the control object acquired from the sensor by other units connected in the horizontal direction. Based on the information on the object and the environment (hereinafter referred to as second input information), the previous unit When a force command is an evaluation means for evaluating whether a positive impact for the local unit, the evaluation unit by evaluation of the results was evaluated as for the local unit had a positive effect, Last entered The command, the previous first and second input information, Last output Combination data including storage means for storing combination data with a command, and current input commands input from the first and second input information of this time and a unit of a higher hierarchy is stored in the storage means. Search means for searching whether or not there is past combination data, and combination data including the current input command and the current first and second input information as a result of the search by the search means, When there is past combination data stored in the means, the output command in the corresponding combination data is determined as a command to be output to the lower hierarchy unit. As a result of the search by the search means, when combination data including the current input command and the current first and second input information does not exist in the past combination data stored in the storage means, In the past combination data stored in the storage means, a search is performed as to whether or not there is similar data of combination data including the current input command and the current first and second input information. As a result, when similar data exists in the past combination data, an output command in the similar data is determined as a command to be output to a unit in a lower hierarchy, and the similar data is included in the past combination data. If it does not exist, the randomly generated output command is determined as the command to be output to the lower hierarchy unit. And determining the type of coupling between each unit in the unit group and the input side sensor or other unit when the signal from the input side sensor or other unit is on. Excitability coupling that is a positive evaluation factor for each unit, inhibitory coupling that is a negative evaluation factor for each output unit, and a positive evaluation factor and a negative evaluation factor for each output unit And the evaluation means is the type of coupling with the corresponding input side sensor or other unit when the signal input from the input side sensor or other unit to the own unit is on. If the coupling is excitatory, a predetermined value is added to the evaluation value, and if the coupling type with the corresponding input sensor or other unit is inhibitory coupling, the predetermined value is mapped from the evaluation value. If the total of the evaluation values is calculated by negative, and the total is positive, the command that the previous unit output last time for the command that was previously input from the higher-level unit has a positive effect on the local unit. It was made to evaluate that it brought about.
[0006]
In the above configuration, information on the control object and environment acquired from the sensor connected to the own unit, and information on the control object and environment acquired by the other unit connected in the horizontal direction from the sensor, etc. Based on the above, it is possible to evaluate whether or not the command previously output by the own unit with respect to the command previously input from the upper layer unit has a positive effect on the own unit. Therefore, by repeating this evaluation, each unit can learn and output an output command that has a positive effect on the unit for each type of input command. As a result, even if the number of controlled objects increases, an output command that has a positive effect on the increased controlled objects with a simple procedure of adding and connecting units corresponding to the increased controlled objects in the horizontal direction. Therefore, the system can be easily expanded according to the increase in the number of control objects. Even when the system is expanded in this way, each unit in the unit group can perform the learning process for each unit using the CPU in each unit, so that the learning process can be speeded up. Can do. Furthermore, since a connection structure having a vertical hierarchical structure is used as the connection structure of each unit in the unit group, it is obtained from a sensor connected to the own unit or other units connected in the horizontal direction. Based on information such as the control object and the environment, a command input from a higher-level unit can be expanded into a command representing a more detailed command corresponding to the control object and the environment.
[0007]
In addition, the coupling between each unit in the unit group and the input side sensor or other unit is an excitement that becomes a positive evaluation factor for each output side unit when the signal from the input side sensor or other unit is on. The combination of sex, the inhibitory combination that is a negative evaluation factor for each unit on the output side, and the neutral combination that is neither a positive evaluation factor nor a negative evaluation factor for each unit on the output side Also good. As a result, each unit on the output side uses an inhibitory coupling and a neutral coupling in addition to the excitatory coupling that is used in a computer system having a conventional learning function. Since the information about the control object and the environment from the unit can be received, in addition to the commands preferable for each unit on the output side, it is possible to learn commands not preferable for each unit on the output side. Thereby, a complicated learning process can be performed. In addition, since each unit on the output side can receive information from a neutrally-coupled sensor or the like that is unclear as to whether or not it is preferable at that time, learning processing can be performed based on more information.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, a computer system having a learning function according to an embodiment of the present invention will be described with reference to the drawings. The computer system according to the present embodiment is incorporated in a robot or the like and used for controlling an actuator such as an arm. FIG. 1A is a diagram individually showing units as basic units of the computer system according to the present embodiment, FIG. 1B is a diagram showing a state in which the units are connected in a vertical / horizontal network, and FIG. It is a figure which shows the format of the combination data of the input data and output command which are stored. A unit body 2 of the unit 1 stores a CPU, RAM, and ROM (not shown), and an input port 3 and an output port 4 are provided on the input side and output side of the unit 1, respectively. ing. The CPU in the unit main body 2 receives from the input port 3 commands input from the higher-level unit 1 and information on the control target and environment from the sensor system connected to the own unit and other units. Then, using a learning processing program stored in the ROM, data of a combination of input data and output command that has a positive effect on the unit shown in FIG. 1C is accumulated in the RAM. Then, based on the accumulated data, an output command preferable for the unit corresponding to the input data is output from the output port 4. Each unit 1 constituting the vertical / horizontal network 10 shown in FIG. 1B repeats such processing, so that each unit 1 learns and outputs a command corresponding to the environment and the controlled object. From the output port 4 of the unit 1 of the uppermost layer (third layer) in the vertical / horizontal network 10, a preferred output command for all the units 1 is sent to a drive device such as an arm. Can be output.
[0009]
As a sensor used in the sensor system shown in FIG. 1 (b), sensors such as temperature, humidity, contact, luminous intensity, and voice can be considered, and information necessary for grasping the environment and the state of a controlled object is captured. There is no particular limitation as long as it can be used. However, sensors having an ON / OFF contact output are used for these sensors. Further, the indirect output shown in FIG. 1B is used as an indicator for grasping the status of the unit 1 in each layer.
[0010]
Next, input terminals of the input port 3 will be described. In each input terminal, binary data represented by 0 or 1 is input. In the case of a terminal for inputting information from a sensor, 1 represents ON and 0 represents OFF. In the case of a command input terminal, an instruction code expressed in binary number is input. Data input from the sensor input terminal is used as an evaluation value of the command output immediately before. The sensor input terminals include three types of input terminals: excitement terminals, suppression terminals, and neutral terminals. When the data input to the excitement terminal is 1 (ON), it is evaluated that the command output by the unit 1 for the previously input data has a positive effect on the unit 1. On the other hand, when the data input to the suppression terminal is 1, it is evaluated that the command output by the unit 1 for the previously input data has a negative effect on the unit 1. Further, data input to the neutral terminal is not treated as an evaluation value for the unit 1. This neutral terminal is mainly used as a connection port of the vertical / horizontal network 10 and is used for command transmission, but for connection with a sensor that detects factors that are not positive for the unit 1 at present. Also used for.
[0011]
Next, a learning program incorporated in the ROM in each unit 1 will be described with reference to the flowchart of FIG. The program shown in the figure resides in all units 1 in the vertical / horizontal network 10. The CPU in each unit 1 receives input data composed of sensor information and a command, and outputs a command preferable for its own unit. First, the CPU reads ON / OFF of the input data input to the input port 3 (# 1), and based on the sensor information input from the excitement terminal and the suppression terminal, the unit 1 detects the previous input data. It is evaluated whether or not the command outputted in this way has brought a positive environment for its own unit 1 (# 2). Specifically, when the data input to the excitement terminal is 1, 1 is added to the evaluation value, and when the data input to the suppression terminal is 1, 1 is subtracted from the evaluation value. As a result, the previous input / output data stored in the temporary memory in the unit 1 according to the data format shown in FIG. 1 (c) only when the sum of the evaluation values of the data input to the terminals is positive. Are stored in the RAM inside the unit 1.
[0012]
Next, the CPU searches whether the same input data as the input data read this time is in the past combination data in the RAM (# 3), and when the same input data exists (# 4) YES), an output command that is paired with the corresponding input data in the past combination data is set as bit data corresponding to the command in the output data (# 5). If the same input data does not exist in the past combination data in the RAM (NO in # 4), a search is performed to determine whether or not similar data exists in the past combination data (# 6). Is present (YES in # 7), the output command in the similar data is set as bit data corresponding to the command in the output data (# 5), and if there is no similar data (# 7), an output command is randomly created and set as bit data corresponding to the command in the output data (# 8). Next, the CPU edits bit information corresponding to the sensor information in the output data based on the sensor information in the input data read in # 1, and outputs the output data including the output command and the sensor information to each unit. The output is output to the output port 4 while synchronizing the output between them (# 9). At this time, the combination data of the current input data and output data is stored in a temporary memory inside the unit 1.
[0013]
When the control object and the environment become wide, and when the control and environment for the control object become complicated, it is necessary to increase the number of units 1 to be connected. This connection method includes a vertical connection and a horizontal connection. First, a data flow at the time of vertical connection will be described with reference to FIG. FIG. 3 is a diagram illustrating a data flow in the vertical network. The vertical network 11 expands the commands input from the upper layer unit 1 into more detailed commands according to the information regarding the environment and the control target input from the sensor connected to the unit 1 of each layer. Thus, it is possible to create a purposeful output command according to the environment and the controlled object.
[0014]
As shown in FIG. 3, the vertical network 11 is connected in series in a hierarchy. The data including the command is output from the upper layer unit 1 to the lower layer unit 1, that is, from the (n + 2) th layer to the (n + 1) th layer and the nth layer. As shown in the figure, if a command “01” is output from the (n + 2) th layer to the (n + 1) th layer, the CPU of the (n + 1) th layer that has received this “01” as input data The data is searched, and an output command “0100” corresponding to the input data “01” is output to the nth layer. Similarly, the n-th layer CPU searches past data in the RAM in its own unit and outputs an output command “01101110” corresponding to the input data “0100” to the control object. This uses the hierarchical structure of the vertical network 11 and past data in the RAM to compress the output pattern output from the n + 2 layer of the top layer from 8 digits to 2 digits to 1/4. It can be said. For example, the operation of “walking” instructed in the (n + 2) th layer is decomposed into operations of “moving the right foot back and forth” and “moving the left foot back and forth” in the (n + 1) th layer. This leads to decomposition into the simplest output unit patterns (output commands) such as “contract the right quadriceps”, “contract the right soleus”, “left...”, Etc. Even when the same “walk” command (“01”) is output from the (n + 2) th layer, it is possible to select and create an output command by taking into account information input from the sensor in the unit 1 of each layer. A command representing an accurate “walking” operation suitable for the user can be output from the unit 1 in the lower hierarchy. For example, when the “walking” place is a slope, “walk with changing heel angle” Thus, it is possible to realize an accurate “walking” operation. On the other hand, since unit 1 in the higher hierarchy can prepare a set of actions such as “walking” and “turn”, a command corresponding to sensor input information in unit 1 in the lower hierarchy It is possible to select a purposeful action according to the object and the environment.
[0015]
The difference between the units 1 of each layer in the vertical network 11 is that the length of processing time required for one loop is different. In the lower layer, the time required for one loop is the time in the higher layer. It is shorter than that. Each unit 1 automatically sets this time. The position of the command input terminal in the input port 3 in each unit 1 is determined in advance.
[0016]
Next, the flow of data during horizontal connection will be described with reference to FIG. FIG. 4 is a diagram illustrating a connection example of the horizontal network. When two or more units 1 have partially the same object or environment to be controlled, they are connected horizontally in this way to provide information to each other or unilaterally. The number of sensors connected to the unit 1 is increased, and the combined units 1 can accurately determine the output command based on more information.
[0017]
In FIG. 4, the unit 1 of the m-th station and the unit 1 of the m + 1-th station are connected in the horizontal direction, the output of the m-th station is used as the input of the m + 1-th station, and the output of the m + 1-th station is m-th. They are mutually input as station inputs. In the case of FIG. 4, a value “0” is input from the m-th station to the m + 1-th station, and a value “1” is input from the m + 1-th station to the m-th station. It is not necessary to grasp all the information from the sensor connected to the m + 1st station, that is, the situation where the m + 1st station is located in the mth station. This is because each unit 1 can respond by autonomously learning the status of other units from past data. The number of units 1 related to the horizontal connection in the horizontal network 12 may be two or more, and may be any number as long as the number of terminals permits. However, since each unit 1 repeats learning in the relationship of (input data-output command), a minimum external sensor input terminal for grasping the situation necessary for each unit 1 itself is left. There is a need. As the connection terminal, any type of excitement / suppression / neutral terminal can be used in the same manner as the sensor connection terminal. When input information about a certain control object or environment has an effect to synchronize for each of the two connected units, it is connected to the exciter terminal, and if it has a contradictory effect, it is connected to the suppression terminal. If not, the basic point is to connect to the neutral terminal, but this is not a limitation. Just because one is connected to the excitement terminal, the other need not be connected to the excitement terminal, and may be connected to another type of terminal or not connected to any terminal.
[0018]
Next, a method for designing the vertical / horizontal network 10 will be described with reference to FIGS. 5A and 5B are diagrams showing design examples of the vertical / horizontal network 10 using the connection terminal total table and the connection terminal total table, respectively. When designing the vertical / horizontal network 10, it is necessary to grasp the connection relationship of the input / output terminals between the units 1. The connection terminal total table shown in FIG. 5A is a table for clarifying the number of input / output connection terminals and connection relationships of each unit 1 and for use in the design of the vertical / horizontal network 10. 5B, the command is connected according to the system of the vertical network 11, and the sensor is a sensor directly connected according to the system of the horizontal network 12. Connect with the number of connections and the connection to the connection unit in the horizontal direction. It is easier to understand if the unit name in the connection terminal total table is entered so that the level and the horizontal position can be understood. In the case of FIG. 5B, the hierarchical position is indicated by a number, and the horizontal position is indicated by an alphabet. The connection number column in the connection terminal total table is for entering the number of terminals. A breakdown of the number of terminals is entered at the point where the output from one unit 1 branches and the point where the outputs from two or more units 1 merge. These breakdowns must match without contradicting the number of input terminals and the number of output terminals.
[0019]
As described above, according to the computer system having the distributed learning function according to the present embodiment, each unit in the unit group is input based on information on the current environment of the own unit obtained from the sensor and other units. It is possible to evaluate whether or not the previous output command of the own unit for the issued command has a positive effect on the own unit. Therefore, by repeating this evaluation, each unit can learn and output an output command that has a positive effect on the unit for each type of input command. As a result, even if the number of controlled objects increases, an output command that has a positive effect on the increased controlled objects with a simple procedure of adding and connecting units corresponding to the increased controlled objects in the horizontal direction. Therefore, the system can be easily expanded according to the increase in the number of control objects. In addition, by using a connection structure having a vertical hierarchical structure as a connection structure of each unit in the unit group, such as an environment acquired from a sensor connected to the own unit or other units connected in the horizontal direction, etc. Based on the information, a command input from a higher-level unit can be expanded and output as a command representing a more detailed command corresponding to the control object or the environment.
[0020]
The present invention is not limited to the above embodiment, and various modifications are possible. For example, in the above embodiment, when the previous output command for the previous input data is positive evaluation, the combination data of the previous input data and the output command is unconditionally saved in the RAM in the unit 1. In addition to the previous combination data, the evaluation value when the corresponding output command is issued is also stored in the RAM, and when the combination data corresponding to the previous input data is already stored in the RAM, By comparing the evaluation value when the output command is issued with the evaluation value when the previous output command is issued, the combination of the input data and output command stored in the RAM is determined alternatively. May be. As a result, it is possible to more accurately learn a combination of input data and output command preferable for the unit.
[0021]
【The invention's effect】
As described above, according to the present invention, each unit in the unit group can evaluate whether or not the previous output command of the own unit with respect to the previously input command has a positive effect on the own unit. Therefore, by repeating this evaluation, each unit can learn and output an output command that has a positive effect on the unit for each type of input command. As a result, even if the number of controlled objects increases, an output command that has a positive effect on the increased controlled objects with a simple procedure of adding and connecting units corresponding to the increased controlled objects in the horizontal direction. Therefore, the system can be easily expanded according to the increase in the number of control objects. Even when the system is expanded in this way, each unit in the unit group has a CPU, and the learning process can be performed for each unit. Even when a complicated learning process is performed based on information about a control object and an environment, the learning process can be speeded up. Furthermore, by using a connection structure having a vertical hierarchical structure as a connection structure of each unit in the unit group, a command input from a higher-level unit is acquired from a sensor or other unit. Based on information such as the object and the environment, it can be expanded into a command representing a more detailed command corresponding to the control object and the environment and output.
[0022]
In addition, the coupling between each unit in the unit group and the input side sensor or other unit is an excitement that is a positive evaluation factor for each output side unit when the signal from the input side sensor or other unit is on. And a combination of restraints that is a negative evaluation factor for each unit on the output side, and a neutral combination that is neither a positive evaluation factor nor a negative evaluation factor for each unit on the output side. By doing so, each unit on the output side can learn a command that is preferable and a command that is not preferable for the unit. Thereby, since complicated learning processing can be performed, it becomes possible to adapt to a complicated environment. In addition, since each unit on the output side can receive information from a neutrally-coupled sensor or the like, which is not preferable at that time, it is possible to perform accurate learning processing based on more information it can.
[Brief description of the drawings]
FIG. 1A is a diagram individually showing units as basic units of a computer system according to an embodiment of the present invention. FIG. 1B is a diagram showing a state in which units are connected in a vertical / horizontal network. ) Is a diagram showing a format of combination data of input data and output command stored in the unit.
FIG. 2 is a flowchart showing control of a learning program built in a ROM in each unit.
FIG. 3 is a diagram showing a data flow in a vertical network.
FIG. 4 is a diagram illustrating a connection example of a horizontal network.
FIGS. 5A and 5B are diagrams illustrating design examples of a vertical / horizontal network using a connection terminal total table and a connection terminal total table, respectively.
[Explanation of symbols]
1 unit
11 Vertical network (concatenation with hierarchical structure in the vertical direction)
12 Horizontal network (horizontal parallel connection)

Claims (1)

In a computer system having a learning function according to the controlled object and environment,
Consists of a unit group in which a plurality of units having a CPU are connected,
The connection structure of each unit in the unit group includes a parallel connection structure in the horizontal direction and a connection structure having a hierarchical structure in the vertical direction,
Of the units in the unit group, each unit excluding the unit of the highest hierarchy and the unit of the lowest hierarchy,
Control object and environment information (hereinafter referred to as first input information) acquired from the sensor connected to the own unit, and control object and environment information acquired from the sensor by other units connected in the horizontal direction. (Hereinafter referred to as “second input information”), whether the command output by the unit last time with respect to the command previously input from the upper layer unit has a positive effect on the unit. An evaluation means to evaluate;
As a result of the evaluation by the evaluation means, when it is evaluated that the unit has a positive effect, the previous input command, the previous first and second input information, and the previous output command Storage means for storing combination data;
Whether or not combination data including the current first and second input information and the current input command input from a higher-level unit exists in the past combination data stored in the storage means. A search means for searching;
As a result of the search by the search means, when combination data including the current input command and the current first and second input information are present in the past combination data stored in the storage means, An output command in the combination data is determined as a command to be output to a lower hierarchy unit, and as a result of the search by the search means, combination data including the current input command and the current first and second input information is obtained. When the combination data stored in the storage means does not exist in the past combination data, the current input command and the first and second input information of the current combination are stored in the past combination data stored in the storage means. A search is performed to determine whether there is similar data in the combination data to be included. As a result, similar data exists in the past combination data. In this case, the output command in the similar data is determined as a command to be output to a unit in a lower hierarchy, and when similar data does not exist in the past combination data, an output command created at random is Determining means for determining as a command to be output to a unit in a lower hierarchy ,
A positive evaluation factor for each unit on the output side when the type of coupling between each unit in the unit group and the sensor or other unit on the input side is ON when the signal from the sensor or other unit on the input side is ON. Excitability coupling, suppression coupling that is a negative evaluation factor for each unit on the output side, and neutral coupling that is neither a positive evaluation factor nor a negative evaluation factor for each unit on the output side Configured,
The evaluation means includes
When the signal input from the sensor on the input side or other unit to the unit is on and the type of coupling with the sensor on the input side or other unit is excitatory coupling, the evaluation value is a predetermined value. If the type of coupling with the sensor or other unit on the input side is inhibitory coupling, the total of the evaluation values is obtained by subtracting a predetermined value from the evaluation value, and this total is added. In this case, the distributed learning function is characterized in that the command output by the previous unit from the previous command input from the upper layer unit has been evaluated as having a positive effect on the own unit. A computer system.

Images