JP6044691B2 - Logic inference system, calculation learning device, and logic inference method - Google Patents

Description

  The present invention relates to a logical reasoning system and a logical reasoning method in the field of artificial intelligence, and a calculation learning device that executes logical reasoning or arithmetic operations and logical operations.

  So far, in the technical field of “AI (Artificial Intelligence)”, (a) knowledge representation method, (b) deduction reasoning method based on knowledge, and (c) knowledge based on examples. The method of inductive reasoning to be made up has been studied.

  Here, artificial intelligence research was born in the 1950s. For decades since then, this is one of the most actively researched areas of information science. In particular, the 5th generation computer project promoted in Japan in the 1980s adopted predicate logic as a knowledge description format and Prolog (a language that processes predicate formulas as programs) as a deductive reasoning method. Based on this, many studies have been conducted to parallelize and speed up knowledge processing.

However, although the 5th generation computer project mentioned above has finally achieved a certain result, strictly speaking, until it produces an outcome that is commensurate with the scale of its investment, for example, an outcome that transforms society. It must be said that it was not successful. The reason is that the problem that must be truly solved by artificial intelligence is (c) the development of inductive reasoning (learning) method for creating knowledge in the above-mentioned research field of artificial intelligence ( b) This may be due to the focus on speeding up the method of deductive reasoning based on knowledge and focusing on developing huge hardware computers.

If the conventional approach as described above is “narrowly defined” AI (symbol processing AI) that explicitly handles symbols / symbols, the symbol / symbol is explicitly handled in “broadly defined” AI. This includes non-technical techniques (typically artificial neural network research). These two technologies, symbol processing AI and neural network, have different fields of expertise, and so far there has been little exchange or fusion of research.

  Here, in the narrow AI described above, research has been conducted on symbolic logic, particularly “predicate logic”.

  Here, “symbol logic” is to convert meaning contents expressed in words into symbols (symbols) and try to grasp them by their logical relationship, and to study logic by mathematics. Therefore, it is also called “Mathematical logic”.

  “Symbolic logic” includes “propositional logic” and “predicate logic”. Unlike propositional logic, “predicate logic” is a symbolic logic that uses variables in a proposition to capture true / false depending on the value of the variable in order to discuss the contents of each proposition.

In other words, propositional logic asks whether a statement is true or false, and cannot infer about the contents expressed in the statement. For example,
(P1) Everyone likes peace.

(P2) Taro is a person.
Then (p3) Taro likes peace.

  In the framework of the propositional logic, it cannot be asserted that the proposition (p3) is always true even if the propositions (p1) and (p2) are true. This is because, under the propositional logic, attention is not paid to the contents relating to the individuals included in the proposition, but only the authenticity of the proposition itself.

  On the other hand, in predicate logic, attention is paid to individual variables included in a proposition, and the properties and states of individual variables in the proposition are inferred using predicates.

Examples of elements constituting the predicate logic include the following.
Individual constant: a symbol representing a specific individual (expressed by a constant such as a, b,...)
Individual variable: A symbol representing an arbitrary individual (represented by variables such as x, y,...)
-Predicate symbols: Symbols representing the properties and states of individuals (represented by symbols such as p, q, ...)
Function symbol: Symbol representing the relationship between individuals (expressed by f in f (x), etc.)
・ Logical symbols: Negation (¬ or ~), logical product (∧), logical sum (∨), implication (→), etc. are used in the same way as logical symbols of propositional logic.

  Limit symbol: A symbol indicating a range of a target solid in the target region, a generic symbol (∀), and an existence symbol (∃) are used. A variable defined by a quantifier is called a bound variable, and a variable that is not a bound variable is called a free variable.

  A term that is an argument of a predicate expresses an element in a target area that is a world targeted by predicate logic, and corresponds to an argument in the predicate. A term can be an individual constant, an individual variable, or a function itself.

Since the definition of the predicate logical expression is well known, the description thereof is omitted here.
In addition, the target predicate logical expression that proves its authenticity generally has a negative expression of a constant expression including a universal symbol and an existence symbol. However, it is known that the existence symbol is erased formally while preserving the authenticity, and it is expressed by a predicate logical expression of only the universal symbol. It is called “Scolem standard form”.

For example, negative form to P ₀ of the predicate logic formulas P ₀ is assumed to be represented by Skolem normal form as follows.

  Here, the prime expression or the negation of the prime expression is called “literal”. A logical expression composed of only a logical OR of literals or only a logical product of literals is called a “clause”. When a clause is composed only of a logical OR of literals or only a logical product of literals, a logical expression composed of only the logical product of clauses or only the logical sum of clauses is called a "clause form" .

In the Skolem standard form described above, all individual variables are constrained by a universal symbol. Therefore, a set C = {C ₁ ,... C _n } consisting of clauses constituting a matrix is referred to as a “node set”. Call. A clause set is true means that all C ₁ ,... C _n are true at the same time. Further, the phrase set is false means that at least one of C ₁ ,... C _n is false.

  A clause set is said to be unsatisfiable when the clause set is false for all interpretations (ie, not all are true at the same time). If the clause set C is unsatisfiable, it can be proved that the Skolem standard form is a false expression and that the target predicate logical expression P0 is a true expression.

Now, assume that the clauses C _i and C _j in the clause set C = {C ₁ ,... C _n } include a certain literal and its negative literals ~ P, and are expressed as follows.

An inference that leads to the following new clause C _ij from these two clauses is called “derivation”.

At this time, the clauses C _i and C _j are referred to as parent clauses, and the derived clause is referred to as a derived clause.
Derivation is a valid inference form.

  In a clause set, if derivation is performed and then adding the derived clause to the clause set is repeated, the final “empty clause” can be derived, which means that the clause set is unsatisfiable. Can prove the identity of the predicate formula. Here, an empty clause refers to a clause consisting of zero literals.

Such a procedure for determining the identity is called “derivation principle”.
Further, in the derivation for the predicate logical expression, the derivation cannot be performed unless the individual variables included in the predicate logical expression are appropriately treated in advance.

  Therefore, an operation for replacing an individual variable with another term is referred to as “singleization”. Once the single substitution is performed, the individual variable is bound to that term in the subsequent derivation process. Derivation is not possible if there is no unification substitution. When an empty clause is finally derived by derivation under the unification replacement s, it is said to be “proven under the unification replacement s”.

  As will be described in more detail later, “predicate logic and derivation”, “Prolog”, “production system”, “semantic network”, “high-level Petri net”, etc. related to predicate logic It is described in Patent Document 7.

On the other hand, in neural network research, knowledge is not expressed as symbols or symbols but as network connection information created by imitating neuro elements in the brain. By modifying the artificial network according to a predetermined learning rule based on the case, the knowledge behind the case is recursively constructed. Based on this method, various network topologies and learning rules have been proposed so far, and examples of practical use have been reported in the field of pattern recognition and the like (see Non-Patent Document 8 to Non-Patent Document 11).

FIG. 44 is a conceptual diagram showing another example of predicate logic and derivation.
In FIG. 44, the following logical expressions are targeted.

1) Taro is a man. : Man (Taro).
2) A man is a human being. : Human (x) ← Man (x).

3) Who is a human being? : ← Expressed as Human (q).
Where 1) is the fact clause, 2) is the rule clause, and the purpose of the logic program here is to determine what the query variable (q) in the goal clause of 3) is .

  In this case, as shown on the right side of FIG. 44, it is derived that the query variable is “Taro” by repeating the derivation until an empty node is obtained.

  On the other hand, FIG. 45 is a conceptual diagram showing the contents for executing derivation for the fact clause, rule clause, and goal clause similar to those in FIG. 44 by Prolog, which is a logical programming language that handles predicate logic.

  In Prolog, derivation is executed by a method called SLD (Selective Linear definite resolution) derivation for a clause including only one literal called a horn clause. In Prolog, when unification is unsuccessful in the derivation process, it returns to the section just before the failure, performs derivation with other parent section candidate sections, and has a mechanism to continue processing. This is called “backtrack”. Therefore, Prolog will perform backward inference with depth-first search.

  FIG. 46 is a conceptual diagram showing the contents for executing derivation for the fact clause, rule clause, and goal clause similar to those in FIG. 44 by the production system that handles predicate logic.

  The production system generates facts in which the recognition behavior cycle is the answer by forward reasoning for the facts in the working memory based on the rule base.

  FIG. 47 is a conceptual diagram showing the contents for executing derivation for the fact clause, rule clause, and goal clause similar to those in FIG. 44 by the high-level Petri net that handles predicate logic.

  In a high-level Petri net, there are two nodes called places and transitions, and logic is expressed by a directed graph obtained by connecting these nodes with labeled arcs. Firing is then performed as forward reasoning, and finally a fact token is generated.

  FIG. 48 is a conceptual diagram showing the contents for executing derivation for the fact clause, rule clause, and goal clause similar to those in FIG. 44 by the semantic network that handles predicate logic.

  Semantic networks express knowledge by describing concepts as relationships between them. The expressed knowledge is expressed by a directed graph with a label in which a concept node is connected to a value node by an arrow indicating an attribute. In a semantic network, deduction is performed by matching structures between graphs.

  Further, FIG. 49 is a conceptual diagram showing the contents of executing the derivation for the fact clause, the rule clause, and the goal clause similar to those in FIG. 44 by the stochastic logic programming that handles the predicate logic, in particular, the Bayesian network.

  A Bayesian network is a probabilistic inference model in which causal relationships existing between events are quantitatively expressed as probabilities, and visually expressed in the form of a network using a directed acyclic graph.

  When causal relations are expressed quantitatively by probability, they are called “Bayesian networks” because they are based on Bayes' theorem.

  In Bayesian networks, knowledge ambiguity can be expressed by probabilistic inference based on weighted rules.

  On the other hand, there is a method of “ATN (Algorithmically Transitive Network)” proposed in Patent Document 1 by the inventors of the present application in order to realize a “computation processing device that creates a program”. In this method, the algorithm is modified by learning while processing the program with a network-type graph representation.

International Publication No. 2010/047388

Lloyd, Sato, Morishita, basics of logic programming. Industrial books (1987) Ikuo Taihara: Fundamental knowledge of artificial intelligence. Modern Science (1988) Kenji Sugawara: Artificial Intelligence (Introduction to Information Engineering Series). Morikita Publishing (1997) Furukawa Koichi and Mukai Kuniaki: Mathematical Logic. Corona (2008) Peterka, C, Murata, T .: Proof procedure and answer extraction in Petri net model of logic programs, IEEE Transactions on Software Engineering V.15 N.2 (1989) 209-217 DOI: 10.1109 / 32.21746 Tadao Murata: Petri net analysis and application. Modern Science (1992) Richardson, M, Domingos, P .: Markov logic networks. Machine Learning V.62 N.1-2 (2006) 107-136 Masatoshi Sakawa and Masahiro Tanaka: Introduction to neurocomputing. Morikita Publishing (1997) Kumazawa Ikuo: Learning and Neural Networks (Electronic Information and Communication Engineering Series). Morikita Publishing (1998) YOSHITOMI Yasunari: Neural network (Introduction to the series nonlinear science). Asakura Shoten (2002) Haykin, S .: Neural networks and learning machines. Prentice-Hall. Inc. (2009)

  FIG. 50 is a diagram showing a typical model that has been proposed so far in the field of symbol processing AI.

  The second column in FIG. 50 indicates whether the knowledge is represented in a graph. The third and fourth columns show the factors that determine the efficiency of deductive reasoning.

  Here, in the second column, “◯” indicates that it is desirable from the viewpoint of calculation efficiency, and “Δ” indicates that the graph is expressed but does not directly lead to the calculation process. “X” indicates that no graph representation is used.

In the third and fourth columns, “○” means that the graph representation is in the process of deduction processing, and in principle, a processing method with high calculation efficiency is adopted, and “△” means deduction. In principle, efficient / inefficient means that it cannot be said in general, and “x” indicates that the principle of deduction is rather inefficient.

  The reason why backward reasoning is higher than that of forward reasoning is that there is no explosive generation of facts in backward reasoning because the search is narrowed down with the goal in mind.

On the other hand, in neural network research, knowledge is not expressed as symbols or symbols but as network connection information created by imitating neuro elements in the brain. By modifying the artificial network according to a predetermined learning rule based on the case, the knowledge behind the case is recursively constructed. Based on this method, various network topologies and learning rules have been proposed so far, and examples of practical use have been reported in the field of pattern recognition and the like.

  The model of the symbol processing AI in FIG. 50 is designed mainly from the viewpoints of (a) knowledge weaving expression and (b) deductive reasoning, and (c) it is difficult to incorporate a learning function into this.

  The machine learning research that has been conducted as "learning from examples" has challenged it, but there are many ways to fill the gap between the discreteness of symbols and the ambiguity and noise of case information. Not done.

  Further, the bottom stochastic logic programming in FIG. 50 generates the ambiguity of knowledge by generating a Paysian network based on the predicate logic that is “weighted” and performing probability inference on it. . However, in this method, variables are deleted and converted to propositional logic at the stage of network conversion from predicate logic, so general knowledge representation unique to predicate logic cannot be performed, and the scale of the resulting graph is also an element of Elblanc It becomes a huge thing to have everything as nodes. Learning is also the optimization of conditional probabilities of nodes prepared in this way, and it is impossible to generate new predicates or change the logical structure.

On the other hand, the neural network is a system specialized in the function of (C) learning. The difficulty of this model is rather the indefiniteness of the completed knowledge. Since the knowledge constructed by learning cannot be interpreted and extracted by humans, it can be modified and used by humans or ported to another architecture. I can't do that.

  The present invention has been made to solve the above-described problems, and its purpose is to perform deductive inference while incorporating a learning function, and the structure of the constructed knowledge is clearly shown in a graph. A logical reasoning system, a calculation learning device, and a logic reasoning method are provided.

  Another object of the present invention is to provide a calculation learning device that creates a program that outputs a calculation result desired by a user by learning a logical program for deductive reasoning or a program that executes arithmetic operations and logical operations. That is.

  According to one aspect of the present invention, there is provided a logical reasoning system for executing inference based on knowledge expressed in logic, wherein the knowledge is a data flow graph in the form of backward reasoning, This is expressed by using a data flow graph composed of multiple operation nodes and a plurality of edges each connecting between the operation nodes. The plurality of operation nodes include operation nodes corresponding to various logical operations and are associated with the operation nodes. Expands the data flow graph into an AND / OR graph by token propagation based on the firing of each node on the data flow graph, to which a value converted to a preset value is assigned to a term in logic Inference execution unit that performs derivation on logic by means of token propagation based on means and firing of each node on AND / OR graph Equipped with a.

  Preferably, the inference execution unit in the AND / OR graph fires each node in parallel and independently, generates a token to search for a necessary fact clause, and flows a token representing correction information in the reverse direction. And inference means for performing deductive inference by repeatedly executing backpropagation processing for eliminating contradiction between variables by making a probabilistic selection.

  Preferably, the data flow graph includes conversion means for expressing knowledge representing logic by predicate logic and generating a data flow graph representing the structure of the logic based on the plurality of operation nodes and a plurality of edges connecting between the nodes. .

  Preferably, the inference execution unit includes constant learning means for making a constant in the data flow graph closer to an optimum value by correcting a numerical value stochastically based on correction information back-propagated as a token.

  Preferably, the data flow graph further includes first topology changing means for adding a subtree corresponding to a randomly generated clause.

  Preferably, the data flow graph further includes second topology changing means for deleting a subtree corresponding to the node generating the false branch.

  According to another aspect of the present invention, there is provided a calculation learning device for executing forward propagation calculation using a data flow graph representing a calculation procedure by a plurality of operation nodes and a plurality of edges respectively connecting between the operation nodes. Correction information for calculating correction information for satisfying a teacher value and constraints given from the outside based on the calculation result of the calculation execution unit. Based on the correction information, the tokens representing the local correction information are propagated in the reverse direction in the data flow graph based on the correction information. And a processing unit that performs calculation.

  Preferably, in the calculation learning device, local correction information held by the token is obtained by back calculation at each node.

  Preferably, in the calculation learning device, local correction information held by the token is obtained by propagation of a differential coefficient at each node.

Preferably, the calculation procedure is a logic program representing knowledge.
Preferably, the calculation procedure is a program for executing processing of arithmetic operation and logical operation.

  Preferably, the calculation procedure is a linear or non-linear simultaneous equation obtained by combining arithmetic operations.

According to yet another aspect of the present invention, there is provided a logical reasoning method for executing inference based on knowledge expressed in logic, wherein the knowledge is a data flow graph in the form of backward reasoning, This is expressed by using a data flow graph composed of multiple operation nodes and a plurality of edges each connecting between the operation nodes. The plurality of operation nodes include operation nodes corresponding to various logical operations and are associated with the operation nodes. Is a step of assigning a value converted into a preset numerical value to a term in logic and expanding the data flow graph into an AND / OR graph by token propagation based on the firing of each node on the data flow graph And performing derivation on the logic by token propagation based on the firing of each node on the AND / OR graph. That.

  Preferably, the step of performing the derivation for the logic includes a forward propagation process in which each node fires independently in parallel, generates a token and searches for a necessary fact clause, and a token representing correction information flows in the reverse direction. It includes the step of performing deductive inference by repeatedly executing backpropagation processing that resolves contradictions between variables by making a probabilistic selection.

  Preferably, in the data flow graph, conversion means for expressing knowledge representing logic in predicate logic and generating a data flow graph representing the structure of the predicate logic based on the plurality of operation nodes and a plurality of edges respectively connecting between the nodes. Prepare.

  According to still another aspect of the present invention, there is provided a calculation learning method for executing forward propagation calculation using a data flow graph representing a calculation procedure by a plurality of operation nodes and a plurality of edges respectively connecting between the operation nodes. A plurality of operation nodes including operation nodes corresponding to various operations of the program, and calculating correction information for satisfying a teacher value and constraints given from the outside based on a calculation result of forward propagation calculation; Propagating tokens representing local correction information in the data flow graph based on the correction information in the reverse direction, and performing variable learning based on constant learning and constraints by performing probabilistic selection on these tokens And further comprising.

  Preferably, in the step of calculating correction information, local correction information held by the token is obtained by back calculation at each node.

  Preferably, in the step of calculating correction information, local correction information held by the token is obtained by propagation of a differential coefficient at each node.

  According to the present invention, deductive reasoning is executed while incorporating a learning function, and the structure of the constructed knowledge is clearly shown in the graph.

  According to the present invention, a logic program for deductive reasoning, or a program that outputs a calculation result desired by the user is created by learning a program that executes arithmetic operations and logic operations.

It is a figure showing the structure of the calculation processing apparatus 100 which concerns on this Embodiment in block diagram format. It is a figure which shows the functional structure of the calculation processing apparatus 100 which concerns on this Embodiment in block diagram format. It is a conceptual diagram which shows the procedure of inference and learning in a forward inference type network similar to KTN. It is a figure which shows the example which converted predicate logic into the graph expression in a high level Petri net. It is a figure which shows the example which converted predicate logic into the network expression in KTN. In KTN, it is a conceptual diagram which shows the process which expands and quantifies the "backward data flow graph" demonstrated in FIG. It is a conceptual diagram which shows the procedure which digitizes a symbol / symbol column. It is a figure which shows the calculation (operation) matched with a node. It is a 1st figure which shows what kind of graph the predicate logic of a logic program is converted in KTN. It is a 2nd figure which shows what kind of graph the predicate logic of a logic program is converted in KTN. It is a figure which shows the example which constructed | assembled KTN. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. It is a figure which shows the flow of the symbol token in KTN sequentially. FIG. 25 is a diagram illustrating a result of expanding the data flow graph described in FIGS. 12 to 24 and converting the data flow graph into an AND / OR graph. FIG. 26 is a diagram showing the AND / OR graph shown in FIG. 25 as an AND / OR tree with the internal structure of each section omitted. It is a figure which shows the result of having judged whether search and unification are possible from the leaf corresponding to branch BR1-1 about tree search and unification. It is a figure which shows the result of having judged whether search and unification are possible for the tree search and unification from the leaf corresponding to branch BR3-1 and the leaf corresponding to branch BR1-2. It is a figure which shows the result of having judged whether search and unification are possible for the tree search and unification from the leaf corresponding to branch BR3-2 and the leaf corresponding to branch BR1-2. It is a conceptual diagram for demonstrating the concept of a stochastic proximity method. It is a conceptual diagram which shows the case where constant learning is performed by the probabilistic proximity method. It is a figure which shows the process of the stochastic proximity method in the data flow graph showing Formula (1) mentioned above. It is a figure for demonstrating the procedure in the case of solving such simultaneous equations using a data flow graph (data flow graph expanded to AND / OR tree). Thus, it is a figure explaining the process at the time of carrying out the back propagation of the token produced | generated by the equivalence node 300 or 302. FIG. The data flow graph of FIG. 12 and the like is expanded, and in the graph shown by the AND / OR tree, the token symbol (when the initial value of q is 0.5 and the initial value of z0 is also 0.5 is forwardly propagated) It is a figure which shows x, r). It is a figure which shows the state which performed back propagation from the state of FIG. It is the figure which expanded the part shown by "enlarged drawing 1" in FIG. It is the figure which expanded the part shown by "enlarged figure 2" in FIG. FIG. 37 is a diagram showing a state in which forward propagation is performed again after updating the initial value from the back propagation state of FIG. 36. It is a figure which shows the state which performed the back propagation further from the state of FIG. It is a figure for demonstrating supervised learning. It is a figure explaining the concept in the case of modifying a topology. It is a flowchart for demonstrating the process which the calculating part 220 performs. It is a conceptual diagram which shows another example about predicate logic and derivation. It is a conceptual diagram which shows the content which performs derivation | leading-out about the fact clause, rule clause, and goal clause similar to FIG. 44 by Prolog which is a logic type programming language which handles predicate logic. It is a conceptual diagram which shows the content which performs derivation | leading-out about the fact clause, rule clause, and goal clause similar to FIG. 44 by the production system which handles predicate logic. It is a conceptual diagram which shows the content which performs derivation | leading-out about the fact clause, rule clause, and goal clause similar to FIG. 44 by the high level Petri net which handles predicate logic. It is a conceptual diagram which shows the content which performs derivation | leading-out about the fact clause, rule clause, and goal clause similar to FIG. 44 by the semantic network which handles predicate logic. It is a conceptual diagram which shows the content which performs derivation | leading-out about the fact clause, rule clause, and goal clause similar to FIG. 44 by the stochastic logic programming which handles predicate logic, especially a Bayesian network. It is a figure which shows the typical model which has been proposed until now in the field of symbol processing AI. It is a conceptual diagram which shows the procedure in the case of calculating a derivative.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

(1. Overview)
The calculation processing apparatus 100 according to the present embodiment executes a derivation process for a program given from outside (hereinafter also referred to as an initial program), in particular, a logic program based on “predicate logic”.

  The program is not limited to such a logical program based on “predicate logic”, but is also a program composed of arithmetic operations and logical operations as disclosed in the above-mentioned Patent Document 1 (International Publication No. 2010/047388). But you can. In the following, however, the derivation process for a logic program based on “predicate logic” will be mainly described.

  Further, as will be described later, calculation processing apparatus 100 according to the present embodiment can also perform processing for correcting an initial program by learning and creating a program that outputs a more appropriate calculation result.

The calculation processor roughly performs the following three processes.
(1) Conversion of logic program into network (data flow graph) (2) Execution of calculation using network (3) Correction of network The outline of each process will be described below.

[(1) Conversion of program to network]
The calculation processing device 100 creates a graph representation of a network type program, more specifically, a data flow graph based on the given initial program. That is, the computer 100 creates a network that expresses the structure of the predicate logic of the initial program that is a logic program.

  In the present embodiment, the graph representation of the program created from the initial program by the computer 100 is referred to as a KTN (Knowledge Transitive Network) type network. The network in the KTN format represents a predicate logic structure by a plurality of operation nodes each representing an operation process and a plurality of edges connecting any two operation nodes among the plurality of operation nodes. The edge has a direction representing the operation order.

[(2) Execution of calculation using network]
The calculation processing apparatus 100 executes calculation using the created network. That is, the calculation processing device 100 performs an operation by the operation node on the input value according to the order represented by the edge.

  In the present embodiment, calculation processing apparatus 100 performs an operation on each shuttle node with respect to a shuttle agent (hereinafter simply referred to as an agent) having information necessary for the operation. The calculation node receives input of one or a plurality of agents, performs calculation on the agent input from the edge based on a predetermined rule, and outputs the calculated agent to the edge. The agent output from the calculation node is input to the calculation node at the destination of the edge. Details of the agent and details of processing rules for the agent will be described later.

[(3) Network correction]
The calculation processing apparatus 100 corrects the network based on the calculation result using the network using a learning algorithm. A network correction method will be described later.

  The calculation processing apparatus 100 can execute (2) inference (derivation) for predicate logic by executing a calculation using a network.

  Furthermore, the calculation processing apparatus 100 can create a network (and therefore a logical program) that outputs a value desired by the user by repeating (3) network correction as necessary.

(2. Hardware configuration)
A hardware configuration of the calculation processing apparatus 100 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of a calculation processing apparatus 100 according to the present embodiment.

  The calculation processing device 100 includes a computer main body 102, a monitor 104 as a display device, and a keyboard 110 and a mouse 112 as input devices. The monitor 104, keyboard 110, and mouse 112 are connected to the computer main body 102 via the bus 105.

  The computer main body 102 includes a flexible disk (hereinafter referred to as “FD”) drive 106, an optical disk drive 108, a CPU (Central Processing Unit) 120, a memory 122, a direct access memory device such as a hard disk 124, and the like. Communication interface 128. These components are connected to each other by a bus 105.

  The FD drive 106 reads and writes information from and to the FD 116. The optical disc drive 108 reads information on an optical disc such as a CD-ROM (Compact Disc Read-Only Memory) 118. The communication interface 128 exchanges data with the outside.

The CD-ROM 118 may be another medium, such as a DVD-ROM (Digital Versatile Disc) or a memory card, as long as it can record information such as a program installed in the computer main body. In this case, the computer main body 102 is provided with a drive device that can read these media. The bus 105 may be connected to a magnetic tape device that is detachably loaded with a cassette type magnetic tape.

  The memory 122 includes a ROM (Read Only Memory) and a RAM (Random Access Memory).

  The hard disk 124 stores an initial program 131, a network creation program 132, a network modification program 133, network information 134, a node calculation definition 135, and learning information 136.

  The initial program 131 is a logical program that is a basis for derivation in the logical program (or creation by modification of the logical program from the initial program). The hard disk 124 can store a logical program created by the user as the initial program 131. The initial program 131 may be supplied by a storage medium such as the FD 116 or the CD-ROM 118, or may be supplied by another computer via the communication interface 128.

  The network creation program 132 creates a network (data flow graph, expanded data flow graph) corresponding to the initial program 131 based on the initial program 131. The network creation program 132 also stores network information 134 relating to the created network in the hard disk 124.

  The network correction program 133 corrects the network created by the network creation program 132.

  The network information 134 includes information about the network created by the network creation program 132 and information created by the network modification program 133 when performing computations or network modifications by the network.

  The node operation definition 135 represents a rule for operations performed by operation nodes included in the network. The node operation definition 135 includes information (operation code) for specifying an operation node, the number of edges input to the operation node (number of inputs), and the like. Details of the node operation definition 135 will be described later.

  The learning information 136 is information necessary for derivation processing in the network and network correction. The learning information 136 includes, for example, a teacher value desired as a calculation result by the user, a constant value updated during learning, a value of a numerical token, a correction value for the token, and information on the history of firing and transfer on the data flow graph. (Referred to as “tokenfire genealogy”).

  Here, as the node calculation definition 135, a user created using an input device can be used. However, these may be supplied by a storage medium such as the FD 116 or the CD-ROM 118, or may be supplied via the communication interface 128 by another computer.

  The network creation program 132 and the network correction program 133 may be supplied by a storage medium such as the FD 116 or the CD-ROM 118, or may be supplied by another computer via the communication interface 128.

  The CPU 120 functioning as an arithmetic processing unit executes processing corresponding to each program described above using the memory 122 as a working memory.

  The network creation program 132 and the network correction program 133 are software executed by the CPU 120 as described above. Generally, such software is stored and distributed in a storage medium such as the CD-ROM 118 and the FD 116, read from the storage medium by the optical disk drive 108 or the FD drive 106, and temporarily stored in the hard disk 124. Alternatively, when the computing device 100 is connected to the network, it is temporarily copied from the server on the network to the hard disk 124. Then, the data is further read from the hard disk 124 to the RAM in the memory 122 and executed by the CPU 120. In the case of network connection, the program may be directly loaded into the RAM and executed without being stored in the hard disk 124.

  The computer hardware itself shown in FIG. 1 and its operating principle are general. Therefore, an essential part for realizing the functions of the present invention is software stored in a storage medium such as the FD 116, the CD-ROM 118, and the hard disk 124.

(3. Functional configuration)
With reference to FIG. 2, a functional configuration of calculation processing apparatus 100 according to the present embodiment will be described. FIG. 2 is a block diagram showing the functional configuration of the calculation processing apparatus 100 according to the present embodiment.

  The calculation processing apparatus 100 includes an input unit 210 that receives data from the outside, a calculation unit 220, an output unit 230 that outputs the calculation result of the calculation unit 220 to the outside, and a storage unit 240.

  The calculation unit 220 performs a calculation on the data stored in the storage unit 240. The calculation unit 220 includes a network creation unit 222, a calculation execution unit 226, and a network update unit 228.

  The network creation unit 222 creates network information 134 based on the initial program 131 stored in the storage unit 240. The network creation unit 222 includes a format conversion unit 224. As will be described later, the format conversion unit 224 converts the logic program into an expanded data flow graph for executing backward deduction with a breadth-first search.

  The calculation execution unit 226 executes calculation using the network based on the network information 134 and the node calculation definition 135. Details of the processing performed by the calculation execution unit 226 will be described later.

  The network update unit 228 corrects the network information 134 using a learning algorithm. More specifically, the network update unit 228 includes a topology learning unit 228a. In addition, the network update unit 228 may include a constant learning unit that corrects the constant value of the operation node included in the network based on the error included in the learning information 136. The topology learning unit 228 executes network update processing such as generation / deletion of operation nodes and generation / deletion of edges as needed based on a predetermined rule. Details of processing performed by the network update unit 228 will be described later.

  In the above description, the calculation processing device 100 has been described as a single computer device. However, since the arithmetic processing is executed using an agent, the calculation processing device 100 is executed by a plurality of computers. You may be comprised by the system which performs a distributed process.

In the calculation processing apparatus 100 according to the present embodiment, as will be apparent from the following description, a derivation process for a logic program using the above-described two techniques, a new AI model that fills the gap between the symbol processing AI and the neural network, or Network modification (that is, creation of a logical program by changing the topology of the network) is executed.

This network-based model is designed to be able to move learning rules developed in the field of neural networks while explicitly handling symbols / symbols with reliability. b) New reasoning that combines the three models of deduction and (c) learning
A learning system can be constructed. As described above, the network for executing this model is called a knowledge transition network (KTN). As will be apparent from the following description, this model is a model disclosed in International Publication No. 2010/047388 of Patent Document 1 that has been filed earlier and has already been published, and performs arithmetic and logical operations. Although it is based on the Algorithmic Transtcive Network (ATN) for the program to be executed, as described below, improvements to handle symbols in predicate logic are made, and the learning method is also “probabilistic proximity”. It applies a new algorithm called "Law".

  FIG. 3 is a conceptual diagram showing inference and learning procedures in a forward inference network similar to KTN, but not KTN itself.

FIG. 3 shows, as an example, a case where inference and learning are performed on the logic “If x = A then y = B”. Here, the concept of processing for an inference network will be described only briefly, and thus the description of individual processing will be omitted.

  First, three constant term nodes (c in the figure) are fired simultaneously, and a token is output from these nodes. Here, the token is expressed as a set of a symbol and a value indicating the reliability of the symbol.

  When a plurality of tokens from each node reach the final node, they are compared with the teacher data t, and the tokens whose reliability is equal to or higher than a predetermined value, that is, tokens that are correct, are traced back to the firing history of this token. , The correction information propagates back.

  In such a forward inference network, multiple answers including contradiction propagate forward along with reliability, so it is possible to capture the ambiguity of inference, and in learning, select a highly reliable token Therefore, the situation in which the number of tokens that must be processed explosively increases during learning can be prevented to some extent.

(Predicate logic and network conversion)
FIG. 4 is a diagram illustrating an example in which predicate logic is converted into a graph representation in a high-level Petri net.

In FIG. 4, the following predicate logic is handled.
Factual paragraph: Taro is a man. Expressed as Man (Taro).

Rule section: A man is a human. Expressed as Human (x) ← Man (x).
Goal Section: Who is a human being? (Who is a human?) ← Expressed as Human (q).

  In a high-level Petri net, predicate logic is expressed as a forward directed graph with a labeled term attached to an arc.

FIG. 5 is a diagram illustrating an example in which predicate logic is converted into a network representation in KTN.
As the predicate logic, the same one as in FIG. 4 is taken as an example.

  First, consider the case where the predicate logic is represented by a forward-looking data flow graph. A directed edge goes from a start node (Begin) that starts firing to a constant node with a symbol of Taro. Next, the directed edge is directed from the constant node (Taro) to the merge node corresponding to Man, and further, the directed edge is directed from the merge node (Man) to the merge node corresponding to Human. From the merge node (Human), the directed edge is directed to the merge node corresponding to the query variable q, and from the merge node (q), the directed edge is directed to the terminal node indicating stop of the search.

  On the other hand, in KTN, it is expressed as a backward data flow graph obtained by inverting the forward data flow graph described above by the processing of the network creation unit 222. First, a directed edge is directed from a start node (Begin) that starts firing to a constant node with a symbol q. Next, the directed edge is directed from the constant node (q) to the merge node corresponding to Human, and further, the directed edge is directed from the merge node (Human) to the merge node corresponding to Man. From the merge node (Man), one is directed to a constant node corresponding to Taro, and the directed edge is directed to one input of an equivalence node that generates true when two inputs are equal. On the other hand, the directed edge goes from the constant node (Taro) to the other input of the equivalence node. The result of the equivalence node is passed to the end node.

  As described above, in KTN, knowledge (predicate logic) is expressed by a graph topology and, unlike a high-level Petri net, only labels for constants and undetermined variables are given to nodes.

  FIG. 6 is a conceptual diagram showing a process for further deducing and dequantizing the “backward data flow graph” described in FIG. 5 in KTN.

  As shown in FIG. 6, the “backward data flow graph” is expanded by the network creation unit 222 and expressed as an AND / OR graph, and symbol tokens are expressed as numerical tokens.

  Here, if Taro is expressed by the numerical value “1”, for example, the value of the query variable q being 1 in the back propagation of the token is equivalent to finally obtaining an answer. To do.

  As tokens are digitized, the two outputs of the merge node corresponding to Man are connected by AND, and the equivalence node functions as a binding to give a logical truth.

  With the above configuration, KTN performs backward deduction by breadth-first search by parallel firing of nodes.

FIG. 7 is a conceptual diagram showing a procedure for digitizing a symbol / symbol string.
As shown in FIG. 7, the symbol / symbol column is represented by a number. As shown in FIG. 7, such quantification is performed by previously associating terms (individual constants, individual variables, functions) in the predicate logic with natural numbers and using them as global tables (tables that can be referenced globally). Define in advance.

  As shown in FIG. 7, if Taro is a natural number, it is 1 when converted to a numerical value with reference to the table. If the function wife corresponds to 7, the function wife (Taro) (Taro's wife) is coded as 71.

  Similarly, if 8 is associated with the function child, child (wife (Taro)) is coded as 871.

  In general, if there are N terms, a symbol corresponding to the term can be coded in N-ary, for example.

  In such conversion from symbol to numerical value, if an N-ary gray code is used, the gray code has high symmetry. It can have the feature of being interchangeable.

FIG. 8 is a diagram illustrating an operation associated with a node.
In FIG. 8, “synchronous / asynchronous” means that when there are two or more inputs, in synchronization, the nodes are fired after waiting for the tokens to become operands on all input edges. Means to be executed.

(Conversion from logic program to KTN)
FIG. 9 is a first diagram showing what kind of graph the predicate logic of the logic program is converted into in KTN.

  FIG. 10 is a second diagram showing what graph the predicate logic of the logic program is converted into in KTN.

  As shown in FIGS. 9 and 10, the predicate logic is converted to KTN depending on whether the argument of the predicate is a query variable, an individual variable, or an individual constant, and the logical relationship or function.

FIG. 11 is a diagram illustrating an example in which a KTN is constructed.
In the example of FIG. 11, the following logic program is targeted.

J (x, 0).
J (x, x).
K (x, x, x).
K (s (x), x, y) ← J (z, y), K (x, z, y).
← K (2,1, q).
12 to 24 are diagrams sequentially showing the flow of symbol tokens in the KTN shown in FIG.

  The execution of the token propagation process for KTN starts from the firing of the Begin node, similar to ATN. After that, each node executes the operation coded using the data of the token arriving at the input edge as an operand and flows the resulting token on the output edge in parallel and independently, just like a normal data flow calculator Yes, so that node firing (fire) and token propagation occur in a cascade in the network.

  First, referring to FIG. 12, the start node fires and the token propagates to the constant nodes (symbols are 2, 1, and q, respectively).

  Next, as shown in FIG. 13, the tokens that have passed through the constant node are accompanied by symbols (2), (1), and (q), respectively.

As shown in FIG. 14, when the token passes through the argument synthesis node, the tokens are combined into one, and this is accompanied by the symbol (2, 1, q) and propagates to the merge node.

  Next, as shown in FIG. 15, the merge node outputs a token accompanied by the symbol (2, 1, q) as two outputs.

  Further, as shown in FIG. 16, symbols (2, 1, q) propagated from the merge node to the argument decomposition node of the first branch BR1 are symbol (2), symbol (1), symbol (q ) Is divided into accompanying tokens and output.

  On the other hand, the symbols (2, 1, q) propagated to the argument decomposition node of the second branch BR2 are also divided into tokens accompanied by the symbols (2), (1), and (q), respectively, and output. Is done.

  As shown in FIG. 17, in the first branch BR1, since one input does not match in the upper equivalence node, the output is false (F: False), and in the lower equivalence node, q If = 1, the output is true (T: True).

  In the second branch, at the function f node, the token 2 symbol changes to s, which propagates to the inverse function node.

  On the other hand, when the token (q) passes through the constant node, the symbol of this token becomes z0. Therefore, the token from the inverse function node and the token (1) are propagated to the equivalent node. Token (1), token (z0), and token (q) are propagated to the upper argument synthesis node, and token (z0) and token (q) are propagated to the lower argument synthesis node.

  Next, as shown in FIG. 18, in the first branch BR1, since false (F) is propagated to one input of the logical product node, the output of the logical product node becomes false (F).

  On the other hand, in the second branch BR2, the inputs to the equivalence node are both “1”, the token (1, z0, q) from the upper argument synthesis node, and the token (z0, q) from the lower argument synthesis node. q) propagates respectively.

  Further, as shown in FIG. 19, in the second branch BR2, the output from the equivalence node to the upper gate node and the lower gate node becomes true (T). Therefore, the token (1, z0, q) from the upper argument synthesis node and the token (z0, q) from the lower argument synthesis node propagate through the upper gate node and the lower gate node, respectively.

  As shown in FIG. 20, the token that has passed through the upper gate node propagates to the merge node on the input side of the first and second branches. The token that has passed through the lower gate node is propagated to the merge node on the input side of the third branch.

  As shown in FIG. 21, two tokens (1, z0, q) are output from the merge nodes on the input side of the first and second branches. Two tokens (z0, q) are output from the merge node on the input side of the third branch.

  As shown in FIG. 22, in the third branch, the token (z0) and token (q) are propagated from the upper argument decomposition node, and the token (z0) and token (q) are also transmitted from the lower argument decomposition node. ) Propagates.

  On the other hand, in the first branch BR1, the token (1), token (z0), and token (q) are propagated from the upper argument decomposition node, and the token (1) and token (1) are also transmitted from the lower argument decomposition node. z0) and the token (q) are propagated.

  As shown in FIG. 23, in the first branch BR1, “T if z0 = 1” is output from the upper equivalence node, and “T if q = z0” is output from the lower equivalence node. .

  In the second branch BR2, at the node of the function f, the symbol of token 1 is changed to s, which is propagated to the inverse function node.

  On the other hand, when the token (q) passes through the constant node, the symbol of this token becomes z1. Therefore, the token from the inverse function node and the token (z0) propagate to the equivalent node. Token (z0), token (z1), and token (q) are propagated to the upper argument synthesis node, and token (z1) and token (q) are propagated to the lower argument synthesis node.

  In the third branch BR3, token (0) and token (q) are input to the upper equivalence node, and “T if q = 0” is output from the lower equivalence node.

  As shown in FIG. 24, in the first branch BR1, “T if z0 = 1, q = z0” is output from the AND node. Further, in the second branch BR2, a token (z0, z1, q) is received from the upper argument synthesis node, and a token (z1, q) is received from the lower argument synthesis node, respectively, as an upper gate node and a lower gate node. Propagate to. From the gate node, if “T if z0 = 0” is established in the equivalent node, the token passes.

  FIG. 25 is a graph showing the AND / OR graph obtained by developing the data flow graph described with reference to FIGS.

  In FIG. 25, nodes are surrounded by dotted lines, and arguments are surrounded by alternate long and short dash lines. In each section, a variable / constant AND node is inserted for expansion into an AND / OR graph.

  FIG. 26 is a diagram showing the AND / OR graph shown in FIG. 25 as an AND / OR tree with the internal structure of each section omitted.

  In the root clause, a query variable q is generated, and the merge node as an argument functions as an OR node. A node corresponding to the branch BR1-1 is expressed as a leaf. The node corresponding to the branch BR2-1 functions as an AND node that generates the variable z0.

  In the leaves corresponding to the branch BR3-1 and the branch BR3-2 having (z0, q) as arguments, there are finally binding conditions (“T if q = 0” and “T if q = z0”). ).

On the other hand, in the leaf corresponding to the branch BR1-2 having (1, z0, q) as an argument, a binding condition finally exists (“T if z0 = 1, q = z0”). On the other hand, in the leaf corresponding to the branch BR2-2, there is a constraint ("T if z0 = 0") that is not on the leaf (determined in the previous leaf). Further, the variable z1 is generated in the leaf corresponding to the branch BR1-2.

Here, the purpose of the above logic program (that is, KTN) is to determine whether the goal clause holds or what the query variable (q) in the goal clause is. The process of determining whether or not such predicate logic is a true expression or obtaining a query variable is called “derivation” as described above.

  In the case of KTN, the problem of extracting this answer is that, from the generated AND / OR graph, Condition 1) Variable binding conditions have consistent solutions (ie can be unified), Condition 2) The root of the tree is true This results in the problem of finding a partial tree.

  FIG. 27 shows the result of determining whether the search and unification of the tree search and unification are possible from the leaves corresponding to the branch BR1-1.

  As shown in FIG. 27, since the leaf corresponding to the branch BR1-1 is false (F), the root is also false (F), and the condition 2 is not satisfied. I can't.

  FIG. 28 shows a result of determining whether the search and unification can be performed from the leaf corresponding to the branch BR3-1 and the leaf corresponding to the branch BR1-2 for the tree search and unification.

  As shown in FIG. 28, there is no q that simultaneously satisfies the leaf constraint condition corresponding to the branch BR3-1 and the leaf constraint condition corresponding to the branch BR1-2, that is, the constraint (constraint expression) of these two leaves. ) Is a simultaneous equation, and there is no solution (cannot be unified), so condition 1 is not satisfied, so this subtree cannot be a solution.

  FIG. 29 shows a result of determining whether a search and unification can be performed from a leaf corresponding to the branch BR3-2 and a leaf corresponding to the branch BR1-2 for tree search and unification.

  As shown in FIG. 29, there exists q that simultaneously satisfies the leaf constraint condition corresponding to the branch BR3-2 and the leaf constraint condition corresponding to the branch BR1-2, that is, the constraint (constraint expression) of these two leaves. ) Is a simultaneous equation (q = 1), so condition 1 and condition 2 are not satisfied, so this subtree is a solution.

  In other words, after token propagation, the answer of deductive reasoning is obtained by solving simultaneous equations between variables.

In order to solve such a problem, the calculation processing apparatus 100 according to the present embodiment solves the simultaneous equations numerically and distributedly using a network, called “probabilistic approach method (PA method)”. Apply.

  The probabilistic proximity method examines the logical consistency of the subtrees in the genealogy during the solution, and based on this, the process of growing the tip of promising branches can be used to grow the It is possible to search for solutions while organically connecting fake checks and variable unification.

FIG. 30 is a conceptual diagram for explaining the concept of such a probabilistic proximity method.
In other words, the probabilistic proximity method is a numerical solution that solves simultaneous equations sequentially, and calculates a correction value such as “calculate a parameter correction value to satisfy the constraint equation using back calculation or differentiation” and “ The solution is sequentially obtained by repeating the probabilistic proximity of “probably jump to the correction value according to the proximity”.

  In the probabilistic proximity method, solution candidates with small contradiction between constraints on the basis of proximity survive, so that parameter contradiction gradually converges to zero and a solution is obtained.

  As will be described below, the probabilistic proximity method can also be used when performing learning in back propagation (constant learning).

Hereinafter, the process of the probabilistic proximity method will be described with a more simplified example.
(Constant learning process)
First, the case where constant learning is performed by the probabilistic proximity method will be described as an example.

  FIG. 31 is a conceptual diagram showing a case where constant learning is performed by the probabilistic proximity method.

a is an answer, and s is a sensor that is substituted on a trial basis in the process of finding a solution.
Therefore, here, it is necessary to perform a procedure for determining the values of the constants ν0 and ν1 so that the relationship between a and s becomes the function form of the final solution.

Here, the teacher value set gives a group of simultaneous equations whose constants are unknown.
FIG. 32 is a diagram illustrating processing of the probabilistic proximity method in the data flow graph representing the above-described formula (1).

  First, it is assumed that the initial values of the constants ν0 and ν1 are 1.5 and 1.0, respectively. It is assumed that the value 1.0 is used as the sensor and the teacher value is 1.0. If the token propagates according to the data flow graph, 2.0 is obtained as a solution.

  However, since the teacher value is 1.0, first, in propagation learning from the plus node in the reverse direction, when the input is adjusted so that the plus becomes the teacher value only with the input above the node, Find the value of. Since the input on the lower side of the node is 1.5, if the input on the upper side is −0.5, it matches the teacher value. Therefore, when calculating what the input of the product node should be with respect to -0.5, since the sensor is 1.0, the value of the constant ν1 should be -0.5. It will be good.

  On the other hand, since the teacher value is 1.0, in the propagation learning from the plus node in the reverse direction, when the input is adjusted so that the plus is only the input below the node and becomes the teacher value. Also find the value. Since the upper input of the node is 0.5, if the lower input is 0.5, the input value matches the teacher value. Therefore, for this 0.5, the value of the constant ν0 may be ν0 ′: 0.5.

After this process, each constant node calculates a proximity factor r = exp (−κ / 4 · (vV) ² ) from the value V returned by the back calculation and the original value ν.

After updating as described above, for example, to update the constant v ₀ , the quotient obtained by dividing the proximity factor r ₀ of v ₀ by the sum Σ _j r _j of the proximity factors of all constant nodes is used as a probability from v _0. Change to V ₀ .

  The same applies to the constant ν1. By performing such a probabilistic update for all constants, the average value of the correction value set calculated by one back propagation is actually reflected in the constant value for the entire network. Is guaranteed.

  Expression (1) is rewritten using the updated constant, and forward propagation and back propagation are further repeated using the sensor and the teacher value. As a result of such repetition, the constant is updated so that the final solution is finally obtained.

(Process to find solutions to simultaneous equations)
Next, the case where the solution of simultaneous equations is obtained by the probabilistic proximity method will be described as an example.

  That is, the problem of determining the value of a variable so as to satisfy a plurality of constraints is solved by the probabilistic proximity method.

As an example, consider the case of solving the following simultaneous equations.
x1 = x0 + 1 (2)
x1 = 2x0-1 (3)
FIG. 33 is a diagram for explaining a procedure for solving such simultaneous equations using a data flow graph (AND / OR graph).

  In solving the simultaneous equations of the above equations (1) and (2), the initial value is (x0, x1) = (0, 0).

  First, “1” and “0” are input to the upper equivalence node 300 through the node x0 side in the data flow graph. As in the case of constant learning, the value must be changed to “0” in order to make it consistent only by changing the upper input, just to change the lower input. In order to make it consistent, the value must be “1”. Therefore, two consistent token pairs (0, 1) a and (1, 1) b are generated for the upper equivalence node 300. Here, (x, r) y for a token is that x is the value of the token, r is the reliability of this value x, and y is an identifier for this token. In this case, the identifier corresponds to any of the points a, b, c, and d in the graph, but actually, any identifier can be used as long as it is an identifier that can identify the token.

In addition, since equivalence nodes are consistent, the reliability is set to 1.
The same applies to the lower equivalence node 302.

  FIG. 34 is a diagram for explaining processing when the token generated in the equivalence node 300 or 302 is back-propagated in this way.

  First, the x0 node is an AND node, and the back-propagated tokens input to this node are (−1, 1) a, (0, 1) b, (0.5, 1) c, (0, respectively. 1) d. Accordingly, all combinations of the number of outputs from the AND node of the token back-propagated to the AND node are considered. That is, ac, ad, bc, and bd are used in the combination of token identifiers. The number of tokens generated by such a combination is expected to divergently increase with the number of variables and the number of constraint equations in the case of a large-scale simultaneous equation. By applying a probabilistic selection described later to the tokens before the combination is performed, processing for appropriately reducing the number of tokens is performed.

  For the tokens combined as described above, the numerical value and reliability are calculated according to the following formula.

(Randomly chosen x _i ) means one randomly taken from the combined x.
Vari _i means the variance for x to be combined, and κ is a predetermined constant. As for the reliability (proximity factor) calculated by this, if the values of the variables to be combined match, the variance is 0 and the reliability is 1. The greater the standard deviation, the less reliable.

  For example, for the combination ac, (x, r) becomes (0.5, 0.55) ac. The same applies to other combinations.

  Similarly, for the node x1, (x, r) can be calculated for all combinations of back-propagated tokens.

  Further, at the node (x0, x1), as the back propagation from the node x0 and the node x1, the token symbol (x0, x1, r) as a result of the final back propagation can be calculated for each identifier of the token. Here, the reliability r in this case is the minimum value of the reliability of the two tokens to be combined.

  As a result, as shown in FIG. 34, four tokens are finally obtained. Therefore, a quotient r / Σr obtained by dividing each reliability by the total reliability Σr is selected as a probability. In the illustrated example, (0, 0, 0.77) bc, which is one of the highest reliability, is selected.

  Next, forward propagation and reverse propagation are repeated with this token (0, 0, 0.77) bc as an initial value.

  By repeating such a procedure, the solutions of Equation (2) and Equation (3) are finally obtained.

  As described with reference to FIG. 29, in KTN, a subtree in which a solution of simultaneous equations exists in the leaf constraint condition is a solution of KTN inference. Specifically, a subtree as a solution can be searched by the same procedure.

  FIG. 35 shows an AND / OR graph obtained by expanding the data flow graph of FIG. 12 and the like. The token symbols (x, It is a figure which shows r).

FIG. 36 is a diagram showing a state in which back propagation is performed from the state of FIG.
It can be seen that the value of z0 has been updated.

  On the other hand, it can also be seen that there is an “undetermined branch”, which is a branch from which a positive r-value token does not come because the clause is undefined.

FIG. 37 is an enlarged view of a portion indicated by “enlarged view 1” in FIG.
In back propagation, the following can be said.

i) The reliability is inherited via the equivalency node.
ii) At the equivalency node, a “consistent” token pair with a unique ID is generated.

  iii) In the AND node, as described with reference to FIG. 34, an operation is performed on a combination of tokens (referred to as “token react”).

iv) In the constant node, only tokens whose x value matches the constant node are passed.
v) The argument decomposition node combines tokens that share an ID. At this time, the reliability is the minimum value of the reliability of the tokens to be combined.

FIG. 38 is an enlarged view of a portion indicated by “enlarged view 2” in FIG.
In back propagation, the following can be said.

vi) The OR node connects tokens with a comma symbol (meaning OR) and passes them.
vii) When there are too many tokens connected by OR, probabilistic selection using reliability r is performed, and tokens with small r are thinned out.

  FIG. 39 is a diagram illustrating a state in which forward propagation is performed again after updating the initial value from the back propagation state of FIG.

FIG. 40 is a diagram showing a state in which reverse propagation is further performed from the state of FIG.
q and z0 are further updated to both 1.

  By repeating such processing, the value of the token arriving at the undetermined variable gradually converges to q = z0 = 1 and r = 1, and unification can be solved approximately.

FIG. 41 is a diagram for explaining a method of learning a constant with supervision.
As is clear from the above description, the probabilistic proximity method is a method that enables constant correction (learning) while performing variable determination (deduction). Therefore, also in KTN, it is possible to perform correction so that the root becomes as true as possible by using probabilistic proximity using the teacher value and modifying the constant value using the correction value calculated in the course. With this learning process, it is possible to bring some constants embedded in the data flow graph closer to optimal values for those that are allowed to change as a problem.

FIG. 42 is a diagram for explaining the concept when the topology is modified.
For example, it is possible to make modifications to add randomly generated clauses to the data flow graph at a predetermined frequency, for example, every predetermined number of forward and reverse propagations, or when the random number exceeds a predetermined value.

  Moreover, it is also possible to delete a clause that generates a false branch more than a predetermined number of times in a plurality of trials of the probabilistic proximity method on the developed data flow graph.

  By such processing, it is possible to generate a logic program that generates a result closer to the teacher value while changing the topology of the corresponding data flow graph with respect to the initial logic program.

  Alternatively, by deleting branches that are not necessarily meaningful for logical reasoning, it is possible to generate a logical program represented by a data flow graph having a simpler topology while maintaining the result of the teacher value.

FIG. 43 is a flowchart for explaining processing executed by the calculation unit 220.
Referring to FIG. 43, when processing of operation unit 220 starts, network creation unit 222 prepares a data flow graph (DFG) corresponding to the initial knowledge expressed in predicate logic (
Step S102).

  Subsequently, the network creation unit 222 adds a subgraph corresponding to the given query / teacher value to the DFG (S104).

Further, the calculation execution unit 226 forwardly propagates the symbol token on the DFG, and grows the expanded DFG one stage ahead of the undetermined branch (S106). As shown in FIGS. 12 to 24, the symbol tokens usually circulate on the DFG in a cyclic manner, but when the processing of S106 is first called, the symbol tokens starting from the firing of Begin If it is called after that, and if it is called after that, the propagation of the symbol token stopped at the end of the undecided branch is further advanced by one cycle.

  Next, the calculation execution unit 226 initializes the variable location (S108) and forwards the numerical tokens in order (S110).

Next, the calculation execution unit 226 reversely propagates the numerical token (S112).
If the token is returned to the root by back propagation, then there is a variable trap, in other words, whether the reliability r of the returned token matches the reliability 1 within a predetermined range. Determination is made by the calculation execution unit 226 (S122).

  If the variables do not match, the calculation execution unit 226 determines whether the number of loops exceeds a predetermined number. If not, the process returns to step S110.

  On the other hand, if the number of loops exceeds the predetermined number (S116), the calculation execution unit 226 determines whether there is an undetermined branch, and if there is an undetermined branch, the network update unit 228 cuts the false branch (S120). The processing returns to step S106.

  On the other hand, if it is not a visible house in S118, an operation answer (false) is output (S130), and the network update unit 228 modifies the topology of the data flow graph by generating or deleting a clause (S126).

  If there is a trap of the variable in S122, the operation answer (q or true) is output, and the process proceeds to step S126.

  When the process of step S126 is completed, the calculation execution unit 226 determines whether there is a new query or a teacher value. If there is, the process returns to step S104, and if not, the process ends (S132). ).

  With the above processing, the logical inference / learning by KTN has the following effects.

(I) Knowledge is represented by the same static graph as the brain or neural network. This in itself does not necessarily have a computational advantage. However, AI's ultimate goal is to create intelligence that is comparable to humans, and future comparison with physiological data obtained from non-invasive measurement techniques of the human brain that have made significant progress recently. When thinking about the possibilities, it is very meaningful to express knowledge with the same mathematical architecture as the brain.

(Ii) A series of deductive and learning processes such as genealogy creation and unification are realized by token propagation on the graph. Therefore, KTN execution programs that implement them can be expected to increase efficiency by porting them to parallel / distributed machines.

(Iii) In deduction, backward inference is performed based on the goal clause. Therefore, it is possible to narrow the search range even in the case of deep reasoning, and it is more efficient than forward reasoning that generates facts without darkness.

(Iv) On the network representing knowledge, each symbol is represented by a continuous real number, and in addition, the final solution is presented together with the reliability r, thereby expressing the ambiguity of knowledge. This is an ambiguity representation of knowledge by a method different from the rule weighting performed in probabilistic logic programming.

(V) Variables in the predicate logic are explicitly handled using identifiers on the network, which makes it possible to use a graph with a compact scale compared to other approaches that graph the predicate logic by propositional logic. Realizes general knowledge expression. In addition, topology change (knowledge modification learning) for graphs provides a framework for generating new nodes (predicates) and changing logical structures.

In the above description, the calculation of the correction value for the stochastic proximity method is all performed using back calculation. However, as disclosed in International Publication No. 2010/047388 of Patent Document 1, if back propagation on the network is used, the difference between the calculation result <x>, the correct result (teacher value), and the calculation result The energy error E determined based on this and the differential coefficients D _x and D _r of the energy error E can be calculated, and by using them, the correction value can be approximately calculated in the first order range of the Taylor expansion. .

FIG. 51 is a conceptual diagram showing a procedure for calculating such a differential coefficient.
In the figure, x represents the current value of an arbitrary node, and the answer a output from the network is written as a = G (x) as this function. Now, if this x is corrected to x _T and the network answer becomes a teacher value t, the request can be written as t = G (x _T ).
If these two formulas are drawn side by side and transformed using the energy error definition formula E = 1/2 (ta) ² , finally x _T is the current node value x, energy error E, and differential coefficient D. use the _x, written as _{x T = x-2E / D} x. The correction value for the probabilistic proximity method can also be calculated using this equation.

  Further, although the target program is a logic program, as disclosed in International Publication No. 2010/047388 of Patent Document 1, the source code of a program for performing arithmetic operations and logical operations is in GIMPLE format. It is also possible to perform back-propagation learning by the probabilistic proximity method on ATN obtained by converting into an intermediate code and converting the intermediate code into a network.

  In the above description, conceptual processing has been mainly described. However, as a use example in which the effect of the present embodiment is most exerted, for example, a manufacturing condition determination problem from an order specification is assumed.

In the field of manufacturing steel and steel sheets, there is a problem that appropriate manufacturing conditions are determined from order specification data such as JTS standards, applications, and sheet thicknesses. The order specification is a list of hundreds of items with discrete symbols or continuous numerical values. Based on this, hundreds of conditions prepared in advance (hundreds of different symbols) Only one appropriate item needs to be selected. At present, this problem is performed manually by skilled workers, but only a rough manual (workbook) can be created because the judgment results differ from person to person and the rules cannot be written precisely. Are difficult to pass on, and the manual itself cannot be improved.

In the past, there has been an example in which ID3 (Iterative Dichotomiser 3), a method for generating a classification decision tree based on a hypothesis, has been applied as a research on this problem. However, the judgment result by the classification tree of about 3,000 leaves synthesized from the accumulated judgment data of about 29,000 cases is
Only about 80% of expert results were reproduced. This is due to the fact that ID3 has the above-mentioned problem of machine learning--the problem of gaps between the discreteness of symbols and the ambiguity and noise of case information. The correct answer rate was rather higher in the method of assigning to the latest data tried separately, but of course this method can not lead to manualization.

The problem is that the implicit rule can be written as “if (order specification) then (manufacturing conditions)” or (manufacturing conditions) ← (order specifications), which is suitable for expressing knowledge in a logical program. It is a problem. The present invention (KTN) and the Paysian network can be considered as a method for deducting and learning knowledge expressed in a logical program while taking into account ambiguity.
Writing knowledge in KTN that explicitly handles variables is more versatile and easier to manual. There is also a possibility that the scale of the graph itself can be made smaller than that of the Paysian network in which nodes that are made constant for each item of the order specification are prepared.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

DESCRIPTION OF SYMBOLS 100 Computation processing apparatus, 102 Computer main body, 104 Monitor, 105 Bus, 106 drive, 108 Optical disk drive, 110 Keyboard, 112 Mouse, 118 ROM, 122 Memory, 124 Hard disk, 128 Communication interface, 131 Initial program, 132 Network creation program, 133 Network modification program, 134 network information, 135 node calculation definition, 136
Learning information, 210 input unit, 220 arithmetic unit, 222 network creation unit, 224 format conversion unit, 226 calculation execution unit, 228 network update unit, 228a topology learning unit, 230 output unit, 240 storage unit, 300, 302 equivalence node, BR1 first branch, BR2 second branch, BR3 third branch.

Claims (2)

A computational learning device,
A calculation execution unit for executing forward propagation calculation using a data flow graph representing a calculation procedure by a plurality of operation nodes and a plurality of edges respectively connecting between the operation nodes;
The plurality of operation nodes include operation nodes corresponding to various operations of the program,
A correction information calculation unit for calculating correction information for satisfying a teacher value and constraints given from the outside based on a calculation result of the calculation execution unit;
Based on the correction information, tokens representing local correction information in the data flow graph are propagated in the reverse direction, and by making a stochastic selection to the tokens, learning of constants and calculation of variable values based on constraints are performed. A calculation learning device, further comprising: a processing unit to perform.   The calculation learning apparatus according to claim 1, wherein in the calculation learning apparatus, the local correction information of the token is obtained by back calculation at each node.

Images