JPH06309293A - Constituting method for multilayer structure type neural network - Google Patents

Abstract

PURPOSE:To easily constitute in a little learning processing time and a little processes for even the multilayer structure type neural network of four layers or more having a complicated classifying function by operating a learning processing by using an appropriate learning sample for each small-scaled neural network in which logically organized initial classifying functions are set. CONSTITUTION:This device is equipped with a partial neural network 108 and a connecting neural network 109, and a total assembling processing 800 is performed by each output. Then, a learning processing 700 is performed to each partial neural network 108 by using a corresponding partial sample group 107. Afterwards, the connecting neural network 109 is connected with the partial neural network 108, and the total assembling processing 800 of the multilayer structure type neural network is performed, or the total assembling processing 800 is performed and then a learning processing 900 is performed to each partial neural network 108 by using the corresponding partial sample group 107.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for constructing a multi-layered neural network, and more particularly to a method for constructing a multi-layered neural network for classifying target data represented by multidimensional feature vectors.

[0002]

2. Description of the Related Art Conventionally, for the purpose of data classification processing,
A multilayer structure type neural network is used.
The configuration of such a multilayer structure neural network will be described.

FIG. 9 is a diagram showing an example of the structure of a multilayer structure type neural network.

The neural network shown in FIG. 1 has an input layer 11 for inputting vector elements of classification target data, an output layer 12 for outputting classification results, and one or more layers between the input layer 11 and the output layer 12. It is composed of the intermediate layer 13. The input of the unit 10 in each layer is coupled to the output of each unit in the preceding layer. The output of each unit is determined according to the following equation.

[0005]

[Equation 1]

Here, o _j ^(k) is k layers (k ≧ 1, k =
1 is the output value of the j-th unit in the input layer), w
_ij ^(k) is the coupling load from the i-th unit of the k-1 layer to the j-th unit of the k layer, and N ^(k-1) is the total number of units of the k-1 layer. However, w _0j ^(k) is a coupling weight for biasing the j-th unit in the k-layer, and o ₀
^(k-1) is always 1.

Each unit in the input layer (k = 1) outputs the input vector element value as it is. In order to cause such a neural network to perform the classification process of the target data, when the vector element of the sample of the target data is given to the unit of the input layer 11, only the output layer unit corresponding to the classification category to which the sample belongs is high. It is necessary to set the above-mentioned coupling load value w _ij ^(k) between each unit so that the value is output and the other output layer units output a low value.

Therefore, as a first conventional method for constructing a multilayered neural network, an error between an actual output value and a desired output value when a vector element of a sample of target data is input is reduced. In addition, an inverse error propagation learning method is widely used in which the coupling weight value w _ij ^(k) is repeatedly adjusted in small amounts.

Of the samples of the target data used in this method, those used for learning are called a learning sample group.

Here, the first conventional method will be described.

FIG. 10 shows a flowchart of the first conventional method. First, as a structure setting process, the number k of layers of the neural network and the number N of units in each layer are set (step 21). Next, a random initial value is set to the connection weight value w _ij ^(k) of each unit as a connection weight random setting process (step 22). Next, case learning is performed using the learning sample group by the inverse error propagation learning method (learning processing), and the classification function is set (step 23). Here, the structure setting process (step 21) and the joint load value random setting process (step 2) in the conventional first method are performed.
2) is a random type forming process (step 20).

As described above, according to the first conventional method, the structure of the neural network including the number of layers and the number of units of the neural network is set based on experience, and the classification function for classifying the target data is set as a case study. Done in.

Next, the second conventional method will be described.

The second conventional method has an initial classification function equivalent to the rough classification rule when the rough classification rule expressed by the comparison and product-sum logic that is considered to be almost correct for the target data is known. As described above, the structure of the neural network and the connection weight value are initialized, and after that, case learning using the learning sample group is performed by the inverse error propagation learning method, the connection weight value is adjusted, and the classification function is refined. (Japanese Patent Laid-Open No. 4-76660 "Neural Network Learning Method").

FIG. 11 shows a flowchart of the second conventional method.

Now, vector elements x1, x of the target data
The sample of the target data from 2, x3, x4 is the category y
Which of 1 and y2 it belongs to is classified. It is assumed that the following roughly correct rough classification rules are known. IF (x1 ≧ a1) or (x2 ≧ a2) THEN y ′ (3) IF (y′and (x3 ≧ x4) or (x4 ≧ a3) THEN y ′ (4) IF not y'or not (x4 ≧ a3 ) THEN y2 (5) (a1, a2, a3 are constant values) Rule (3) is “(x1 ≧ a1) or (x2 ≧ a
If it is 2), it means that the sample of the target data belongs to the subcategory y '. Rule (4) is “if the sample of the target data belongs to subcategory y ′ and (x3 ≧ x4) or (x4 ≧ a3), then the sample belongs to category y1”, rule (5) is “target data Of the sample does not belong to the sub-category y ′, or if (x4 ≧ a3), the sample belongs to the category y2 ”.

First, in the rule / logical formula conversion processing, these rules are converted into logical formulas with subcategories removed, and further converted into the following sum of products standard form logical formulas (step 31).

Y1 = ((x1 ≧ a1) * (x3 ≧ x4) + ((x2 ≧ a2) * (x3 ≧ x4) + (x4 ≧ a3) (6) y2 = (¬ (x1 ≧ a1) * ¬ (X2 ≧ a2)) + ¬ (x4 ≧ a3) (7) Here, ¬ indicates negative, and is realized by reversing the input polarity of the unit in the neural network.

Next, in the initial structure forming process, the structure of the neural network and the connection weight value are set in the following procedure according to this product-sum standard form logical expression (step 32).

An example of a neural network according to the second conventional method will be described below. FIG. 12 shows an example of a neural network according to the second conventional method.
The structure of the neural network 1 is, as shown in the figure, one input layer 1 for each variable appearing on the right side of the logical operation expression.
1 unit is assigned as an input unit 41, and one unit of the first intermediate layer 13 is assigned for each comparison operation to be a comparison unit 43. One second intermediate layer 1 is assigned for each multiplication term.
4 units are allocated, the units of the output layer 12 are allocated for each additive term, and the additive unit 42 is set. The method for determining the coupling load w is described in detail in Japanese Patent Laid-Open No. 4-76660, but here the calculation formula for setting is shown.

1. Additive Unit 42 (Output Layer 12) Consider an n-input additive unit. Input signal I_i, (1 ≦
i ≦ n) is −1 ≦ I _i“False” if ≦ −d, d ≦ I_i
In the case of ≦ 1], it is “true”. The output signal o is −1 ≦ o ≦
-False if d ', true if d'≤o≤1]
It Under the above conditions, to realize the additive function,
w_i(1 ≦ i ≦ n) and bias w₀Set as follows
To do.

[0022]

[Equation 2]

However, when the input is a negative term (y = a +
In b) of b, w _i = −w. Also, d is d>
It is necessary to satisfy the condition of (n-1) / (n + 1).

2. Multiplication unit 44 (second intermediate layer 14) n, d, d ′ are defined in the same manner as in the case of addition. In order to realize the multiplicative function, the coupling weight w _i (1 ≦ i ≦ n) and the bias w ₀ are set as follows.

[0025]

[Equation 3]

However, when the input is a negative term (y = a ·
In b) of b, w _i = −w. Also, d is d>
It is necessary to satisfy the condition of (n-1) / (n + 1).

3. Comparison unit 43 (first intermediate layer 13) The relation between the coupling load w ₁ and the bias w ₀ of the unit that outputs “true” when the input I is larger than the constant A is set as follows.

W ₀ = −w ₁ A (12) The input unit 41 is set so that the input is output as it is, as in the first method of the related art. Also,
There may be extra units and couplings between the units that do not correspond to the above formula, and these couplings are set to random values whose absolute values are sufficiently smaller than the coupling loads set above. Further, when the logical expression does not include the multiplication operation or the addition operation, the layer is omitted.

Next, a learning process is performed. In the learning process, the obtained neural network 1 is learned by the same inverse error propagation learning method as the conventional first method using a learning sample group, and the weighting value is adjusted to classify the classification function in detail. To convert. In this learning process, generally, the parameter for changing the connection weight and the number of times of learning of the same learning sample are set to be smaller than those in the case of the first conventional method (step 23).

Here, the rule in the second conventional method /
The logical expression conversion process (step 31) and the initial structure formation process (step 32) are referred to as a function embedded type formation process (step 30).

As described above, in the second conventional method, the structure of the neural network having the initial classification function is uniquely set, and the case learning is used only for refining the classification function.

[0032]

However, of the above-mentioned conventional methods of constructing a multilayered neural network, the first method is to set a structure consisting of the number of layers and the number of units of the neural network based on experience, Since all the classification functions are set by case learning, there is a problem that a large amount of learning processing time and man-hours are required to converge learning before obtaining appropriate classification accuracy. In particular,
Prepared in advance when the dimension number of the feature vector expressing the target data, the number of categories to be classified, the variation range of vector element values for each category, the space of feature vectors, etc. are large and the classification function to be set is complicated. There is a problem that the learning does not converge in the learning sample group of the target data in the range, and the expected classification function cannot be set in some cases.

On the other hand, the second conventional method is a method of simply constructing a multilayer structure type neural network when the rough classification rule of the classification target data is known. In this method, the structure of the neural network having the initial classification function is uniquely set, and the case learning is used only for refining the classification function. Therefore, compared to the first method, the learning processing time and man-hours are significantly reduced. However, when the space formed by the feature vectors is large and the classification function to be set is complicated, the classification function of the learning sample group of the target data in the range prepared in advance is set as in the case of the first conventional method. There is a problem that the learning for refinement does not converge and the classification accuracy may deteriorate rather.

Further, the number of layers of the multilayer structure type neural network which can set the function by the inverse error propagation learning method used for case learning in the above-mentioned first and second conventional methods is generally about four layers including the input layer. Yes, some new inventions are needed to easily construct more multilayer neural networks.

The present invention has been made in view of the above points, and solves the above-mentioned problems of the related art, and even if it is a multilayer structure type neural network of four or more layers having a complicated classification function, a small learning processing time is required. It can be easily realized by the number of man-hours, and further, a multilayer structure type neural network for classification processing when observing an object in various aspects such as shape and color can be easily configured by directly reflecting the logical nature of the classification processing. It is an object of the present invention to provide a method for constructing a multilayered neural network.

[0036]

FIG. 1 is a diagram for explaining the principle of the present invention.

When the rough classification rule of the hierarchical structure represented by the comparison and product-sum logic is known for the target data represented by the feature vector composed of a plurality of vector elements,
A method for constructing a multi-layered neural network for classifying target data, comprising: a multi-layered neural network as a whole including a plurality of partial neural networks and a connection neural network having outputs of the partial neural networks as inputs. In such a configuration, the vector space 101 formed by the feature vectors is divided into a plurality of subspaces 102 according to the type of vector element or the level of vector element according to the hierarchical structure of the rough classification rule.
In accordance with the vector space division processing 100 that divides into, the rough classification rule 103, the connection rule 104 corresponding to the mutual relation of the subspaces, and the subrule 105 corresponding to the subspaces.
And a step of the general rule sorting processing 200
The step of the learning sample group division processing 300 for dividing the learning sample group 106 of the target data into the partial sample groups 107 corresponding to the respective subspaces, the number of the partial spaces 102, the partial learning sample groups 107, and the original learning sample group 106.
Based on the relationship between the number of partial neural networks to be configured and the connection relationship between the partial neural network and the connection neural network, the overall configuration determination processing 400 steps is performed, and among the connection neural network 109 and the partial neural network 108, If there is a corresponding partial rule, connect rule 1
04 and the partial rule 105 according to the logical structure,
By combining the neuron units for which the logical functions of product and sum are set by the connection weight value,
For the step of the function-embedded forming process 500 for forming a neural network having an initial classification function equivalent to the connection rule 104 and the partial rule 105, and for a partial neural network for which no corresponding partial rule exists, connect neuron units. The step of random type formation processing 600 for installing a neural network in which weight values are set randomly is performed, and the learning processing 700 is performed for each partial neural network 108 using the corresponding partial sample group 107. , The connection neural network and the partial neural network are connected to assemble the entire multilayer structure type neural network, or the steps of the overall assembly processing 800 are performed, or the steps of the entire assembly processing 800 are performed, and thereafter, For each of the partial neural network, it performs the steps of the learning process 900 using the corresponding parts sample group 107.

[0038]

According to the present invention, when the rough classification rule of the hierarchical structure expressed by comparison and logical operation is known for the target data expressed by the feature vector composed of a plurality of vector elements, the target data is classified. The overall structure of the multilayer structure type neural network is configured to include a plurality of partial neural networks and a connection neural network that receives the outputs of the partial neural networks as inputs, in accordance with the hierarchical structure of the rough classification rule. This is a method of dividing the case learning so as to correspond to each of the partial neural networks by utilizing the fact that the initial classification rule can be set in the connected neural network by the method of. In this way, by dividing the space formed by the feature vectors according to the hierarchical structure of the rough classification rule, the classification function to be set for the whole of the multilayer neural network is divided according to its logical property, and the small-scale part is divided. It is assigned to a neural network and a connection neural network. At this time, since the same rule of the rough classification rule includes vector elements that are logically related to each other, the classification function logically organized is assigned to the partial neural network and the connection neural network. Further, corresponding to the divided space, a rough classification rule for initial structure setting for configuring these neural networks, and a learning sample group used for learning processing are divided into partial sample groups and uniquely assigned. .

As described above, since the learning process is performed by using the appropriate learning sample for each small neural network for which the logically integrated initial classification function is set, the learning process should be set. The function is greatly simplified, and the learning itself can be extremely easily converged.

[0040]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 2 shows an example of the multilayer structure type neural network of the present invention. The neural network shown in the figure includes an input layer 11, a connection neural network 51,
It is composed of partial neural networks 52 and 53. In the example of the same figure, the whole of the multilayer structure type neural network is composed of two partial neural networks 52 and 53 and one connection neural network which inputs these outputs. The number of networks can be expanded.

In the method of the present invention, the connection neural network 51 and the partial neural network 52,
As for 53, basically, the one in which the initial classification function is set by the function-embedded forming process 30 in the above-mentioned second conventional method is used, but a part of the partial neural network uses the above-mentioned first conventional method. It is possible to combine those whose functions are set only by case learning by the random type forming processing 20 in the method. In this example, the partial neural network 52 is a neural network in which the initial classification function is set and then the classification function is refined by case learning according to the above-described second conventional method, and the partial neural network 53 is described above. This is a neural network in which a classification function is set only by learning a case according to the first conventional method.

In the figure, the neuron unit 41 is an input unit, the neuron unit 42 is an addition unit, the neuron unit 43 is a comparison unit, and the neuron unit 44 is a multiplication unit.

<< First Embodiment >> FIG. 3 shows an example of target data when the classification processing of the first embodiment of the present invention is performed.
This figure shows an example of target data (vector element and learning sample group) represented by a feature vector composed of a plurality of vector elements that perform classification processing by the multilayer structure type neural network configured by the above method.

Here, a neural network for classifying whether or not a sample whose characteristics are given by two kinds of vector elements (characteristic elements) of "vertical width" x1 and "horizontal width" x2 is included in one kind of category y1. Configuration is a problem.

For the above target data, it is assumed that the following approximate classification rules having a substantially correct hierarchical structure expressed by comparison and sum-of-products logic are known.

IF vertical or horizontal THEN y1 (13) IF (vertical width> 0.8 and horizontal width <0.2) THEN vertical (14) Here, the rule (13) is the rule of the upper layer, the rule (1).
4) is a lower layer rule. The variable terms “vertical” and “horizontal” in the conditional part of rule (13) are subcategories, and the rule (13) defines a schematic logical relationship between these subcategories.

On the other hand, in the rule (14) of the lower hierarchy, the schematic logical relationship for classification is defined by using the vector elements "height" x1 and "width" x2 for only the subcategory "height" as variables (input). Is specified.

FIG. 4 shows a flow chart of a method for constructing a multilayer structure type neural network according to the first embodiment of the present invention.

Step 1) First, according to the hierarchical composition of the rough classification rule, the vector space division processing 60 for dividing the space formed by the feature vectors into a plurality of subspaces for each type of vector element or each level of vector element. I do.
In the present embodiment, two subcategories relating to the levels of vector elements “vertical” and “horizontal” are used as variables in the upper layer rule (13), and one of these subcategories is set in the lower layer rule (14). Gives the rules regarding. Therefore, as shown in FIG. 3, the space formed by the feature vectors is divided into two subspaces 72 for each level of vector elements.
Divide into 73. The method of dividing the vector space is reflected in the general rule sorting processing 61 to the overall configuration determination processing 63 described below.

Step 2) Set the rough classification rule to the subspace 7
Subspace 7 and connection rules corresponding to the mutual relationship of 2, 73
A general rule sorting process 61 is performed for sorting into partial rules corresponding to Nos. 2 and 73. In this embodiment, the rule (13) is classified into a connection rule and the rule (14) into a partial rule.

Step 3) A learning sample group division process 62 for dividing the learning sample group of the target data into corresponding partial sample groups of each subspace is performed. In this embodiment, as shown by the target data in FIG. 3, the learning sample group 71 corresponding to the space formed by the feature vectors is divided into two partial sample groups 72 and 73 corresponding to the two divided partial spaces. To divide.

Step 4) An overall procedure for determining the number of partial neural networks to be constructed based on the number of subspaces and the relationship between the partial sample group and the original learning sample group, and the connection relationship between the partial neural network and the connection neural network. The configuration determination process 63 is performed. In the present embodiment, as shown in FIG. 5, two partial neural networks 52 and 53 having the same input layer 11 are provided, and these outputs are used as the input of the connection neural network 51.

Step 5) Forming process 81 of connection neural network 51, partial neural network 5
The formation processes 82 and 83 of 2 and 53 are performed.

Here, the connection neural network 5
1 and the partial neural networks 52 and 53 having corresponding partial rules, the respective logical functions of comparison, product, and sum are set in advance by the connection weight value according to the logical structure of the connection rule and the partial rule. By combining certain neuron units, a function-embedded forming process 30 for forming a neural network having an initial classification function equivalent to the connection rule and the partial rule is performed. In this embodiment, since the connection rule (13) corresponds to the connection neural network 51 and the partial rule (14) corresponds to the partial neural network 52, this processing is performed for these neural networks.

For a partial neural network that does not have a corresponding partial rule, a random type forming process 20 for installing a neural network in which the connection weight values of the neuron units are randomly set is performed. In this embodiment, the partial neural network 53
This processing is performed because there is no corresponding partial rule for.

Next, in this embodiment, in the partial neural network forming processes 82 and 83, the learning process 23 of each of the partial neural networks 52 and 53 is individually performed using the corresponding partial sample groups 72 and 73.
Here, the learning process 23 for the partial neural network 52 is a case learning for detailing the classification function initialized in the function embedded type forming process 30, and the learning process 23 for the partial neural network 53.
Is for setting the classification function only in the case learning to the neural network in which only the structure is set in the random type forming process 20.

The present embodiment is an example in which the space formed by the feature vectors is divided into two subspaces according to the levels of vector elements in the vector space division processing 60, and these are created by the learning sample group division processing 62. In the partial sample groups 72 and 73 included in the partial space of, as shown in FIG. 3, information of the category y1 is uniquely added to each partial sample group. That is, it is possible to regard the category y1 as a subcategory. Therefore, by using these partial sample groups to which the category y1 is added,
It is possible to learn partial neural networks individually.

Step 6) Thereafter, the connection neural network 51 and the partial neural network 52,
Multilayer structure type neural network 1 by connecting with 53
The whole assembling process 64 for assembling the whole is performed.

With the above processing, in the multilayer structure type neural network 1 shown in FIG. 5, the initial classification function is set in the connection neural network 51, and the detailed classification function by case learning is set in the partial neural networks 52 and 53. It is composed of

Step 7) Next, in the additional learning process 84, the learning process 23 for detailing the connection neural network 51 connected to the whole of the multilayer structure neural network assembled by using the learning sample group 71 is performed. . It should be noted that this additional learning process 84 can be performed for the entire multilayer structure type neural network 1, and can be omitted if the connection neural network 51 has a simple classification function. .

With the above processing, the construction process of the multilayer structure type neural network 1 is completed.

As described above, in the present embodiment, it is possible to individually carry out case learning when constructing a multilayer structure type neural network for each of the partial neural network and the connection neural network which are its constituent elements. Can be easily converged.

<< Second Embodiment >> Next, a second embodiment of the present invention will be described.

The present embodiment is an example in which the space formed by the feature vectors expressing the target data is divided according to the types of vector elements. The first embodiment described above is performed for a connection neural network and a partial neural network. The order of performing the learning process 23 and the overall assembling process 64 is different.

FIG. 6 shows an example of target data when the classification processing of the second embodiment of the present invention is performed. This figure shows an example (vector element and learning sample group) of target data represented by a feature vector composed of a plurality of vector elements that perform classification processing by the multilayer structure type neural network configured in the present embodiment. For the vector element (feature element), "vertical width"
x1, "width" x2, "hue" x3, "lightness" x4 4
There are types. Further, there are y1 and y2 as categories to which the samples are classified. Here, there are 2 samples whose features are given by these four types of vector elements (feature elements).
The problem is to construct a neural network that classifies whether or not the type is included in the categories y1 and y2.

Regarding the above target data, it is assumed that the following general rule of the hierarchical structure represented by the comparison and product-sum logic is known.

IF (vertical and (light blue or purple)) THEN y1 (15) IF ((vertical or horizontal) and purple) THEN y2 (16) IF (vertical> 0.8 and horizontal <0.2) THEN vertical ( 17) IF (length <0.2 and width> 0.8) THEN width (18) Here, rules (15) and (16) are general classification rules of the upper hierarchy, and rules (17) and (18) are It is a rough classification rule of the lower hierarchy. The variable terms “vertical” and “horizontal” in the condition part of the upper layer rules (15) and (16) are subcategories, and the vector element “vertical width” is set for these subcategories according to the lower layer rules (17) and (18). "X1" and "width" x
A general logical relationship for classification with 2 as a variable (input) is defined. On the other hand, the condition parts of the upper layer rules (15) and (16) include the variable terms “light blue” and “purple”. These variable terms are vector elements "hue" x3
Is a subcategory defined by “lightness” x4, but there is no general classification rule of the lower hierarchy that defines these relationships.

FIG. 7 shows a multi-layer structure type neural network constructed in the second embodiment of the present invention. The neural network shown in the figure comprises partial neural networks 52 and 53 having an input layer 11 and a connection neural network 51. In this neural network, the partial neural network 5
The vertical width x1 and the horizontal width x2 are input to the second input layer 11,
It is compared in the comparison unit 43, multiplied in the multiplication unit 44, added in the addition unit 42, and output to the connection neural network 51. Further, the hue x3 and the brightness x4 are input to the input layer 11 of the partial neural network 53, and the other parts are the same as those of the partial neural network 52.

In the connection neural network 51, the outputs from the partial neural networks 52 and 53 are input to the multiplication unit 44, and the addition unit 42 outputs the classification results y1 and y2.

FIG. 8 is a flow chart of the method of constructing the multilayer structure type neural network of the second embodiment of the present invention.

The specific procedure will be described below with reference to these figures.

(Step 100) First, the vector space division processing 60 is performed. In this embodiment, the rule (1
In 5) and (16), the subcategories related to the shapes “vertical” and “horizontal” and the unknown subcategories related to the colors “light blue” and “purple” are used as variables. In addition, rules (17) and (18) in the lower hierarchy give rules for two subcategories regarding shapes. Therefore, as shown in FIG. 6, the space formed by the feature vectors is divided into two subspaces x1 and x2 regarding the shape and xx3 and x4 regarding the color according to the vector element type. This way of dividing the vector space is reflected in the general rule sorting process 61 to the overall configuration determining process 63 described below, as in the case of the first embodiment.

Step 101) General rule sorting processing 61
Then, rules (15) and (16), rules (17) and (1
8) is sorted as it is into a connection rule and a partial rule.

Step 102) In the learning sample group division processing 62, as shown in FIG. 6, the learning sample group 71 corresponding to the space formed by the feature vectors is divided into two corresponding to the partial space.
It is divided into individual partial sample groups 72 and 73.

Step 103) In the overall configuration determining process 63, as shown in FIG. 7, two partial neural networks 52 and 53 each having its own input layer 11 are provided, and these outputs are connected to the connection neural network 51. Input it.

Step 104) Next, the formation processing 81 of the connection neural network 51 and the formation processing 82 and 83 of the partial neural networks 52 and 53 are performed. Here, the function-embedded formation processing 3 is performed for all of the connection neural network and the partial neural network.
0 or only the random type forming process 20 is performed. The random type forming process is performed on the partial neural network 53 for which the corresponding partial rule does not exist.

Step 105) Next, before carrying out the learning process 23, an entire assembling process 64 for assembling the whole of the multilayer structure type neural network 1 by connecting the connection neural network 51 and the partial neural networks 52 and 53. To do.

Step 106) Partial sample groups corresponding to the two sets of the partial neural network 52 and the connection neural network 51, and the partial neural network 53 and the connection neural network 51 included in the whole of the assembled multilayer structure type neural network 1. The learning process 23 is simultaneously and synchronously performed by using 72 and 73. Here, the learning process 23 for the set including the partial neural network 52 is a case learning for refining the classification function of the partial neural network and the connection neural network 51, and the learning process 23 for the set including the partial neural network 53. Has two meanings: to set the classification function in the partial neural network 53 in which only the structure is set in the random type formation processing 20 by only case learning, and to make the connection neural network 51 detailed.

In the present embodiment, the reason for taking the above learning processing procedure is that the space formed by the feature vectors is divided into two subspaces according to the vector element type in the vector space division processing 60. As shown in 6, the partial sample groups 72 and 73 included in these partial spaces created by the learning sample group dividing process 62 are assigned to the category y1.
This is because the information of y2 and y2 cannot be added individually. That is, when the space formed by the feature vectors is divided according to the types of vector elements, the categories y1 and y2 cannot be regarded as subcategories. Therefore, the learning process 23
Is performed using the partial sample groups 72 and 73 to which the categories y1 and y2 are added while the connection neural network 51 is connected to the individual partial neural networks.

In the present embodiment, the learning processing of the connected neural network is not included in the above, and thus additional learning is not necessary.
Is completed.

Also in the second embodiment described above, the learning process 2 is performed.
When performing 3, the initial classification function is set in the connected neural network, and each partial neural network performs case learning using the corresponding partial sample group. Therefore, as in the case of separating and learning each of the partial neural networks described in the first embodiment, by dividing the case learning, the convergence of learning can be facilitated.

In the first and second embodiments, the case where the number of partial neural networks is two and the number of connected neural networks is one has been described. However, in general, the number of partial neural networks should be expanded. In a multilayer structure type neural network in which small-scale neural networks are further combined in multiple stages, the relationship between the connection neural network and the partial neural network described in the present embodiment is repeatedly applied to form the whole, and a function-embedded forming process is performed. It is possible to combine the random type formation processing with the same partial neural network or the same connection neural network.

[0084]

As described above, according to the present invention, it is possible to facilitate the convergence of the case learning by dividing the case learning.
Even a multi-layered neural network having four or more layers having a complicated classification function, which has not been realized by the conventional technique, can be easily realized with a short learning processing time and man-hours.

Further, in each of the partial neural networks, a classification function can be classified and set according to the nature of the target data reflected in the type of vector element (feature element) and the level of element value. Therefore, it is possible to obtain an effect that a multilayer structure type neural network for classification processing when observing an object from various aspects such as shape and color can be easily configured by directly reflecting the logical nature of the classification processing.

[Brief description of drawings]

FIG. 1 is a diagram for explaining the principle of the present invention.

FIG. 2 is a diagram showing an example of a multilayer structure type neural network of the present invention.

FIG. 3 is a diagram showing an example of target data when performing a classification process according to the first embodiment of this invention.

FIG. 4 is a flow chart of a method for constructing a multilayer structure type neural network according to the first embodiment of the present invention.

FIG. 5 is a diagram showing a multilayered neural network constructed in the first embodiment of the present invention.

FIG. 6 is a diagram showing an example of target data when performing a classification process according to a second embodiment of the present invention.

FIG. 7 is a diagram showing a multi-layered neural network constructed in a second embodiment of the present invention.

FIG. 8 is a flow chart of a method of constructing a multilayer structure type neural network according to a second embodiment of the present invention.

FIG. 9 is a diagram showing a configuration example of a multilayer structure type neural network.

FIG. 10 is a flowchart of a first conventional method.

FIG. 11 is a flowchart of a second conventional method.

FIG. 12 is a diagram showing an example of a neural network according to a second conventional method.

[Explanation of symbols]

 1 Neural Network 10 Neurons 11 Input Layer 12 Output Layer 13 Intermediate Layer 41 Input Layer Unit 42 Additive Unit 43 Comparison Unit 44 Multiplication Unit 51 Connection Neural Network 52, 53 Partial Neural Network 60 Vector Space Division Processing 61 General Rule Sorting Processing 62 Learning Samples Group division processing 63 Overall configuration determination processing 64 Overall assembly processing 71 Learning sample groups 72, 73 Partial sample groups 81 Connection neural network formation processing 82, 83 Partial neural network formation processing 84 Additional learning processing 100 Vector space division processing 101 Vector space 102 subspace 103 rough classification rule 104 connection rule 105 partial rule 106 learning sample group 107 partial sample group 108 partial neural network Over click 109 connected neural network 200 schematically rules sorting process 300 the learning sample group division process 400 overall configuration determination process 500 functions embedded form process 600 randomly-type forming process 700 learning process 800 whole assembled process 900 learning process

Claims (1)

[Claims] 1. When a rough classification rule of a hierarchical structure represented by comparison and product-sum logic is known for target data represented by a feature vector composed of a plurality of vector elements, the target data is classified. A method for constructing a multi-layered neural network, wherein the whole of the multi-layered neural network is configured to include a plurality of partial neural networks and a connection neural network whose output is an input of the partial neural networks. In this case, according to the hierarchical structure of the general classification rule, in accordance with a vector space division process for dividing the space formed by the feature vector into a plurality of subspaces according to the type of vector element or the level of the vector element, The rough classification rule corresponds to the connection rule corresponding to the mutual relation of the subspaces and the subspace. A general rule sorting process for sorting into a rule and a learning sample group splitting process for splitting a learning sample group of the target data into partial sample groups corresponding to each of the subspaces; Performing an overall configuration determination processing step of determining the number of partial neural networks to be configured based on the relationship between the partial sample group and the original learning sample group, and the connection relationship between the partial neural network and the connection neural network, For the connection neural network and the one having the corresponding partial rule among the partial neural networks, according to the logical structure of the connection rule and the partial rule,
A function-embedded type that forms a neural network having an initial classification function equivalent to the connection rule and the partial rule by combining neuron units in which logical functions of comparison, product, and sum are set in advance by connection weight values. The step of formation processing and the step of random formation processing of installing a neural network in which the connection weight value of the neuron unit is randomly set are performed for the partial neural network having no corresponding partial rule. A learning process is performed on each of the partial neural networks using the corresponding partial sample group, and thereafter, the connection neural network and the partial neural network are connected to assemble the entire multilayer structure type neural network. Overall assembly Or a step of the whole assembling process, and thereafter, a learning process is performed for each of the partial neural networks by using the corresponding partial sample group. Method for constructing multi-layered neural network.