JPH03263167A - Setting processing system for network structure in neural network - Google Patents

Abstract

PURPOSE:To simply describe the repeated coupling even if notch width of two layers is different by designating independently repeated notch width of a fundamental unit in each of two layers to be coupled in a setting expressing of the repeated coupling. CONSTITUTION:The system is provided with an input file 1, a network data setting processing part 2, a temporary file 3, and an output file 4. In such a state, a setting expression of a fundamental unit coupling in a setting expression of a network structure designates a symbol name of the fundamental unit cou pling, and a head position of each unit range and width of the range of two layers for executing a coupling, and a setting expression of a repeated coupling designates a symbol name of the fundamental unit coupling for executing a repetition, and a start point and an end point of a repetition in one layer, notch width of a repetition and a notch mode in both layers. In such a way, even in the case the repeated notch width of the fundamental unit coupling in two layers to be coupled or the same layer is different, the network structure can be set easily.

Description

[Detailed Description of the Invention] [Summary] This invention relates to a processing method for efficiently setting a network structure in a neural network.

Even if the repetition step widths of the basic unit bonds in two or the same layers to be bonded are different.

The purpose is to make the network structure easily configurable.

Basic unit combination operation section that stores basic unit combination data, such as the number of each unit in the unit range specified as 9 basic units, in a temporary file according to the basic unit combination setting formula, and settings for repeating combinations. Depending on the formula, call the basic unit combination data specified for repetition from the temporary file 2
A repeating join operation section that creates network data for repeating connections according to specified repeating conditions and stores them in an output file, and a repeating join operation section that creates network data for repeated connections according to the specified repetition conditions and stores them in an output file. It is equipped with a control unit that operates a corresponding one of the operation section and the repeating connection operation section, and among the setting expressions for the network structure, the setting expression for the repeating connection is the identification name of the basic unit connection and the identification name of the two or the same unit to be connected. The width of the repetition increments in each layer is specified.

[Industrial application field]

The present invention relates to a processing method for efficiently setting up a network structure in a two-neural network.

Neural network (neural circuit!i1) is.

Each of the plurality of units (corresponding to neurons) and the other plurality of units have a structure having predetermined connections.

As the scale of a neural network increases and the number of connections between units increases, the description of the network structure becomes complex and enormous. The present invention is.

We propose a means for configuring such a network structure in a flexible and simple manner.''

[Technical background] A neural network is an imitation of the neural network in the brain. What corresponds to each nerve cell is uni-
shall be They have a structure of several layers,
A neural network is formed by connections between units between the eyebrows and units on the same layer. Then, it performs various movements by learning like humans.

In recent years, using the concept of neural networks,
Cases of pattern recognition and automatic control are increasing. Neural networks.

It has a simple structure and is easy to learn. Therefore.

It is expected to be applied in fields where control has been difficult using conventional programming methods.

The basic models of neural networks are the Berceptron model and the Hopfield model.

The Bersebtron model has a hierarchical structure as illustrated in FIG. 9, and multiple connections are made between units within one hierarchy.

The Hopfield model has a structure in which each unit is interconnected, as illustrated in FIG. 1O.

The basic model of neural networks is detailed in the following literature.

David E., Rumelhart, James
L., McClelland, et al.

”Parallel Distributed Pr
cessing Explorations in
the Microstructure of C
ignition” published by the MIT press (
(1986) models have been published in addition to this document, and their structures often differ depending on the purpose and target. Furthermore, there are many different learning methods for neural networks. From this point of view, the network structure of neural networks is truly unique.

Then, a problem arises. It suits each purpose and use. It is a means of building a network.

Conventionally, in order to set the network connections of a neural network, it was necessary to set the corresponding units one by one. This is because there are three problems. The first is that the way neural networks are connected is complicated.

Second, when the neural network model changes, the connections must change accordingly; 3.
The first is that the connections in neural networks involve a lot of repetition.

However, when building such a complex network,
The user has to set the positions of the connected units one by one, which is very time consuming.

[Conventional technology]

In the above-mentioned berceptron model, the positions of the units in each layer are arranged as 1 to 3 as shown in FIGS. 11(a) to 11(C).
Represented in a dimensional coordinate system.

In the conventional network structure setting method, neural
This is achieved by defining one connection between the eyebrows of the network as a basic unit connection and repeating its definition. The main terms used here will be explained below.

A "basic unit connection" is a total connection from a unit in a certain area of a certain layer to a unit in a certain area of that layer or a different layer.

``Fully connected'' refers to a state in which all units are all connected, as shown in FIG.

“Repetition °” means that the basic unit bond is the starting point, the ending point,
This is repeated by giving parameters for step width and step mode.

The "starting point" is the coordinate of the first unit position in the range of basic unit bonds at the beginning of the repetition (see Figure 11 (b)), and the "end point" is the coordinate of the position of the first unit in the range of basic unit bonds at the beginning of the repetition. These are the coordinates of the position of the first unit 7 (see FIG. 11(b)).

The "step width °" indicates the interval between the coordinates of repeating positions in which the unit range of the basic unit combination is the unit.

“'Step mode°” means that in the case of repetition, you can either repeat the step width in the x and y directions between one beat point and the end point (Figure 13), or use the x and y values of the step width as components of hectors. This is a flag indicating whether to repeat at a fixed position (FIG. 14). Here, "filling" means copying the basic unit combination.

In traditional network configuration, a user configures connections using a network description language that is automatically translated into an internal representation of the network.

It was done in the following steps.

First, let the rectangle indicating the range of the basic unit be the mouth (X, Y, XX, YY), let (X, Y) be the coordinate position, and (xx, yy) be the width in the horizontal and vertical directions. Then, there is a quadrilateral opening (XI
, Yl, XXI, YYI)
．． XX2. If it consists of a bond with YY2), the basic unit bond B = (mouth (XI, Yl, XXI, YYI)
It can be expressed as (X2.Y2.XX2.YY2) (D bond). For example, when XX2=YY2=1.

Each unit becomes a many-to-l combination.

Next, the repeating combinations shown in Figures 13 and 14 are created using the basic unit B: CB, (XS, VS), (XL, YL), (M
XMY), MODE). Here, (XS, YS) is the starting point (I/), and (XL, YL) is the ending point. (MX, M
Y) is the step width, and MODE is the step mode. The repeating pattern is the basic unit bond B.

(Problems to be Solved by the Invention) Networks have conventionally been described using one or more ways of thinking.However, there are problems with this method.

For example, consider the case of network connection as shown in FIG. In this figure, both the L layer and the R layer are shown in one dimension. In the connections in FIG. 15, the basic unit connections in both (a) and (b) are two L layer units and one RIW unit. In FIG. 15(a), the step width of both the Lli and R layers is 2, but in FIG. 15(b), the step width of the 1L layer is 2. The step convergence of R1'i is 1.

FIG. 15(b) can be described using the conventional setting method as follows.

That is, the setting formulas for the basic unit bond and repeat bond described above are for one-dimensional representation.

Basic unit combination B-[mouth (XLXXI) and mouth (X2.
Since this is a repeated combination (B, XS, XL, MX, MODE), the example shown in FIG. 15(a) is written as the following equation.

B-(combination of mouth (0,2) and mouth (0,1)) (B,
(0), (5), (2), fill in) However, in the example of Fig. 15(b), since the repetition step width is different between the L layer and the R layer, the above-mentioned setting formula for the repetition connection is used. It was not possible to describe both of them, so it was necessary to describe them separately. Therefore, the amount of description of the network structure increases,
Network configuration requires large memory capacity.

The problem was that the work load associated with changes was large and errors were likely to occur.

An object of the present invention is to make it possible to easily set up a network structure even when the repetition step widths of basic unit connections in two layers to be connected are different.

(Means for Solving the Problems) The present invention, in describing the network structure of a neural network, independently specifies the repetition step width of the basic unit in each of the two layers to be combined in the setting formula for the repetition combination. This makes it possible to easily describe repeated connections even if the step widths of the two layers are different.

Here, the unit will be explained using two-dimensional x and y coordinates.

First, a rectangle indicating the range of the basic unit is defined as the mouth (X, Y, XX, YY), and (XY) is the coordinate value L. (XX, YY) is the width in the horizontal and vertical directions.

Then, a certain basic unit bond B has a rectangular opening (X
I', Yl, XXI, YYl) and square mouth with R layer (
X2. Y2. XX2, YY2), the basic unit bond B- [The opening in the L layer (Xi, Yl, XXI, YYI) and the opening in the R layer (X2.Y2.XX2.YY2) 17+ bond) It can be expressed as For example, when XX2=YY2=1.

Each unit becomes a many-to-one combination.

Next, use the basic unit B for repeating bonds as shown in Figure 6.

CB, (XS, YS), (XL, YL).

(MXI, MYI), (MX2.MY2)MODE
) can be written as Here, (XS, YS) is the starting point and (XL, YL) is the ending point. (MXI.

MYI) is the rectangular step width of the L layer (MX2.
MY2) is the rectangular step width of the R layer, and MODE
It is in increments mode. The repeating pattern is the basic unit bond B. Also.

The square convergence (MX2.MY2) of the R layer is.

It can be omitted, in which case the width of the rectangular increments of the R layer is
It is equal to the rectangular step width (MXI, MYl) of the L layer. The network will be described using the above ideas.

FIG. 1A is a diagram showing the basic configuration of the present invention.

In the figure, l is an input file in which a setting formula for a network structure written using a network description language is stored. The setting formula is.

Basic unit connection setting formula B = [combination of rectangular opening in 1st layer and rectangular opening in 2nd layer] Includes repeating connection setting formula (repetition of B, starting point, ending point, step width 5 modes). Here, °'B°" represents the identification name (name) of the basic unit bond.

Reference numeral 2 denotes a network data setting processing unit which converts a repeating connection setting formula for a network structure described in a network description language into an expression format for non-repetitive connections. In other words, the basic unit connections are repeatedly copied and expanded into network structure data so that they correspond to the actual connections.

3 is a -time file and is used as one work file.

4 is an output file in which network structure data converted by the network data setting processing unit 2 is stored;
Make it available for use in neural network processing.

Reference numeral 5 denotes a reading unit, which performs a process of reading the setting formula of the network structure from the input file l.

Reference numeral 6 denotes an interpreter which determines whether the read setting formula is a basic unit combination setting formula or a repeating combination setting formula.

Reference numeral 7 denotes a control section, which operates a corresponding one of a basic unit connection operation section and a repeat connection operation section, which will be described later, according to the result determined by the interpretation section 6.

Reference numeral 8 denotes a basic unit connection operation section, which stores the unit 7) data in the temporary file 3 when the basic unit connection setting formula is read.

Reference numeral 9 denotes a repeating combination operation section which, when a repeated combination setting formula is read, retrieves the unit data of the preceding basic unit combination setting formula from the temporary file 3 using the identification name (B) of the repeated basic unit combination. , the starting point specified as the repetition condition, P: point, increment width 5 Repeatedly copy the unit data of the basic unit combination according to whether the increment mode is °'' filling space or "hector" 5
Create network structure data.

10 is a writing unit which writes the network structure data created in 1 to the output file 4;

A neural network processing unit (not shown) is
Read the network structure data of output file 4,
Based on this, a neural network is set up and the process is executed.

[For production] A detailed explanation of the present invention will be given with reference to FIG. 1B.

FIG. 1B is an example in which repeated connections with different step widths are performed between the LJi and R layers of the neural network.

According to the present invention, the setting formula for the repeating bond is the starting point (XS, YS) of the basic unit repeating of LJi in FIG. 1B.
and the end point (XL, YL), and the step width in the X direction MXI
and the step width MYI in the Y direction, and further specify the step width MX2 in the X direction and the step width MY2 in the Y direction of the basic unit repetition of the R layer.

The basic unit combination of repetition is LJli and R layer unit range a0° and bo. It is set about.

Therefore, the stepwise convergence between LH and Rli is different, and MXl
f-MX2. Even if MYI≠MY2, the iterative connection can be described using a single iterative connection setting formula, and the network expression can be simplified.

When converting to network structure data, in the “°fill” mode, increments MXI and MYI are used in LN.
Copy aoO to aO1+ alo+ al l,
In the R layer, the increments are MX2. This is MY2. . bob
,b,. + By copying to bll.

Can create network structure data. In addition.

In the "°vector°" mode, it is only necessary to copy to +all.

[Example] The second embodiment of the network data setting process according to the embodiment of the present invention
This will be explained using the flowchart shown in the figure.

■ Set the dimension in which to enter the coordinates of each layer's units.

■ Read one line from the input file.

■ If the first character is "a"°, this is an expression that sets the configuration of unit number B, which is the basis of basic unit combination. Their names, X to the array.

y, size in 4 directions (if there is no value in 1 or 2 dimensions - ○)
Store. Go to ■.

■ If the first character is "b"°, check whether the expression contains -. If there is −°゛, go to ■. If not, go to ■.

■ If the first character is ° “e”, it is an end mark. Finish the process.

■ It is an expression that sets the basic unit bond. If a is included, the value of that 'a°° is extracted from array A.

The unit numbers on the input side and output side are respectively calculated from the coordinate values, and the five results are stored in array B. Array B stores the combination of unit numbers to be connected on the name 1 input and output side of the basic unit connection. Go to ■.

■ Otherwise, it is a repeatable description type. Read the unit number of the basic unit combination from the previously set array B. Starting point by step mode.

Calculate the unit number to be combined from the end point and step width values and store it in the output file. If there is only one step width, each layer uses that step width. Go to ■.

Next, explanation will be given using a specific example.

An example of the format of the input file l is shown in FIG. The numbers in parentheses below indicate the formulas in FIG.

Equation (1) defines the range of the unit as a rectangle.

That is, a quadrilateral is defined in advance as "ao", and its size is x=4. It represents that y=5. (1) The first character "a" in the formula indicates a formula for setting a rectangle. Fourth
The figure shows the range of units defined by this equation (1).

Equation (2) sets the basic unit bond BO. 1L means 1 layer, and 2L means 2N. Also, ``1O11'' represents the coordinate values 10 and 11, and ``1012°'' represents 10
~12 (=10.11.12). In other words, in the case of equation (2), the basic unit bond BO, as shown in FIG.
20, 10) represents that it is combined with the unit within the range indicated by the rectangle (2, 3).

Equations (3) and (4) show repeating bonds of basic unit bonds BO. Equations (3)' and (4)', which will be given later, are forms in which the parentheses of equations (3) and (4) are omitted, respectively. As shown in Figure 6, equation (3) is for the case where the step mode is "fill" (MODE=O), and equation (4) is,
As shown in FIG. 7, the case where the step mode is "vector" (MODE=1) is shown. This uses the basic unit bond BO, and the starting point in one layer is (10, 10) and the ending point (3
0,40); This is a description of repeated bonding when the step width is (10°15) and the step width of two layers is (10,10).

Basic unit bond BO = [Opening in layer 1 (10, 10, 4, 5) and opening in layer 2 (2
0, to, 2. 3) combination)...(2)
' [BO, (10, 10), (30, 40).

(10,15L (10,10), O)...
(3)'(BO, (10,10), (30,40)(1
0,15), (10,10), l)・・・・・・(
4)'Finally, an example of the format of the output file 11 is shown in FIG.

Here, the units to be combined and the serial numbers of the units from layer 0 to the final layer are written. In this example, unit number 0 is unit number 100, 101, 102.
Unit number 100 is combined with unit number 20.
This shows that it is connected to numbers 0, 201, 202, and so on.

(Effects of the Invention) By using the network structure setting method for neural networks according to the present invention, it is possible to easily set the connections of complex networks with different step widths, and it is also possible to easily change the model, thereby improving efficiency and reliability. It can improve performance and economy.

[Brief explanation of drawings]

Fig. 1A is a diagram showing the basic configuration of the present invention, Fig. 1B is an explanatory diagram of an example of repeating connections when the step width is different, Fig. 2 is a flowchart of the network setting process according to the embodiment of the present invention, and Fig. 3 is Figure 4 is an explanatory diagram of an example of the format of an input file. Figure 4 is an explanatory diagram of an example of a range definition. Figure 5 is an explanatory diagram of an example of basic unit combination. Figure 6 is an example of repeated combination in "Fill" mode. Fig. 7 is an explanatory diagram of an example of repeating combination in "'Hector゛° mode," Fig. 8 is an explanatory diagram of an example of the output file format, Fig. 9 is an explanatory diagram of an example of the Berceptron model, and Fig. 10 The figure is an explanatory diagram of an example of the Hopfield model, Fig. 11 is an explanatory diagram of the coordinate system of the unit position, Fig. 12 is an explanatory diagram of an example of fully connected, and Fig. 13 is a repeated description of "increment mode - fill". FIG. 14 is an explanatory diagram of an example of repeated description of "' step mode - hector °", and FIG. 15 is an explanatory diagram of an example of step width. i in FIG. 1A: Input file 2: Non-work data setting processing section 3; -time file 4: Output file 5: Reading section 6; Interpretation section 7; Control section 8: Basic unit combination operation section 9: Repeated combination operation section IO: Writing section

Claims (1)

[Claims] In a neural network processing system, an input file storing a network structure setting formula including a basic unit connection setting formula and a repeating connection setting formula, and reading the network structure setting formula from the input file. a reading section, an interpretation section that interprets the loaded setting formula and determines whether the setting formula is for a basic unit bond or a repeating bond, and a A basic unit combination operation section that stores basic unit combination data such as the number of each unit in the unit range in a temporary file, and basic unit combination data whose repetition is specified according to the repetition combination setting formula. is called from a temporary file,
A repeating join operation section that creates network data for repeating connections according to specified repeating conditions and stores them in an output file; It is equipped with a control unit that operates a corresponding one of a combination operation section and a repeating combination operation section, and among the above network structure setting formulas, the basic unit combination setting formula includes the identification name of the basic unit combination and the two names that perform the combination. Specify the start position and range width of each unit range in the layer, and the repeating combination setting formula is the identification name of the basic unit combination that performs the repetition, the starting point and end point of the repetition in one layer, and the setting formula for the repeating connection. 1. A method for setting a network structure in a neural network, characterized in that it specifies a step width of repetition and a step mode in a neural network.