JP5682403B2 - Neural network design method and program - Google Patents

Description

  The present invention relates to a neural network design method for designing a neural network applicable to a physical model, and a program for causing a computer to design a neural network. The present invention also relates to a computer-readable storage medium storing such a program.

  Recurrent Neural Network (RNN) is a neural network developed for application to robot motions, but not only robot motions, but also various electronic devices such as pedometers and voice analysis devices, etc. It can be applied to the physical model. The RNN itself is proposed in, for example, Patent Document 1, Non-Patent Document 1, and the like.

The RNN is a network having connections with neurons, and the relational expression between the neurons and input / output connections is given as shown in FIG. In FIG. 1, ε _i is a delay parameter, y _i and y _j are neuron state quantities, C _ij is a weighting factor, t is time, and y _i representing neuron state quantities may be identified with neurons. .

As an extension of the relational expression in FIG. 1, a relational expression as shown in FIG. Figure 2 shows that the configuration was added value g _i of FIG. 1 is input to the neuron y _i, the value g _i is a value obtained by non-neuronal the RNN circuit.

FIG. 3 is a diagram illustrating an example of an RNN circuit having two neurons and four connections. In FIG. 3, ε _i = 1, and “1” and “−1” attached to the connection represent weighting factors. The right side of FIG. 3 represents a differential equation of this RNN circuit, and when this differential equation is solved, x ₁ = sin (t) and y ₁ = cos (t) are obtained.

FIG. 4 is a diagram illustrating an example of bilinear coupling. The bilinear coupling in FIG. 4 is an example of a coupling other than those in FIGS. 1 and 2. As shown in FIG. 4, the bilinear combination represents multiplication, and the result of multiplying the output of “y ₁ ” and the output of “y ₂ ” is input to “y ₃ ”. ε ₃ is a delay parameter. Further, when the result of multiplying the output of “y ₁ ” and the output of “y ₂ ” is output to the outside without being input to the neuron, it is defined as “bilinear output” to distinguish it from bilinear coupling. Since the bilinear combination is multiplication, it can be used to obtain the distance (or the square of the vector distance) and inner product of vectors output from sensors such as an acceleration sensor.

FIG. 5 is a diagram illustrating an example of obtaining the inner product. In the example of FIG. Two RNN circuits shown in FIG. 3 are provided, and two vectors (x ₁ , y ₁ ) = (sin (t), cos (t)), (x ₂ , y ₂ ) = (cos (t), sin ( t)) is output. By combining these vector outputs with a bilinear combination, the output of neuron “i” is the inner product of two vectors i = x ₁ · x ₂ + y ₁ · y ₂ = 2sin (t) · cos (t) Can be obtained.

  Further, by using bilinear coupling, more various outputs can be obtained. FIG. 6 is a diagram illustrating an example of an RNN circuit having two bilinear couplings proposed in, for example, Non-Patent Document 1. The RNN circuit shown in FIG. 6 represents Lorenz's Equations and can obtain chaos.

  FIG. 7 is a diagram for explaining the relationship between an RNN circuit having bilinear coupling and an RNN circuit not having bilinear coupling. An RNN circuit having a bilinear coupling can obtain a wider variety of outputs than an RNN circuit having no bilinear coupling.

  However, an RNN circuit having a bilinear coupling as in the example of the RNN circuit shown in FIG. 6 cannot generally obtain a theoretical solution by theoretical analysis, and a theoretical solution can be obtained only by simulation. Moreover, the simulation for obtaining the theoretical solution takes time, and the reliability of the theoretical solution obtained by the simulation is low. On the other hand, an RNN circuit that does not have a bilinear coupling can be converted into a linear simultaneous differential equation, so that a theoretical solution can be obtained, but output variations are limited.

Japanese Patent Laid-Open No. 6-149771 WO2007 / 135723 WO 2004/104917

Shiro Nagashima, “Robot Software System Using Bilinear Time Delay Neural Network”, Journal of the Robotics Society of Japan, Vol.24, No.6, pp.53-64, 2006

  The conventional neural network design method has a problem that it is difficult to obtain an RNN circuit having bilinear coupling that can obtain various variations of outputs and can obtain a theoretical solution.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a neural network design method and program capable of obtaining an RNN circuit having a bilinear combination capable of obtaining various variations of output and obtaining a theoretical solution. To do.

  According to an aspect of the present invention, a computer-aided neural network design method for determining whether or not a theoretical analysis of a recurrent neural network (RNN) circuit having a bilinear connection including neurons and connections is possible. Judgment procedure, division procedure for dividing RNN circuit determined to be capable of theoretical analysis into a plurality of partial RNN circuits having no bilinear coupling and storing them in the storage unit, and theoretical analysis of each partial RNN circuit To obtain a theoretical solution of the RNN circuit having the bilinear combination by combining the analysis procedure for storing the logical solution in the storage unit and the theoretical solution of each partial RNN circuit. There is provided a neural network design method characterized by causing the computer to execute an output procedure.

  According to one aspect of the present invention, a program for causing a computer to design a neural network, whether or not a theoretical analysis of a recurrent neural network (RNN) circuit having a bilinear connection including neurons and connections is possible. A determination procedure for determining, an RNN circuit that is determined to be theoretically analyzable, a division procedure for dividing the RNN circuit into a plurality of partial RNN circuits having no bilinear coupling and storing them in the storage unit; A theoretical solution is obtained by performing a theoretical analysis, and an analysis procedure for storing the logical solution in the storage unit and the theoretical solution of each partial RNN circuit are combined to obtain a theoretical solution of the RNN circuit having the bilinear coupling. A program is provided that causes the computer to execute an output procedure to be output.

  According to the disclosed neural network design method and program, it is possible to obtain an RNN circuit having bilinear coupling that can obtain various variations of outputs and can obtain a theoretical solution.

It is a figure explaining the relational expression of a neuron and input-output connection. It is a figure explaining the expansion of the relational expression of FIG. It is a figure which shows an example of the RNN circuit which has two neurons and four connections. It is a figure explaining an example of a bilinear coupling. It is a figure explaining an example which calculates | requires an inner product. It is a figure which shows an example of the RNN circuit which has two bilinear couplings. It is a figure explaining the relationship between the RNN circuit which has bilinear coupling, and the RNN circuit which does not have bilinear coupling. It is a block diagram which shows an example of the electronic device carrying an RNN circuit. It is a figure which shows an example of the RNN circuit which has a bilinear coupling without the coupling | bonding which returns to self. It is a flowchart explaining the process which judges whether the RNN circuit which has a bilinear coupling is theoretically analyzable. It is a figure which shows the other example of the RNN circuit which has a bilinear coupling without the coupling | bonding which returns to self. It is a figure which shows a partial RNN circuit. It is a figure which shows a partial RNN circuit. It is a figure which shows a partial RNN circuit. It is a figure which shows the RNN circuit which has a bilinear coupling. It is a figure explaining the process which changes all the bilinear couplings of FIG. 15 to an external input function. It is a figure which shows the RNN circuit in which a theoretical analysis is possible. It is a figure which shows the converted partial RNN circuit. It is a figure which shows the converted partial RNN circuit. It is a figure which shows the converted partial RNN circuit. It is a figure which shows an example of a comparatively large RNN circuit. FIG. 6 is a diagram showing an RNN circuit that combines an RNN circuit having a bilinear coupling capable of theoretical analysis of FIG. 5 and “x ₃ ”. It is a block diagram which shows an example of a computer system.

  The disclosed neural network design method and program determine whether or not a theoretical analysis of an RNN circuit having a bilinear coupling is possible, and the RNN circuit determined to be capable of the theoretical analysis has a bilinear coupling. It is divided into a plurality of partial RNN circuits that are not present. Since the partial RNN circuit having no bilinear coupling can be theoretically analyzed, the theoretical solution of the RNN circuit having the original bilinear coupling is obtained by combining the theoretical solutions of the partial RNN circuits. Compared to the case where the theoretical solution of the RNN circuit having the original linear combination is obtained by simulation, it does not take time to obtain the theoretical solution as in the case of simulation, and the reliability of the theoretical solution is as in the case of simulation. It will never go down.

  Hereinafter, embodiments of the disclosed neural network design method and program will be described with reference to the drawings.

FIG. 8 is a block diagram illustrating an example of an electronic device equipped with an RNN circuit. The electronic device is, for example, a mobile phone having a pedometer function. In the following description, as in the description of FIGS. 1 to 6, ε _i is a delay parameter, y _i and y _j are neuron state quantities, C _ij is a weighting factor, t is time, and neuron state quantities In some cases, y _i representing と is _identified with a neuron. G _i is a value input to the neuron y _i .

In the example of FIG. 8, for convenience of explanation, it is assumed that the RNN circuit realizes the function of a pedometer for counting the number of steps. 8 includes an acceleration sensor 11, a memory 12, a CPU (Central Processing Unit) 13 which is an example of a processor (or a computer), and a display device 14. The acceleration sensor 11 includes, for example, three sensors that detect triaxial accelerations orthogonal to each other. The output value g _i of the acceleration sensor 11 is input to the memory 12, and the CPU 13 calculates the RNN circuit and outputs the step count result to the display device 14. In the memory 12, the initial value of each y _i neuron (or state quantity) constituting the RNN circuit and each output value g _i of the acceleration sensor 11 are substituted and stored. The memory 12 stores an execution program for the RNN circuit. The execution program includes a program that executes bilinear combination processing for obtaining the inner product and distance of vectors. For each clock, the values of y _i and g _i are calculated by the CPU 13 by the execution program. Each y _i value calculated by the calculation of the CPU 13 is stored in the memory 12, and the next g _i value is substituted into the memory 12. After such processing is executed a predetermined number of times, the value of the neuron (or state quantity) desired as an output, that is, the step value is read from the memory 12 by the CPU 13 and displayed on the display device 14 to display the number of steps. A series of processing is completed.

  When the RNN circuit designed for the electronic device 10 is an RNN circuit having a bilinear coupling as in the example of the RNN circuit shown in FIG. 6, it is generally not possible to obtain a theoretical solution by theoretical analysis. Therefore, in the present embodiment, RNN circuits having bilinear coupling that can obtain a theoretical solution are classified with respect to RNN circuits having bilinear coupling that cannot be theoretically analyzed in general.

  Since the RNN circuit in FIG. 6 outputs chaos, it has a bilinear connection that cannot be theoretically analyzed, but has a bilinear connection that is input to the neuron “z”. These are the neurons “x” and “y”. It is output from. Here, focusing on the neuron “y”, the result output from the neuron “z” is input to the neuron “y”. In this way, when there is a connection that returns to the neuron “z” itself when going back from the bilinear connection to the neuron “z”, the theoretical analysis is generally difficult. Therefore, in the present embodiment, if there is a connection that returns from the bilinear connection to the neuron when going back to the neuron, it is determined that the theoretical analysis is impossible.

  On the other hand, for example, the RNN circuit having the bilinear coupling of FIG. 9 has one bilinear coupling, but the neuron “z” can be connected even if the connection with the neuron is traced back from the bilinear coupling to the neuron “z”. There is no connection that returns to itself. FIG. 9 is a diagram illustrating an example of an RNN circuit having a bilinear coupling with no coupling back to itself. As described above, when there is no connection that returns to itself even if the connection with the neuron is traced back from the bilinear connection, the RNN circuit determines that the theoretical analysis is possible.

  FIG. 10 is a flowchart for explaining processing for determining whether or not an RNN circuit having bilinear coupling is theoretically analyzable. The processing in FIG. 10 determines whether or not an RNN circuit having a given bilinear combination is capable of theoretical analysis, and obtains a set of partial neurons from the set of whole neurons. The set of partial neurons obtained by this processing is used when theoretical analysis is performed in the design process of the neural network.

(When the RNN circuit of FIG. 6 is a processing target)
As a specific example, a case where the process of FIG. 10 is applied to the RNN circuit of FIG. 6 will be described. First, in step S1, as an initial setting, I is set to an empty set φ (I = φ), w is set to w = ′ ′, and a set X _I of all neurons of the RNN circuit to be processed is obtained. The set Sub _I of neurons having bilinear coupling as input in the set X _I of all neurons is obtained. In this case, a set X _φ = {x, y, z} of the entire neurons of the RNN circuit and a set of neurons Sub _φ = {z} having bilinear connections in the set X _φ as inputs are obtained. Step S2 determines whether or not Sub _I = φ. In this case, since Sub _φ = {z} ≠ φ and the determination result is NO, the process proceeds to step S3. Step S3 extracts and fixes one neuron from the set Sub _I , finds I = I） {b}, adds 'b' to w, and goes back to X _I = b from the bilinear connection. Find the resulting set of neurons. In this case, since Sub _φ = {z} is a set containing only one neuron, “z” is extracted and fixed, and I is set to I = I∪ {z} = φ∪ {z} = {z} Update and update w to w = '' + 'z' = 'z'. The set of neurons obtained from the bilinear connection to “z” is the set of all neurons obtained by tracing the connection with the neuron from the bilinear connection to neuron “z”. In this case, the neurons “x” and “y” that are output to the bilinear connection and the neuron “z” obtained by tracing back the connection from the neuron “y” are the targets of retrogression, and X _{z} = {x , y, z}.

Step S4 decides whether or not include "b" to the X _I. In this case, it is determined whether or not z is included in X _{z} . Since z is included in X _{z} , the determination result is YES. In step S5, the RNN circuit of FIG. 6 determines that the theoretical analysis is impossible, and outputs such as displaying a message indicating that the theoretical analysis is impossible or storing it in the storage unit. finish.

(When the RNN circuit in FIG. 9 is a processing target)
Next, a case where the process of FIG. 10 is applied to the RNN circuit of FIG. 9 will be described as a specific example. First, in step S1, as initial settings, I = φ, w = ′ ′, set X _φ = {x, y, z}, and set Sub _φ are obtained. In this case, the set Sub _φ is a set of neurons having a bilinear connection as an input in the set X _φ , so Sub _φ = {z}. The determination result of step S2 is NO because Sub _φ = {z} ≠ φ. Since Sub _φ = {z} is a set including only one neuron, step S3 extracts and fixes “z” and sets I to I = I = {z} = φ∪ {z} = {z} And w is updated to w = '' + 'z' = 'z'. The set of neurons obtained from the bilinear connection to “z” is the set of all neurons obtained by tracing the connection with the neuron from the bilinear connection to neuron “z”. In this case, the neurons “x” and “y” that are output to the bilinear connection are retroactive, and X _{z} = {x, y}. Since the determination result of step S4 is NO because X _{z} does not include {z}, the process proceeds to step S6.

In step S6, X _I is stored, a set of neurons having a bilinear connection in Sub _I = X _I as an input is obtained, and Sub _A = Sub _A- {for any A⊂I- {b}. b}. In this case, save the X _{z}, although Sub _{z} is a set of neurons with the input bilinear bond in X _{z}, the input bilinear bond in X _{z} Since there is no neuron, we get Sub _{z} = φ, and I- {z} = {z}-{z} = φ, so it is only when A = φ that satisfies this for any A⊂φ Update Sub _φ to Sub _φ = Sub _φ- {z} = {z}-{z} = φ. After step S6, the process returns to step S2 to determine whether Sub _{z} = φ. In this case, since the determination result of step S2 is YES, the process proceeds to step S7.

In step S7, it is determined whether I = φ. In this case, since I = {z} ≠ φ, the determination result in step S7 is NO, and the process proceeds to step S8. In step S8, the last character 'b' of w is acquired and removed, and I = I- {b} is obtained. In this case, the character 'z' is removed from w = 'z', w is updated to w = '', and I is updated to I = I- {z} = {z}-{z} = φ. After step S8, the process returns to step S2 to determine whether or not Sub _φ = φ. In this case, since the determination result of step S2 is YES, the process proceeds to step S7. Furthermore, since I = φ, the determination result in step S7 is YES, and the process ends. Therefore, in this case, since there is no output that cannot be theoretically analyzed, the RNN circuit of FIG. 9 determines that theoretical analysis is possible, and displays or stores a message indicating that theoretical analysis is possible, for example. The process ends after outputting such as storing in the section.

(When the RNN circuit of FIG. 11 is a processing target)
Next, a case where the process of FIG. 10 is applied to the RNN circuit of FIG. 11 will be described as a specific example. FIG. 11 is a diagram illustrating another example of an RNN circuit having a bilinear coupling that does not return to itself. First, step S1 is an initial setting. I = φ, w = ′ ′, set X _φ = {a, b, c, d, e, f, g}, and set Sub _φ are obtained. In this case, the set Sub _φ is a set of neurons having a bilinear connection as an input in the set X _φ , so Sub _φ = {a, b, e}. The determination result of step S2 is NO because Sub _φ = {a, b, e} ≠ φ. In step S3, one neuron is extracted from Sub _φ = {a, b, e}. Here, “a” is extracted and fixed, and I is set to I = I∪ {a} = φ∪ {a} = { Update to a} and update w to w = '' + 'a' = 'a'. The set of neurons obtained from the bilinear connection to “a” is the set of all neurons obtained by tracing the connection with the neuron from the bilinear connection to neuron “a”. Therefore, in this case, X _{a} = {b, c, d}. Since the determination result of step S4 is NO because X _{a} does not include {a}, the process proceeds to step S6.

In step S6, X _{a} is stored, and Sub _{a} is determined as Sub _{a} = {b} because Sub _{a} is a set of neurons having a bilinear connection in X _{a} as an input. Also, from I- {a} = {a}-{a} = φ, for any A⊂φ, only this is satisfied when A = φ, so Sub _φ is Sub _φ = Sub _φ- {a } = {a, b, e}-{a} = Update to {b, e}. After step S6, the process returns to step S2 to determine whether Sub _φ = φ. In this case, since the determination result in step S2 is NO because Sub _{a} = {b} ≠ φ, the process proceeds to step S3.

Step S3 is taken out one neuron from _{Sub {a} = {b}} , because Sub _{a} is the set has only one neuron, fixed removed "b", the I I = I∪ { b} = {a} ∪ {b} = Update to {a, b} and update w to w = 'a' + 'b' = 'ab'. The set of neurons obtained from the bilinear connection to “b” is the set of all neurons obtained by tracing the connection with the neuron from the bilinear connection to neuron “b”. Therefore, X _{{a, b}} = {c, d}. The determination result of step S4 is NO because X _{{a, b}} does not include {b}, and the process proceeds to step S6. In step S6, X _{{a, b}} is stored, and since there is no set of neurons having a bilinear connection in X _{{a, b}} as an input, Sub _{{a, b}} = φ. Also, from I- {b} = {a, b}-{b} = {a}, this is satisfied for any A⊂ {a} when A = φ, {a}. Therefore, when A = φ, Sub _φ is updated to Sub _φ = Sub _φ- {b} = {b, e}-{b} = {e}, and when A = {a}, Sub _{{a} is changed} to Sub. Update to _{a} = Sub _{a} -{b} = {b}-{b} = φ.

After step S6, the process returns to step S2, but since Sub _{{a, b}} = φ, the determination result in step S2 is YES, and the process proceeds to step S7. Furthermore, since I = {a, b} ≠ φ, the determination result in step S7 is NO, and the process proceeds to step S8. In step S8, the character 'b' is removed from w = 'ab' and updated to w = 'a', and I is updated to I = I- {b} = {a, b}-{b} = {a} Update to

After step S8, the process returns to step S2. In this case, since Sub _{a} = φ, the determination result of step S2 is YES. The determination result of step S7 is NO because I = {a} ≠ φ. Accordingly, step S8 removes the character 'a' from w = 'a', updates w to w = '', and sets I to I = I- {a} = {a}-{a} = φ. Update.

After step S8, the process returns to step S2, but since Sub _φ = {e} ≠ φ, the determination result in step S2 is NO, and the process proceeds to step S3. In step S3, one neuron is extracted from Sub _φ = {e}, but since there is only one neuron, “e” is extracted and fixed, and I is set to I = I∪ {e} = φ∪ { Update to e} = {e} and update w to w = '' + 'e' = 'e'. The set of neurons obtained from the bilinear connection to “e” is the set of all neurons obtained by tracing the connection with the neuron from the bilinear connection to neuron “e”. Therefore, X _{e} = {f, g}. The determination result of step S4 is NO because X _{e} does not include {e}, and the process proceeds to step S6. In step S6, X _{e} is stored, and since there is no set of neurons having a bilinear connection in X _{e} as an input, Sub _{e} = φ. Further, from I- {e} = {e}-{e} = φ, this is satisfied for an arbitrary A⊂ {e} when A = φ. Therefore, when A = φ, Sub _φ is updated to Sub _φ = Sub _φ − {e} = {e} − {e} = φ.

After step S6, the process returns to step S2, but since Sub _{e} = φ, the determination result in step S2 is YES, and the process proceeds to step S7. The determination result of step S7 is NO because I = {e} ≠ φ, and the process proceeds to step S8. In step S8, the character 'e' is removed from w = 'e', w is updated to w = '', and I is updated to I = I- {e} = {e}-{e} = φ. .

After step S8, the process returns to step S2, but since Sub _φ = φ, the determination result in step S2 is YES, and the process proceeds to step S7. The determination result of step S7 is YES because I = φ, and the process ends. In this case, since there is no output that cannot be theoretically analyzed, the RNN circuit of FIG. 11 determines that theoretical analysis is possible, and displays, for example, a message indicating that theoretical analysis is possible or stores it in the storage unit. The process ends after the output such as.

(Partial RNN circuit)
Next, the partial RNN circuit will be described. The set of partial neurons obtained by performing the processing of FIG. 10 on the RNN circuit of FIG. 11 is X _{a} = {b, c, d}, X _{{a, b}} = {c, d} , X _{e} = {f, g}. An RNN circuit obtained by taking out all neurons included in a set of partial neurons and taking out only a connection in which those neurons are connected is defined as a partial RNN circuit of the original RNN circuit. At this time, the partial RNN circuit corresponding to X _{a} = {b, c, d}, X _{{a, b}} = {c, d}, X _{e} = {f, g} is shown in FIG. 13 and FIG. 12 to 14 are diagrams showing partial RNN circuits.

  Next, with respect to the original RNN circuit and all partial RNN circuits, all bilinear combinations are converted (or changed) from the state of FIG. 15 to an external input function as shown in FIG. However, such conversion is not performed for the bilinear output.

  The results obtained by performing this conversion on the original RNN circuit of FIG. 11 and the partial RNN circuits of FIGS. 12 to 14 are shown in FIGS. 17 to 20, respectively. FIG. 17 is a diagram showing an RNN circuit capable of theoretical analysis, and FIGS. 18 to 20 are diagrams showing converted partial RNN circuits. The resulting RNN circuits of FIGS. 17 to 20 are all RNN circuits that do not have a bilinear coupling, so that a theoretical analysis can be performed if a function of an external input is known.

In order to perform a theoretical analysis of the original RNN circuit, it is only necessary to perform a theoretical analysis of all the partial RNN circuits. First, analysis is performed from the set of sub-neurons obtained first, with the largest number of neurons included in the subscript set. This is because the RNN circuit obtained from the set of partial neurons having the largest number of neurons included in the set of subscripts has no conversion to the external input. In this case, among the set of partial neurons X _{a} , X _{{a, b}} , X _{e} , the set of partial neurons with the largest number of neurons in the set of subscripts is X _{{a, b}} Therefore, the analysis is performed from the partial RNN circuit of FIG. 19 corresponding to the partial neuron set X _{{a, b}} . Since the partial RNN circuit of FIG. 19 is an RNN circuit having no external input, a theoretical solution of c and d can be obtained.

Next, a partial neuron set having the second largest number of neurons included in the subscript set is analyzed. In this case, the partial neuron set having the second largest number of neurons included in the subscript set is X _{a} and X _{e} . The partial RNN circuit corresponding to the partial neuron set X _{a} is the partial RNN circuit of FIG. 18 and has one external input F ₂ . F ₂ is obtained from the partial RNN circuit of FIG. 19 corresponding to X _{{a, b}} . Since the partial RNN circuit of FIG. 19 can be theoretically analyzed, the theoretical solution of F ₂ can be obtained. Accordingly, the partial RNN circuit of FIG. 18 can perform theoretical analysis, and can obtain theoretical solutions of b, c, and d.

The partial RNN circuit corresponding to the partial neuron set X _{e} is the partial RNN circuit of FIG. Since the partial RNN circuit of FIG. 20 is an RNN circuit having no external input, a theoretical solution of f and g can be obtained.

As described above, since the theoretical analysis of all the partial RNN circuits can be performed, it is possible to finally analyze the original RNN circuit. The RNN circuit of FIG. 17 has three external inputs F ₁ , F ₂ and F ₃ . The theoretical solution of F ₁ is obtained from the partial RNN circuit of FIG. 18 corresponding to X _{a} , the theoretical solution of F ₂ is obtained from the partial RNN circuit of FIG. 19 corresponding to X _{{a, b}} , and F ₃ Is obtained from the partial RNN circuit of FIG. 20 corresponding to X _{e} . Therefore, it was confirmed that the RNN circuit of FIG. 17 can be theoretically analyzed, and the original RNN circuit of FIG. 11 can also be theoretically analyzed.

  Further, by combining a plurality of RNN circuits having bilinear coupling that can be theoretically obtained, obtained by the above processing, an RNN circuit having a relatively large scale as shown in FIG. 21 can be created. FIG. 21 is a diagram illustrating an example of a relatively large RNN circuit. In FIG. 21, P1 to P4 are all RNN circuits having bilinear coupling and capable of theoretical analysis.

FIG. 22 is a diagram illustrating an example of an RNN circuit in which “X ₃ ” is combined with an RNN circuit having a bilinear coupling that can be theoretically analyzed in FIG. 5. FIG. 22 shows an RNN circuit that obtains an output such that the speed of x ₃ (t) is dx ₃ / dt (t) = 2sin (t) cos (t).

  In this way, by combining multiple RNN circuits with bilinear coupling that can be theoretically analyzed, various variations of output can be obtained, and RNN circuits with bilinear coupling that can obtain theoretical solutions are designed. Can be created. Also, when analyzing the entire RNN circuit obtained by combining, do not perform simulation analysis from the beginning, but perform analysis by using as much as possible the part that can be theoretically analyzed by the method of the above embodiment. Thus, the RNN circuit can be designed efficiently.

  FIG. 23 is a block diagram illustrating an example of a computer system. A computer system 100 illustrated in FIG. 23 has a configuration in which a CPU 101, a storage unit 102, an interface (I / F) 103, an input device 104, and a display unit 105 are connected by a bus 106. The CPU 101 controls the entire computer system 100 by executing a program stored in the storage unit 102. The storage unit 102 can be formed of a semiconductor storage device, a magnetic recording medium, an optical recording medium, a magneto-optical recording medium, and the like. The storage unit 102 stores the above programs and various data, and also performs intermediate results and calculation results of calculations executed by the CPU 101. It also functions as a temporary memory for temporarily storing etc. The I / F 103 can receive a program and data stored in the storage unit 102 from a network (not shown). The input device 104 can be formed by a keyboard or the like. The display unit 105 can be formed by a display or the like. The input device 104 and the display unit 105 may be formed of an input / output device having functions of both the input device and the display unit, such as a touch panel.

  The CPU 101 executes the program stored in the storage unit 102 to cause the computer system 100 to function as a device for designing a neural network. The program may cause the CPU 101 to execute at least a neural network design processing procedure, and may be stored in an appropriate computer-readable storage medium including the storage unit 102. Further, the program may cause the CPU 101 to execute a process of determining whether or not at least an RNN circuit having the bilinear coupling shown in FIG. In other words, a program that causes the CPU 101 to execute a process for determining whether or not the RNN circuit having the bilinear coupling shown in FIG. 10 is theoretically analyzable is a program that causes the CPU 101 to execute a neural network design process. It is good also as a structure which can be plugged in.

  Information regarding the RNN circuit to be processed, that is, the neuron of the RNN circuit to be processed, is stored in the storage unit 102 via the I / F 103 from the outside of the computer system 100, but is input from the input device 104 and stored. Even if stored in the unit 102, it may be generated during the neural network design process performed by the CPU 101 and stored in the storage unit 102. In addition, the intermediate results and calculation results of the calculations executed by the CPU 101 stored in the storage unit 102 include the intermediate results and calculation results of the calculations obtained in each procedure of the processing described with reference to FIG. Information displayed on the display unit 105 includes a message indicating that the RNN circuit cannot perform theoretical analysis, a message indicating that the RNN circuit can perform theoretical analysis, and the like.

  In the computer system 100, needless to say, the connection between the CPU 101 and other parts is not limited to the connection via the bus 106 shown in FIG.

  In the above embodiment, the RNN circuit is applied to the pedometer. However, the RNN circuit designed according to the present invention is not limited to the pedometer, and includes a drive circuit for a robot, a control circuit for various devices, a voice analysis circuit, and the like. Needless to say, the present invention can be applied to various physical models to which an RNN having a bilinear coupling is applicable.

The following additional notes are further disclosed with respect to the embodiment including the above examples.
(Appendix 1)
A neural network design method using a computer,
A determination procedure for determining whether a theoretical analysis of a recurrent neural network (RNN) circuit having a bilinear connection including a neuron and a connection is possible;
A division procedure for dividing an RNN circuit that is determined to be theoretically analyzable into a plurality of partial RNN circuits having no bilinear coupling and storing them in a storage unit;
An analysis procedure for obtaining a theoretical solution by performing a theoretical analysis of each partial RNN circuit and storing the logical solution in the storage unit;
An output procedure for obtaining and outputting the theoretical solution of the RNN circuit having the bilinear coupling by combining the theoretical solutions of the respective partial RNN circuits;
The neural network design method is characterized in that the computer is executed.
(Appendix 2)
In the determination procedure, when there is no connection that returns from the bilinear connection of an arbitrary neuron of the RNN circuit having the bilinear connection to the arbitrary neuron itself when the connection with the neuron is traced back, the RNN circuit The neural network design method according to appendix 1, wherein it is determined that a theoretical analysis is possible.
(Appendix 3)
The division procedure defines a circuit obtained by taking out only a connection in which neurons are connected from an RNN circuit that is determined to be capable of the theoretical analysis as a partial RNN circuit. Or the neural network design method described in 2.
(Appendix 4)
In the dividing procedure, for the partial RNN circuit, all bilinear couplings are converted into external input functions to obtain a plurality of partial RNN circuits that do not have the bilinear coupling. 4. The neural network design method according to appendix 3, wherein no conversion is performed.
(Appendix 5)
If the determination procedure has a connection that returns from the bilinear connection of any neuron of the RNN circuit having the bilinear connection to the neuron itself when the connection with the neuron is traced back, the RNN circuit is The neural network design method according to any one of appendices 1 to 4, wherein it is determined that theoretical analysis is impossible.
(Appendix 6)
A program that allows a computer to design a neural network,
A determination procedure for determining whether a theoretical analysis of a recurrent neural network (RNN) circuit having a bilinear connection including a neuron and a connection is possible;
A division procedure for dividing an RNN circuit that is determined to be theoretically analyzable into a plurality of partial RNN circuits having no bilinear coupling and storing them in a storage unit;
An analysis procedure for obtaining a theoretical solution by performing a theoretical analysis of each partial RNN circuit and storing the logical solution in the storage unit;
A program for causing the computer to execute an output procedure for combining the theoretical solutions of the partial RNN circuits to obtain and output a theoretical solution of the RNN circuit having the bilinear coupling.
(Appendix 7)
In the determination procedure, when there is no connection that returns from the bilinear connection of an arbitrary neuron of the RNN circuit having the bilinear connection to the arbitrary neuron itself when the connection with the neuron is traced back, the RNN circuit The program according to appendix 6, characterized in that it is judged that theoretical analysis is possible.
(Appendix 8)
Supplementary note 6 is characterized in that, in the dividing procedure, a circuit obtained by extracting only a connection in which neurons are connected from an RNN circuit determined to be capable of the theoretical analysis is defined as a partial RNN circuit. Or the program of 7.
(Appendix 9)
In the dividing procedure, for the partial RNN circuit, all bilinear couplings are converted into external input functions to obtain a plurality of partial RNN circuits that do not have the bilinear coupling. The program according to appendix 8, wherein conversion is not performed.
(Appendix 10)
If the determination procedure has a connection that returns from the bilinear connection of any neuron of the RNN circuit having the bilinear connection to the neuron itself when the connection with the neuron is traced back, the RNN circuit is The neural network design method according to any one of appendices 6 to 9, wherein it is determined that theoretical analysis is impossible.
(Appendix 11)
A computer-readable storage medium storing the program according to any one of appendices 5 to 10.

  Although the disclosed neural network design method and program have been described with the embodiments, it is needless to say that the present invention is not limited to the above embodiments, and various modifications and improvements can be made within the scope of the present invention. Yes.

10 Electronic Device 11 Acceleration Sensor 12 Memory 13 CPU
14 display device 100 computer system 101 CPU
102 storage unit 103 I / F
104 Input device 105 Display unit

Claims (5)

A program that allows a computer to design a neural network,
A determination procedure for determining whether a theoretical analysis of a recurrent neural network (RNN) circuit having a bilinear connection including a neuron and a connection is possible;
A division procedure for dividing an RNN circuit that is determined to be theoretically analyzable into a plurality of partial RNN circuits having no bilinear coupling and storing them in a storage unit;
An analysis procedure for obtaining a theoretical solution by performing a theoretical analysis of each partial RNN circuit and storing the logical solution in the storage unit;
An output procedure for obtaining and outputting the theoretical solution of the RNN circuit having the bilinear coupling by combining the theoretical solutions of the respective partial RNN circuits;
Is executed by the computer.   The determination procedure is based on the assumption that the RNN circuit is connected when there is no connection that goes back from the bilinear connection of any neuron of the RNN circuit having the bilinear connection back to the neuron itself when the connection is made with the neuron. The program according to claim 1, wherein it is determined that analysis is possible.   The division procedure defines, as a partial RNN circuit, a circuit obtained by taking out only a connection in which neurons are connected from an RNN circuit determined to be capable of the theoretical analysis. The program according to 1 or 2.   In the dividing procedure, for the partial RNN circuit, all bilinear couplings are converted into external input functions to obtain a plurality of partial RNN circuits that do not have the bilinear coupling. 4. The program according to claim 3, wherein no conversion is performed. A neural network design method using a computer,
A determination procedure for determining whether a theoretical analysis of a recurrent neural network (RNN) circuit having a bilinear connection including a neuron and a connection is possible;
A division procedure for dividing an RNN circuit that is determined to be theoretically analyzable into a plurality of partial RNN circuits having no bilinear coupling and storing them in a storage unit;
An analysis procedure for obtaining a theoretical solution by performing a theoretical analysis of each partial RNN circuit and storing the logical solution in the storage unit;
A method for designing a neural network, comprising: causing the computer to execute an output procedure for obtaining and outputting a theoretical solution of an RNN circuit having the bilinear combination by combining the theoretical solutions of each partial RNN circuit.

Images