JP2690755B2 - Optical neural network - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Outline] The present invention relates to an optical neural network based on an optical neurochip that enhances oscillation when the received optical frequency matches, and oscillates when the received optical frequency or light emission, light receiving timing, etc. match. In order to build an arbitrary network by optical communication without using the wiring between neuron models by using the optical neuron model that enhances the optical neuron model, in the neural network constructed based on the optical neuron model, each optical neuron model Input an optical signal having a plurality of optical information in a time-divisional or space-divisional manner, increase the output of an optical signal having optical information corresponding to the optical information previously assigned to the optical neuron model, The output of the optical signal that has different optical information from the existing optical information is reduced. Then, the network is sequentially formed by outputting in a time-divisional or space-divisional manner.

[Industrial applications]

The present invention relates to an associative network that fires even when incomplete information close to stored complete information is input, and more specifically, it is based on an optical neurochip that enhances oscillation when the received optical frequency matches. It relates to a constructed optical neural network.

In order to cope with the limit of parallel processing computers based on Neumann type sequential computers and the crisis that a large amount of software is required to determine the processing contents by programs, Neumann type computers have completely no processing method. Different neurocomputers have received attention. The network that constitutes the neurocomputer is called a neural network. The neurocomputer is based on the method of changing the structure of the network by itself based on the information input on the neural network and, so to speak, self-organizing the network suitable for the input information. In order to break the risk of the program, this neurocomputer makes processing decisions by learning, and when information with ambiguity is input, it is restored based on the information stored at the time of learning. By this, it performs associative and recognition similar to human beings according to massively parallel processing, which is the basis of the next-generation computer architecture. There is a growing need to construct such neural networks using physical devices.

[Conventional technology]

Traditionally, this kind of neural network is an electronic circuit,
In particular, it is constructed based on an analog circuit, and the basic circuit for constructing a network is based on a formal neuron model. This formal neuron model has multiple inputs and 1
With a book output, add all the signals weighted to the voltage of each input signal, and find the difference between this and the stored threshold value,
If the difference is positive, the output signal is strengthened, and if the difference is negative, the output signal is weakened. Conventionally, such a formal neuron model is constructed by using a basic circuit such as an operational amplifier and is connected to each other to construct a neural network. In such a neural network, in order to make the weighting coefficient of the connection that connects each neuron model variable at the time of learning, the weights are digitized, MOS transistors whose weights are variable are used, or between the neuron models. The presence or absence of coupling was configured using a switch. That is, in the conventional network, wiring is performed in advance on the integrated circuit, and the weight on the wiring is made variable to perform self-organization.

Hopfield type, backpropagation type, and the like are considered as methods for forming a conventional neural network. A Hopfield neural network integrates a differential variable with an integrator, field-quantizes the value of the output, and forms an output variable.
A plurality of these output variables are connected to each other at the connecting portion, the weighting factors at the connecting points are multiplied, and the sum is taken as the differential variable of each neuron. In this network, the energy function is controlled so as to be minimized, so that the associative operation is enabled by approaching the output patterns at a plurality of local minimum points.

On the other hand, in a multilayer neural network based on backpropagation, layers of multiple formal neuron models are arranged in multiple layers so that the output pattern obtained from the output layer approaches the teacher signal with respect to the pattern given to the input layer. , The weighting factor between each signal is decided at the time of learning,
At the time of association, the given incomplete information is matched with the internal pattern corresponding to the weighting coefficient determined at the time of learning to restore the complete information.

[Problems to be solved by the invention]

Therefore, in these conventional neural networks,
The wiring between the formal neuron models is preconfigured on the integrated circuit, and the network is self-organized by changing the value of the physical element on the wiring by an internal voltage or the like. Therefore, in order to carry out learning for any purpose for the application of association and recognition, it is necessary to have a complete graph structure in which each neuron is connected in the Hopfield type connection part. In the neural network, the connection between the next layer and the subsequent layer is such that one neuron in the previous stage is wired to all neurons in the subsequent stage. Therefore, this conventional method has a drawback that the amount of wiring is extremely large and it is difficult to construct a large neural network.

The present invention does not require wiring between neuron models by using an optical neuron model in which oscillation is enhanced when received light frequency or light emission, light reception timing, etc. are matched,
The purpose is to build an arbitrary network by optical communication.

[Means for solving the problem]

 FIG. 1 is a diagram illustrating the principle of the present invention.

The input means 3 of each optical neuron model 1 inputs a plurality of optical information, for example, an optical signal having an optical frequency, an emission intensity, and an emission timing as a reception light 7 in a time division manner, an optical frequency division manner, or a space division manner, The processing means 5 processes it as an electric signal.

The storage means 2 stores the optical information corresponding to the optical information assigned to the optical neuron model 1 in advance.

The associative transmission means 4 receives the output of the electronic processing means 5, makes the optical signal distinguishable from the optical signal having other optical information, and oscillates the frequency of the received light that matches the optical frequency stored in the storage means 2, for example. It is characterized in that the networks are sequentially formed by increasing the frequency and weakening the oscillation of other frequencies to output as the first optical output 6 or the second optical output in a time division or space division manner.

[Action]

In the optical neural network of the present invention, information such as the light emission frequency, the light emission intensity, and the light emission timing is stored, and when the light emission frequency that matches the received light frequency received by one optical neuron model is stored, It has the function of increasing the oscillation of the optical neuron and weakening the oscillation when a non-matching frequency is received. By optically coupling a plurality of optical neuron models having these functions with each other, self-organization can be arbitrarily performed. It will be possible.

〔Example〕

Next, an embodiment of the present invention will be described with reference to the drawings. Second
The figure is an embodiment of the configuration of an optical neural network based on an optical neuron model based on the optical time division multiplexing system of the present invention. In the figure, the same parts as those in FIG. 1 are indicated by the same symbols. In the figure, 11 is an optical neurochip, 12 is an electronic memory control unit that stores the emission frequency, emission intensity, emission timing, etc., 13 is a photodetector that detects the received light received from another neuron model, and 14 is an optical detector. It is a variable-frequency LD-type associative transmission unit that scans from frequencies f ₁ to f _x , enhances oscillation at the received optical frequency when it matches within a certain range, and weakens oscillation at frequencies that do not match. Reference numeral 15 is an electronic bandpass filter for band-limiting the received optical frequency of reception. In this optical neuron model, light having the received optical frequency received by the photodetector 13 is converted into an electronic signal of a certain intermediate frequency, that is, a heterodyne received signal, and this received signal is input to the electronic bandpass filter 15. . Then, the signal is given to the electronic storage control unit 12 by band limiting within a certain frequency range,
Matching of the light emission frequency, the light emission intensity, the light emission timing, etc. stored in the electronic memory control unit 12 is performed. In addition, the variable frequency LD section 14 detects the light of different frequencies
13 and the matching is performed after converting to a heterodyne reception signal by the difference between the reception light frequency of the reception light 7 and the photodetector 13, and only the optical frequency is converted to an electronic signal, and after that conversion It is also possible to use a method of comparing the stored frequency in the electronic storage control unit 12 and giving the matched signal frequency information to the frequency variable LD unit 14. If there is a matching frequency, the frequency variable LD section 14 intensifies the oscillation at the frequency corresponding to the received optical frequency, and if the frequency does not match, weakens the oscillation at that frequency. That is, the variable frequency LD section 14 outputs the light having the oscillation intensity up to the optical frequencies f _{1 to} f _x as the first optical output 16 to other neurochips in a time division manner. Therefore, in a neural network using this type of optical neuron model, the optical network from the optical neuron model in the previous stage to the optical neuron model in the subsequent stage is output in a form in which the frequencies from f _{1 to} f _x are time-division multiplexed, Each frequency f _i corresponds to an item associated with each neuron model. For example, in this neuron model, at a specific frequency f _i , when a reception frequency almost equal to that f _i is input as the received light 7, only the optical frequency having that f _i from this neuron model becomes strong oscillation. ,
Oscillations of light of other frequencies are weakened. Therefore, this neuron model is associated with the item corresponding to f _i . For example, if f _i is associated with the item corresponding to the color “blue”, the oscillation is strengthened at the optical frequency of f _i corresponding to the item blue, and the light is transmitted to the next-stage optical neuron model. Is transmitted. This light transmission goes straight in the space and straight in the unwired space, so it can be directly transmitted to the desired optical neuron model, and as a result, the whole neural network becomes a network suitable for the purpose. It will be self-organized.

FIG. 3 is an embodiment diagram of the configuration of an optical neural network based on the optical neuron model based on the spatial multiplexing system of the present invention. In the figure, 13 is a photodetector for receiving light, 15 is an electronic bandpass filter for band limiting the received optical frequency, 12 is an electronic storage controller for storing oscillation frequency, emission intensity, emission timing, etc., 14 ' Has a function to increase the oscillation of light of the frequency when it matches with the received optical frequency within a certain range and weaken the oscillation of light of the frequency that does not match, and it is generated from multiple laser diodes (LDS). An associative transmission unit that simultaneously multiplexes and transmits optical frequencies f _{1 to} f _x .

It can also be referred to as time multiplexing in the sense that multiple frequencies are multiplexed at the same time.

In this method, a smaller number of optical frequency multiplexes is sufficient than the number of neurons. The frequencies f _{1 to} f _M are used with priority from f ₁ (first optical output, that is, the downstream direction). The optical signal from the firing neuron determines the optical coupling strength (positive or negative) of the network and organizes the network.

Second light output, that is, rising direction (feedback)
On the contrary, the frequency of use of f _M is increased, and the frequency is reduced to the minimum at f ₁ , to reduce crosstalk. Others function similarly to the downstream direction.

The time-division multiplexing method of FIG. 2 is a time-division multiplexing method in which light of each frequency having oscillation strength is sequentially transmitted, but the spatial multiplexing method of FIG. 3 has these oscillation strengths. It transmits all the light of a frequency at the same time, so to speak, it can communicate with many other optical neuron models at the same time.
Complex neural networks can be constructed.

FIG. 4 is a conceptual diagram of a waveguide network in which the optical neuron models of the present invention are mutually coupled. In the figure
The mark indicates the optical neuron model shown in FIG. 2 or FIG. 3. For example, in the case of transmission, a variable frequency laser is used, and in the case of reception, one using a frequency selective light receiving element and a gate element is used. , 8 are waveguide plates that limit the range of the optical transmission path, and 9 is an optical path. In the present waveguide network, it is possible to form an arbitrary optical path between the optical neuron model in the layer on the left side of the waveguide plate 8 and the optical neuron model in the layer on the right end of the waveguide plate.

FIG. 5 is a conceptual diagram of the configuration of the space distribution type network according to the present invention. In the figure, the circles indicate the neuron model of FIG. 2 or 3, and in this case, the optical device uses a surface emitting type frequency tunable laser for the transmitting device and a surface receiving type of frequency for the receiving device. A selective light receiving element is used. Optical communication from the optical neuron model in the previous layer on the left side to the optical neuron model in the next layer on the right side follows the optical path 10 indicated by the arrow, so to speak, the optical path is formed in a three-dimensional spatial region. Therefore, it is possible to build a complicated network.

FIG. 6 is a conceptual diagram of the configuration of the fiber type network of the present invention. In the figure, the mark ◯ is the optical neuron model of the present invention shown in FIG. 2 or 3, 21 is an optical distributor, and 22 is an optical fiber. This fiber type network of the present invention constructs a multilayer neural network, and further uses an optical fiber, so that it has a predetermined purpose-oriented optical wiring structure. The light transmitted from each optical neuron model is transmitted to the next-stage optical neuron model via the optical distributor 21, and is further transmitted between the optical distributors using the optical fiber, so that the interference of the light in space is prevented. There is none, and thus it is possible to construct a network with extremely high reliability.

As described above, the present invention has an associative function by strengthening the oscillation at the received light frequency within a certain range and weakening the oscillation at the non-matching frequency, and has such a function. The optical neuron model can be installed on various networks, and for example, a multilayer neural network can be constructed and self-organized.

FIG. 7 shows a concrete example of a neural network based on the optical neuron model of the present invention. Here, it is assumed that each neuron model is based on the spatial multiplexing method shown in FIG. In the figure,
x _{1 to} x _n are input signals, x marks describing x _{1 to} x _n are input layer neuron models, f _{1 to} f _p are circle marks, middle layer neuron models, S _{1 to} S _{The circles} indicated by _m are optical neuron models in the output layer. The input signals x _{1 to} x _n input to the input layer have a pattern corresponding to, for example, an item of “a bitter apple and an insect apple”.

Each optical neuron model in the input layer extracts or associates the characteristics of the input information, and is fired for each input signal to the input signals x ₁ to x _n , that is, for each specific information of the input. For example, 23 is a neuron model that is associated with “types” such as ○, □, and Δ, 24 is for detecting “color” such as red, green, and blue, and 25 is for “size” of large, medium, and small. 26 is associated with the "inclination" of the line segment, 27
Is associated with "perspective", 28 is spatial "view direction"
29 is associated with "like and dislike". In each neuron model in these input layers, the degree of optical coupling is increased in proportion to the size of the match of the input information input, and the input information is assigned to the optical frequencies f _{1 to} f _p in order. One common term uses one optical frequency. In addition, for a given input of x ₁ to x _n , n optical neuron models corresponding thereto may perform optical modulation corresponding to the information. in any case,
The feature of the input information is extracted by each optical neuron model, and the light corresponding to the feature is transmitted. For example, in the first optical neuron model 13, a frequency corresponding thereto when x ₁ is input, the oscillation is strengthened as f ₁ (x _1), the frequency f ₁ corresponding to the other input x ₂ ~x _n ( x ₂ ) ・
..., f ₁ (x _n ) is transmitted as a weak excitation. The same applies to other input layers. In the P optical neuron model in the intermediate layer, f from the optical neuron model in the input layer
₁ (x ₁ ), f ₁ (x ₂ ), etc. are received, and light with different intensity of excitation is received, and the main information of these signals is f ₁ (x ₂ ) etc.,
It is returned to the optical neuron model of the input layer, which in turn excites common information such as f ₁ (x ₂ ), that is, the frequency f ₁ that represents the “type” of x ₁ and also the specific “color” of x _2. As the common information to be ANDed with the item with, only f ₁ (x ₂ ) is excited, and this is filtered information f ₁ (x ₁ , x ₂ ,
x ₃ ... x _n ). That is, as the common information of x ₁ and x _2, the common information that the “type” is round and the “color” is green is excited from the optical neuron model of f ₁ . This is given to the neuron model in the output layer. In each neuron model of the output layer, f ₁ (x ₁ , x ₂ ...
・ X _n ), f ₂ (x ₁ ,, x ₂ ... x _n ), ・ ・, f _p (x ₁ ,, x ₂ , ・ ・
・ And the common information group shown by x _n ) etc. is further taken,
The result is collated with the stored information, affirmative,
Negatives will be judged. These stored concepts are in the S ₁ -S _n neurons of each output layer. That is,
In S _i of the output layer, the teacher signal is S _i , and f _i (x ₁ , x ₂ ,
x _3, ··· x _p) and the minimum squared error between the teacher signal S _i is calculated, the difference is ignited as smallest S _i is associative, so that it behavior control.

In such a network, sub-networks to be associated are created by the number of characteristic points of interest, the total number of sub-networks is P, and these are divided by the optical frequencies f _{1 to} f _p . At the time of learning, it is assembled around the salient features of the input information and self-organized. For example, in the figure, if red is noticeable as the color of sentence pattern input information x ₁ ,
In the x ₂ neuron, light modulation is performed corresponding to “red” in RGB and assigned as an optical frequency. This optical frequency is selected in the order of f _{1 to} f _p , and the frequency of f ₁ (x ₂ ) is excited. This means that the neuron in the middle layer amplifies the f ₁ (x ₂ ) signal and distributes it to the surroundings for broadcasting. Therefore, the input information is filtered by a red filter, and the related terms related to the common term, f ₁ (x ₂ ), such as the circled f ₁ (x ₁ ), and its viewing direction information f ₁ (x ₆ ), Etc. are broadcast at the same frequency f ₁ , which is summarized by a specific small number of optical neuron models in the middle layer, from which f ₁ (x ₁ , x ₂ , x ₃ , ...
_Xn ) information is transmitted to the neurons in the output layer.
Similarly, neurons emitting at f ₂ is the optical frequency f ₂
To extract another salient information. In this case, if the information is the information indicating the biting part of the apple, this pattern emits as f ₁ (x ₁ ) in the input layer. Then, neurons in the middle layer emit light with emphasis at f ₂ (x ₁ ), and are further filtered to focus on the common information and summarize the contents at f _2.
Excited at f ₂ . This causes the feature of interest to be extracted and transmitted to the neurons in the output layer as f ₂ (x ₁ , x ₂ , ... x _n ) in the neurons in the intermediate layer. Similar to f ₁ and f ₂ , features up to f _p are formed on the network of FIG. 7, respectively. Further, in the case of correlation information, f ₁ (x ₁ ,
... x _n ) and f ₂ (x ₁ ... x _n ) are combined. Then, in the output phase, the neuron models having the same concept S _i are fired, and the concept of information is sequentially extracted and output. The set of input information of the neuron model in the output layer is represented by Σ (f _i (x ₁ , x ₂ ... x _p )). Further, the teacher signal is S _i = S _i (x ₁ , x ₂ , ... x _n ) as stored concept information. The set of input information is compared to stored information, S _i , Is calculated. Here, 1 () is a quantization function.

D _i is the result of collation with S _i . For example, if 1 (x) is a quantization function that outputs 1 if x is positive, 0 if 0, and -1 if negative, then D _{i If i} is 1, it means positive, and if -1, it means negative. For example _{D 1 = 1 (Σf 1 (} x 1, ·· x n) -S 1) from apple _{D 2 = 1 (Σf 1 (} x 1, ·· x n) -S 2) from eating chipping D ₃ = 1 It becomes an insect from (Σf ₁ (x ₁ , ··· x _n ) −S ₃ ).

Further, the correlation of these AND information is that the output neuron model outputs an optical signal of the corresponding optical frequency as "apple with insect bite and lack of bite" due to the association with the commonality of other neuron models.

Such a multilayer neural network can be constructed by using the optical neuron model of the time division multiplexing system shown in FIG. 2, and in this case, the frequencies f _{1 to} f _p are switched over time. Therefore, the P networks are configured by time division, and the neuron model groups of the input layer, the intermediate layer, and the output layer need the function of the frame memory by the time division switching method.

〔The invention's effect〕

According to the present invention, a neural network can be efficiently constructed without the need for wiring in advance.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the principle of the present invention, FIG. 2 is a block diagram showing an embodiment of the configuration of an optical neural network by an optical neuron model based on the optical time division multiplexing system of the present invention, and FIG. FIG. 4 is a block diagram showing an embodiment of the configuration of an optical neural network based on an optical neuron model based on the spatial multiplexing method. FIG. 4 is a conceptual diagram of a waveguide network for mutually connecting optical neuron models of the present invention. FIG. 5 is a conceptual diagram of the configuration of the spatial distribution type network according to the present invention, FIG. 6 is a conceptual diagram of the configuration of the fiber type network of the present invention, and FIG. 7 is a concrete example of a neural network based on the optical neuron model of the present invention. It is an explanatory view of a typical example. 1 ... Optical neuron model, 2 ... Storage means, 3 ... Input means, 4 ... Associative transmission means, 5 ... Electronic processing means, 6 ... First optical output, 2 ... Received light.

Claims (4)

(57) [Claims] 1. In a neural network constructed on the basis of an optical neuron model (1), each optical neuron model (1) inputs an optical signal having a plurality of optical information in a time division or space division manner (3 ), Increasing the output of an optical signal having optical information corresponding to the optical information (2) assigned to the optical neuron model (1) in advance, and assigning the assigned optical information (2)
An optical neural network characterized in that the network is sequentially constructed by reducing the output of an optical signal having optical information different from that of (4) and outputting in a time-divisional or space-divisional manner (4). 2. An optical neural network based on an optical neuron model (1), wherein said optical neuron model detects the received light (7) received and converts it into an electronic signal, and associates it beforehand during learning. A storage means (2) for storing optical information corresponding to an item to be stored as a storage signal, and the electronic signal and the storage signal are compared at the time of association to enhance oscillation of a frequency corresponding to the matched storage signal, An optical neural network having an associative transmission means (4) for weakening oscillation and outputting light formed. 3. An optical neural network based on an optical neuron model (1), wherein the optical neuron model detects received light (7) received, and uses an optical heterodyne method from optical frequencies scanned in a time division manner. An input means (3) for converting into an electronic signal, an electronic processing means (5) for obtaining a desired processed signal from the electronic signal, and a storage means for storing optical information corresponding to an item to be associated beforehand in learning as a storage signal. (2) When associatively, the electronic signal and the stored signal are compared, and the oscillation of the optical frequency corresponding to the matched stored signal is strengthened and the oscillation at another frequency is weakened, and the formed light is output in a time division manner. And an associative transmission means (4) for sending to the input means in a time division manner. 4. An optical neural network based on an optical neuron model (1), wherein the optical neuron model detects received light (7) received, and electronically uses an optical heterodyne method from a spatially scanned optical frequency. Input means (3) for converting into a signal, and processing means (5) for obtaining a desired processed signal from the electronic signal
A storage means (2) for storing, as a storage signal, optical information corresponding to an item to be associated in advance during learning; and an oscillation of a frequency corresponding to the matched storage signal by comparing the electronic signal with the storage signal during association. An optical neural network, comprising: associative transmission means (4) for outputting light formed by strengthening and weakening oscillations at other frequencies in a space-division manner and sending it to the input means in a space-division manner.