JP2008299381A - Parallel arithmetic unit and parallel arithmetic method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a parallel arithmetic unit and a parallel arithmetic method for simplifying wiring between a plurality of arithmetic units. <P>SOLUTION: The parallel arithmetic unit is equipped with: a plurality of units daisy-chained through a daisy-chain control bus in a predetermined sequence; and an amplifier for receiving an output value to be output by any one of the plurality of units through a unit output bus, and for outputting the received output value as an input value through a unit input bus to each of the plurality of units. Each of the units is configured to perform an arithmetic operation by an arithmetic method predetermined for each unit based on the input value from the amplifier to be input through the unit output bus, and to, according as a token is input from the previous unit through the daisy-chain control bus, output the token through the daisy-chain control bus to the next unit, and to output the arithmetic result through a unit output bus to the amplifier as the output value. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a parallel arithmetic device having a plurality of arithmetic units, and more particularly to a parallel arithmetic device that makes it possible to simplify connection between a plurality of arithmetic units.

  A neural network is known as one of parallel operations. This neural network is a mathematical model that aims to express some characteristics found in brain function. Since this neural network process can reduce the amount of information in the input data, it is a relatively small amount of computation for problems that cannot be linearly separated such as images and statistics. In many cases, a good solution can be obtained. Therefore, neural networks are applied in various fields such as pattern recognition and data mining.

  Here, if an attempt is made to realize a large-scale neural network, the amount of calculation becomes enormous and processing in a realistic time becomes difficult. As a method for solving this, (1) a method for increasing the computing power of a single processor itself, (2) a method using a parallel computing method using a plurality of processors, and (3) a function is implemented by hardware using an LSI or the like. A method, etc. can be considered. The above methods (1) and (2) are intended to deal with a huge amount of calculation by increasing the capacity of the processor, and it is possible to cope with various neural network algorithms by changing the program. it can.

  Here, in the method (1), conventionally, the computing power of the single processor itself has been increased by increasing the clock frequency of the single processor, that is, by Moore's law. However, in recent years, increasing the clock frequency has increased the amount of heat generation, and since microfabrication dimensions are reaching the physical limit, Moore's Law is failing, increasing the computing power of the single processor itself. Is getting harder. Therefore, the development of a method for increasing the computing power of a processor has shifted from the method (1) to the method (2), and it is possible to achieve both high performance to increase computing power and suppression of heat generation. To go further, the focus is on having a larger cache and multiple compute cores. Development of this processor is also progressing so that it operates at a low clock frequency and operates at low power.

  However, in the method (2), it is difficult to efficiently operate a large number of processors, that is, to increase the degree of parallelism and to configure a network that enables huge data communication between a plurality of processors. As a general problem. Therefore, it is difficult to increase the parallel computation efficiency of a large-scale neural network by the method (2) that uses a parallel computing technique using a plurality of processors.

On the other hand, regarding the hardware implementation of the method (3), there are limitations on the neural network algorithm that can be implemented by hardware, but for specific applications, compared to the method (1) or (2), Even in the case of a low frequency, it is possible to exhibit the performance of an order of magnitude. Patent Documents 1 and 2 are known as such techniques for implementing parallel computation in hardware.
JP-A-6-195454 Japanese Patent Laid-Open No. 2006-39790

  However, in the prior arts of Patent Document 1 and Patent Document 2 relating to hardware as the method of (3), there is a wiring problem, and the circuit scale becomes enormous due to wiring, or wiring between circuits is There is a problem that it cannot be done. For example, in a multilayer neural network, it is necessary to input the output from a node with an output layer to the input of all nodes in the input layer. As the number of nodes in each layer increases, the amount of wiring rapidly increases. It will increase.

  Also, for example, in the case of an actual brain neural network, neurons are arranged and wired in a three-dimensional space, whereas in a hardware configuration such as LSI, the arrangement of components is basically two-dimensional. The wiring problem cannot be solved essentially. Even if an attempt is made to make the components three-dimensional in a laminated structure or the like, the problem of wiring remains, so the application is limited to a use where limited wiring (coupling with only the vicinity) is sufficient.

  In addition, the problem of wiring between a plurality of arithmetic units is not limited to a neural network, and is a problem when it is computationally necessary to input the outputs of all arithmetic units to all arithmetic units. It becomes. For example, in addition to neural networks, there are self-organizing maps, gravity many-body problems, and simulation of charged many-body particles.

  The present invention has been made in view of such circumstances, and one of its purposes is to provide a parallel arithmetic device and a parallel arithmetic method that can simplify wiring between a plurality of arithmetic units. It is in.

  The present invention has been made to solve the above-described problem. A parallel processing device includes a plurality of units daisy-chained via a daisy-chain control bus in a predetermined order, and the plurality of units. An output value output from any one of the plurality of units via a unit output bus, and a relay unit that outputs the input output value to each of the plurality of units as an input value via the unit input bus. A token input unit for inputting a token from a unit in the previous order in which the unit is daisy chained via the daisy chain control bus, and a token is input to the token input unit. The token is transferred to the next unit in the daisy chain order via the daisy chain control bus. A predetermined token output unit, a data input unit for inputting an input value from the relay unit via the unit output bus, and an input value input to the data input unit. A unit calculation unit for calculating by the calculation method, and a calculation result that is a result of calculation by the unit calculation unit in response to a token being input to the token input unit as the output value. And a data output unit that outputs the data through an output bus.

  According to the present invention, it is possible to simplify the connection between the plurality of arithmetic units of the parallel arithmetic device. In addition, since a plurality of units perform operations in parallel, there is an effect that operations can be performed at high speed.

  Further, according to the present invention, the unit includes a unit output storage unit in which the calculation result is stored, the unit calculation unit stores the calculation result in the unit output storage unit, and the data output unit includes: In response to a token input to the token input unit, the calculation result is read from the unit output storage unit, and the read calculation result is output as the output value to the relay unit via the unit output bus. This is a parallel computing device characterized by that.

  According to the present invention, the unit of the parallel arithmetic device stores the operation result and can output the operation result at an appropriate timing in response to the token input to the token input unit. Play.

  In the present invention, in the case where the parallel arithmetic device is a parallel arithmetic device that calculates a hierarchical neural network, the unit arithmetic unit is a layer for identifying an output layer among the layers of the hierarchical neural network. A layer information storage unit in which information is stored; a function storage unit in which an output function of the unit and the layer information are associated with each other; and an output from the unit of the output layer identified by the layer information Stored is a weight storage unit in which a predetermined coupling load multiplied by the value and the layer information are stored in association with each other, and product-sum calculation primary value information obtained by multiplying the input value from the output unit and the coupling load. The product-sum calculation primary value storage unit and the connection load corresponding to the layer information read from the layer information storage unit are read from the weight storage unit, the read connection load and the data A product-sum calculation primary value calculation unit that multiplies the input value input to the input unit, adds the multiplied value to the product-sum calculation primary value information stored in the product-sum calculation primary value storage unit, and stores the sum. And reading the product-sum calculation primary value information from the product-sum calculation primary value storage unit, reading an output function corresponding to the layer information read from the layer information storage unit from the function storage unit, and reading the output function to the output function An output value is calculated by substituting the read product-sum calculation primary value information, and a unit output calculation unit that stores the calculated output value in the unit output storage unit. is there.

  According to the present invention, there is an effect that the calculation of the hierarchical neural network can be performed by the parallel arithmetic device in which the connection between the plurality of arithmetic units is simplified. In addition, since the connection between a plurality of arithmetic units is simple, there is an effect that a wiring problem hardly occurs even when the number of units that are nodes of each layer is large in the hierarchical neural network.

  Further, according to the present invention, a unit identification information storage unit that stores unit identification information indicating which unit outputs as an output unit among the units belonging to the output layer. And the weight storage unit includes a predetermined combined load multiplied by an output value from the unit of the output layer identified by the layer information and the unit identification information, the layer information, and the unit identification. Information is stored in association with each other, and the combined load corresponding to the unit identification information read from the unit identification information storage unit and the layer information read from the layer information storage unit by the product-sum calculation primary value calculation unit Is read from the weight storage unit, the read connection weight is multiplied by the input value input to the data input unit, and the multiplied value is multiplied by the product-sum calculation primary Was added to the product sum calculation primary value information stored in the storage unit is stored, it is a parallel arithmetic apparatus according to claim.

  According to the present invention, there is an effect that the hierarchical neural network can be calculated by multiplying the connection weight according to the output from each node of each layer.

  Further, according to the present invention, the unit calculation unit increments a value of unit identification information stored in the unit identification information storage unit every time an input value is input to the data input unit, and the layer The unit number storage unit in which the information and the number of units of the hierarchical neural network belonging to the layer identified by the layer information are stored in association with each other, and the layer information is read from the layer information storage unit, and the read layer information The number of units corresponding to is read from the unit number storage unit, unit identification information is read from the unit identification information storage unit, the value of the read unit identification information is compared with the read unit number, and the comparison result Indicates that the value of the read unit identification information matches the number of read units. A unit number comparison unit that outputs a comparison coincidence signal, and the unit output calculation unit receives the comparison coincidence signal from the unit number comparison unit, and receives the comparison coincidence signal from the product-sum calculation primary value calculation unit. Read the product-sum calculation primary value information, read the output function corresponding to the layer information read from the layer information storage unit from the function storage unit, and substitute the read product-sum calculation primary value information into the read output function Thus, the output value is calculated, and the calculated output value is stored in the unit output storage unit.

  According to the present invention, there is an effect that each unit can detect whether or not a unit, which is all nodes in the layer that outputs, can complete the output.

  The present invention is also characterized in that the unit number comparison unit resets the value of the unit identification information stored in the unit identification information storage unit when the comparison result matches. Device.

  According to the present invention, there is an effect that each unit can perform the calculation of the next layer after the calculation of the certain layer is completed.

  In addition, according to the present invention, the unit calculation unit increments the value of the layer information stored in the layer information storage unit in response to the comparison coincidence signal input from the unit number comparison unit. A parallel computing device characterized by comprising:

  According to the present invention, there is an effect that each unit can calculate and obtain layer information of the next layer after the operation of a certain layer is completed.

  In the invention, the unit calculation unit includes a data output flag storage unit that stores a data output flag indicating whether or not the data output unit has already output the output value. The unit sets the data output flag of the data output flag storage unit as output in response to a token input to the token input unit, and the token output unit inputs the input value to the data input unit. In response, the parallel output device reads the data output flag from the data output flag storage unit, and outputs the token when the read data output flag has been output. is there.

  According to the present invention, there is an effect that each unit can output a token at a timing according to the case of the hierarchical neural network.

  Further, the present invention is the parallel arithmetic device, wherein the token output unit outputs the token and sets the data output flag of the data output flag storage unit as not output.

  According to the present invention, there is an effect that each unit can output a token at the same timing as the hierarchical neural network even in the next layer.

  Further, according to the present invention, the unit output calculation unit reads product sum calculation primary value information from the product sum calculation primary value calculation unit, and then stores the product sum calculation primary value stored in the product sum calculation primary value calculation unit. A parallel arithmetic device characterized in that the value of the value information is reset to zero.

  According to the present invention, there is an effect that each unit can perform the calculation of the next layer after the calculation of the certain layer is completed.

  Further, in the present invention, when the parallel arithmetic device is a parallel arithmetic device that calculates a self-organizing map, each of the plurality of units is previously identified by coordinates identified by network coordinates, A unit calculation unit, a unit network coordinate storage unit in which unit network coordinates that are network coordinates of the unit are stored in advance, an input network coordinate storage unit in which input network coordinates that are input network coordinates are stored, A weight storage unit storing a unit connection weight vector, a unit norm storage unit storing a norm, an input norm storage unit storing an input norm, and an input value input to the data input unit If it is an input data vector, the input data vector A norm calculation unit that calculates a norm with the combined weight vector read from the weight storage unit, stores the calculated norm as a unit norm in the unit norm storage unit, and an input value input to the data input unit includes a network In the case of a combination of coordinates and norm, the input network coordinates are stored as input network coordinates in the input network coordinate storage unit, and the input norm is stored as input norm in the input norm storage unit. The unit norm read from the unit norm storage unit is compared with the input norm read from the input unit norm storage unit in response to the input of the token via the token input unit. The comparison result between the norm comparison unit and the norm comparison unit is When the unit norm is smaller than the input norm, the unit norm read from the unit norm storage unit and the unit network coordinates read from the unit network coordinate storage unit are output as a set via the data output unit, If the read unit norm is greater than or equal to the input norm, the input norm read from the input norm storage unit and the input network coordinates read from the input network coordinate storage unit are combined as a set via the data output unit. And a selection output unit for outputting.

  According to the present invention, there is an effect that the calculation of the self-organizing map can be performed by the parallel arithmetic device in which the connection between the plurality of arithmetic units is simplified. In addition, since the connection between the plurality of arithmetic units is simple, there is an effect that the problem of wiring hardly occurs even when the number of units in each layer is large in the self-organizing map.

  Further, in the present invention, when the unit calculation unit is a combination of network coordinates and proximity determination identification information that is identification information indicating that the proximity determination is executed, to the data input unit. Is a result of determination by the proximity determination unit that determines whether the unit network coordinates read from the unit network coordinate storage unit are in the vicinity of the input network coordinates that are the input network coordinates, and the determination result of the proximity determination unit And a weight updating unit that updates a connection weight vector stored in the weight storage unit when the is a neighborhood.

  According to the present invention, there is an effect that each unit can learn by updating the connection weight vector.

  Further, according to the present invention, the unit calculation unit includes an input data storage unit that stores an input data vector that is input, a learning speed storage unit that stores a learning speed coefficient that is a coefficient for determining a learning speed, When the input value input to the data input unit is an input data vector, the data input unit stores the input data vector input to the input data storage unit, and the weight update unit A value obtained by multiplying the difference between the input data vector read from the input data storage unit and the combined weight vector read from the weight storage unit by the learning speed coefficient read from the learning speed storage unit is stored in the weight storage unit. The parallel weight vector stored in the weight storage unit is updated by adding to the stored joint weight vector. It is a device.

  According to the present invention, there is an effect that each unit can learn by updating the connection weight vector according to the learning speed.

  Further, according to the present invention, the learning speed storage unit stores a learning step number indicating the number of times of learning and the learning speed coefficient in association with each other, and the unit arithmetic unit stores an input data vector in the unit. Each time the input data vector is input, the counter counts up the number of steps that is the number of times the input data vector is input to the unit, and the weight update unit reads the step number from the counter as the learning step. A parallel computing device, wherein a learning speed coefficient corresponding to a number is read from the learning speed storage unit, and a connection weight vector stored in the weight storage unit is updated based on the read learning speed coefficient. is there.

  According to the present invention, there is an effect that each unit can learn at a learning speed corresponding to the number of learning steps.

  Further, according to the present invention, the unit calculation unit includes a distance storage unit in which a reference distance that is a reference for determining whether or not the distance between the network coordinates is a neighborhood is stored in advance. If the input value input to the data input unit is a combination of network coordinates and proximity determination identification information that is identification information indicating that proximity determination is performed, a reference distance from the distance storage unit And determining whether or not the distance between the unit network coordinates read from the unit network coordinate storage unit and the input network coordinates that are the input network coordinates is equal to or less than the read reference distance. And determining whether or not the unit network coordinates are in the vicinity of the input network coordinates. A.

  According to this invention, there exists an effect that it can be determined whether each unit is the vicinity of a winner unit based on a reference distance.

  Further, according to the present invention, the distance storage unit stores the learning step number and the reference distance in association with each other, and the number of steps read from the counter by the proximity determining unit is the learning step number. The parallel computing device is characterized in that a reference distance corresponding to the above is read from the learning speed storage unit, and based on the read reference distance, it is determined whether or not the unit network coordinates are in the vicinity of the input network coordinates. It is.

  According to this invention, there exists an effect that it can be determined whether each unit is the vicinity of a winner unit based on the reference distance according to the number of learning steps.

  Further, the present invention is the parallel arithmetic device, wherein the token output unit outputs a token in response to a token input to the token input unit.

  According to the present invention, there is an effect that each unit can sequentially determine a winner unit by passing a token to the next unit.

  Further, according to the present invention, the unit calculation unit includes a token output flag storage unit that stores a token output flag indicating whether or not the token output unit has already output a token, and a token is input to the token input unit. In response to the input, the token output flag is read from the token output flag storage unit, and when the read token output flag has not been output, the input token is output to the norm comparison unit. And a token output flag determination unit that outputs the input token via the token output unit.

  According to the present invention, when the winner unit is determined, each unit can participate in the tournament that determines the winner unit only once, and can be prevented from participating in the tournament multiple times.

  Further, according to the present invention, the token output unit sets the unit norm read out from the unit norm storage unit and the unit network coordinates read out from the unit network coordinate storage unit as a set through the data output unit. Is output from the input norm storage unit and the input network coordinates read out from the input network coordinate storage unit as a set by the selection output unit via the data output unit. In response to this, the parallel processing device outputs the token.

  According to the present invention, there is an effect that each unit can output a token at a timing suitable for the self-organizing map.

  Further, the present invention is the parallel processing device, wherein the token output unit updates the token output flag stored in the token output flag storage unit to the output completed after outputting the token. .

  According to the present invention, there is an effect that each unit can be prevented from outputting a token or an output value a plurality of times.

  Further, according to the present invention, in the first redundancy order which is one of the predetermined orders, half of the plurality of units are daisy chained by the first redundancy daisy chain control bus. And the unit daisy chained by the first redundant daisy chain control bus transfers the token to the next unit in the redundancy order in response to the token being input to the token input unit. Stored is a redundant token output unit that outputs via the first redundant daisy chain control bus, and token output selection information that indicates which of the token output unit and the redundant token output unit to select. In response to a token input to the token selection information storage unit and the token input unit, the token selection information storage unit Based on the token output selection information read from the unit, either the token output unit or the redundant token output unit is selected, and the token is output via the selected token output unit or the redundant token output unit. And a token output unit selector.

  According to the present invention, even when a part or a part of a unit of the daisy chain control bus is defective due to the redundant daisy chain control bus, the parallel arithmetic device can operate without any problem. .

  The present invention further includes a redundant token input unit for inputting a token from a unit in the previous redundant order in which the unit is daisy chained via the redundant daisy chain control bus, and the token output unit Is a parallel processing device that outputs the token in response to a token input to the token input unit or the redundant token input unit.

  According to the present invention, even when a part of the daisy chain control bus or a part of the unit is defective due to the redundant daisy chain control bus, each unit can operate without any problem.

  Further, according to the present invention, the remaining half of the plurality of units that are not daisy chained by the first redundant daisy chain control bus in the order of skipping one of the predetermined orders, A unit that is daisy chained by a redundant daisy chain control bus and that is daisy chained by the second redundant daisy chain control bus uses the second redundant daisy chain control bus as the first redundant daisy chain control bus. A parallel processing device having the redundant token output unit, the token selection information storage unit, and the token output unit selection unit.

  According to the present invention, even when a part of the daisy chain control bus or a part of the unit is defective due to the redundant daisy chain control bus, each unit can operate without any problem.

  Further, according to the present invention, the unit that is daisy chained by the second redundant daisy chain control bus uses the second redundant daisy chain control bus as the first redundant daisy chain control bus. It is the parallel arithmetic unit characterized by having.

  According to the present invention, even when a part of the daisy chain control bus or a part of the unit is defective due to the redundant daisy chain control bus, each unit can operate without any problem.

  The present invention also provides a plurality of units daisy-chained via a daisy chain control bus in a predetermined order, and an output value output from any one of the plurality of units via the unit output bus. And a relay unit that outputs the input output value to each of the plurality of units as an input value via a unit input bus. The unit inputs a token from the previous unit in the daisy chain via the daisy chain control bus, and the token is daisy chained in response to the token being input. Output to the next unit through the daisy chain control bus, and the unit output The input value from the relay unit is input via the network, based on the input value input, calculated by a predetermined calculation method for each unit, and according to the token input, The parallel calculation method is characterized in that a calculation result which is the calculation result is output as the output value to the relay unit via the unit output bus.

<Basic configuration>
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a parallel arithmetic device according to an embodiment of the present invention. Here, a case where the number of units U1 to U10 is ten will be described. Details of this unit will be described later.

  The plurality of units U1 to U10 are daisy chained via the daisy chain control bus 1 in a daisy chain order that is a predetermined order. Here, this daisy chain order will be described assuming that the order of unit U1, unit U2, unit U3,..., Unit U10 is predetermined. The daisy chain control bus 1 connects adjacent units in the daisy chain order. For example, the daisy chain control bus 1 connects the unit U1 and the unit U2, the unit U2 and the unit U3,..., The unit U10 and the unit U1. The daisy chain control bus 1 connects units U1 to U10 in a daisy chain sequence. Therefore, the unit U10 and the unit U1 are connected via the daisy chain control bus 1.

  Each of the units U1 to U10 transmits and receives tokens between the units via the daisy chain control bus 1 in the daisy chain order. This token is a signal indicating that any one of the daisy chained units U1 to U10 has a unit, and that the unit having this token is an output unit. As for the unit, only a unit having a token becomes an output unit, and all units become input units.

  Each unit performs an operation by a predetermined operation method for each unit based on an input value from the amplifier 4 input via the unit output bus 2. Also, each unit receives a token from the previous unit via the daisy chain control bus 1 in the daisy chain order, and passes the token to the next unit via the daisy chain control bus 1 in the daisy chain order. At the same time, the calculated result is output as an output value to the amplifier 4 via the unit output bus 2.

  The amplifier (repeater) 4 receives an output value output from any one of the plurality of units U1 to U10 via the unit output bus 2, and inputs the input output value to each of the plurality of units U1 to U10. And output as an input value via the unit input bus 3. The amplifier 4 electrically amplifies the input output value signal and outputs it as an input value.

  One end of the unit output bus 2 is connected to the output terminal P1 of the parallel arithmetic device. Further, the input terminal of the amplifier 4 is connected to the input terminal P2 of the parallel arithmetic device. The unit U1 is connected to the trigger input terminal P3 through the trigger input line 5. The output terminal P1, the input terminal P2, and the trigger input terminal P3 are connected to a control device outside the parallel arithmetic device. The control device inputs data to the parallel arithmetic device via the output terminal P1, the input terminal P2, and the trigger input terminal P3, acquires the arithmetic result from the parallel arithmetic device, and controls the parallel arithmetic device.

  Next, the configuration of each unit will be described with reference to FIG. Since units U1 to U10 have the same configuration, only the configuration of unit U2 will be described here. The unit U2 includes a token input unit 12, a token output unit 13, a data input unit 11, a data output unit 14, a unit calculation unit 15, and a unit output storage unit 16.

  The token input unit 12 inputs a token from the unit in the previous order in the daisy chain via the daisy chain control bus 1. For example, the token input unit 12 of the unit U2 inputs a token from the unit U1 which is the unit in the previous order in the daisy chain via the daisy chain control bus 1. Note that the token input unit 12 of the unit U1 not only receives a token from the unit U10 via the token daisy chain control bus 1, but also receives a token from the control device via the trigger input terminal P3 and the trigger input line 5. Is input via.

  The token output unit 13 outputs the token via the daisy chain control bus 1 to the next unit in the order in which the tokens are daisy chained in response to the token input to the token input unit. For example, the token output unit 13 of the unit U2 outputs a token to the unit U3 which is the unit that is the next unit in the daisy chain order via the daisy chain control bus 1.

  The data input unit 11 inputs an input value from the amplifier 4 via the unit output bus 2. The unit output storage unit 16 stores a calculation result that is a result of calculation by the unit calculation unit 15. The unit calculation unit 15 calculates based on the input value input to the data input unit 11 by a calculation method predetermined for each unit. Further, the unit calculation unit 15 causes the unit output storage unit 16 to store a calculation result that is a calculation result.

  The data output unit 14 outputs, as an output value, a calculation result that is a result of calculation by the unit calculation unit 15 to the amplifier 4 via the unit output bus 2 in response to the token input to the token input unit 12. . Further, the data output unit 14 reads out the calculation result from the unit output storage unit 16 in response to the token input to the token input unit 12, and uses the read calculation result as an output value to the amplifier 4 for the unit output bus 2. Output via.

  As will be described later, the parallel arithmetic device inputs a token that is a trigger to the unit U1 via the trigger input terminal P3, and the tokens are sequentially passed between the units in order from the unit U1, thereby the parallel arithmetic device. The processing in each unit is executed. The operation of the parallel arithmetic device will be described by taking as an example a case where the parallel arithmetic device described below is applied to a hierarchical neural network.

<First Embodiment>
<Unit configuration when applied to hierarchical neural network>
Next, a configuration when the parallel arithmetic device is applied to a hierarchical neural network will be described. Also in the hierarchical neural network, the configuration of the entire parallel arithmetic device is the same as that of the parallel arithmetic device in FIG. 1, and only the configuration of the unit is different. Therefore, the configuration of the unit when applied to the hierarchical neural network will be described with reference to FIG.

  The unit includes a data input unit 101, a token input unit 102, a token output unit 103, and a data output unit 104. The unit includes a weight storage unit 130, a layer information storage unit 133, a unit number storage unit 134, a product-sum calculation primary value storage unit 131, a function storage unit 132, a unit output storage unit 135, and a data output. A flag storage unit 136 and a unit identification information storage unit 137 are provided. The unit includes a counter 114, a product-sum calculation primary value calculation unit 111, a layer information calculation unit 113, and a unit output calculation unit 112.

  Here, the data input unit 101, token input unit 102, token output unit 103, and data output unit 104 of FIG. 3 are connected to the data input unit 11, token input unit 12, token output unit 13, and data output unit 14 of FIG. , Respectively. Further, the unit output storage unit 135 in FIG. 3 corresponds to the unit output storage unit 16 in FIG. Descriptions of common functions in the configurations corresponding to those in FIGS. 2 and 3 are omitted.

  Also, the weight storage unit 130, the layer information storage unit 133, the unit number storage unit 134, the product-sum calculation primary value storage unit 131, the function storage unit 132, the data output flag storage unit 136, the unit of FIG. The identification information storage unit 137, the counter 114, the product-sum calculation primary value calculation unit 111, the layer information calculation unit 113, and the unit output calculation unit 112 correspond to the unit calculation unit 15 in FIG.

  The layer information storage unit 133 stores layer information for identifying an active layer (output layer) among the layers of the hierarchical neural network. The unit identification information storage unit 137 stores unit identification information indicating which unit of the units belonging to the active layer is outputting. In the function storage unit 132, the output function of the unit and the layer information are stored in advance in association with each other. The output function is a predetermined function such as a sigmoid function, a step function, or a piecewise linear function.

  In the weight storage unit 130, a predetermined coupling load multiplied by the output value from the unit of the active layer identified by the layer information is stored in association with the layer information. Further, the weight storage unit 130 stores a predetermined coupling load multiplied by an output value from the unit of the layer identified by the layer information and the unit identification information, and the layer information and the unit identification information in association with each other. ing.

  In the unit number storage unit 134, the layer information and the number of units of the hierarchical neural network belonging to the layer identified by the layer information are stored in advance in association with each other. The product-sum calculation primary value storage unit 131 stores product-sum calculation primary value information obtained by multiplying the input value from the unit corresponding to the preceding unit and the coupling load. The data output flag storage unit 136 stores a data output flag indicating whether or not the data output unit 104 has already output an output value. The counter 114 increments the value of the unit identification information stored in the unit identification information storage unit 137 every time an input value is input to the data input unit 101.

  The unit number comparison unit 115 reads layer information from the layer information storage unit 133, reads the number of units corresponding to the read layer information from the unit number storage unit 134, and reads and reads unit identification information from the unit identification information storage unit 137. The unit identification information value is compared with the number of read units, and if the comparison results match, a comparison match signal indicating that the read unit identification information value matches the read unit number is output. . Further, the unit number comparison unit 115 compares the read unit identification information value with the read unit number, and if the comparison result matches, the unit identification information stored in the unit identification information storage unit 137 Reset the value.

  The product-sum calculation primary value calculation unit 111 reads the connection load corresponding to the layer information read from the layer information storage unit 133 from the weight storage unit 130, the read connection load, and the input value input to the data input unit 101 , And the multiplied value is added to and stored in the product-sum calculation primary value information stored in the product-sum calculation primary value storage unit 131.

  Further, the product-sum calculation primary value calculation unit 111 reads out and reads the combined load corresponding to the unit identification information read from the unit identification information storage unit 137 and the layer information read from the layer information storage unit 133 from the weight storage unit 130. The combined weight and the input value input to the data input unit 101 are multiplied, and the multiplied value is added to and stored in the product-sum calculation primary value information stored in the product-sum calculation primary value storage unit 131.

  The unit output calculation unit 112 reads the product-sum calculation primary value information from the product-sum calculation primary value storage unit 131, reads the output function corresponding to the layer information read from the layer information storage unit 133 from the function storage unit 132, and reads The output value is calculated by substituting the read product-sum calculation primary value information into the output function, and the calculated output value is stored in the unit output storage unit 135.

Further, the unit output calculation unit 112 reads the product sum calculation primary value information from the product sum calculation primary value storage unit 131 in response to the input of the comparison coincidence signal from the unit number comparison unit 115, and the layer information storage unit 133 The output function corresponding to the layer information read out from is read from the function storage unit 132, the output value is calculated by substituting the read product-sum calculation primary value information into the read out output function, and the calculated output value is stored in the unit output Stored in the unit 135.
The unit output calculation unit 112 reads the product-sum calculation primary value information from the product-sum calculation primary value storage unit 131, and then stores the value of the product-sum calculation primary value information stored in the product-sum calculation primary value storage unit 131. Is reset to 0.

  The layer information calculation unit 113 increments the value of the layer information stored in the layer information storage unit 133 in response to the comparison match signal input from the unit number comparison unit 115. The data output unit 104 sets the data output flag of the data output flag storage unit 136 as output in response to the token input to the token input unit 102.

  The token output unit 103 reads the data output flag from the data output flag storage unit 136 in response to the input value being input to the data input unit 101, and the read data output flag is set to have been output. Output a token (token output procedure). In addition, the token output unit 103 outputs a token and sets the data output flag of the data output flag storage unit 136 to non-output.

  The token output unit 103 reads the data output flag from the data output flag storage unit 136 in response to the input value being input to the data input unit 101, and the read data output flag is set to have been output. In this case, instead of the token output procedure of outputting a token, the data output flag storage unit 136 determines that the input value from the data input unit 101 is input to the product-sum calculation primary value calculation unit 111. If the data output flag is read and the read data output flag is set to be output, a token may be output.

<Unit operation when applied to hierarchical neural network>
Next, the operation of the unit when applied to the hierarchical neural network of FIG. 3 will be described with reference to FIGS. First, the operation when an input value is input to the unit will be described with reference to FIG.

  First, an input value is input from the amplifier 4 to the data input unit 101 via the unit input bus 3 (step S100). Next, in step S100, in response to the input value being input to the data input unit 101, the counter 114 increments the value of the unit identification information stored in the unit identification information storage unit 137 by one (step S100). S101).

  In addition, in step S100, in response to the input value being input to the data input unit 101, the product-sum calculation primary value calculation unit 111 reads the unit identification information and the layer information storage unit 133 read from the unit identification information storage unit 137. The combined weight corresponding to the layer information read out from is read out from the weight storage unit 130, the read out combined weight is multiplied by the input value input into the data input unit 101, and the multiplied value is stored as a product-sum calculation primary value storage. The product-addition calculation primary value information stored in the unit 131 is added and stored.

  Next, the unit number comparison unit 115 reads the layer information from the layer information storage unit 133, reads the number of units corresponding to the read layer information from the unit number storage unit 134, and receives the unit identification information from the unit identification information storage unit 137. A comparison match signal indicating that the value of the read unit identification information matches the number of read units when the read and read unit identification information values are compared with the number of read units and the comparison results match. Is output (step S103). Further, the unit number comparison unit 115 compares the read unit identification information value with the read unit number, and the unit identification information stored in the unit identification information storage unit 137 when the comparison result matches. Reset the value of.

  Next, when the comparison results in step S103 match, the layer information calculation unit 113 stores the comparison match signal from the unit number comparison unit 115 and stores it in the layer information storage unit 133. The layer information value is incremented (step S104).

  Next, when the comparison results in step S103 match, the unit output calculation unit 112 responds to the input of the comparison match signal from the unit number comparison unit 115, and the product-sum calculation primary value storage unit 131. The product sum calculation primary value information is read out, the output function corresponding to the layer information read out from the layer information storage unit 133 is read out from the function storage unit 132, and the read product sum calculation primary value information is substituted into the read out output function. Then, the output value is calculated, and the calculated output value is stored in the unit output storage unit 135 (step S105). Next, the unit output calculation unit 112 reads the product-sum calculation primary value information from the product-sum calculation primary value storage unit 131, and then stores the product-sum calculation primary value information stored in the product-sum calculation primary value storage unit 131. The value of is reset to 0.

  Next, or when the result of comparison in step S103 is inconsistent, the token output unit 103 determines that the data output flag is in accordance with the input value input to the data input unit 101 in step S100. A data output flag is read from the storage unit 136, and it is detected whether or not the read data output flag has been output (step S106). If the data output flag is set to output, a token is output. (Step S107).

  In addition, the token output unit 103 outputs a token in step S107, sets the data output flag of the data output flag storage unit 136 to non-output (step S108), and ends the process. On the other hand, when the data output flag is set to non-output in step S106, the token output unit 103 ends the process.

  Next, the operation when a token is input to the unit will be described with reference to FIG. First, tokens from the unit in the previous order in the daisy chain via the daisy chain control bus 1 are input to the token input unit 102 (step S200).

  Next, the data output unit 104 reads the output value (calculation result) from the unit output storage unit 135 in response to the token input to the token input unit 102, and outputs the read output value (calculation result) as the output value. Is output to the amplifier 4 via the unit output bus 2 (step S201). Next, the data output unit 104 sets the data output flag of the data output flag storage unit 136 to “already output” in response to the token input to the token input unit 102 (step S202), and ends the process. .

<Overall configuration of hierarchical neural network>
Next, the overall configuration of the hierarchical neural network according to the present embodiment will be described.
Here, as shown in FIG. 6, a case where there are three layers of an input layer L1, a hidden layer L2, and an output layer L3 as a hierarchical neural network will be described. Each layer is identified by the layer information, and the layer information of the layers is identified such that the layer information increases in order of the layers. For example, the layer information of the input layer L1 is identified as 1, the layer information of the hidden layer L2 is 2, and the layer information of the output layer L3 is identified as 3.

  The input layer L1 has three nodes P101, P102, and P103, and the hidden layer L2 has ten nodes P201, P202, P203, P204, P205, P206, P207, P208, P209, and P210. A case where there are nodes and the output layer L3 has three nodes P301, P302, and P303 will be described. The output of each node of the input layer L1 is input to the input of each node of the hidden layer L2, and the output of each node of the hidden layer L2 is input to the input of each node of the output layer L3.

  In each layer, each node is identified by unit identification information, and the unit identification information of each node is set so that the unit identification information is increased by one in the predetermined node arrangement order. For example, P101 has unit identification information of 1, P102 has unit identification information of 2, and P103 has unit identification information of 3.

In the present embodiment, each unit functions as a node of the hidden layer L2 and the output layer L3. In this example, the units U1 to U3 function as nodes of the hidden layer L2 and the output layer L3, and the units U4 to U10 function as nodes of the hidden layer L2. Here, the control device sequentially outputs the output value as the input layer L1 to each unit as the hidden layer L2 via the amplifier 4 and the unit output bus 2. Therefore, here, each unit functions as a hidden layer L2 and an output layer L3.
For example, the unit U1 functions as the node P201 in the hidden layer L2 and the node P301 in the output layer L3. The unit U3 functions as a node P203 in the hidden layer L2 and a node P303 in the output layer L3. The unit U4 functions as the node P204 of the hidden layer L2.

  Here, when a value is input to a certain unit, the input from the input layer L1, the hidden layer L2, or the output layer L3 is stored in the layer information storage unit 133 of each unit. It is determined based on certain layer information.

<Overall operation of hierarchical neural network>
Next, an overall operation as an example of the hierarchical neural network according to the present embodiment will be described. Here, a case will be described in which each unit functions as a node of the output layer L3, and any one unit to which a token is input functions as an active node of the hidden layer L2. In this case, since each unit functions as a node of the output layer L3 and the active layer is the hidden layer L2, 2 is stored as the value of the layer information in the layer information storage unit 133 of each unit.

  First, the unit U1 which is the node P201 reads the output value from the unit output storage unit 135 and outputs the output value when the token is input from the control device, and sets the data output flag of the data output flag storage unit 136. Set as output completed (procedure A100). Next, the output value from the node P201 is input to the data input unit 101 of each unit of the units U1 to U10 via the unit output bus 2, the amplifier 4, and the unit input bus 3.

Next, each of the units U1 to U10 has unit identification information and a layer information storage unit 133 read from the unit identification information storage unit 137 by the product-sum calculation primary value calculation unit 111 based on the input value from the unit U1. The combined weight corresponding to the layer information read out from is read out from the weight storage unit 130, the read out combined weight is multiplied by the input value input into the data input unit 101, and the multiplied value is stored as a product-sum calculation primary value storage. The product-addition calculation primary value information stored in the unit 131 is added and stored (procedure A101).
Here, the product-sum calculation primary value calculation unit 111 of each unit uses the weight storage unit 130 to determine the combined load corresponding to the unit identification information read from the unit identification information storage unit 137 and the layer information read from the layer information storage unit 133. By reading from, each unit functions as a node of the output layer L3 based on the layer information, and identifies the layer to which the active unit belongs as the hidden layer L2 based on the layer information. Based on the identification information, each node in the active hidden layer L2 is identified, and the coupling load corresponding to the active node in the hidden layer L2 is set as an output value from the node in the active hidden layer L2. It is possible to multiply.

  Here, since the data output flag of the data output flag storage unit 136 of the unit U1 is set as already output, the token output unit 103 of the unit U1 responds to the input value being input to the data input unit 101. Then, the token is output and the data output flag of the data output flag storage unit 136 is set as not output (procedure A102).

  Next, the token output from the unit U1 is input via the daisy chain control bus 1 to the token input unit 12 of the unit U2 which is the next unit in the daisy chain order (procedure A103). .

  In response to the input of the token to the token input unit 12, the unit U2 outputs the unit U1 as the node P101 in the procedure A100 described above from the unit output storage unit 135 in the same manner as when the token is input. The value is read, the output value is output, and the data output flag in the data output flag storage unit 136 is set as already output (procedure A104).

  Thereafter, similarly, the procedure A101 to the procedure A103 are repeated, and the unit U10 that is the node P210 of the hidden layer L2 outputs the output value and sets the data output flag of the data output flag storage unit 136 as already output.

  Next, the units U1 to U3 corresponding to the nodes P301 to P303 of the output layer L3 are similar to the procedure A101, based on the input value from the unit U10, the product-sum calculation primary value calculation unit 111 of each unit. However, the combined load corresponding to the unit identification information read from the unit identification information storage unit 137 and the layer information read from the layer information storage unit 133 is read from the weight storage unit 130, and the read combined load and the data input unit 101 are The inputted input value is multiplied, and the multiplied value is added to and stored in the product-sum calculation primary value information stored in the product-sum calculation primary value storage unit 131 (procedure A105).

Here, in response to the input value from the unit U10 being input to the data input unit 101 of each unit, the counter 114 of each unit displays the value of the unit identification information stored in the unit identification information storage unit 137. And the value of the unit identification information becomes 3 (procedure A106).
The unit number storage unit 134 of each unit stores in advance the layer information and the number of units of the hierarchical neural network belonging to the layer identified by the layer information in association with each other. For example, in the case of the hierarchical neural network of FIG. 6, the value 1 which is the layer information of the input layer L1 and the number of units of the hierarchical neural network, that is, the number of nodes of the input layer L1 are associated with each other. Stored in advance. Further, for example, in the case of the hierarchical neural network of FIG. 6, the value 2 that is the layer information of the hidden layer L2 and the number of units of the hierarchical neural network, that is, the number of nodes of the hidden layer L2 are 10 Pre-stored in association.

  Next, the unit number comparison unit 115 of each unit reads the layer information from the layer information storage unit 133, reads the number of units corresponding to the read layer information from the unit number storage unit 134, and reads the unit number from the unit identification information storage unit 137. The identification information is read out, the value of the read unit identification information is compared with the number of read units, and the comparison result matches, so that the value of the read unit identification information matches the number of units read. A comparison coincidence signal is output, and the value of the unit identification information stored in the unit identification information storage unit 137 is reset (procedure A107).

  Next, the unit output calculation unit 112 of each unit reads the product sum calculation primary value information from the product sum calculation primary value storage unit 131 in response to the input of the comparison coincidence signal from the unit number comparison unit 115, and An output function corresponding to the layer information read from the information storage unit 133 is read from the function storage unit 132, an output value is calculated by substituting the read product-sum calculation primary value information into the read output function, and the calculated output value Is stored in the unit output storage unit 135 (procedure A108).

  Here, the unit output calculation unit 112 of each unit reads the output function corresponding to the layer information read from the layer information storage unit 133 from the function storage unit 132, and therefore, by the output function for each unit corresponding to the output layer L3, The output value of the unit can be calculated. In addition, since the output value calculated as the node of the output layer L3 is stored in the unit output storage unit 135 of each unit, when each unit receives a token, each unit becomes an active node of the output layer L3. An output value can be output.

  Further, the layer information calculation unit 113 of each unit increments the value of the layer information stored in the layer information storage unit 133 in response to the input of the comparison coincidence signal from the unit number comparison unit 115, and The information value is 3. For this purpose, each unit outputs an output value as an active node of the output layer L3 after that any one unit to which a token is input.

  In the above, a token is output from the unit U3 to the unit U4 in the output layer L3, and then the unit U4 to which the token is input outputs an output value as a node of the output layer L3. Since there is no node corresponding to the unit U4 as a node of the output layer L3 in the network, it is desirable that the unit U4 does not output an output value as a node of the output layer L3. Alternatively, the unit U4 may output the output value as 0 as a node of the output layer L3. Alternatively, in each unit, the connection load multiplied by the output value from the unit U4 as the output layer L3 may be set to 0 and stored in the weight storage unit 130 in advance.

  Thus, the unit U4 output value or the connection load of each unit that receives the output value from the unit U4 is set in advance in accordance with the number of nodes in each layer of the hierarchical neural network. On the other hand, even when there is no node corresponding to the unit in a certain layer, the parallel arithmetic device of this embodiment can operate as a hierarchical neural network without any problem.

As described above, when this embodiment is applied to a hierarchical neural network, the number of neurons that fire at one time is limited to one, that is, the node that outputs an output value in the output layer, By limiting to one unit to which a token is input, an effect of reducing an increase in wiring between units can be obtained.
Further, since the output value from the unit which is a node of one output layer is distributed and inputted to the unit which is a node which becomes the input layer at a time via the unit input bus 3, it is necessary for the distribution. It has the effect of reducing time. In addition, since a plurality of units perform in-unit operations in parallel, the overall operation is fast.

  For example, when calculating a hierarchical neural network, if a conventional processor is used, the calculation time is approximately calculated as calculation time = (number of synapses × number of steps required for one calculation ÷ clock frequency). On the other hand, when the parallel arithmetic device according to the present embodiment is used, calculation time = (number of neurons × one neuron firing cycle). Here, since the number of synapses is proportional to the square of the number of neurons, the calculation time of the hierarchical neural network using the parallel arithmetic device according to this embodiment is calculated as compared with the case of using a conventional processor. Time can be greatly reduced.

  In addition, since each unit changes the layer of the hierarchical neural network and functions as a node on the input side and a node on the output side, it is possible to perform multi-layer node operations with a small number of units. There is a possible effect.

  In the first embodiment described above, the control device sequentially outputs output values as the input layer L1, and therefore, each unit functions as a node of the hidden layer L2 and the output layer L3. Each unit may function as a node of the input layer L1. For example, the control device sequentially outputs output values corresponding to the nodes of the input layer L1, and the unit sequentially stores output values from the control device in the unit output storage unit 135 as nodes of the input layer L1. Next, the unit to which the token is input outputs an output value as an active unit of the input layer L1. In this way, each unit can also function as a unit of the input layer L1, the hidden layer L2, and the output layer L3.

  In the first embodiment, the case where there are three layers of the input layer L1, the hidden layer L2, and the output layer L3 as the hierarchical neural network has been described. However, in the present embodiment, the hierarchical neural network The hierarchy is not limited to three layers, and can be applied to any hierarchy.

<Second Embodiment>
<Unit structure when adapted to the self-organizing map>
Next, the case where the parallel arithmetic device of this embodiment is applied to a self-organizing map will be described. The self-organizing map learns the statistical properties of multivariate data by performing the following processes, arranges similar data in close proximity, and visualizes the data: It is possible.

  Each unit has a unit connection weight vector. First, an input data vector is input to each unit, and each unit calculates a norm that is the difference between the input data vector and the connection weight vector (procedure A200). Next, the output unit with the smallest norm is selected as the winner unit (procedure A201). Next, learning is performed by changing the value of the coupling weight vector with a predetermined function so that the selected winner unit and its neighboring units are closer to the input data vector (procedure A203). This procedure A200 to procedure A203 are repeated.

  When the parallel computing device of this embodiment is applied to a self-organizing map, the data input to each unit is different in the above-described procedure A200, procedure A201, and procedure A202. The input data is identified by combining identification information for identifying the procedure A200, the procedure A201, and the procedure A202.

  As shown in FIG. 9A, the input value from the unit input bus 3 has a norm part and a data part. Further, as shown in FIG. 9B, when the value of the norm part is −2, the norm part of the input value is an identifier indicating that “the value of the data part is an input vector”, and the data part Is the input vector. As shown in FIG. 9C, when the value of the norm part is −1, the norm part is “neighboring determination identification information”, and the data part is network coordinates. Further, as shown in FIG. 9D, when the value of the norm part is zero or a positive value, the value of the norm part is the norm, and the data part is the network coordinates. Details of these will be described later.

  Next, with reference to FIGS. 7 and 8, the configuration of the units of the parallel arithmetic device when applied to the self-organizing map will be described. Here, each of the plurality of units is identified in advance by coordinates identified by network coordinates. Since each unit has the same configuration, only one unit will be described.

  7 or 8, the data input unit 201 corresponds to the data input unit 11, the data output unit 204 corresponds to the data output unit 14, and the token input unit 202 corresponds to the token input unit 12. The token output unit 203 corresponds to the token output unit 13. The unit norm storage unit 236, the input norm storage unit 237, the unit network coordinate storage unit 234, and the input network coordinate storage unit 233 correspond to the unit output storage unit 16. The other configuration corresponds to the unit calculation unit 15.

  In the unit network coordinate storage unit 234, unit network coordinates, which are unit network coordinates, are stored in advance. The input network coordinate storage unit 233 stores input network coordinates that are input network coordinates. The weight storage unit 231 stores unit connection weight vectors in advance. The input data storage unit 230 stores input input data vectors. The unit norm storage unit 236 stores the unit norm. The input norm storage unit 237 stores the input norm.

  The learning speed storage unit 235 stores a learning speed coefficient that is a coefficient for determining the learning speed in advance. In addition, the learning speed storage unit 235 stores a learning step number indicating the number of times of learning and the learning speed coefficient in association with each other in advance. The distance storage unit 232 stores in advance a reference distance that is a reference for determining whether or not the distance between the network coordinates is near. The distance storage unit 232 stores the number of learning steps and the reference distance in association with each other.

The data input determination unit 222 determines whether or not the input value input to the data input unit 201 is an input data vector. If the input value is an input data vector, the input data that is an input value to the data extraction unit 210 is determined. When the vector is output and not the input data vector, the input value is output to the network coordinate extraction unit 211 and the norm extraction unit 212.
The data input determination unit 222 is input to the data input unit 201 by determining whether the norm value of the input value is −2 that is a value indicating that the input value is an input data vector. It is determined whether the input value is an input data vector.

  If the input value input to the data input unit 201 is an input data vector, the data extraction unit 210 causes the input data storage unit 230 to store the input data vector input. Further, the data extraction unit 210 outputs the input data vector that has been input to the norm calculation unit 214. In the data extraction unit 210, the counter 213 outputs a count-up signal each time an input data vector is input.

  Each time an input data vector is input to the unit, the counter 213 counts up the number of steps that is the number of times the input data vector is input to the unit. The counter 213 counts up the number of steps each time a count up signal is input from the data extraction unit 210.

  When the input value input to the data input unit 201 is an input data vector, the norm calculation unit 214 calculates a norm between the input data vector input and the combined weight vector read from the weight storage unit 231. The calculated norm is stored in the unit norm storage unit 236 as a unit norm.

  When the input value input to the data input unit 201 is a combination of network coordinates and norm, the input norm unit 250 stores the input network coordinates as input network coordinates in the input network coordinate storage unit 233. At the same time, the input norm is stored in the input norm storage unit 237 as an input norm. The input norm unit 250 includes a network coordinate extraction unit 211, a norm extraction unit 212, and an input norm determination unit 218.

  The network coordinate extraction unit 211 extracts network coordinates from the input values that have been input, and stores the extracted network coordinates in the input network coordinate storage unit 233 as input network coordinates. The norm extraction unit 212 extracts a norm from the input value that has been input, and outputs the extracted norm to the input norm determination unit 218.

The input norm determination unit 218 determines whether the input norm is a norm based on the norm value input from the norm extraction unit 212 or is identification information indicating identification information indicating that the network coordinates and the proximity determination are executed. Whether it is identification information or not is determined. If it is a norm, the input norm is stored in the input norm storage unit 237 as an input norm. Conversely, when the input norm is the proximity determination identification information, the input norm determination unit 218 outputs the proximity determination identification information to the proximity determination unit 215.
For example, when the norm value input from the norm extraction unit 212 is −1, the input norm determination unit 218 determines that the input norm is proximity determination identification information, and the norm extraction unit 212 If the input norm value is not -1, it is determined as the norm.

  The norm comparison unit 217 compares the unit norm read from the unit norm storage unit 236 with the input norm read from the input norm storage unit 237 in response to the token input via the token input unit 202.

  The selection output unit 251 determines that the unit norm read from the unit norm storage unit 236 and the unit read from the unit network coordinate storage unit 234 when the comparison result of the norm comparison unit 217 is smaller than the input norm. When the unit norm output from the data output unit 204 as a pair with the network coordinates and read out is equal to or larger than the input norm, the input norm read from the input norm storage unit 237 and the input network coordinate storage unit 233 are read. The input network coordinates are combined and output via the data output unit 204. The selection output unit 251 includes a network coordinate selection unit 219, a norm selection unit 220, and a data synthesis output unit 221.

  When the comparison result of the norm comparison unit 217 indicates that the read unit norm is smaller than the input norm, the norm selection unit 220 reads the unit norm from the unit norm storage unit 236, and uses the read unit norm as the data synthesis output unit 221. Output to. Further, the network coordinate selection unit 219 reads the input norm from the input norm storage unit 237 when the comparison result of the norm comparison unit 217 indicates that the read unit norm is greater than or equal to the input norm, and synthesizes the read input norm Output to the output unit 221.

  When the result of comparison by the norm comparison unit 217 indicates that the read unit norm is smaller than the input norm, the network coordinate selection unit 219 reads the unit network coordinates from the unit network coordinate storage unit 234 and uses the read unit network coordinates as data. The result is output to the composite output unit 221. The network coordinate selection unit 219 reads the input network coordinates from the input network coordinate storage unit 233 when the unit norm read by the norm comparison unit 217 is greater than or equal to the input norm, and reads the input network The coordinates are output to the data composition output unit 221.

  The data synthesis output unit 221 synthesizes a unit norm input from the norm selection unit 220 or a norm that is an input norm and a network coordinate that is a unit network coordinate or input network coordinate input from the network coordinate selection unit 219. Then, the norm synthesized as a set and the network coordinates are output via the data output unit 204.

  When the input value input to the data input unit 201 is a set of network coordinates and proximity determination identification information that is identification information indicating that the proximity determination is executed, the proximity determination unit 215 stores the unit network coordinate storage. It is determined whether the unit network coordinates read from the unit 234 are in the vicinity of the input network coordinates that are the input network coordinates.

  Further, the neighborhood determination unit 215 receives the unit network coordinates read from the unit network coordinate storage unit 234 in response to the input of the neighborhood determination identification information from the input norm determination unit 250 or the input norm determination unit 218. It is determined whether or not the input network coordinates are coordinates.

  The proximity determination unit 215 stores the distance when the input value input to the data input unit 201 is a set of network coordinates and proximity determination identification information that is identification information indicating that the proximity determination is executed. The reference distance is read from the unit 232, and it is determined whether or not the distance between the unit network coordinates read from the unit network coordinate storage unit 234 and the input network coordinates that are the input network coordinates is equal to or less than the read reference distance. By doing so, it is determined whether or not the unit network coordinates are in the vicinity of the input network coordinates.

  Further, the proximity determination unit 215 reads a reference distance associated with the number of learning steps that matches the number of steps read from the counter 213 from the distance storage unit 232, and the unit network coordinates are calculated based on the read reference distance. It is determined whether it is in the vicinity of the input network coordinates.

  The weight update unit 216 updates the value of the coupling weight vector stored in the weight storage unit 231 when the result determined by the proximity determination unit 215 is a neighborhood. Further, the weight update unit 216 calculates the difference between the input data vector read from the input data storage unit 230 and the combined weight vector read from the weight storage unit 231, and reads the calculated difference from the learning speed storage unit 235. A value obtained by multiplying the learning speed coefficient is added to the connection weight vector stored in the weight storage unit 231, thereby updating the value of the connection weight vector stored in the weight storage unit 231.

  Also, the weight update unit 216 reads from the learning speed storage unit 235 the learning speed coefficient associated with the number of learning steps that matches the number of steps read from the counter 213, and the weight storage unit 231 based on the read learning speed coefficient. Update the value of the connection weight vector stored in.

  The token output flag storage unit 238 stores a token output flag indicating whether or not the token output unit 203 has already output a token. The token output flag determination unit 223 reads the token output flag from the token output flag storage unit 238 in response to the token input to the token input unit 202, and when the read token output flag has not output a token. The input token is output to the norm comparison unit 217, and the input token is output via the token output unit 203.

  The token output unit 203 outputs a token in response to the token input to the token input unit 202. The token output unit 203 also includes a data output unit that includes the unit norm read from the unit norm storage unit 236 and the unit network coordinates read from the unit network coordinate storage unit 234 by the data synthesis output unit 221 or the selection output unit 251 as a set. Output via the data output unit 204 in response to the output via the data output unit 204 or as a combination of the input norm read from the input norm storage unit 237 and the input network coordinates read from the input network coordinate storage unit 233 Depending on, the token is output. Further, after outputting the token, the token output unit 203 updates the token output flag stored in the token output flag storage unit 238 to be output.

<Unit operation when applied to a self-organizing map>
Next, with reference to FIGS. 10 and 11, the operation of the units of the parallel arithmetic device when applied to the self-organizing map will be described. First, the operation when an input value is input to the data input unit of the unit will be described with reference to FIG.

  First, data that is an input value is input to the data input unit 201 (step S400). Next, the data input determination unit 222 determines whether or not the norm value of the input value is −2 that is a value indicating that the input value is an input data vector. It is determined whether or not the input value is an input data vector (step S401).

  Next, when the determination result in step S401 is that the norm value is −2 and an input data vector, the data extraction unit 210 stores the input data vector input in the input data storage unit 230 ( A count-up signal is output to the counter 213 together with step S402). Next, the counter 213 counts up the number of steps in response to the input of the count up signal from the data extraction unit 210 (step S403).

  Next, the norm calculation unit 214 calculates a norm between the input data vector input via the data extraction unit 210 and the combined weight vector read from the weight storage unit 231, and uses the calculated norm as a unit norm. The data is stored in the storage unit 236 (step S404), and the process ends.

  On the other hand, if the determination result in step S401 is that the norm value is not −2 and is not an input data vector, the network coordinate extraction unit 211 extracts network coordinates from the input values that have been input, and extracts the extracted network. The coordinates are stored in the input network coordinate storage unit 233 as input network coordinates (step S405), and the norm extraction unit 212 extracts the norm from the input value that has been input, and outputs the extracted norm to the input norm determination unit 218. To do.

  Next, the input norm determination unit 218 determines whether or not the norm value input from the norm extraction unit 212 is −1 (step S406), and the determination result is the norm input from the norm extraction unit 212. If the value of is not −1, it is determined as a norm, the input norm is stored as an input norm in the input norm storage unit 237 (step S407), and the process is terminated.

  On the other hand, if the determination result in step S406 is that the norm value input from the norm extraction unit 212 is -1, the input norm determination unit 218 determines that the input norm is the proximity determination identification information. The proximity determination identification information is output to the proximity determination unit 215. Next, the neighborhood determining unit 215 inputs the unit network coordinates read from the unit network coordinate storage unit 234 in response to the input of the neighborhood determining identification information from the input norm determining unit 218 being the input network coordinates. It is determined whether or not it is in the vicinity of the network coordinates (step S408).

  If the determination result in step S408 is near, the weight updating unit 216 updates the value of the coupling weight vector stored in the weight storage unit 231 (step S409), and the process ends. On the other hand, if the determination result in step S408 is not close, the process ends.

  Next, the operation when a token is input to the token input unit of the unit will be described with reference to FIG. First, a token is input to the token input unit 202 (step S500). Next, the token output flag determination unit 223 reads the token output flag from the token output flag storage unit 238 in response to the token input to the token input unit 202, and the read token output flag indicates that the token has not been output. It is determined whether or not there is a token (step S501). If the read token output flag indicates that the token has not been output, the input token is output to the norm comparison unit 217, and the input token is output as a token. The data is output via the unit 203.

  Next, the norm comparison unit 217 calculates the unit norm read from the unit norm storage unit 236 and the input norm read from the input norm storage unit 237 in response to the token input via the token input unit 202. Compare (step S502).

  When the comparison result of the norm comparison unit 217 indicates that the read unit norm is smaller than the input norm, the norm selection unit 220 reads the unit norm from the unit norm storage unit 236, and the read unit norm is the data synthesis output unit 221. (Step S503), the network coordinate selection unit 219 reads the unit network coordinates from the unit network coordinate storage unit 234, and outputs the read unit network coordinates to the data composition output unit 221 (step S504).

  Next, the data combination output unit 221 combines the unit norm input from the norm selection unit 220 and the unit network coordinates input from the network coordinate selection unit 219 as a set, and combines the unit norm and unit network combined as a set. The coordinates are output via the data output unit 204 (step S505).

  On the other hand, if the result of comparison by the norm comparison unit 217 in step S502 indicates that the read unit norm is greater than or equal to the input norm, the norm selection unit 220 reads the input norm from the input norm storage unit 237 and reads the read input norm. Is output to the data composition output unit 221 (step S508), and the network coordinate selection unit 219 reads the input network coordinates from the input network coordinate storage unit 233, and outputs the read input network coordinates to the data composition output unit 221 ( Step S509).

  Next, the data synthesis output unit 221 synthesizes the input norm input from the norm selection unit 220 and the input network coordinates input from the network coordinate selection unit 219 as a set, and combines the input norm and the input network combined as a set. The coordinates are output via the data output unit 204 (step S510).

  Next, in step S <b> 505, the token output unit 203 outputs data by combining the unit norm read from the unit norm storage unit 236 and the unit network coordinates read from the unit network coordinate storage unit 234 by the data composition output unit 221. The data output unit 204 sets the input norm read from the input norm storage unit 237 and the input network coordinates read from the input network coordinate storage unit 233 as a set in response to the output via the unit 204 or in step S510. In response to the output via, a token is output (step S506).

  Next, after outputting the token, the token output unit 203 updates the token output flag stored in the token output flag storage unit 238 to have been output (step S507), and ends the process.

<Operation of entire parallel computing device when adapted to self-organizing map>
Next, the operation of the entire parallel computing device when applied to the self-organizing map will be described. First, the control device inputs an input value composed of an input vector and a norm having a value of −2 to all units via the unit input bus 3 (procedure A300).

  Next, the norm calculation unit 214 of each unit that has received the input vector calculates the norm between the input vector that has been input and the connection weight vector that the unit has, and stores the calculated norm in the unit norm storage unit 236. (Procedure A301).

  Next, the control device inputs a token to the unit U1 (procedure A302). Next, the norm comparison unit 217 of the unit U1 receives the unit norm read from the unit norm storage unit 236 and the input norm storage unit 237 in response to the token input via the token input unit 202 of the unit U1. The read input norm is compared (procedure A303).

  Here, the input norm storage unit 237 of the unit U1 will be described assuming that a norm having a sufficiently large value is stored in advance as compared with the norm calculated by the unit U1 in step A301.

  Here, since the unit norm read from the unit norm storage unit 236 is smaller than the input norm read from the input norm storage unit 237, the selection output unit 251 of the unit U1 reads the unit norm read from the unit norm storage unit 236. And the unit network coordinates read from the unit network coordinate storage unit 234 are output as a set via the data output unit 204 (procedure A304).

  Next, the norm output from the data output unit 204 of the unit U1 and the network coordinates are input as a set to the data input unit 201 of each unit via the unit output bus 2, the amplifier 4, and the unit input bus 3. (Procedure A305).

  Next, since the norm value of the input value input to the data input unit 201 is not −1, the input norm unit 250 of each unit stores the input network coordinates as input network coordinates in the input network coordinate storage unit 233. The input norm is stored in the input norm storage unit 237 as an input norm (procedure A306). Here, the network coordinates stored in the input network coordinate storage unit 233 and the norm stored in the input norm storage unit 237 are the network coordinates and norm of the unit U1.

  Next, the token output unit 203 of the unit U1 includes the unit norm read from the unit norm storage unit 236 and the unit network coordinates read from the unit network coordinate storage unit 234 by the data synthesis output unit 221 or the selection output unit 251 of the unit U1. And a token is output to the unit U2 via the daisy chain control bus 1 (procedure A307).

  Next, similarly to the unit U1 in the procedure A303, the norm comparison unit 217 of the unit U2 receives the token from the unit U1 via the token input unit 202 of the unit U2, and then the unit norm storage unit 236. Is compared with the input norm read from the input norm storage unit 237 (procedure A308).

  Here, since the input norm stored in the input norm storage unit 237 is the norm of the unit U1, and the unit norm read from the unit norm storage unit 236 is the norm of the unit U2, the norm of the unit U2 The comparison unit 217 compares the norm of the unit U2 with the norm of the unit U1. Thereafter, based on the comparison result of the norm comparison unit 217 of the unit U2, the processing from the procedure A304 to the procedure A306 is executed similarly (procedure A309).

  Therefore, when the token is input to the unit U2, the norms of the unit U1 and the unit U2 are compared, the unit with the smaller norm is selected as the winner unit, and the winner unit norm and unit network coordinates are all Are stored in the input norm storage unit 237 and the input network coordinate storage unit 233 of the unit (step A310).

  Next, similarly to the procedure A307, the token is transmitted from the unit U2 to the unit U3, and then the unit U3 executes the procedure A303 to the procedure A307, thereby winning the unit U1 and the unit U2. The norm of the unit is compared with the norm of the unit U3, and the winner unit is selected from the units U1 to U3 (procedure A311).

  Thereafter, each time tokens are delivered to the units in the daisy chain order, the winner unit among the units to which the tokens are input is selected (procedure A312). Thereafter, when a token is input to the unit U10 which is the last unit in the daisy chain order, the data output unit 204 of the unit U10 outputs the norm and network coordinates of the winner unit in the unit U10 from the unit U1 ( Procedure A313).

  The control device receives the norm and network coordinates output from the data output unit 204 of the unit U10 as the norm and network coordinates of the winner unit (procedure A314). Thereby, the control apparatus can receive the norm of the winner unit and the network coordinates with respect to the input vector.

  Next, the control device sends the input value that is a combination of the network coordinates received from the data output unit 204 of the unit U10 and the norm with a value of −1 to all units via the unit input bus 3. Input (procedure A315).

  Next, each unit to which an input value that is a combination of the network coordinate and the norm with a value of −1 is input, the network coordinate extraction unit 211 of each unit extracts the network coordinate from the input value. The extracted network coordinates are stored in the input network coordinate storage unit 233 as input network coordinates (procedure A316), and the input norm determination unit 218 of each unit has a norm value of −1 input from the norm extraction unit 212. Therefore, it is determined that the input norm is the proximity determination identification information, and the proximity determination identification information is output to the proximity determination unit 215 (procedure A317).

  Next, the unit network coordinates read from the unit network coordinate storage unit 234 by the proximity determination unit 215 of each unit in response to the input of the proximity determination identification information from the input norm determination unit 250 or the input norm determination unit 218, It is determined whether or not the input network coordinate is read from the input network coordinate storage unit 233 (step A318).

  Next, the weight update unit 216 of each unit updates the value of the combined weight vector stored in the weight storage unit 231 of the unit when the result determined by the proximity determination unit 215 of the unit is a neighborhood. (Procedure A319). Thereby, the connection weight vector of the unit that is in the vicinity of the network coordinates of the winner unit is updated, and learning is executed.

  In the parallel processing device adapted to the self-organizing map described above, all the units are connected to the unit input bus 3, so that the control device only outputs the input vector to the unit input bus 3 once. Thus, input vectors can be input to all units. Further, since each unit can calculate the norm in parallel with respect to the input vector that has been input, the norm can be calculated in a short time for the entire unit.

  Further, since all the units are connected to the unit input bus 3, the control device only outputs the network coordinates of the winner unit once to the unit input bus 3, and inputs the network coordinates of the winner unit to all the units. be able to. In addition, since each unit determines and learns in parallel whether or not each unit is in the vicinity of the network coordinates of the input winner unit, the entire unit can be learned in a short time.

  Also, by passing tokens between units via the daisy chain control bus 1, the winner unit can be determined in order of daisy chain. That is, the winner unit can be determined by simple wiring of the daisy chain control bus 1.

<Parallel processing unit with redundancy>
Next, the configuration of the parallel arithmetic unit when redundancy is provided to the daisy chain control bus 1 will be described with reference to FIG.
Each unit has a first redundancy order that is a skipping order in a daisy chain order that is a predetermined order, and half of the plurality of units are connected to the first redundant daisy chain control bus 6. Daisy chained by. For example, the unit U 1, the unit U 3, the unit U 5, the unit U 7 and the unit U 9 are daisy chained by the first redundant daisy chain control bus 6.

  Further, in the daisy chain order that is a predetermined order, the remaining half of the plurality of units that are not daisy chained by the first redundant daisy chain control bus 6 in the order of one skip are The two redundant daisy chain control buses 7 are daisy chained. For example, the unit U 2, the unit U 4, the unit U 6, the unit U 8, and the unit U 10 are daisy chained by the second redundant daisy chain control bus 7.

  Next, the configuration of units daisy-chained by the first redundant daisy chain control bus 6 will be described with reference to FIG. That is, for example, the configuration of the unit U1, the unit U3, the unit U5, the unit U7, and the unit U9 will be described. In order to describe only the configuration that is changed with respect to redundancy, only the configuration that is changed in the configuration of the unit in FIG. 2 will be described.

  In FIG. 13, FIGS. 13A and 13C show the configuration of the unit shown in FIG. 2 before being changed. By making the parallel computing device have redundancy, the configuration of the unit in FIG. 13A is changed to the configuration in FIG. Further, the configuration of the unit in FIG. 13C is changed to the configuration of the unit in FIG.

  As shown in FIG. 13B, the unit includes a token output unit A13_1, a token output unit B13_2, a token output unit selection unit 17, and a token selection information storage unit 18. Here, the token output unit A13_1 corresponds to the token output unit 13 of FIG.

  The token output unit B13_2 (redundant token output unit) sends the token to the next unit in the redundant order via the first redundant daisy chain control bus 6 in response to the token input to the token input unit 12. Output. The token selection information storage unit 18 stores token output selection information indicating which one of the token output unit A13_1 and the token output unit B13_2 is selected.

  Based on the token output selection information read from the token selection information storage unit 18 in response to the token input to the token input unit 12, the token output unit selection unit 17 performs the token output unit A13_1 and the token output unit B13_2. Is selected, and a token is output via the selected token output unit A13_1 or token output unit B13_2.

  Moreover, as shown in FIG.13 (d), a unit has token input part A12_1 and token input part B12_2 (redundant token input part). Here, the token input unit A12_1 corresponds to the token input unit 12 of FIG.

The token input unit B12_2 inputs a token from the unit in the previous redundancy order that is daisy chained via the first redundant daisy chain control bus 6.
Further, the token output unit 13 outputs a token in response to the token input to the token input unit A12_1 or the token input unit B12_2.

  Similarly to the units daisy chained by the first redundant daisy chain control bus 6, the units daisy chained by the second redundant daisy chain control bus 7 also use the second redundant daisy chain control bus 7. The first redundant daisy chain control bus 6 includes a token input unit B12_2 (redundant token output unit), a token selection information storage unit 18, a token output unit selection unit 17, and a token input unit B12_2 (redundant token input unit). Yes.

  Here, the token selection information storage unit 18 has been described assuming that the token output selection information indicating which one of the token output unit A13_1 and the token output unit B13_2 is selected is stored. The token output selection information may be information indicating whether or not the next unit is defective in the daisy chain order, for example.

  When the token output selection information is information indicating whether or not the next unit is defective in the daisy chain order, the token output unit selection unit 17 is based on the token output selection information read from the token selection information storage unit 18. If there is no defect in the next unit in the daisy chain order, the token output unit A13_1 is selected and a token is output. Conversely, if there is a defect, the token output unit B13_2 is selected and a token is output. To do.

<Defect check method>
Next, a defect check method will be described. Here, a case where the parallel arithmetic device is a hierarchical neural network will be described. First, the parallel processing unit of the neural network executes the calculation as a hierarchical neural network having the same coupling coefficient and two layers of the input layer and the output layer without the intermediate layer, and the control device outputs the calculation result. Get the value. The control device determines the output value, and if normal, completes the process. When the control device determines the output value and detects that there is an abnormality, it writes defect information in the token selection information storage unit 18 of the preceding unit of the abnormal unit.

  Next, the determination method and its processing when the control device is normal and abnormal will be described in detail. First, when all the units obtain the same output from all units, the control device determines that the unit is normal. Next, the control device determines that the output from the unit stops in the middle or when the output value from the unit is different from the expected value as an abnormality. In this case, the control device writes the defect information in the token selection information storage unit 18 of the preceding unit that is the previous unit in the daisy chain order of the abnormal unit that is the unit detected as abnormal, performs the defect check again, The defect check is executed until the judgment result becomes normal. At this time, a mechanism for electrically disconnecting the circuit may be provided by providing a high-impedance switch in the input / output unit of the unit in case the electrical disconnection is required due to a defect.

  As described above, the parallel arithmetic device has redundancy by using the first redundant daisy chain control bus 6 and the second redundant daisy chain control bus 7, thereby enabling the daisy chain control bus 1 or any one of them. Even when a defect occurs in the unit, the parallel arithmetic device can operate without any problem.

  Here, a case has been described in which the redundant wirings are the first redundant daisy chain control bus 6 and the second redundant daisy chain control bus 7 which are skipped by one in the daisy chain order. It is possible to add two redundant daisy chain control buses.

Each of the storage units having the configuration shown in FIG. 2, FIG. 3, FIG. 7 and FIG. 8 of the above embodiment includes a nonvolatile memory such as a hard disk device, a magneto-optical disk device, a flash memory, or a CR-ROM. And the like, a volatile memory such as a RAM (Random Access Memory), or a combination thereof.
2 may be realized by dedicated hardware, or may be realized by a memory and a microprocessor.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes a design and the like within the scope not departing from the gist of the present invention.

It is a block diagram which shows the structure of the parallel arithmetic unit by one Embodiment of this invention. It is a block diagram which shows the structure of the unit of the parallel arithmetic unit by one Embodiment of this invention. It is a block diagram which shows the structure of the unit of FIG. 2 in a hierarchical neural network. FIG. 4 is a first flowchart showing the operation of the unit of FIG. 3. It is a 2nd flowchart figure which shows operation | movement of the unit of FIG. It is explanatory drawing explaining a neural network. FIG. 3 is a first block diagram showing a configuration of a unit of FIG. 2 in a self-organizing map. It is a 2nd block diagram which shows the structure of the unit of FIG. 2 in a self-organization map. It is explanatory drawing which shows the data structure in a self-organization map. FIG. 9 is a first flowchart showing the operation of the unit of FIGS. 7 and 8. It is a 2nd flowchart figure which shows operation | movement of the unit of FIG. 7 and FIG. It is a block diagram which shows the structure of the parallel arithmetic unit which has a redundant circuit. It is a block diagram which shows the structure of the unit of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Daisy chain control bus 2 Unit output bus 3 Unit input bus 4 Amplifier 5 Trigger input line 6 1st redundant daisy chain control bus 7 2nd redundant daisy chain control bus 11, 101, 201 Data input part 12, 102, 202 Token input unit 13, 103, 203 Token output unit 14, 104, 204 Data output unit 16 Unit output storage unit 17 Token output unit selection unit 18 Token selection information storage unit 111 Product sum calculation primary value calculation unit 112 Unit output calculation unit 113 Layer information calculation unit 114 Counter 130, 231 Weight storage unit 131 Product sum calculation primary value storage unit 132 Function storage unit 133 Layer information storage unit 134 Unit number storage unit 135 Unit output storage unit 136 Data output flag storage unit 137 Unit identification information storage Part 21 0 data extraction unit 211 network coordinate extraction unit 212 norm extraction unit 213 counter 214 norm calculation unit 215 proximity determination unit 216 weight update unit 217 norm comparison unit 218 input norm determination unit 219 network coordinate selection unit 220 norm selection unit 221 data synthesis output unit 222 data input determination unit 223 token output flag determination unit 230 input data storage unit 232 distance storage unit 233 input network coordinate storage unit 234 unit network coordinate storage unit 235 learning speed storage unit 236 unit norm storage unit 237 input norm storage unit 238 token output Flag storage unit 250 Input norm unit 251 Selection output unit L1 Input layer L2 Hidden layer L3 Output layer P1 Output terminal P2 Input terminal P3 Trigger input terminal U1, U2, U3, U4, U5, U , U7, U8, U9, U10 unit

Claims (25)

Parallel computing device
A plurality of units daisy-chained via a daisy-chain control bus in a predetermined order;
An output value output from any one of the plurality of units is input via a unit output bus, and the input output value is output as an input value to each of the plurality of units via a unit input bus. A relay section;
Have
The unit is
A token input unit for inputting a token from a unit in the previous order of being daisy chained via the daisy chain control bus;
A token output unit configured to output the token to the next unit in the order in which the tokens are daisy chained via the daisy chain control bus in response to a token being input to the token input unit;
A data input unit for inputting an input value from the relay unit via the unit output bus;
Based on the input value input to the data input unit, a unit calculation unit for calculating by a predetermined calculation method for each unit;
A data output unit that outputs, as the output value, a computation result that is a result of computation by the unit computation unit in response to a token being input to the token input unit, to the relay unit via the unit output bus; ,
A parallel computing device comprising: The unit has a unit output storage unit in which the calculation result is stored,
The unit calculation unit stores the calculation result in the unit output storage unit,
The data output unit reads the calculation result from the unit output storage unit in response to a token input to the token input unit, and outputs the read calculation result as the output value to the relay unit as the unit output. Output via bus,
The parallel arithmetic apparatus according to claim 1, wherein: When the parallel computing device is a parallel computing device that computes a hierarchical neural network,
The unit calculation unit is
Of the layers of the hierarchical neural network, a layer information storage unit storing layer information for identifying a layer to be output;
A function storage unit in which the output function of the unit and the layer information are associated and stored in advance;
A weight storage unit in which a predetermined combined load multiplied by an output value from the unit of the output layer identified by the layer information is stored in association with the layer information;
A product-sum calculation primary value storage unit for storing product-sum calculation primary value information obtained by multiplying an input value from the output unit and a combined load;
The connection load corresponding to the layer information read from the layer information storage unit is read from the weight storage unit, the read connection load is multiplied by the input value input to the data input unit, and the multiplied value is obtained. A product-sum calculation primary value calculation unit for adding and storing the product-sum calculation primary value information stored in the product-sum calculation primary value storage unit;
The product-sum calculation primary value information is read from the product-sum calculation primary value storage unit, the output function corresponding to the layer information read from the layer information storage unit is read from the function storage unit, and the read to the output function is read. A unit output calculation unit that calculates an output value by substituting product-sum calculation primary value information, and stores the calculated output value in the unit output storage unit;
The parallel arithmetic apparatus according to claim 2, wherein: The unit calculation unit has a unit identification information storage unit in which unit identification information indicating which unit outputs as an output unit among the units belonging to the output layer is stored,
In the weight storage unit, a predetermined coupling load to be multiplied by an output value from the unit of the output layer identified by the layer information and the unit identification information, the layer information and the unit identification information, Associated with it,
The product-sum calculation primary value calculation unit reads the combined load corresponding to the unit identification information read from the unit identification information storage unit and the layer information read from the layer information storage unit from the weight storage unit, and reads the read Multiplying the combined weight by the input value input to the data input unit, and adding the multiplied value to the product-sum calculation primary value information stored in the product-sum calculation primary value storage unit,
The parallel arithmetic apparatus according to claim 3. The unit calculation unit is
A counter that increments a value of unit identification information stored in the unit identification information storage unit each time an input value is input to the data input unit;
A unit number storage unit in which the layer information and the number of units of the hierarchical neural network belonging to the hierarchy identified by the layer information are stored in advance,
The layer information is read from the layer information storage unit, the number of units corresponding to the read layer information is read from the unit number storage unit, the unit identification information is read from the unit identification information storage unit, and the unit identification information A unit that compares a value with the number of read units and outputs a comparison match signal indicating that the value of the read unit identification information and the number of read units match when the comparison result matches A number comparison section;
Have
The unit output calculation unit reads product sum calculation primary value information from the product sum calculation primary value calculation unit and reads from the layer information storage unit in response to a comparison coincidence signal input from the unit number comparison unit. An output function corresponding to the layer information is read from the function storage unit, an output value is calculated by substituting the read product-sum calculation primary value information into the read output function, and the calculated output value is Store in the output storage unit,
The parallel arithmetic apparatus according to claim 4, wherein:   6. The parallel operation according to claim 5, wherein the unit number comparison unit resets the value of the unit identification information stored in the unit identification information storage unit when the comparison result matches. apparatus.   The unit calculation unit includes a layer information calculation unit that increments a value of layer information stored in the layer information storage unit in response to a comparison match signal input from the unit number comparison unit. The parallel arithmetic apparatus according to claim 5 or 6, characterized in that The unit calculation unit includes a data output flag storage unit in which a data output flag indicating whether or not the data output unit has already output the output value is stored;
In response to the token input to the token input unit, the data output unit sets the data output flag of the data output flag storage unit as already output,
In response to the input value being input to the data input unit, the token output unit reads the data output flag from the data output flag storage unit, and when the read data output flag has been output, Outputting the token,
The parallel arithmetic device according to claim 3, wherein   9. The parallel arithmetic apparatus according to claim 8, wherein the token output unit outputs the token and sets a data output flag of the data output flag storage unit as an unoutput.   After the unit output calculation unit reads the product-sum calculation primary value information from the product-sum calculation primary value calculation unit, the unit-sum calculation primary value information stored in the product-sum calculation primary value calculation unit is set to 0. The parallel arithmetic apparatus according to claim 3, wherein the parallel arithmetic apparatus is reset to When the parallel computing device is a parallel computing device that computes a self-organizing map, each of the plurality of units is identified in advance by coordinates identified by network coordinates,
The unit calculation unit is
A unit network coordinate storage unit in which unit network coordinates, which are network coordinates of the unit, are stored in advance;
An input network coordinate storage unit that stores input network coordinates that are input network coordinates;
A weight storage unit in which the unit weight vector is stored;
A unit norm storage unit for storing the norm;
An input norm storage unit for storing the input norm;
When the input value input to the data input unit is an input data vector, a norm between the input data vector input and the combined weight vector read from the weight storage unit is calculated, and the calculated norm A norm calculation unit that stores the unit norm in the unit norm storage unit;
When the input value input to the data input unit is a set of network coordinates and norm, the input network coordinates are stored in the input network coordinate storage unit as input network coordinates and the input is performed. An input norm part that stores the obtained norm as an input norm in the input norm storage part;
A norm comparison unit that compares a unit norm read from the unit norm storage unit with an input norm read from the input unit norm storage unit in response to a token being input via the token input unit;
When the comparison result of the norm comparison unit indicates that the read unit norm is smaller than the input norm, the unit norm read from the unit norm storage unit and the unit network coordinates read from the unit network coordinate storage unit are obtained. When the unit norm is output as a set and the read unit norm is greater than or equal to the input norm, the input norm read from the input norm storage unit and the input network read from the input network coordinate storage unit A selection output unit that outputs coordinates as a set via the data output unit;
The parallel arithmetic device according to claim 1, wherein the parallel arithmetic device is provided. The unit calculation unit is
When the input value input to the data input unit is a combination of network coordinates and proximity determination identification information that is identification information indicating execution of proximity determination, the unit read from the unit network coordinate storage unit A proximity determination unit that determines whether network coordinates are in the vicinity of the input network coordinates that are the input network coordinates;
If the result of the determination by the proximity determination unit is a neighborhood, a weight update unit that updates the connection weight vector stored in the weight storage unit;
The parallel processing device according to claim 10, comprising: The unit calculation unit is
An input data storage unit for storing input input data vectors;
A learning speed storage unit in which a learning speed coefficient that is a coefficient for determining a learning speed is stored in advance;
When the input value input to the data input unit is an input data vector, a data extraction unit that stores the input data vector input in the input data storage unit;
Have
The weight update unit multiplies the difference between the input data vector read from the input data storage unit and the combined weight vector read from the weight storage unit by the learning speed coefficient read from the learning speed storage unit, Updating the connection weight vector stored in the weight storage unit by adding to the connection weight vector stored in the weight storage unit;
The parallel arithmetic apparatus according to claim 12, wherein: In the learning speed storage unit, a learning step number indicating the number of times of learning and the learning speed coefficient are stored in association with each other in advance.
The unit calculation unit includes a counter that counts up the number of steps that is the number of times the input data vector is input to the unit each time an input data vector is input to the unit.
The weight update unit reads from the learning speed storage unit the learning speed coefficient corresponding to the number of steps read from the counter and stored in the weight storage unit based on the read learning speed coefficient. Update the connected weight vector,
The parallel arithmetic apparatus according to claim 12 or 13, The unit calculation unit includes a distance storage unit in which a reference distance that is a reference for determining whether or not the distance between the network coordinates is a neighborhood is stored in advance.
If the input value input to the data input unit is a set of network coordinates and proximity determination identification information that is identification information indicating that the proximity determination is performed, the distance storage unit A reference distance is read from the unit network coordinate storage unit, and it is determined whether or not a distance between the unit network coordinates read from the unit network coordinate storage unit and the input network coordinates that are the input network coordinates is equal to or less than the read reference distance. To determine whether the unit network coordinates are in the vicinity of the input network coordinates,
15. The parallel arithmetic apparatus according to claim 12, wherein the parallel arithmetic apparatus is characterized in that In the distance storage unit, the number of learning steps and the reference distance are stored in advance in association with each other,
The proximity determination unit
A reference distance corresponding to the learning step number read from the counter is read from the learning speed storage unit, and the unit network coordinates are in the vicinity of the input network coordinates based on the read reference distance. To determine,
The parallel arithmetic apparatus according to claim 15.   17. The parallel computing device according to claim 11, wherein the token output unit outputs a token in response to a token input to the token input unit. The unit calculation unit is
A token output flag storage unit storing a token output flag indicating whether or not the token output unit has already output a token;
When a token is input to the token input unit, a token output flag is read from the token output flag storage unit, and when the read token output flag is a token not output, the input token is A token output flag determination unit that outputs to the norm comparison unit and outputs the input token via the token output unit;
The parallel processing device according to claim 11, further comprising: The token output unit
In response to the output of the selection output unit via the data output unit as a set of the unit norm read from the unit norm storage unit and the unit network coordinates read from the unit network coordinate storage unit, or the input In response to the output from the selection output unit via the data output unit as a set of the input norm read from the norm storage unit and the input network coordinates read from the input network coordinate storage unit, the token is output.
The parallel arithmetic apparatus according to claim 11, wherein the parallel arithmetic apparatus is characterized in that The token output unit
After outputting the token, update the token output flag stored in the token output flag storage unit to have been output,
The parallel arithmetic apparatus according to claim 11, wherein the parallel arithmetic apparatus is characterized by that. In a first redundancy order, which is one of the predetermined orders, half of the plurality of units are daisy chained by a first redundancy daisy chain control bus;
Units that are daisy chained by the first redundant daisy chain control bus are:
A redundant token output unit that outputs the token to the next unit in the redundant order via the first redundant daisy chain control bus in response to a token being input to the token input unit;
A token selection information storage unit storing token output selection information indicating which of the token output unit and the redundant token output unit to select;
Select one of the token output unit and the redundant token output unit based on the token output selection information read from the token selection information storage unit in response to a token input to the token input unit A token output unit selection unit that outputs a token via the selected token output unit or redundant token output unit,
20. The parallel arithmetic device according to claim 1, further comprising: The unit is
A redundant token input unit for inputting tokens from a unit in the redundant order that is daisy chained via the redundant daisy chain control bus;
The token output unit
In response to a token input to the token input unit or the redundant token input unit, the token is output.
The parallel arithmetic apparatus according to claim 21, wherein: The remaining half of the plurality of units that are not daisy chained by the first redundant daisy chain control bus in the order of skipping one of the predetermined orders are transferred by the second redundant daisy chain control bus. Daisy chained,
The units daisy chained by the second redundant daisy chain control bus have the second redundant daisy chain control bus as the first redundant daisy chain control bus, and the redundant token output unit and token selection information storage unit. And a token output unit selection unit,
The parallel processing device according to claim 21 or 22, wherein The unit daisy chained by the second redundant daisy chain control bus has the redundant token input unit with the second redundant daisy chain control bus as the first redundant daisy chain control bus.
24. The parallel arithmetic apparatus according to claim 21, wherein the parallel arithmetic apparatus is characterized in that A plurality of units daisy-chained via a daisy chain control bus in a predetermined order, and an output value output from any one of the plurality of units is input via the unit output bus, and the input A parallel processing method used in a unit of a parallel processing device having a relay unit that outputs the output value as an input value to each of the plurality of units via a unit input bus,
The unit is
Enter tokens from the previous unit in the daisy chain via the daisy chain control bus;
In response to the token being input, the token is output to the next unit in the daisy chain order via the daisy chain control bus.
Input the input value from the relay unit via the unit output bus,
Based on the inputted input value, it is calculated by a predetermined calculation method for each unit,
In response to the input of the token, the calculation result that is the calculation result is output as the output value to the relay unit via the unit output bus.
A parallel operation method characterized by that.

Images