JP2006154992A - Neuro-processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a neuro-chip wherein initialization of a coupling load to a neuron is easy. <P>SOLUTION: Intermediate layer modules 103-105, and output layer modules 106 and 107 respectively hold a coupling load with respect to each input. Connection relationships from three input terminals 111-113 of an input layer to each input IN1-IN3 of the intermediate modules 103-105 are different in each of the intermediate modules 103, 104, and 105. Connection relationships from outputs of the intermediate modules 103-105 to each input IN1-IN3 of the output layer modules 106 and 107 are also different in each of the output layer modules 106 and 107. By shifting connection relationships between modules of the same layer in such manner, even when a set of the same coupling load is initialized in each module, learning progresses since output of each module is different with respect to input from a higher order. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a neuroprocessor that implements a neural network with a hardware circuit.

  As a processor in which a neural network is implemented as hardware, for example, there are those disclosed in Patent Documents 1 to 4.

  Of these, those shown in Patent Documents 1 and 2 are implemented by hardware only for recognition processing of a neural network, and learning is performed by an external general-purpose computer.

  Moreover, the thing shown by patent document 3 and 4 implement | achieves both the recognition process and the learning process as hardware. In particular, in Patent Document 3, each neuron in the input layer, intermediate layer, and output layer of a hierarchical neural network is configured as one chip, and the connection load of synaptic connections with each neuron in the upper layer is stored in the memory in each chip. Are stored, and forward propagation (back propagation) and back propagation (back propagation) are processed using these coupling weights.

JP 7-192073 A JP-A-7-210532 Japanese Patent Laid-Open No. 5-159087 JP-A-5-346914

  When learning the neural network, if the initial value of the connection weight between the neurons is not properly set, the convergence of the learning may be remarkably slowed or the learning may not converge depending on the case. For each chip in the same layer, if the same value is set between the chips as the initial value of the connection load with each neuron in the upper layer, each chip outputs the same output value for the same input value. The neural network remains in a uniform state and learning does not progress.

  However, setting the connection load of each synapse connection not to be the same between neurons (chips) in the same layer is very troublesome as the number of neurons increases. Such measures can be reduced by taking measures such as preparing an initial value setting program that prevents the initial values from being the same between chips, or incorporating similar functions into the processor as hardware circuits. The cost for the countermeasure is not necessarily small.

  In one aspect of the present invention, a neuroprocessor is provided in which an initial value of a connection load for each neuron module can be easily set.

  In another aspect of the present invention, a neuroprocessor capable of alternately executing forward propagation processing and back propagation processing for learning is provided.

  The neuroprocessor according to the present invention includes an upper layer including a plurality of signal sources and a lower layer including a plurality of neuron modules, and each neuron module has a plurality of input terminals to which signals are input from the signal sources. A plurality of input terminals arranged in a predetermined order, a coupling load storage unit for storing a coupling load for each of the input terminals, a signal input from each input terminal, and a coupling load corresponding to each input terminal are used. And a forward propagation arithmetic circuit for performing forward propagation computation, wherein the connection relationship between each signal source and each input terminal arranged in a predetermined order of the neuron module is different for each neuron module. And

  In this configuration, the signal source may be a simple signal input terminal or may be a neuron module in an upper layer. For example, in the case of a neuroprocessor having a three-layer structure of one input layer, one intermediate layer, and one output layer, if the intermediate layer is considered as the “lower layer”, the input layer becomes the “upper layer” and the output layer becomes the “lower layer” The middle class is the “upper class”.

  In a preferred aspect, each neuron module in the lower layer has a connection load for each of the input terminals stored in the connection load storage unit based on the operation result of the forward propagation operation circuit and a given back propagation signal. And a back-propagation operation circuit that generates back-propagation signals for the signal sources of the upper layer, and back-propagation output terminals that output the respective back-propagation signals, each back-propagation output terminal, Each is connected to a back propagation input terminal of the corresponding signal source.

  In this aspect, the back propagation signal is a teacher signal given from the outside in the case of the output layer, and is an error back propagation signal that is back propagated from the lower layer in the case of other layers.

  In another preferred aspect, the neuroprocessor includes a first clock generation circuit that generates a learning forward propagation clock and a learning back propagation clock from a given reference clock in the learning mode, and each of the upper layers Each signal source is a neuron module, and each neuron module in the upper layer and each neuron module in the lower layer perform forward propagation operation and back propagation operation according to the learning forward propagation clock and the learning backward propagation clock. Execute alternately.

  In a further preferred aspect, the learning forward propagation clock and the learning backward propagation clock have a predetermined period equal to or longer than a time required for a series of forward propagation operations and backward propagation operations in the upper layer and the lower layer. The learning back-propagation clock is delayed in phase by a predetermined delay time greater than the time required for a series of forward propagation operations in the upper layer and the lower layer with respect to the learning forward propagation clock. The module executes a forward propagation operation using the falling edge of the learning forward propagation clock as a trigger, and also executes a backward propagation operation using the rising edge of the learning back propagation clock as a trigger. Forward propagation calculation is triggered by the rising edge of the propagation clock, and back propagation calculation is triggered by the falling edge of the backward propagation clock for learning. That.

  The best mode for carrying out the present invention (hereinafter referred to as “embodiment”) will be described below with reference to the drawings.

  First, the system configuration of the neuroprocessor 101 of this embodiment will be described with reference to FIG.

  The neuroprocessor 101 is a hardware chip formed from a hierarchical neural network with one intermediate layer, and includes a clock generator 102, intermediate layer modules 103 to 105, and output layer modules 106 to 107. A learning rule based on a back propagation method is applied to this chip learning method.

  The neuroprocessor 101 illustrated in FIG. 1 is a network constructed for the purpose of extracting human skin regions from digital color images of the RGB (red, green, blue) color system. The input layer is provided for a total of three units, one for each of R, G, and B colors. Specifically, the units of the input layer are an input terminal 111 for R color data, an input terminal 112 for G color data, and an input terminal 113 for B color data. Further, the output layer obtains a probability that the input color (R, G, B) is a skin color (referred to as skin chromaticity) and a probability that it is not a skin color (referred to as non-skin chromaticity). Thus, it has a two-unit configuration. For example, the output layer module 106 outputs skin chromaticity, and the output layer module 107 outputs non-skin chromaticity. That is, the neuroprocessor 101 accepts R, G, and B values indicating the color of one pixel as input, and outputs the skin chromaticity and non-skin chromaticity of that color, respectively. In the example of FIG. 1, the intermediate layer has a three-unit configuration of intermediate layer modules 103 to 105.

  However, such a unit configuration is merely an example, and the number of units in each layer can be freely changed according to the problem area to be applied.

  R, G, and B input data input from the input terminals 111 to 113 on the left side of FIG. 1 are distributed to the intermediate layer modules 103 to 105, respectively. In this embodiment, the distribution function of the input layer is realized by the dedicated bus 108 that connects the input terminals 111 to 113 and the intermediate layer modules 103 to 105. As can be seen from the connection structure in FIG. 1, the input data (that is, all of R, G, and B) of all units in the input layer are input to each of the intermediate layer modules 103 to 105.

  Here, in this embodiment, the connection relationship between the input terminals 111 to 113 of the input layer and the three inputs IN1 to IN3 of the intermediate layer modules 103 to 105 is different for each intermediate layer module. That is, R, G, and B signals are input to the input IN1, IN2, and IN3 in this order in the intermediate layer module 103, but in the intermediate layer module 104, R is IN2, G is IN3, and B is IN1. In the intermediate layer module 105, R is input to IN3, G is input to IN1, and B is input to IN2.

  Further, the intermediate layer modules 103 to 105 and the output layer modules 106 and 107 are connected by a dedicated forward propagation bus 109 and a backward propagation dedicated bus 110. All the forward propagation outputs of the intermediate layer modules 103 to 105 are input to the output layer modules 106 and 107. Further, the back propagation outputs of the output layer modules 106 and 107 are input to the intermediate layer modules 103 to 105 one by one.

  Here, in this embodiment, the connection relationship between the three forward propagation outputs from the intermediate layer modules 103 to 105 and the three inputs IN1 to IN3 of the output layer modules 106 and 107 is different for each intermediate layer module. Yes. In other words, the signals of the intermediate layer modules 103 to 105 are input to the output layer module 106 in this order with respect to the inputs IN1, IN2, and IN3. In the output layer module 107, the output of the intermediate layer module 103 is set to IN2. The output is input to IN3 and the output of 105 is input to IN1.

  Further, in the circuit configuration of FIG. 1, the back propagation output of the output layer modules 106 and 107 and the back propagation input of each of the intermediate layer modules 103 to 105 are connected according to such a connection relationship. That is, in FIG. 4, as can be seen from the fact that the back propagation output DW1 of the output layer module 401 outputs the back propagation value obtained using the forward propagation input IN1, the back propagation outputs DW1 to DW3 correspond to each other. The back propagation value obtained from the input value of the forward propagation input of the number is output. That is, the output layer module uses the forward propagation input value from the intermediate layer module to generate a back propagation output for the intermediate layer module. Accordingly, the back propagation outputs DW1 to DW3 of the output layer modules 106 and 107 are connected to the modules to which the forward propagation inputs IN1 to IN3 having the corresponding numbers are connected, among the intermediate layer modules 103 to 105, respectively.

  The processing result by the neuroprocessor 101 is output from the output terminals 114 and 115 on the right side of FIG. The output terminal 114 outputs the forward propagation output of the output layer module 106, and the output terminal 115 outputs the forward propagation output of the output layer module 107, respectively. The neuroprocessor 101 also has teacher signal input terminals 116 and 117. The teacher signal input from the former is input to the output layer module 106, and the teacher signal from the latter is input to the output layer module 107.

  The clock generator 102 receives a reference clock CLK from the outside, and from this reference clock CLK, a forward propagation clock CLK_FP used for operation control of the intermediate layer modules 103 to 105 and the output layer modules 106 and 107 at the time of forward propagation processing, A back-propagation clock CLK_BP that is used to control the operations during back-propagation processing is generated. The forward propagation clock CLK_FP and the backward propagation clock CLK_BP are respectively supplied to the intermediate layer modules 103 to 105 and the output layer modules 106 and 107 via dedicated clock signal lines. Further, the clock generator 102 generates different clock signals in the learning mode and the recognition mode. The mode in which the neuroprocessor 101 performs the learning operation is the learning mode, and the mode in which the pattern recognition operation is performed is the recognition mode. The designation of the mode is input from the outside as a mode signal MODE. Details of the clock generator 102 will be described later.

  The schematic configuration of the neuroprocessor 101 has been described above. Next, before describing the details of the intermediate layer modules 103 to 105 and the output layer modules 106 and 107 and the operation of the neuroprocessor 101, an engineering model of the hierarchical neural network employed in this embodiment will be described. explain.

  FIG. 2 shows an engineering model of a hierarchical neural network. FIG. 2A shows an example of a hierarchical neural network in which the input layer 201 has 3 units (neurons), the intermediate layer 202 has 3 units, and the output layer 203 has 2 units. FIG. 4B schematically shows the relationship between one unit 206 in the lower layer and the unit 204 in the upper layer connected thereto. For example, if the lower layer is an intermediate layer, the upper layer is an input layer, and if the lower layer is an output layer, the upper layer is an intermediate layer. In this model, each unit 206 in the lower layer is coupled to all the units 204 in the upper layer, each of which has a combined load 205. In the lower layer unit 206, a product-sum operation (Σ) of the forward propagation output value output from each previous upper layer unit 204 and each coupling load 205 is performed, and the result is a predetermined transfer function. By passing through (f), the output value of the unit 206 is determined. A sigmoid function is often used as a transfer function.

  Next, a hardware model of the internal configuration of the intermediate layer modules 103 to 105 shown in FIG. 1 will be described with reference to FIG. In this embodiment, the hardware internal configurations of the intermediate layer modules 103 to 105 are common, and this is shown as an intermediate layer module 301 in FIG. In the figure, the signal flow is indicated by arrows, the signal flow during forward propagation processing is indicated by solid arrows, and the signal flow during reverse propagation processing is indicated by broken arrows.

  The intermediate layer module 301 has three forward propagation inputs IN1, IN2, and IN3. For each of these inputs, forward propagation input signals 309 from the input terminals 111 to 113 of the input layer via the bus 108 are provided. 310 and 311 are input. The intermediate layer module 301 includes a load storage unit 302 that stores a coupling load w1 corresponding to the forward propagation input IN1, a load storage unit 303 that stores a coupling load w2 corresponding to the forward propagation input IN2, and a coupling load corresponding to the input IN3. A storage unit 304 that stores w3 is included. The forward propagation input signal 309 input from the input IN1 during forward propagation is multiplied by the coupling weight w1 by the multiplier 320. Similarly, the signals 310 and 311 input from the inputs IN2 and IN3 are multiplied by the coupling loads w2 and w3 by the multipliers 321 and 322, respectively. The threshold storage unit 305 stores neuron thresholds (offsets). The multiplication results of the multipliers 320 to 322 and the threshold value from the threshold value storage unit 305 are added by the Σ operation block 306. The above is the product-sum operation in the forward propagation process.

  The sigmoid function LUT unit (lookup table) 307 stores a table showing sigmoid functions that are transfer functions of neurons. The sigmoid function LUT unit 307 receives the output value of the Σ operation block 306, reads out the sigmoid function value corresponding to the input value from the table, and outputs it. By passing through the sigmoid function LUT unit 307 in this way, the product-sum operation result is normalized to a range of 0-1. Thus, the value output from the sigmoid function LUT unit 307 becomes the output value 312 of the intermediate layer module 301. This output value 312 is output from the forward propagation output OUT of the intermediate layer module, and is supplied to the output layer modules 106 and 107 via the dedicated bus 109 for forward propagation. The above is the configuration for forward propagation in the intermediate layer module 301.

  Next, the configuration at the time of back propagation processing indicated by a broken line in the figure will be described. The back propagation process is executed in the learning process. In the learning process, back propagation signals 313 and 314 are propagated from the output layer modules 106 and 107. The back propagation signal 313 is input from the back propagation input DW1, and the back propagation signal 314 is input from the back propagation input DW2. These back propagation signals 313 and 314 are added by an adder 323 and input to a multiplier 324. The output value of the sigmoid differential function LUT unit 308 is input to the other input of the multiplier 324. The sigmoid differential function LUT unit 308 stores a table indicating the differential function of the sigmoid function that the sigmoid function LUT unit 307 has. The sigmoid differential function LUT unit 308 receives the product-sum operation result obtained by the Σ operation block 306 during forward propagation in the learning process, and outputs a sigmoid differential function value corresponding thereto. An error signal A is obtained by multiplying the sum of the back propagation signals 313 and 314 by the multiplier 324 by this sigmoid differential function value. The error signal A is input to an adder 325 and multipliers 330, 332, and 334.

  The adder 325 calculates the threshold update value by adding the error signal A to the threshold output from the threshold storage unit 305. The obtained threshold update value is written in the threshold value storage unit 305, whereby the threshold value is updated.

  The multiplier 330 multiplies the input signal 309 from the forward propagation input IN1 by the error signal A. The result of this multiplication is added by the adder 331 to the combined load w1 from the load storage unit 302. Based on the addition result, the value of the combined load w1 of the load storage unit 302 is updated.

  Similarly, the error signal A is multiplied by the input signals 310 and 311 by the multipliers 332 and 334, respectively, and these multiplication results are added to the original combined weights w2 and w3 by the adders 333 and 335, respectively. The values of the combined loads w2 and w3 that the storage units 303 and 304 have are updated.

  The above is the configuration of the intermediate layer module 301 and the processing of forward propagation and back propagation by this configuration.

  Next, a hardware model of the output layer modules 106 and 107 will be described with reference to FIG. The hardware internal configurations of the output layer modules 106 and 107 are common, and this is shown as the output layer module 401 in FIG.

  The output layer module 401 has three forward propagation inputs IN1, IN2, and IN3. For each of these inputs, forward propagation input signals 409, 410, and 411 from the three intermediate layer modules 103 to 105 are input. Is done. The output layer module 401 includes a load storage unit 402 that stores a coupling load w1 corresponding to the forward propagation input IN1, a load storage unit 403 that stores a coupling load w2 corresponding to the forward propagation input IN2, and a coupling load corresponding to the input IN3. It has the memory | storage part 404 which memorize | stores w3. The signals 409, 410, and 411 input from the inputs IN1, IN2, and IN3 at the time of forward propagation are multiplied by coupling loads w1, w2, and w3 by multipliers 420, 421, and 422, respectively. The threshold storage unit 405 stores neuron thresholds (offsets). The Σ operation block 406 sums the multiplication results of the multipliers 420 to 422 and the threshold value from the threshold value storage unit 405 and outputs the result.

  The sigmoid function LUT unit 407 stores a table indicating a sigmoid function similar to that of the intermediate layer module. The output value of the Σ operation block 406 is input, and the sigmoid function value corresponding to the input value is obtained from the table. Output. The value normalized in the range of 0 to 1 through the sigmoid function LUT unit 407 in this way becomes the output value 412 of the intermediate layer module 401 and is output from the forward propagation output OUT. The above is the configuration for forward propagation in the output layer module 401.

  Next, the configuration of back propagation will be described. The back propagation process is executed in the learning process, similar to the intermediate layer module. In the learning process, a teacher signal 413 is input from the teacher signal input TCH. The teacher signal 413 is supplied to one input of the adder 423. A value obtained by inverting the sign of the forward propagation output value output from the sigmoid function LUT unit 407 by the inverter 418 is input to the other input of the adder 423. Therefore, the adder 423 outputs a difference value obtained by subtracting the forward propagation output of the output layer module 401 from the teacher signal 413. This difference value is given to one input of the multiplier 424. The output value of the sigmoid differential function LUT unit 408 is input to the other input of the multiplier 424. The sigmoid differential function LUT unit 408 stores a table indicating the differential function of the sigmoid function of the sigmoid function LUT unit 407, receives the product-sum operation result obtained by the Σ operation block 406 at the time of forward propagation, The sigmoid differential function value corresponding to this is output. The error signal A is obtained by multiplying the sum of the back propagation signals 413 and 414 by the multiplier 424 with this sigmoid differential function value. The error signal A is input to an adder 425 and multipliers 430, 432, and 434.

  The adder 425 calculates an updated value of the threshold by adding the error signal A to the threshold output from the threshold storage unit 405. The obtained update value of the threshold value is written in the threshold value storage unit 405, whereby the threshold value is updated.

  Multipliers 430, 432, and 434 multiply error signals A by input signals 409, 410, and 411 from forward propagation inputs IN1, IN2, and IN3, respectively. The multiplication results are added to the combined loads w1, w2, and w3 from the load storage units 402, 403, and 404 by adders 431, 433, and 435, respectively. Based on the addition result, the values of the combined loads w1, w2, and w3 of the load storage units 402, 403, and 404 are updated.

  The error signal A is multiplied by the coupling loads w1, w2, and w3 by the multipliers 436, 437, and 438, respectively. These multiplication results are output as back propagation values 414, 416, and 417 from back propagation outputs DW1, DW2, and DW3, respectively. The output back propagation value is supplied to the intermediate layer modules 103 to 105 via the dedicated back propagation bus 110.

  The above is the configuration of the output layer module 401 and the processing of forward propagation and back propagation by this configuration.

  Next, with reference to FIG. 5, the clock signal generated by the clock generator 102 in each mode and the operation of each module based on this will be described.

  The clock generator 102 generates two types of clocks, forward propagation clocks 502 and 504 for controlling forward propagation, and backward propagation clocks 503 and 505 for controlling backward propagation, from a reference clock 501 given to the CLK input from the outside. The learning and recognition (test) processing of the neurons of the intermediate layer modules 103 to 105 and the output layer modules 106 and 107 are controlled by the forward propagation clock 502 or 504 and the backward propagation clock 503 or 505. The clock generator 102 switches between the learning mode and the recognition mode in accordance with a mode setting signal input from the outside with respect to the MODE input.

  First, in the learning mode, the clock generator 102 generates the forward propagation clock 502 and the backward propagation clock 503, outputs the former from the forward propagation clock output CLK_FP, and outputs the latter from the backward propagation clock output CLK_BP.

  In the example of FIG. 5, learning is performed once with 4 reference clocks. That is, the forward propagation clock 502 is first lowered at the first rising edge of the reference clock 501. Using the falling edge of the forward propagation clock 502 as a trigger, the intermediate layer modules 103 to 105 perform forward propagation arithmetic processing in parallel. Then, the forward propagation clock 502 is raised at the second rising edge of the reference clock 501. The output layer modules 106 and 107 perform forward propagation arithmetic processing in parallel with the rising of the forward propagation clock 502 as a trigger.

  Subsequently, the back-propagation clock 503 falls at the third rise of the reference clock 501. The output layer modules 106 and 107 perform reverse propagation calculation processing using the falling edge of the reverse propagation clock 503 as a trigger. Then, the back propagation clock 503 is raised at the rise of the fourth clock of the reference clock 501, and the intermediate layer modules 103 to 105 perform back propagation processing using this as a trigger.

  In this way, learning processing is performed once in one cycle of the forward propagation clock 502 and the backward propagation clock 503, that is, the learning data is given to the input layer of the neuroprocessor 101 to be forward propagated to obtain the output from the output layer, and the output A process is performed in which a teacher signal is given to the layer, and the threshold signal and the coupling load of the output layer module and the intermediate layer module are updated by propagating the teacher signal. Therefore, the clock cycle may be a period that is equal to or greater than the total time required for forward propagation processing in each layer in the neuroprocessor 101 and subsequent back propagation processing.

  In the example of FIG. 5, one period of the forward propagation clock 502 and the backward propagation clock 503 is a length of four reference clock periods. The forward propagation clock 502 is at a period L (low) level for a quarter of the period (that is, one period of the reference clock), and the remaining 3/4 period is at an H (high) level. In this example, since the forward propagation clock 502 is generated by the rising of the reference clock, the L level period of the forward propagation clock 502 is one cycle of the reference clock 501, but this is only an example. In principle, the length of the L level period of the forward propagation clock 502 is that the intermediate layer modules 103 to 105 perform an operation by an internal forward propagation arithmetic circuit on the input signals input from the inputs IN1 to IN3. The time required to generate the output signal output from the output OUT is sufficient. Similarly, the length of the L level period of the back propagation clock 504 is basically the same as the internal back propagation arithmetic circuit using the teacher signal input from the teacher signal input TCH by the output layer modules 106 and 107. It suffices if it is equal to or longer than the time required to generate the output signal output from the back propagation outputs DW1 to DW3. Further, in principle, the interval between the rising of the forward propagation clock 502 and the next falling of the backward propagation clock 503 is relative to the input signals input from the input IN1 to IN3 by the output layer modules 106 and 107. It is sufficient that the time is longer than the time required to perform an operation by the internal forward propagation arithmetic circuit and generate an output signal output from the output OUT. Therefore, the phase of the back propagation clock 503 only needs to be delayed from the forward propagation clock 502 by at least the total time required for forward propagation of each layer in the neuroprocessor 101.

  By using the forward propagation clock 502 and the backward propagation clock 503 as described above, the forward propagation arithmetic processing for one learning data and the reverse propagation arithmetic processing based on the corresponding teacher signal are alternately performed, in other words, once. Can be executed simultaneously in parallel for each learning. Since forward propagation and back propagation can be executed alternately during learning in this way, it becomes easy to associate the result of forward propagation processing for the input learning data with the teacher signal for the learning data. For example, in the apparatus of Patent Document 3, the forward propagation results for all the learning data are obtained once and stored in the storage device, and then the teacher data corresponding to the forward propagation results accumulated thereafter are sequentially given to perform error back propagation learning. However, in this method, it is necessary to manage the correspondence between the accumulated forward propagation results and the teacher data. On the other hand, if forward propagation and reverse propagation are alternately performed as in the present embodiment, such association is unnecessary.

  Of course, the forward and backward propagation clocks at the time of learning may be those obtained by inverting the H and L levels as shown in FIG. In this case, the forward propagation clock causes the intermediate layer module to perform forward propagation processing at the falling edge, and the output layer module performs forward propagation processing at the falling edge, respectively. What is necessary is just to perform a back propagation process.

  Next, processing in the recognition mode will be described. In the recognition mode, the clock generator 102 generates the forward propagation clock 504 and the backward propagation clock 505, outputs the former from the forward propagation clock output CLK_FP, and outputs the latter from the backward propagation clock output CLK_BP.

  In this example, the recognition operation is performed once with two reference clocks, that is, the forward propagation clock 502 is first lowered at the first rising edge of the reference clock 501. With this as a trigger, the intermediate layer modules 103 to 105 perform forward propagation arithmetic processing in parallel. Next, the forward propagation clock 502 is raised at the second rising edge of the reference clock 501. With this as a trigger, the output layer modules 106 and 107 execute forward propagation arithmetic processing. In the recognition processing, no reverse propagation is performed, so that the reverse propagation clock 505 is a flat signal.

  In this example, the forward-propagation clock 504 at the time of recognition has two cycles of the reference clock 501 as one cycle. However, in principle, the cycle is equal to or greater than the sum of the time required for the forward-propagation processing in each layer in the neuroprocessor 101. If it is. In principle, the L level period of the forward propagation clock 504 is longer than the time required for the forward propagation operation of the intermediate layer modules 103 to 105, and the H level period is longer than the time required for the forward propagation operation of the output layer modules 106 and 107. If it is.

  The circuit configuration and operation of the neuroprocessor 101 have been described above. As can be seen from the description, in this embodiment, the respective layers are connected by a dedicated bus, and the common forward propagation clock and the backward propagation clock are supplied to the modules 103 to 107 of each layer, so that each module is completely configured. It can be operated in parallel. In the learning mode, by using a forward propagation clock and a back propagation clock having an appropriate phase relationship, recognition of learning data and error back propagation learning using teacher data can be executed alternately. .

  Further, in the present embodiment, as described above, the connection relationship between the input terminals 111 to 113 of the input layer and the inputs IN1 to IN3 of the intermediate layer modules 103 to 105 is shifted for each intermediate layer module. Since the connection relationship between the forward propagation outputs 103 to 105 and the forward propagation inputs IN1 to IN3 of the output layer modules 106 and 107 is shifted for each output layer module, it is easy to set the initial value of the coupling load for each module. There is an advantage. This point will be described with reference to FIG.

  FIG. 6 shows a case where the connection relationship between the input layer and the intermediate layer is not shifted as in the conventional case when the same value is set between the three intermediate layer modules 103 to 105 as the initial value of the coupling load (a). And how the output value changes in the case of shifting (b) as in the present embodiment. In the illustrated example, w1 = 0.2, w2 = 0.6, and w3 = 0.5 are set as initial values of the coupling loads for all the intermediate layer modules 103 to 105.

  In such a case where the initial values are set and the connection relationship is not shifted (a), if the input of R, G, B is given from the input layer, both the intermediate layer module 103, the intermediate layer module 104, and the intermediate layer All the modules 105 also output the same value (0.42 in this example). In this way, if the output values of all the intermediate layer modules 103 to 105 are the same, even if learning is performed by back propagation, all the intermediate layer modules are connected to the same output value and the same back propagation value based on the combined load. Since the threshold value is changed, the neural network remains in a uniform state only by changing all the same, and learning does not proceed.

  On the other hand, if the connection relationship is shifted as shown in (b), even if the coupling loads in the intermediate layer modules 103 to 105 are the same among the modules, the output values with respect to the inputs are different between the modules. In this way, if the output value differs between the intermediate layer modules, the state of the neural network can vary, and learning proceeds.

  Therefore, in the present embodiment, if a common set of loads (w1, w2, w3) is set for the plurality of intermediate layer modules 103 to 105, an initial setting in which learning proceeds can be performed. For this purpose, the load storage units 302 of the intermediate layer modules 103 to 105 are connected with the same initial value setting line, and similarly, the 303 and 304 are connected with the same initial value setting line. The initial value of the corresponding load may be given to each. Unless a set of initial values satisfying w1 = w2 = w3 is given, variations in the forward propagation calculation results are likely to occur for each module due to the effect of shifting the connection relationship, so that learning is likely to proceed.

  Even in a circuit configuration that does not shift the connection relationship, if an internal circuit that sets the initial values of the three coupling loads in the load storage units 302 to 304 of each intermediate layer module while shifting in the order of the modules is incorporated, the same as described above However, this has the disadvantage that the circuit scale of the neuroprocessor increases by the amount of the internal circuit.

  On the other hand, in the circuit configuration of the present embodiment that shifts the connection relationship, an increase in circuit scale can be suppressed. Also, when the neuroprocessor 101 is configured using a programmable logic device such as an FPGA (Field Programmable Gate Array), the shift of the connection relationship is caused by the input layer and the circuit in the hardware description language such as VHDL. Since the relationship described in the port map between the intermediate layers only needs to be shifted for each intermediate layer module, the implementation is easy.

  The shift of the connection relationship between the input layer and the intermediate layer has been described above, but the same applies to the connection relationship between the intermediate layer and the output layer. In the embodiment, the connection relationship is shifted in both the input layer / intermediate layer and the intermediate layer / output layer, but a certain degree of effect can be obtained by using only one of them.

  In the above embodiment, the neural network has a three-layer structure having only one intermediate layer. However, a neural network having two or more intermediate layers can also be configured in the same manner as in this embodiment.

  Next, a specific example of the neuroprocessor 101 based on this embodiment will be described.

  In one specific example, the neuroprocessor 101 uses a VGA (Video Graphics Array) pixel clock 24.5454 MHz as a reference clock. In this case, since the learning process can be performed once every 4 clocks in the learning process, 100,000 images (24,545,400 ÷ 4 ÷ 60 ≒ 100,000) learning processes per frame are performed for images drawn at 60 frames per second. Can be executed. That is, for example, data of 100,000 pixels in an image of one frame can be given to the input layer one by one in order and can be learned (of course, the value of the teacher signal corresponding to each pixel is also given to the output layer at this time) . If the number of pixels in one frame is greater than 100,000, for example, a region used for learning (the total number of pixels in the region may be 100,000 or less) is set in advance, and the pixels in these learning regions are set. Data may be given to the neuroprocessor 101 in order.

  FIG. 8 shows an example of setting such a learning area. In this example, a 64 × 64 pixel target area 702 is set at the center of a VGA 640 × 480 pixel screen 701, and four non-target areas 703, 704, 705 and 706 of 32 × 32 pixels are set around the target area 702. . The target area 702 is an area where an image of a color desired to be learned as a skin color comes, and the non-target areas 703 to 706 are areas where an image of a color desired to be learned that is not a skin color comes. Therefore, when the pixel data of the target area 702 is given to the input of the neuroprocessor 101 as learning data, for example, “1” is output to the output layer module 106 that outputs skin chromaticity as a teacher signal, and output that outputs non-skin chromaticity. For example, “0” is given to each of the layer modules 106. On the contrary, when the pixel data of the non-target areas 703 to 706 is given to the input of the neuroprocessor 101 as learning data, for example, “0” is set in the output layer module 106 that outputs skin chromaticity as a teacher signal. For example, “1” is given to the output layer module 106 that outputs “1”.

  Also, since the recognition process can be performed once every two clocks in the recognition process, the recognition process is performed on every pixel before up-scanning in the case of an interlaced image at a rate of once every two pixels. It can be performed.

  Also, in one specific example, a 16-bit fixed point is used for the numerical expression in the neuroprocessor 101 as shown in FIG. The breakdown of the 16 bits is that the decimal part 603 is 8 bits, the integer part 602 is 7 bits, and the encoding part 601 is 1 bit. Therefore, the resolution is 0.00390625, the maximum value is 127.99609375, and the minimum value is -127.99609375.

  Here, since the input value to the neuroprocessor 101 is 8 bits for each of R, G, and B, the 8 bits are set to a decimal number from 0 to 1. The intermediate layer modules 103 to 105 to which this has been input are converted to 16-bit fixed-point data by padding the 8-bit upper integer part 7 bits and the code part 1 bit with 0, respectively. Addition and multiplication are performed as 16-bit fixed point data. The joint weight and threshold value are also set in the form of the same 16-bit fixed point data. As described above, the calculation is performed with a 16-bit fixed point in the module as described above, but when output, it is normalized to a value of 0 to 1 by the sigmoid function LUT unit 307 and output as 8-bit data. Upon receiving this, the output layer modules 106 and 107 pad the high-order bits 0 of the 8-bit data as 16-bit fixed point data, and execute internal calculations, as in the intermediate layer.

  For example, when a neural network is described using a high-level language such as C language and executed sequentially by a Neumann computer, a double precision floating point is usually used. On the other hand, the inventors of the present invention have confirmed from simulation experiments that even if a 16-bit fixed-point representation is used, an inferior result can be obtained as compared with the case where a double-precision floating point is used. The result of this comparison will be described later with reference to FIG.

  In addition, with the use of 16-bit fixed point data in this way, in this specific example, the sigmoid function is approximated by a step function, and the differential form of the sigmoid function is approximated by a pyramid type function.

  That is, as shown in FIG. 9A, the sigmoid function (broken line) used for forward propagation was approximated as a 23-step step function (solid line). Further, as shown in FIG. 9B, the sigmoid primary differential function (solid line) used at the time of back propagation was approximated as a 12-stage pyramid function (broken line). Note that the variation width (step increment) in the x-axis direction of each of these functions is 0.5. The sigmoid function LUT units 307 and 407 hold the table representing the step function of FIG. 9A, and the sigmoid differential function LUT units 308 and 408 hold the table representing the pyramid function of FIG. 9B, respectively. It will be.

  The neuroprocessor of the present embodiment is supposed to be provided as an IP (Intellectual Property) core, and is mounted on the FPGA 804 in the example of FIG. This implementation example is in the stage of confirming the effectiveness of a process for learning and extracting a skin region in real time from a general scene image. Therefore, here, a system for extracting a skin region will be described as an example. However, the method of the present embodiment is not limited to a system for extracting a skin region, and can be extended to pattern recognition such as shape and voice in general.

  In this mounting example, an Altera Stratix series EP1S80 is mounted on the board 807 as the FPGA 804. Further, a digital video decoder 805 is mounted as an image input LSI, and a video D / A converter 806 is mounted as an image output LSI. An image input from the CCD camera 808 is input to the image processing block 802 in the FPGA 804 as digital data through the digital video recorder 805. Image data is sequentially delivered to the neuro processor unit 801 in the FPGA 804 pixel by pixel via the image processing block 802. The neuroprocessor unit 801 performs the above-described processing on the received data. In the recognition mode that is input to the image processing block 802, the processing result is obtained by the image processing block 802, for example, from the skin chromaticity and non-skin chromaticity values of each pixel supplied from the neuroprocessor unit 801. It is determined whether or not the pixel data does not belong to the skin color area of the image data input from the digital video decoder 805, and only the pixel values belonging to the skin color area are output. Thereby, for example, an image of a face area that can be used for face recognition can be extracted. The output of the image processing block 802 is converted into analog data by the video D / A converter 806 and displayed on the VGA display 809. Since interlaced image data is input from the digital video recorder 805 and the video D / A converter 806 handles non-interlaced image data, the image processing block 802 is responsible for interlaced to non-interlaced conversion, that is, up-scan processing. ing. The RAM 803 is used as a working memory area when processing the image processing block 802 and the like.

  Finally, referring to FIG. 11, an extraction result 901 obtained by extracting the skin region using the apparatus of the above-described specific implementation example, and an extraction result 902 obtained by a double precision floating point model executed by a normal Neumann computer are used. Compare. In the implementation example of this embodiment, the error rate converges to 1.746094 for 100,000 learnings. In contrast, in the double precision floating point model, the error rate converged to 0.01 after 21,155 learnings. Since learning beyond this could lead to overfitting, learning was terminated at 0.01. In the extraction result 901 according to the implementation example, misdetection particularly in the background region centered on the lower right of the screen, undetected in the lower part of the right eye, the upper part of the nose, or the like. However, since a result comparable to the extraction result 902 by the double-precision floating-point model is obtained, the configuration of this embodiment has a level that is considerably close to that of the software model (double-precision floating-point model), and a neural network. It can be said that this is a hardware model that can perform learning and recognition processing in parallel and at high speed.

It is a figure which shows the structure of the neuroprocessor of embodiment. It is a figure for demonstrating the engineering model of a hierarchical neural network. It is a figure which shows the internal structure of an intermediate | middle layer module. It is a figure which shows the internal structure of an output layer module. It is a figure which shows the clock signal which a clock generator produces | generates. It is a figure for demonstrating the effect of a shift of connection relation. It is a figure explaining the fixed point data expression in each module in a specific example. It is a figure which shows the example of the learning area | region set in a screen. It is a figure which shows the example of the approximate function of the sigmoid function in a specific example, and a sigmoid differential function. It is a figure which shows the specific example of the image processing system using a neuroprocessor. It is a figure which shows the comparative example of the neuro processor of the specific example which performs internal data processing by a fixed point, and the neural network by the software which performs a double precision floating point calculation.

Explanation of symbols

  101 neuroprocessor, 102 clock generator, 103-105 intermediate layer module, 106, 107 output layer module, 108-110 bus, 111-113 input terminal, 114, 115 output terminal, 116, 117 teacher signal input terminal.

Claims (7)

An upper layer containing multiple signal sources;
A lower layer containing multiple neuron modules;
With
Each of the neuron modules is a plurality of input terminals to which signals are input from the signal source and arranged in a predetermined order; a connection load storage unit that stores a connection load for each of the input terminals; A forward propagation computation circuit that performs forward propagation computation using a signal input from each input terminal and a coupling load corresponding to each input terminal,
A neuroprocessor characterized in that a connection relationship between each signal source and each input terminal arranged in a predetermined order of the neuron module is different for each neuron module. The neuroprocessor of claim 1, comprising:
Each neuron module in the lower layer corrects the coupling load for each of the input terminals stored in the coupling load storage unit based on the computation result of the forward propagation computation circuit and the given back propagation signal, and A back propagation operation circuit for generating a back propagation signal for each signal source in the upper layer, and a back propagation output terminal for outputting each back propagation signal,
Each back propagation output terminal is connected to a back propagation input terminal of the corresponding signal source, respectively.
A neuroprocessor characterized by that. A neuroprocessor according to claim 2, wherein
A first clock generation circuit for generating a learning forward propagation clock and a learning backward propagation clock from a given reference clock in the learning mode;
Each signal source in the upper layer is a neuron module,
Each of the upper layer neuron modules and the lower layer neuron modules alternately execute forward propagation operation and back propagation operation according to the learning forward propagation clock and the learning back propagation clock. Neuroprocessor to do. The neuroprocessor according to claim 3, wherein
The learning forward propagation clock and the learning backward propagation clock have a predetermined period equal to or longer than a time required for a series of forward propagation operations and backward propagation operations in the upper layer and the lower layer, and the learning backward propagation clock. Is behind the learning forward propagation clock by a predetermined delay time equal to or longer than the time required for a series of forward propagation operations in the upper layer and the lower layer,
The neuron module of the upper layer executes the forward propagation calculation using the falling edge of the learning forward propagation clock as a trigger, and executes the reverse propagation operation using the rising edge of the learning back propagation clock as a trigger,
The lower layer neuron module executes a forward propagation operation with a rising edge of the learning forward propagation clock as a trigger, and executes a back propagation operation with a falling edge of the learning back propagation clock as a trigger.
A neuroprocessor characterized by that. The neuroprocessor according to claim 3, wherein
The learning forward propagation clock and the learning backward propagation clock have a predetermined period equal to or longer than a time required for a series of forward propagation operations and backward propagation operations in the upper layer and the lower layer, and the learning backward propagation clock. Is behind the learning forward propagation clock by a predetermined delay time equal to or longer than the time required for a series of forward propagation operations in the upper layer and the lower layer,
The neuron module of the upper layer executes a forward propagation operation using a rising edge of the learning forward propagation clock as a trigger, and executes a backward propagation operation using a falling edge of the learning back propagation clock as a trigger,
The neuron module of the lower layer executes a forward propagation calculation using a falling edge of the learning forward propagation clock as a trigger, and executes a reverse propagation operation using a rising edge of the learning back propagation clock as a trigger.
A neuroprocessor characterized by that. A neuroprocessor according to claim 4 or 5, wherein
A second clock generation circuit for generating a recognition forward propagation clock having a predetermined period equal to or longer than a time required for a series of forward propagation operations in the upper layer and the lower layer from the reference clock in the recognition mode;
The upper layer neuron module triggers one of the rising and falling edges of the recognition forward propagation clock, and the lower layer neuron module triggers the other of the rising and falling edges of the recognition forward propagation clock, respectively. Perform forward propagation as
A neuroprocessor characterized by that. A neuroprocessor according to claim 6, wherein
Each neuron module in the upper layer and the lower layer includes a forward propagation clock input terminal and a backward propagation clock input terminal,
The output terminals of the learning forward propagation clock of the first clock generation circuit and the recognition forward propagation clock of the second clock generation circuit are shared, and the output terminals are connected to the upper layer and lower layer neuron modules. Connect to the forward propagation clock input terminal of
Connecting the output terminal of the back propagation clock for learning of the first clock generation circuit to the back propagation clock input terminal of each neuron module of the upper layer and the lower layer;
Operating the first clock generation circuit in the learning mode to stop the second clock generation circuit, and in the recognition mode to stop the first clock generation circuit and operate the second clock generation circuit;
A neuroprocessor characterized by that.