JPH06203058A - Sigmoid function arithmetic unit and its computing method - Google Patents

Abstract

PURPOSE:To perform calculation with high accuracy and at high speed without complicating hardware constitution. CONSTITUTION:The square of the output e<x> of a register 25 is taken on a multiplier 25 when it is -x2<=x<-x1, and a result e<2x> is held with the register 25. thence, e<2x>, e<4x>, e<6x>,... are outputted from the multiplier 24 by repeating processing to calculate the product of the output of the multiplier 24 and that of the register 25, and the products -e<2x>, +e<4x>, -e<6x>,... of the output of the multiplier 24 and that of a ROM 30 are calculated by a multiplier 13 in parallel with the above processing, and furthermore, the sum 1 of the output 1 of a ROM 34 and the output 0 of a constant setting circuit 32 is outputted to an adder/substractor 14 first in parallel with the above processing, thence, the sum of the output of the multiplier 13 and that of the adder/subtractor 14 is calculated repeatedly, thereby, g(x)=1-e2x+e<4x>, -e<6x>,... is outputted from the adder/subtractor 14, thence, the product of the output g(x) of the adder/ subtractor 14 and the output e<2x> of the register 25 is calculated by the multiplier 13, and it is outputted as a value of sigmoid function.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sigmoid function computing device used in a neuron model of a neurocomputer and a computing method thereof.

[0002]

2. Description of the Related Art A neural network is shown in FIG.
It consists of many connected neuron models
There is. This neuron model uses the input signal V₁, V₂・
..V _nAnd synapse load W₁, W₂, ..., W_nWith
Sum of products x = V₁W₁+ V₂W₂+ ... + V_nW_nCalculate
And pass this to the sigmoid function (S-shaped function) f (x).
Then, y = f (x) is output. Sigmoid function f (x)
Although various shapes are considered as,
A parallel translation of the hyperbolic tangent function is widely used.
There is.

F (x) = {1 + tan (x / s)} / 2 (1) where s is a constant that determines the slope of the sigmoid function, and in the following, s = Set to 1.

Although a fixed point processor (FPU) or the like is used to speed up the calculation of the sigmoid function, since the calculation takes time, an arithmetic unit for performing the calculation at higher speed is required.

Therefore, the same applicant as the present invention has proposed an arithmetic unit for performing the calculation by the following method (Japanese Patent Application No. 2-5343). That is, for example, x ₁ = 0.7
8, x ₂ = 1.75, in the range of 0 ≦ x <x ₁ ,
Let Σ be the sum of n = 1 to p1, and f (x) = 1/2 + Σ (−1) ⁿ⁻¹ {2 ²ⁿ (2 ²ⁿ −1) B _n x ²ⁿ⁻¹ } / {2 · (2n)! } = 1/2 + Σβ _2n-1 x ^2n-1 (2) In the range of x ₁ ≦ x <x ₂ , Σ is the sum of n = 0 to p2, and f (x) = Σ (-1) ⁿ e ^-2nx ... (3) The approximate expression that is expanded is used.

In the range of x ≧ x ₂ , it is approximated as f (x) = 1 (4), and in the range of x <−x ₂ , it is approximated as f (x) = 0 (5) To do.

In the range of -x ₂ ≤x <0, the following relational expression, f (x) = 1-f (-x) (6) and the above expressions (2) and (3) are used. .

In the right side of the above equation (3), e ^x is k = 0.
Is approximated by the following Chebyshev expansion equation, which is the sum of p3 to p3: e ^x = Σα _k x ^k (7) This expression (8) can be used only in the range of | x | ≦ 1. In the range of | x |> 1, assuming that the mantissa part and the exponent part of x are A and m, respectively, x = A × 2 ^m is expressed as (8), and e ^x is e ^{A to} the 2 ^m- th power. .. (9), and | A | <1. Therefore, after calculating e ^A by the equation (7), e ^x is calculated by using the equation (9).

[0009]

However, in the above equation (6), when the value of f (-x) is very close to 1, 1
The precision of calculation is reduced because a digit is lost in the calculation of −f (x).

In view of the above problems, an object of the present invention is to provide a sigmoid function computing device and a computing method therefor capable of performing highly accurate and high-speed computation without complicating the hardware configuration. To do.

[0011]

According to the present invention, an approximate expansion of a sigmoid function f (x) according to the range of x is applied to an input value x having a mantissa A and an exponent m. Based on the sigmoid function calculating device for calculating and outputting the value of the sigmoid function f (x),
For constants x ₁ and x _{2 such} that 0 <x ₁ <x _2, x is −x ₂ ≦ x <
In the case of −x ₁ , the power of A is calculated, and in parallel with this, the product of the power of A and the expansion coefficients α ₁ , α ₂ , ..., αp α ₁ A, α ₂ A ² , ..., α _p3 A ^p3 is calculated, and in parallel with this, the total sum eA = α ₀ of the first term α ₀ and the product
A first step of calculating a _{+ α 1 A + ··· + α} p3 A p3, a second step of calculating a e ^x by the e ^A 2 ^m-th power is, e based on ^{e x 2x, e 4x, e} 6x, ... e
^{2 (p2) x} is calculated, and in parallel with this, −e ^2x , + e ^4x ,
^{-E 6x, ···, (- 1} ) p2 allowed sought e ^{2 (p2) x,} and further parallel with this, g (x) = 1- e 2x + e 4x -e 6x +
... + (- 1) and the third step of calculating the ^p2 e ^{2 (p2) x,} is calculated the product of g (x) and e ^2x, integrating e ^2x g
A third step of setting (x) to the value of the sigmoid function f (x).

A sigmoid function computing device according to the present invention is
The method is for carrying out the above method, and will be described with reference to the reference numerals of corresponding constituent elements in the embodiment drawings.

This sigmoid function computing device is, for example, as shown in FIG. 1, a register 25 and a first unit for calculating a product of a value supplied to a first input terminal and a value supplied to a second input terminal.
A multiplier 24; a first multiplexer 26 that selects one of the output of the register 25 and the output of the first multiplier 24 and supplies it to the first input terminal of the first multiplier 24; The second multiplexer 27 which selects one of the output of the 1 multiplier 24 and supplies it to the second input terminal of the first multiplier 24, and the coefficient of each term other than the first term when the function is polynomial expanded are set. The coefficient setting means 30, the second multiplier 13 for calculating the product of the value supplied to the first input terminal and the value supplied to the second input terminal, and the first when the function is polynomial expanded. The first term setting means 34 in which a value corresponding to the term is set, the constant setting means 32 in which a constant is set, the sum of the value supplied to the first input end and the value supplied to the second input end, or The adder / subtractor 14 for calculating the difference, the output of the register 25, and the first multiplier 2
No. 4 output and one of the output of the coefficient setting means 30 and the output of the adder / subtractor 14 are selected to select the third multiplexer 28 for supplying to the first input terminal of the second multiplier 13. The fourth multiplexer 29 which supplies the second input terminal of the 2 multiplier 13, the output of the constant setting means 32 and the adder / subtractor 14
And a first multiplexer for selecting one of the outputs of the first and second input terminals to supply the first input terminal of the adder / subtractor 14 to the first input terminal.
4 to select one of the output of the second multiplier 13 and the output of the second multiplier 13 to determine the sixth multiplexer 33 to be supplied to the second input terminal of the adder-subtractor 14 and which expansion formula to obtain the sigmoid function. Determination means 16 to 20 for determining the range of x
Then, the first to sixth multiplexers are switched and controlled according to the judgment result by the judging means 16 to 20, and the first multiplier 24,
And a control unit C for causing the second multiplier 13 and the adder / subtractor 14 to execute an operation to obtain the value of the sigmoid function.

This control means C, for constants x ₁ and x ₂ satisfying 0 <x ₁ <x ₂ , when the judgment result is −x ₂ ≦ x <−x ₁ , for example, in FIGS. As in the operation when -A is replaced with A, the first multiplier 24
Of the output of the register 25 and the output A of the register 25 are repeatedly calculated to calculate the power of A. In parallel with this, the output of the first multiplier 24 and the coefficient setting means 30 are transmitted to the second multiplier 13.
It is calculated the product of the output of the further parallel with this, with respect to the adder-subtracter 14, initially to output the operation result alpha ₀ of the outputs of the constant setting means 32 of the first term setting means 34, then , By repeatedly calculating the sum of the output of the second multiplier 13 and the output of the adder / subtractor 14,
^A = α ₀ + α ₁ A + ... + α _p3 A ^p3 is output and held in the register 25, and then, for example, as in the operation when −A is replaced with A in FIG. 10, the first multiplier 24 As opposed to
The output of register 25 is squared and the result is registered in register 2
5 is repeated to output e ^x, which is the power of e ^A raised to the power of 2 ^m , from the register 25. Next, for example, the operation when -A is replaced with A in FIGS. , The output e of the register 25 to the first multiplier 24
^By squaring ^x and holding the result in the register 25, and then repeating the process of calculating the product of the output of the first multiplier 24 and the output of the register 25, the first multiplier 24 e ^2x , E ^4x , e ^6x , ..., E ^{2 (p2) x} are output, and in parallel with this, the second multiplier 13 outputs the output of the first multiplier 24 and the output of the coefficient setting means 30. Product of −
e ^2x , + e ^4x , −e ^6x , ..., (−1) ^p2 e ^{2 (p2) x} are calculated, and in parallel with this, the adder / subtractor 14 is initially operated by the first term setting means 34. Output and constant setting means 32
Output the calculation result 1 and the second multiplier 1
By repeatedly calculating the sum of the output of 3 and the output of the adder / subtractor 14, g (x) = 1-e ^2x from the adder / subtractor 14 is calculated.
+ E ^4x −e ^6x + ..., + (− 1) ^p2 e ^{2 (p2) x} is output, and then, as shown in FIG. 14, for example, the second multiplier 13
In contrast, the product of the output g (x) of the adder / subtractor 14 and the output e ^{2x of the} register 25 is calculated, and the product e ^2x g (x) is output as the value of the sigmoid function f (x).

When −x ₂ ≦ x <−x ₁ , the conventional method shown in FIG.
13 is performed, and then 1-f (-x) is added to the adder / subtractor 1
In the present invention, the calculation performed in step 4 is, for example, in FIGS.
Since the process of replacing -A with A in 3 is performed, and then the process of FIG. 14 is performed, the sigmoid function f is calculated in the same calculation time as the conventional one, that is, at high speed and with high precision without digit loss. The value of (x) can be obtained.

In comparison with the conventional hardware configuration, the hardware configuration of the present invention conventionally supplies the output of the coefficient setting means 30 directly to the second multiplier 13, but in the present invention, the fourth multiplexer 29. The difference is that the output of the coefficient setting means 30 or the adder / subtractor 14 is selected and supplied to the second multiplier 13 and the control method of the control means C is different, but the sigmoid function computing device as a whole. As a result, the degree of complexity is almost the same as the conventional one.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a sigmoid function computing device according to the present invention will be described below with reference to the drawings.

This sigmoid function computing device calculates the sigmoid function f (x) of the above equation (1) based on an approximate expression that is a power expansion of x, and by pipeline processing with a hardware configuration, Performs calculations at high speed.

In order to perform calculation at high speed and with high accuracy, different exponential expansions of x are used depending on the range of x. 0 ≦ x <
the above equation (2) used in the range of x _1, the above equation (3) used in the range of x ₁ ≦ x <x _2, above equation (4) used in the range of x ≧ x _2, x <-x _In the range of ₂ , the above formula (5) is used, and
It is the same as the conventional one in that ex in Expression (3) is calculated based on Expressions (7) to (9) above. The novel point of this embodiment is that
Using the relationship of the above formula (6) only in the range of −x ₁ ≦ x <0,
In the range of −x ₂ ≦ x <x ₁ , Σ is the sum of n = 0 to p2, and f (x) = 1 / (1 + e ^−2x ) = e ^2x / (e ^2x +1) = e ^2x g (x ) (11) g (x) = Σ (-1) ⁿ e ^2nx (12) and the point where e ^2x is calculated using the above equations (7) to (9) Is.

FIG. 1 shows a block configuration of a sigmoid function computing device. The following various controls are performed based on the control signal from the control circuit C. Also, the above formula (2),
The number of expansion terms of (3), (7) and (12) is given to the control circuit C.

The multiplexer 10 is one of the outputs of x from floating point notation, a constant setting circuit 11 with a value 0 set, a constant setting circuit 12 with a value 1 set, a multiplier 13 and an adder / subtractor 14. Is selected and supplied to the input / output register 15. First, select x. The sign bit determination / inversion circuit 16 checks the sign bit of the mantissa part A of x when x is initially held in the input / output register 15, and sets the sign flag in the flag register 17 when x is a negative number. , X is a positive number, this code flag is reset. The sign bit determination / inversion circuit 16 further inverts the sign bit to obtain a positive number | x | when x is a negative number.
The comparator 18 outputs the | from the sign bit determination / inversion circuit 16.
x | is a constant x set in the constant setting circuits 19 and 20.
_It is compared with ₁ and x _2, and the comparison result is held in the flag register 17 as a 2-bit value.

Exponent part clearing / sign bit inverting circuit 21
When the content of the flag register 17 represents x ₁ <| x | ≦ x ₂ , the exponent part m of | x |
M = 0, that is, x is set to A, and when the sign flag of the flag register 17 indicates a positive number, the sign bit of the mantissa part is inverted and x
Be -A.

The above processing is performed as the operation of the 0th cycle.

The multiplexer 23 selects one of the outputs of the exponent part clearing / sign bit inverting circuit 21, the multiplier 24 and the adder / subtractor 14 and supplies it to the register 25.
The contents of the register 25 are supplied to one input end of the multiplexer 26 and one input end of the multiplexer 27. The output of the multiplier 24 is supplied to the other input end of the multiplexer 26 and the other input end of the multiplexer 27. The outputs of the multiplexers 26 and 27 are supplied to the multiplier 24, and the product of both is calculated. With the components 23 to 27 as described above, a power calculation of x or the like is performed.

The multiplexer 28 is supplied with the outputs of the multiplier 24 and the register 25, while the multiplexer 29 is supplied with the outputs of the adder / subtractor 14 and the ROM 30. The ROM 30 stores the above formulas (2) and (3) of f (x).
And the expansion coefficients of (7) are stored (the above equation (12)).
The expansion coefficient of is the same as the expansion coefficient of the above equation (3)). ROM3
The expansion coefficient read from 0 is determined by the contents of the flag register 17 and the operation cycle described later. The outputs of the multiplexers 28 and 29 are supplied to the multiplier 13, and the product of the power of x and the coefficient thereof is calculated.

The multiplexer 31 is supplied with the outputs of the constant setting circuit 32 in which 0 is set and the adder / subtractor 14,
On the other hand, the multiplexer 33 includes a multiplier 13 and a ROM.
34 outputs are provided. The ROM 34 stores the first term (constant) of the expansion equations of the above equations (2), (3), and (7) of f (x).
1/2, 1 and α ₀ are stored (the first term of the expansion equation of the above equation (12) is the same as the first term 1 of the expansion equation of the above equation (3)).
The outputs of the multiplexers 31 and 33 are supplied to the adder / subtractor 14, and the sum or difference between the product sum and the product or the like is calculated.

The multipliers 24 and 13 and the adder / subtractor 14
Each has a buffer register in the internal output stage.

The operation of the first and subsequent cycles following the 0th cycle will be described below for each of the five cases of the contents of the flag register 17, that is, the range of x.

(A) When x ≧ x ₂ The multiplexer 10 selects the constant setting circuit 12 to hold the value 1 in the input / output register 15 and output it as f (x).

(B) When x≤-x ₂ The constant setting circuit 11 is selected by the multiplexer 10 to hold the value 0 in the input / output register 15 and output as f (x).

(C) When 0 ≦ | x | <x ₁ (see FIG.
5) At this time, | x | is held in the register 25 in the 0th cycle. The expansion formula of the above formula (2) is calculated as follows.

(1) First Cycle As shown in FIG. 2, multiplexers 26, 27, 28, 2
Switching control is performed on 9, 31, and 33. And the multiplier 24
The output | x | of the register 25 is squared with, and in parallel with this, the output | x | of the register 25 and the output β _{1 of the} ROM 30
The product of and is obtained by the multiplier 13, and in parallel with this, the sum of the output 0 of the constant setting circuit 32 and the output 1/2 of the ROM 34 is obtained by the adder / subtractor 14.

(2) Second Cycle As shown in FIG. 3, multiplexers 23, 26, 27, 2
8, 29, 31 and 33 are switch-controlled. The output of the multiplier 24 | x | | output ² and the register 25 | x a product of the calculated with the multiplier 24 and the output of the multiplier 24 before obtaining this | hold ² in register 25 | x Let In parallel with this, the product of the output | x | ² of the multiplier 24 and the output 0 of the ROM 30 is obtained by the multiplier 13, and in parallel with this, the output ½ of the adder / subtractor 14 and the multiplier 13 Output β ₁
The sum with | x | is obtained by the adder / subtractor 14.

(3) Third Cycle As shown in FIG. 4, multiplexers 26, 27, 28, 2
Switching control is performed on 9, 31, and 33. And the multiplier 24
Of the output | x | ³ of the register 25 and the output | x | ² of the register 25 are obtained by the multiplier 24. In parallel with this, the product of the output | x | ³ of the multiplier 24 and the output β _{3 of the} ROM 30 is calculated. Calculated by the multiplier 13, and in parallel with this, the output of the adder / subtractor 14 1 /
The adder / subtractor 14 calculates the sum of 2 + β ₁ | x | and the output 0 of the multiplier 13.

(4) Fourth Cycle As shown in FIG. 5, multiplexers 26, 27, 28, 2
Switching control is performed on 9, 31, and 33. And the multiplier 24
Of the output | x | ⁵ of the register 25 and the output | x | ² of the register 25 are obtained by the multiplier 24. In parallel with this, the product of the output | x | ⁵ of the multiplier 24 and the output β _{5 of the} ROM 30 is calculated. Calculated by the multiplier 13, and in parallel with this, the output of the adder / subtractor 14 1 /
The sum of 2 + β ₁ | x | and the output β ₃ | x | ³ of the multiplier 13 is calculated by the adder / subtractor 14.

Thereafter, by repeating the same operation as the fourth cycle, the calculation result of the above equation (2) is obtained from the adder / subtractor 14 in the (p1 + 2) th cycle. When the sign flag of the flag register 17 indicates negative, the output 1 of the ROM 34 and the output f (-
x) is calculated by the adder / subtractor 14.

Next, the multiplexer 10 of FIG. 1 selects the output of the adder / subtractor 14 to hold the output of the adder / subtractor 14 in the input / output register 15 and output it as f (x).

(D) When x ₁ ≤x <x ₂ (FIGS. 6 to 13) At this time, in the 0th cycle, the mantissa part -A of x is held in the register 25 and the exponent part of x is stored in the exponent part register 22. The part m is retained. In calculating the expansion formula of the above formula (3), first, the expansion formula when x = −A in the above formula (7) is calculated as follows.

(1) First cycle As shown in FIG. 6, multiplexers 26, 27, 28, 2
Switching control is performed on 9, 31, and 33. And the multiplier 24
The output of register 25-A is squared with, and in parallel with this,
Output of register 25-A and output of ROM 30 α₁Product of
Is calculated by the multiplier 13, and in parallel with this, a constant is set.
Output 0 of circuit 32 and output α of ROM 34 ₀Adjust the sum of
Calculated by the calculator 14.

(2) Second Cycle As shown in FIG. 7, multiplexers 26, 27, 28, 2
Switching control is performed on 9, 31, and 33. And the multiplier 24
Of the output (-A) ² of the register 25 and the output -A of the register 25 are obtained by the multiplier 24. In parallel with this, the product of the output (-A) ² of the multiplier 24 and the output α _{2 of the} ROM 30 is calculated. Multiplier 13
And in parallel with this, the output α of the adder / subtractor 14
_The adder / subtractor 1 calculates the sum of ₀ and the output α ₁ (−A) of the multiplier 13.
Find in 4.

(3) Third Cycle As shown in FIG. 8, multiplexers 26, 27, 28, 2
Switching control is performed on 9, 31, and 33. And the multiplier 24
Of the output (-A) ³ of the register 25 and the output -A of the register 25 are obtained by the multiplier 24. In parallel with this, the product of the output (-A) ³ of the multiplier 24 and the output α _{3 of the} ROM 30 is calculated. Multiplier 13
And in parallel with this, the output α of the adder / subtractor 14
The sum of ₀ + α ₁ (−A) and the output α ₂ (−A) ² of the multiplier 13 is calculated by the adder / subtractor 14.

(4) Fourth to (p3 + 1) cycles Hereinafter, by repeating the same operation as the third cycle, when x = -A in the above equation (7) in the (p3 + 1) cycle, The calculation result on the right side is obtained from the adder / subtractor 14. As shown in FIG. 9, the output e ^−A of the adder / subtractor 14 is held in the register 25 via the multiplexer 23.

(6) No. (p3 + 2) to (p3 + m + 1)
Cycle Multiplexers 23, 26 and 27 as shown in FIG.
Switch control. Then, the output of the register 25 is squared by the multiplier 24, held in the register 25, and this is repeated a number of times equal to the content m of the exponent part register 22 of FIG. 1 to obtain e ^{−A in} the (p3 + m + 1) th cycle. 2
^The e ^−x raised to the ^m- th power can be obtained from the multiplier 24. This e ^−x is held in the register 25.

(7) Cycle (p3 + m + 2) The multiplexers 26, 27, 28, as shown in FIG.
29, 31 and 33 are switch-controlled. And the multiplier 2
In step 4, the output e ^−x of the register 25 is squared, and in parallel with this, the product of the output e ^{−x of the} register 25 and the output 0 of the ROM 30 is obtained by the multiplier 13, and in parallel with this, the constant setting is performed. The adder / subtractor 14 calculates the sum of the output 0 of the circuit 32 and the output 1 of the ROM 34.

(8) Cycle (p3 + m + 3) As shown in FIG. 12, multiplexers 23, 26, 27,
28, 29, 31 and 33 are switch-controlled. Then, the square of the output e ^-2x of the multiplier 24 is obtained by the multiplier 24, and in parallel with this, the product of the output e ^-2x of the multiplier 24 and the output -1 of the ROM 30 is obtained by the multiplier 13, and In parallel with this, the adder / subtractor 14 calculates the sum of the output 1 of the adder / subtractor 14 and the output 0 of the multiplier 13.

(9) Cycle (p3 + m + 4) As shown in FIG. 13, multiplexers 26, 27, 28,
29, 31 and 33 are switch-controlled. And the multiplier 2
4 output e ^-4x and register 25 output e ^-2x are calculated by the multiplier 24, and in parallel with this, the output e of the multiplier 24
The product of ^-4x and the output 1 of the ROM 30 is calculated by the multiplier 13,
Further, in parallel with this, the sum of the output 1 of the adder / subtractor 14 and the output −e ^−2x of the multiplier 13 is ^obtained by the adder / subtractor 14.

Thereafter, the same operation as the (p3 + m + 3) th cycle is repeated to obtain the (p3 + m) th cycle.
The calculation result of the right side of the above equation (3) is obtained from the adder / subtractor 14 in + p2 + 3) cycles.

Then, the multiplexer 10 shown in FIG. 1 selects the output of the adder / subtractor 14 to hold the output of the adder / subtractor 14 in the input / output register 15 and output it as f (x).

(E) When -x ₂ ≤x <-x ₁ (FIG. 14) At this time, in the 0th cycle, the mantissa part A of x is held in the register 25 and the exponent part of x is held in the exponent part register 22. m is retained. Since the right side of the above equation (12) is equal to the right side of the above equation (3) in which the sign of x is inverted, the above (D)
By performing the same processing as the first to (p3 + m + p2 + 3) cycles of, the calculation result of the right side of the above equation (11) is obtained from the adder / subtractor 14.

In the (p3 + m + p2 + 4) th cycle,
As shown in FIG. 14, the multiplexers 28 and 29 are switch-controlled. Then, the product of the output g (x) of the adder / subtractor 14 and the output e ^{2x of the} register 25 is obtained by the multiplier 13. Next, the output of the multiplier 13 is selected by the multiplexer 10 of FIG. 1, the output of the adder / subtractor 14 is held in the input / output register 15, and this is output as f (x).

Therefore, when -x ₂ ≤x <-x ₁ ,
Conventionally, with respect to the mantissa part -A and the exponent part m, the first to the above (D)
After performing the processing of the (p3 + m + p2 + 3) cycle, 1-f (-in the (p3 + m + p2 + 4) th cycle
In the present embodiment, x) is calculated by the adder / subtractor 14, but the first through (p) of (D) are calculated for the mantissa part A and the exponent part m.
By performing the same processing as the (3 + m + p2 + 3) cycle and then performing the above-mentioned (p3 + m + p2 + 4) cycle processing, that is, the value of the sigmoid function f (x) can be obtained with high precision without loss of precision in the same calculation time as the conventional one. it can.

In the hardware configuration of FIG. 1, the output of the ROM 30 is directly supplied to the multiplier 13 in the prior art as compared with the prior art, but in the present embodiment, the ROM 30 or the adder / subtractor is provided with the multiplexer 29. The output of 14 is selected and supplied to the multiplier 13. Further, in the conventional example, the input of the multiplexer 10 was 4 points. Although the output is supplied to the multiplexer 10 and the control method of the control circuit C is different, the degree of complexity of the sigmoid function arithmetic device as a whole is almost the same as the conventional one.

Next, specific numerical values of constants used in the sigmoid function computing device of FIG. 1 will be described in relation to calculation accuracy.

The value of the sigmoid function f (x) is compared by the comparator 18
To obtain the bit precision, the above equations (2), (3),
P1, p2, p representing the final terms of the expansions of (7) and (12)
3 and p2 are set to p1 = 8, p2 = 7, p3 = 8, and the constants may be set as follows.

X ₁ = 0.78 x ₂ = 10.75 β ₁ = 0.5 β ₃ = -1.66667 β ₅ = 0.0666667 67 β ₇ = -0.0269841 β ₉ = 0.109347 β ₁₁ = -0.00443162 β ₁₃ = 0.00179606 β ₁₅ = -0.000727917 α ₁ = 1 α ₂ = 1 α ₃ = 0.500006 α ₄ = 0.166668 α ₅ = 0.0416350 α ₆ = 0.00832860 α ₇ = 0.00143927 α ₈ = 0.000204700 Further, in order to obtain the value of the sigmoid function f (x) with a precision of 24 bits of the multiplier, the above equations (2), (3), (7) and (12) ) P1, p2, p3 and p representing the final term of the expansion
2, p1 = 9, p2 = 12, p3 = 9, and the constants may be set as follows.

X ₁ = 0.65 x ₂ = 43.67 β ₁ = 0.5 β ₃ = -1.666667 β ₅ = 0.666667 β ₇ = -0.02698413 β ₉ = 0.1093474 β ₁₁ = -0.0044331618 β ₁₃ = 0.0017996064 β ₁₅ = -0.0007279172 β ₁₇ = 0.0002950137 α ₁ = 1 α ₂ = 0.99999999 α ₃ = 0.5 α ₄ = 0.16666680 α ₅ = 0. 041666689 α ₆ = 0.0083288596 α ₇ = 0.001388276 α ₈ = 0.0002046999 α ₉ = 0.00002549920 The present invention includes various modifications other than the above.

For example, in FIG. 6, the result obtained by calculating the output of the ROM 34 and the output of the constant setter 32 by the adder / subtractor 14 should be the first term α _{0. The} ROM 34 and the constant setter 3
There is a degree of freedom in the setting value of 2.

[0058] Further, in FIG. 11, the output e ^-2x of multiplier 24 is held in the register 25 via the multiplexer 23, 12, and an output e ^-2x output e ^-2x a multiplier 24 of the register 25 It may be supplied to the multiplier 24.

[0059]

As described above, according to the sigmoid function operation device and the operation method thereof according to the present invention, it is possible to calculate with high accuracy and high speed, without making the hardware configuration complicated. It has an excellent effect.

[Brief description of drawings]

FIG. 1 is a block diagram of a sigmoid function calculation device according to an embodiment of the present invention.

FIG. 2 is an operation explanatory diagram of a first cycle when 0 ≦ | x | <x ₁ .

FIG. 3 is an operation explanatory diagram of a second cycle when 0 ≦ | x | <x ₁ .

FIG. 4 is an operation explanatory diagram of a third cycle when 0 ≦ | x | <x ₁ .

FIG. 5 is an operation explanatory diagram of a fourth cycle when 0 ≦ | x | <x ₁ .

FIG. 6 is an operation explanatory diagram of the first cycle when x ₁ ≦ x <x ₂ .

FIG. 7 is an operation explanatory diagram of the second cycle when x ₁ ≦ x <x ₂ .

FIG. 8 is an operation explanatory diagram of a third cycle when x ₁ ≦ x <x ₂ .

FIG. 9 is an operation explanatory diagram of the latter stage of the (p3 + 1) th cycle when x ₁ ≦ x <x ₂ ;

FIG. 10 shows (p3 + 2)-(p when x ₁ ≦ x <x ₂
It is operation | movement explanatory drawing of 3 + m + 1) cycle.

FIG. 11 is an operation explanatory diagram of the (p3 + m + 2) th cycle when x ₁ ≦ x <x ₂ .

FIG. 12 is an operation explanatory diagram of the (p3 + m + 3) cycle when x ₁ ≦ x <x ₂ ;

FIG. 13 is an operation explanatory diagram of the (p3 + m + 4) th cycle when x ₁ ≦ x <x ₂ .

FIG. 14 is the (p3 + m + p) when −x ₂ ≦ x <−x ₁
It is operation | movement explanatory drawing of 2 + 4) cycle.

FIG. 15 is a diagram showing a neuron model.

[Explanation of symbols]

10, 23, 26, 27, 28, 29, 31, 33 Multiplexer 11, 12, 19, 20, 32 Constant setting circuit 14 Adder / subtractor 15 Input / output register 16 Sign bit determination / inversion circuit 17 Flag register 18 Comparator 21 Exponent Part clear / sign bit inversion circuit 22 exponent part register 24, 13 multiplier 25 register 30, 34 ROM C control circuit

Claims (2)

[Claims] 1. A sigmoid function f (x) is calculated based on an approximate expansion formula of the sigmoid function f (x) according to the range of x for an input value x having a mantissa part A and an exponent part m. In a calculation method using a sigmoid function calculation device that calculates and outputs a value, x is − for constants x ₁ and x _{2 such} that 0 <x ₁ <x _2.
When x ₂ ≦ x <−x ₁ , the power of A is calculated, and in parallel with this, the product α ₁ of the power of A and the expansion coefficients α ₁ , α ₂ , ..., α _p3. A, α
₂ A ² , ..., α _p3 A ^p3 is calculated, and in parallel with this, the sum of the first term α ₀ and the product eA = α ₀ + α ₁ A + ...
+ Alpha _p3 A ^p3 a first step of calculating a second step of calculating a e ^x by the e ^A 2 ^m-th power is, e based on ^{e x 2x, e 4x, e} 6x, ···, e 2 ^{(p2) x} is calculated, and in parallel with this, −e ^2x , + e ^4x , −e ^6x ,
..., (- 1) allowed sought ^p2 e ^{2 (p2) x,} and further parallel with this, g (x) = 1- e 2x + e 4x -e 6x + ··· +
(-1) and the third step of calculating the ^{p2 e 2 (p2) x,} g of (x) and to calculate the product of the e ^2x, integrating e ^2x g (x) the sigmoid function f (x) An arithmetic method using a sigmoid function arithmetic device, comprising: 2. An input value x whose mantissa is A and whose exponent is m
And the approximation of the sigmoid function f (x) according to the range of x
The value of the sigmoid function f (x) is calculated based on the expansion formula.
In a sigmoid function operation device for calculating and outputting, a value supplied to a register (25), a first input terminal, and a value supplied to a second input terminal
A first multiplier (24) for calculating the product of and, and selecting one of the output of the register and the output of the first multiplier
And supplies the first multi-input to the first input terminal of the first multiplier.
Selector (26), one of the output of the register and the output of the first multiplier
And then supplies the second multi-input to the second input terminal of the first multiplier.
Plexer (27) and the coefficient of each term other than the first term when the function is polynomial expanded are set.
Coefficient setting means (30), the value supplied to the first input end and the value supplied to the second input end
A second multiplier (13) for calculating the product of and, and a value corresponding to the first term when the function is polynomial expanded are set.
The first term setting means (34), the constant setting means (32) for which a constant is set, the value supplied to the first input end and the value supplied to the second input end.
An adder / subtractor (14) for calculating the sum or difference between and, and selecting one of the output of the register and the output of the first multiplier
And supplies the third input signal to the first input terminal of the second multiplier.
The multiplexer (28) and one of the output of the coefficient setting means and the output of the adder / subtractor is selected.
And a fourth round signal which is selectively supplied to the second input terminal of the second multiplier.
A chipplexer (29) and one of the output of the constant setting means and the output of the adder / subtractor is selected.
A fifth multi-selector for supplying the first input terminal of the adder / subtractor
A plexer (31) and one of the output of the first term setting means and the output of the second multiplier
A sixth circle which is selected and supplied to the second input terminal of the adder / subtractor.
The chipplexer (33) and the expansion equation for determining the sigmoid function
Determination unit (16 to 20) for determining the range of x, and the first to sixth multis according to the determination result by the determination unit.
The first multiplexer, the second multiplier and the multiplier are switched and controlled.
Value of the sigmoid function by causing the adder / subtractor to execute an operation.
And a control means (C) for determining₁<X₂Constant x₁, X₂As opposed to
The determination result is -x₂≤x <-x₁, The output of the first multiplier and the register
The power of A is calculated by repeatedly calculating the product with the output A.
In parallel with this, the first multiplier is applied to the second multiplier.
The product of the output of the instrument and the output of the coefficient setting means,
In parallel with this, the first term for the adder / subtractor is
Calculation result α of the output of the setting means and the output of the constant setting means
₀Is output, and then the output of the second multiplier and the adder / subtractor
By repeatedly calculating the sum with the output of
E from calculator^A= Α₀+ Α₁A + ... + α_p3A^p3Is output
Hold the output in the register, and then, for the first multiplier,
Then, the process of holding the result in the register is repeated.
To allow e from the register^AE raised to the power of 2 m^X
And then output the output e of the register to the first multiplier.^XFlat
The result and hold the result in the register.
1 Calculate the product of the output of the multiplier and the output of the register
By repeating the process, the first multiplier
e^2x, E^4x, E^6x, ... e^{2 (p2) x}And output this
In parallel with the output of the first multiplier to the second multiplier
And the output of the coefficient setting means −e^2x, + E^4x,-
e^6x, ..., (-1)^p2e^{2 (p2) x}And calculate
In parallel with this, the initial term setting is first performed for the adder / subtractor.
The calculation result 1 of the output of the means and the output of the constant setting means is output.
Then the output of the second multiplier and the output of the adder-subtractor
By repeatedly calculating the sum of
G (x) = 1-e^2x+ E^4x-E^6x+ ... + (-1)
^p2e ^{2 (p2) x}Is output, and then the output g (x) of the adder / subtractor is output to the second multiplier.
And the output e of the register ^2xAnd the product e is calculated.^2xg
(X) is output as the value of the sigmoid function f (x).
A sigmoid function operation device characterized by the following.