JP6918358B2 - Approximate calculation device, approximate calculation method and program - Google Patents

Description

The present invention relates to an approximate calculation device, an approximate calculation method and a program, particularly an approximate calculation device, an approximate calculation method and a program suitable for Bayesian identification using a mixed Gaussian model.

Pattern identification of biological signals is used in myoelectric prosthetic hands, brain-machine interfaces, etc., and it is known that Bayesian identification based on the Gaussian Mixture Model (GMM) is effective for this, but GMM's There is a problem that the implementation cost becomes enormous when trying to process the calculation in real time.

On the other hand, there are cases where exponentiation is calculated in real-time processing.
For example, Patent Document 1 describes that a biological signal is input to a neural network to which a mixed Gaussian model is applied to operate an electric prosthesis. When calculating posterior probabilities with a mixed Gaussian model, the calculation including the exponential function is performed.

Japanese Patent No. 3816762

When calculating exponentiation in real-time processing, it is necessary to calculate exponentiation at high speed. If the exponentiation calculation can be replaced with a relatively simple calculation, the calculation can be speeded up, and the calculation can be implemented in hardware relatively easily.

The present invention provides an approximate calculator, an approximate calculation method and a program that can replace the power calculation with a relatively simple calculation.

According to the first aspect of the present invention, the approximate computing device has an exponent separation part that separates an exponent of the real power of 2 into an integer part and a fractional part of 0 or more and less than 1, and a power of 2 for the fractional part. , A power of 2 whose power is the value obtained by dividing the index value indicating the subsection including the fractional portion of the subsections in which the section from 0 or more and less than 1 is divided into a plurality of sections by the number of the subsections. , -1 to the power of 2 with the value divided by the number as the power, the linear coefficient based on the number, and the power of 2 with the value obtained by dividing the index value by the number as the power. , -1 divided by the number is a power of 2, and a fractional partial calculation unit calculated by linear approximation by a linear equation using a constant term based on the index value, and the integer part. It is provided with a multiplication unit for multiplying the calculation result of the power of 2 by the fractional part calculation unit and the calculation result of the power of 2 for the fractional part.

An integer partial calculation unit that calculates a power of 2 for the integer portion by a shift operation may be provided.

The fractional partial calculation unit may perform the linear approximation with a line segment connecting both ends of the power of 2 in the section obtained by dividing the section of 0 or more and less than 1.

It may be provided with a bottom conversion unit that converts the real power of the Napier number into the real power of 2.

The bottom conversion unit may convert the real power of the Napier number in the Bayesian identification based on the mixed Gaussian model into the real power of 2.

According to a second aspect of the present invention, the approximate calculation method, programmable devices, the exponent separation step of separating the real number squared index of 2 to the integer part and zero or more less than one fractional portion, said device However, the power of 2 for the fractional part was divided by the number of the partial sections, which is an index value indicating the partial section including the fractional part among the partial sections in which the sections 0 or more and less than 1 are divided into a plurality of sections. A power of 2 with a value as a power, a power of 2 with a value obtained by dividing -1 by the number, a linear coefficient based on the number, and the index value divided by the number. Calculated by a linear approximation using a linear equation using a power of 2 with a value as a power, a power of 2 with a value obtained by dividing -1 by the number, and a constant term based on the index value. and a fractional part calculating step, the device includes a power of two the calculation result for the integer part, and a multiplication step of multiplying the power of calculation results of 2 for the fractional part in the fractional part calculating step include.

According to a third aspect of the invention, the program provides a programmable device with an exponent separation step that separates an exponent of 2 to the power of an integer into an integer part and a fractional part greater than or equal to 0 and 2 for the fractional portion. The exponentiation is defined as the power obtained by dividing the index value indicating the subsection including the fractional portion of the subsections in which the section from 0 or more and less than 1 is divided into a plurality of sections by the number of the subsections. The power of 2 and the power of 2 with the value obtained by dividing -1 by the number as the power, the linear coefficient based on the number, and the value obtained by dividing the index value with the number as the power 2 A power of 2, a power of 2 whose value is -1 divided by the number, and a fractional partial calculation step calculated by linear approximation by a linear equation using a constant term based on the index value, and the above. This is a program for executing a multiplication step of multiplying the calculation result of the power of 2 for the integer part and the calculation result of the power of 2 for the fractional part in the fractional part calculation step.

According to the present invention, the calculation of exponentiation can be replaced with a relatively simple calculation.

It is a schematic block diagram which shows the functional structure of the approximate calculation apparatus which concerns on embodiment of this invention. It is a figure which shows the example of the approximation ^{of 2 β} in the same embodiment. It is a figure which shows the structural example of the probability neural network based on the approximate GMM of the same embodiment. It is a figure which shows the approximate error rate of the exponential function when the _{approximate function g 1} (z) which concerns on the same embodiment is used. In the case of using an approximate function g _{'1 (z)} according to the embodiment and showing the approximate error rate of the exponential function. Approximation function g ₁ according to the embodiment _(z), it is a diagram illustrating a g _{'1 (z)} each approximation error rate. Is a diagram illustrating an example of the approximation of the exponential function according to the approximation function g _{'1 (z)} according to the embodiment. It is a figure which shows the approximate error rate of a logarithmic function when the _{approximate function g 2} (z) which concerns on the same embodiment is used. It is a figure which shows the example of the approximation of the logarithmic function by the _{approximation function g 2} ( ⁽³⁾ Occ) which concerns on the same embodiment.

Hereinafter, embodiments of the present invention will be described, but the following embodiments do not limit the inventions claimed. Also, not all combinations of features described in the embodiments are essential to the means of solving the invention.
FIG. 1 is a schematic block diagram showing a functional configuration of an approximate calculation device according to an embodiment of the present invention. As shown in FIG. 1, the approximation calculation device 10 includes a bottom conversion unit 11, an exponential separation unit 12, an integer partial calculation unit 13, a decimal partial calculation unit 14, and a multiplication unit 15.

The approximation calculation device 10 performs an approximation calculation of the power.
The bottom conversion unit 11 converts the Napier number (e = 2.71828 ...) to the real power of 2. For example, the bottom conversion unit 11 converts the real power of the Napier number in Bayesian identification based on the Gaussian Mixture Model (GMM) into the real power of 2.
However, the power to which the approximate calculation device 10 should perform the approximate calculation is not limited to the one whose base is the number of Napiers. For example, the approximation calculation device 10 may receive an input of a power with a base of 10, and the bottom conversion unit 11 may convert a power with a base of 10 into a power with a base of 2. Alternatively, the approximation calculation device 10 may perform the approximation calculation by receiving an input of a power whose base should be originally 2.

The exponent separation unit 12 separates the exponent of 2 to the real power into an integer part and a decimal part of 0 or more and less than 1.
The integer partial calculation unit 13 calculates a power of 2 for the integer portion of the exponent by a shift operation (bit shift).
The fractional part calculation unit 14 calculates the power of 2 for the decimal part of the exponent by linear approximation.
The fractional part calculation unit 14 may calculate the power of 2 for the fractional part of the exponent by linear approximation for each section obtained by dividing a section of 0 or more and less than 1. Further, the fractional partial calculation unit 14 may perform linear approximation with a line segment connecting both ends of powers of 2 in a section obtained by dividing a section of 0 or more and less than 1.

The multiplication unit 15 multiplies the calculation result of the power of 2 for the integer part of the exponent by the integer part calculation unit 13 and the calculation result of the power of 2 for the fractional part by the fractional part calculation unit 14.
As a result, the approximation calculation device 10 can replace the power calculation with relatively simple calculations such as four arithmetic operations and shift operations.
The approximate calculation of the power to be performed by the approximate calculation device 10 will be described with reference to the equation.

In the mixed Gaussian model, assuming that the feature vector x is a D-dimensional real number vector, the occurrence probability density function f (x) of the feature vector x is expressed by Eq. (1).

Here, c (c is a positive integer of 1 ≦ c ≦ C) is an identification number for identifying each class, and C indicates the number of classes. m (m is 1 positive integer of ≦ m ≦ M _c) is an identification number that identifies the components that make up the class c, M _c denotes the number of components forming the class c. In the mixed Gaussian model, the class is constructed by superimposing the normal distribution (Gaussian distribution), and the component referred to here is the normal distribution that constitutes the class.
Further, μ _{c, m} , Σ _{c, m} , γ _{c, m} (0 ≦ γ _{c, m} ≦ 1) are the mean vector of the component m of the class c, the variance-covariance matrix, and the mixture with other components, respectively. Shows the ratio. _{It is assumed that the sum of the mixing ratios γ c and m} of all the components m of all the classes c is 1, that is, the equation (2) is satisfied.

The normal distribution M (x; μ _{c, m} , Σ _{c, m} ) is expressed by Eq. (3).

Here, the exponential function exp indicates the power of the Napier number e. D indicates the number of dimensions of the feature vector as described above.
The indices z _{c and m} are expressed by Eq. (4).

Here, the superscript "T" indicates the transpose of a matrix or vector. The superscript "-1" indicates the inverse matrix.
The posterior probabilities for the component m of class c when classifying the feature vector x into classes are shown by Eq. (5).

When the normal distribution is expressed using an exponential function, equation (5) is expressed as equation (6).

The domain of the exponential function exp in Eq. (6) is as wide as [-∞, 0], and it is considered that it takes time to calculate these exponential functions.
Therefore, we construct an approximate expression that can calculate the exponential function at high speed and with high accuracy. In particular, the approximate expression is constructed in consideration of the hardware implementation.
First, when the exponential function exp is expressed by a power of 2, it is expressed as in Eq. (7).

Here, k is a constant expressed as in Equation 8.

The conversion of the exponential function exp to the power of 2 using the equation (7) corresponds to an example of processing performed by the bottom conversion unit 11.
When the zk of the equation (7) is divided into an integer part and a decimal part, it is expressed as the equation (9).

Here, α is an integer part of zk and is expressed as in equation (10).

The function floor indicates truncation to an integer. Specifically, the function floor indicates the maximum integer that does not exceed the argument.
Β in equation (9) is a fractional part of zk and is expressed as in equation (11).

The domain of β is [0,1), that is, 0 ≦ β <1. The process of separating the exponent zk into the integer part α and the decimal part β corresponds to the example of the process of the exponent separation unit 12.
Here, the domain of β is divided into n at equal intervals, and 2 ^β is linearly approximated for each interval based on the Taylor expansion. Assuming that quadratic or higher terms can be ignored, ^{the Taylor expansion of 2 β} around t is given by Eq. (12).

Further, each interval in which the domain [0,1] of β is divided into n at equal intervals is shown by Eq. (13).

That is, (l-1) / n ≦ β <l / n, and l is a positive integer of 1 ≦ l ≦ n.
Assuming that the midpoint of each interval is t _{n, l} , it is expressed as in Eq. (14).

The Taylor expansion of 2 ^β around the midpoint t _{n, l} is given by Eq. (15).

Equation (15) can be expressed as Equation (16).

Here, an _{and l} are expressed as in the equation (17).

Further, b _{n and l} are expressed by the equation (18). Equation (16) is a linear function of β and linearly approximates β.

^{Using the approximation of 2 β} by equation (16), exp (z) can be approximated as in equation (19).

In addition, the coefficients an _{, l} and b _{n, l} are simplified.
Substituting the equation (14) into the equation (17), the coefficients an and _l are shown as in the equation (20).

Further, when the equation (14) is substituted into the equation (18), the coefficients b _{n and l} are shown as the equation (21).

Replace "(2l-1) / 2n" (= t _{n, l} ) in Eqs. (20) and (21) with the natural logarithm "ln2" of 2.
As the value of n is increased, (2l-1) / 2n converges to 1 / n. This is expressed as in equation (22).

Further, as the value of n is increased, ( ^{1-2-1 / n} ) n converges to ln2. This is expressed by equation (23).

Based on the equation (22), the coefficients an _{, l} and b _{n, l} (2l-1) / 2n are replaced with 1 / n. Further, based on the equation (23), ln2 of the coefficients an _{, l} and b _{n, l} is replaced with (1-2-1 ^{/ n} ) n.
The coefficients _{a'n, l} after replacement from the coefficients an _{n, l} are expressed by Eq. (24).

The coefficients b'n _{, l} after replacement from the coefficients b _{n, l} are expressed by Eq. (25).

Substituting coefficients _{a n} of formula _{(16), l} and _{b n,} the _l coefficient a _{'n, l} and b' _n, at _l, approximation 2 ^beta is represented by the equation (26).

Equation (26) approximates the curve of each section obtained by dividing the curve indicated ^{by 2 β into n by a straight line connecting both ends.}
FIG. 2 is a diagram showing an example of approximation of ^{2 β} by the equation (26). FIG. 2 shows an example in which the domain of ^{2 β is divided into two.} The horizontal axis of the graph of FIG. 2 shows the value of β, and the vertical axis shows the value of 2 ^β .

Line L11 shows ^{2 β.} Points P11, P12, and P13 indicate the intersections of the end of the section in which the domain of β is divided and the line L11, respectively. The point P11 indicates the intersection of the straight line β = 0 and the line L11. The point P12 indicates the intersection of the straight line β = 0.5 and the line L11. The point P13 indicates the intersection of the straight line β = 1 and the line L11.
The line L12a and the line L12b approximate the line L11 in the sections [0,0.5) and [0.5,1) of β, respectively. The line L12a is a straight line (line segment) connecting the points P11 and P12. The line L12b is a straight line (line segment) connecting the points P12 and P13.
As described above, according to the equation (26), β ² can be approximated in a manner that is easy to grasp intuitively.

The approximation by equation (26) is applied to the approximation of the exponential function exp. Specifically, by replacing the coefficient _{a n} of formula _{(19), l} and _{b n,} the _l by a factor a _{'n, l} and b' _{n, l,} the approximation of the exponential function exp is as in Equation (27) Shown.

^{2 α} in equation (27) can be calculated by a shift operation (bit shift) in binary representation. Further _{, a'n, l} β + b'n _{, l} are linear functions of β. As described above, according to the equation (27), the exponential function exp can be approximated by the shift operation and the linear equation. According to the equation (27), the calculation of the exponential function exp can be speeded up in that the calculation can be performed by a simple process. In particular, Eq. (27) is easy to implement in hardware because it uses a shift operation and a simple function such as a linear function. By implementing the equation (27) in hardware, the operation of the exponential function exp can be speeded up.
^{The calculation of “2 α} ” in the equation (27) corresponds to the processing example of the integer partial calculation unit 13. _{The calculation of "a'n, l} β + b _{n, l} " in the equation (27) corresponds to the processing example of the fractional partial calculation unit 14. The multiplication of these calculation results corresponds to the example of the processing of the multiplication unit 15.

Next, the approximation of the exponential function exp is applied to the mixed Gaussian model.
By introducing the approximation of equation (27) into equation (3) showing the normal distribution, the normal distribution is approximated as in equation (28).

Here, _An is a real number constant and is expressed as in Eq. (29).

_{The approximate Mapp} (x; μ _{c, m} , Σ _{c, m} ) of the normal distribution shown in the equation (28) is referred to as an approximate Gaussian distribution (or an approximate normal distribution).
Equation (30) holds for the approximate Gaussian distribution _Mapp (x; μ _{c, m} , Σ _{c, m).}

Further, equation (31) holds for the approximate Gaussian distribution _Mapp (x; μ _{c, m} , Σ _{c, m).}

From Eqs. (30) and (31), the approximate Gaussian distribution _Mapp (x; μ _{c, m} , Σ _{c, m} ) is a probability density function.
When the approximate Gaussian distribution M _app (x; μ _{c, m} , Σ _{c, m} ) is applied to Eq. (1), Eq. (32) is obtained.

f _app (x) simulates the probability density function f (x) of a mixed Gaussian model. _{Simulation of a mixed Gaussian model using fapp} (x) is referred to as an approximate mixed Gaussian model (approximate GMM).
Applying the approximation of equation (26) to equation (5), the posterior probability _papp (c, m | x) for the component m of class c in Bayesian estimation based on the approximation GMM is as in equation (33). Shown.

Since each of α and β takes a value for each class c and component m, the subscripts c and m or c'and m'are added.
As described above, according to the approximate GMM, the Bayesian estimation based on the GMM can be approximated without using the exponential function, and the calculation can be speeded up in this respect.

Next, the application of the approximate GMM to the neural network will be described. Hereinafter, the neural network to which the approximate GMM is applied is referred to as a stochastic neural network based on the approximate GMM.
When equation (5) is _{expanded using the mean vector μ c, m} and the covariance inverse matrix Σ _{c, m} ^-1 , a new input vector X and η ^{(c, m) are obtained as shown in equation (34).} The arguments of the exponential function can be apparently linearized using and.

Here, the new input vector X is expressed by Eq. (35).

That is, the new input vector X has the product of two or less elements of _{the elements x 1} , ..., X _{D of the original input vector x as elements.}
Further, η ^{(c, m)} is expressed by the equation (36).

Here, si _{, j} ^{(c, m)} is an element of the covariance inverse matrix Σ _{c, m} ^-1 , and is expressed by Eq. (37).

Further, η ₀ ^{(c, m)} is expressed by the equation (38).

δ _ij is the Kronecker Delta and is expressed as in Eq. (39).

When the formula (24), the formula (25) and the formula (27) are applied to the formula (34), it is expressed as the formula (40).

_{Α c and m} of the formula (40) can be expressed as the formula (41).

_{Β c and m} of the formula (40) can be expressed as the formula (42).

As shown in the equation (6), k = log ₂ e, and this k can be used to set the vector w ^{(c, m)} of the new parameter as in the equation (43).

Comparing Eq. (40) with Eq. (34), the GMM parameters μ _{c, m} , Σ _{c, m} and γ _{c, m} can be represented by the unconstrained vector w ^{(c, m).}
If the value of the vector w ^{(c, m)} can be obtained appropriately, the product of the new input vector X obtained by nonlinearly transforming the input vector x and the vector w ^{(c, m)} is obtained, and a linear equation using a coefficient is obtained. And bit shift can perform Bayesian estimation operations. ^{Therefore, the value of w (c, m)} is determined by machine learning with the vector w ^{(c, m)} as the weight of the neural network.

FIG. 3 is a diagram showing a configuration example of a stochastic neural network based on the approximate GMM. With the configuration shown in FIG. 3, the neural network 20 includes a non-linear conversion unit 21, a first layer 22, a second layer 23, and a third layer 24.
The non-linear conversion unit 21 converts the input vector x = [x ₁ , x ₂ , ..., X _D ] ^T into a new input vector X, as shown in the equation (35). Here, the number of elements of the new input vector X is H, and it is also expressed as _{X = [X 1} , X ₂ , ..., X _H ] ^T. H = 1 + (D + 3) / 2.

The first layer 22 includes the same number of nodes 22-1 to 22-H as the number of elements H of the new input vector X. The node 22-h of the first layer (h is an integer of 1 ≦ h ≦ H) outputs the input Xh to each node of the second layer 23. When a node input to 22-h and ⁽¹⁾ _{I h,} the output from the node 22-h ⁽¹⁾ and _{O h,} is represented by equation (44).

The second layer 23 has the same number of nodes 23-1, 1, ..., 23-1, M1, ..., 23-c, m, ..., 23- as all the components of all classes in the approximate GMM. C, equipped with a _{M C.}
Paths are provided from all the nodes of the first layer 22 to all the nodes of the second layer 23, and weights are set for each route. The output from the node of the first layer 22 is multiplied by the weight and input to the node of the second layer 23, and the node of the second layer receives the input of the weighted total of the output from the first layer. When node 23-c of the second layer 23 from the node 22-h of the first layer 22, the weight of the path to m _w ^{h c,} and ^m, node 23-c, the input to the m ⁽²⁾ _{I c, m} is expressed as in equation (45).

Each node of the second layer 23 performs the calculation of the equation (40), calculates and outputs the posterior probabilities for each class and each component. Outputs from nodes 23-c, m ⁽²⁾ _{Oc, m} are expressed as in Eq. (46).

As shown in equation (46), the calculation performed by each node of the second layer 23 includes an approximate calculation of the power to be performed by the approximate calculation device 10. In this respect, each node of the second layer 23 corresponds to the example of the approximate calculation device 10.
The third layer 24 includes the same number of nodes 24-1 to 24-C as the number C of the class.
Each node of the second layer 23 outputs data (value) to the node corresponding to the class among the nodes of the third layer 24 according to the class corresponding to the node. The nodes 23-c and m of the second layer output to the nodes 24-c of the third layer 24. Each node of the third layer 24 receives the input of the total output from the node of the second layer, and outputs the total.
The inputs to the node 24-c of the third layer 24 and ⁽³⁾ _{I c,} when the output from the node 24-c and ⁽³⁾ _{O c,} is represented by equation (47).

Next, the learning rule in the neural network 20 will be described. Neural network 20 learns the weight _w ^{h c,} the value of ^m.
Consider the case where N input vectors x ⁽ⁿ⁾ (n is a positive integer of 1 ≦ n ≦ N) and the teacher signal vector T ^{(n) shown in the equation (48) are given as training data.} ..

The evaluation function of the neural network 20 is defined as in Eq. (49) on the premise that the parameters of GMM can be acquired by learning by minimizing the Kullback-Leibler Divergence.

The neural network 20 learns so as to reduce J in the equation (49).
Assuming that the correction amount of the weights w _h ^{c, m} _{is Δw h} ^{c, m} and the learning rate is μ, the correction amount Δw _h ^{c, m} is expressed by the batch learning method as shown in the equation (50).

(∂J _n ) / (∂w _h ^{c, m} ) of the formula (50) is expressed as the formula (51).

Here, X _h ⁽ⁿ⁾ is a new input vector obtained from the input vector x ^(n).
Equations (50) and (51) can be calculated by simple four arithmetic operations, and in this respect, they can be calculated at high speed.
Further, an approximation _log ^{2 (3)} so that the _{O c} can perform high-speed calculation of Equation (49). log ₂ ⁽³⁾ _{O c} can be modified as formula (52).

Here, _mc and n _c are both integers, and 1 ≦ _mc <2.
As in the case of the approximation of the exponential function exp, domain of _{m c} a [1,2) divided into n approximates the _log 2 _{m c.} m _c equal intervals n divided sections the domain of is as shown in equation (53).

Points at both ends of the _log 2 _{m c} in this interval is as shown in equation (54).

The log _{2 mc is approximated by the straight line (line segment) g''n, l} connecting the points at both ends shown in the equation (54) as in the equation (55).

Here, cn _{and l} are expressed as in the equation (56).

d _{n and l} are expressed by the equation (57).

Furthermore, when the _{n = 1, l = 1,} c n, l = 1, d n, can be linearly approximated as l = -1. Therefore, the _log 2 _{m c} Equation (58).

In this case, equation (52) is expressed as equation (59).

Further, the equation (49) is expressed as the equation (60).

It is conceivable that the learning rate μ of the equation (50) is updated every learning cycle based on the equation (61).

Here, assuming that the upper limit setting value of the learning convergence time is t _f , μ _k is expressed by Eq. (62).

λ is an arbitrary constant that satisfies 0 <λ <1. Here, assuming that λ = 0.5, μ _k is expressed as in Eq. (63).

When the function h (J) = √J is Taylor-expanded around t and linearly approximated to the second term, it is shown by Equation 64.

Here, assuming that t = 1, it is expressed as in the equation (65).

Equation (65) is a tangent line at J = 1, and the approximation holds only in the vicinity of J = 1. In order to improve the accuracy of the approximation, the equation (65) is modified as the equation (66).

Here, 0.5 ≦ J ′ ≦ 2. Also, q is a vector of natural numbers.
In this way, learning can be realized with relatively simple calculations.
Next, the accuracy of the above-mentioned approximation will be examined.
_{The error rate E 1 of the approximate function g 1} (z) = 2α (an _{, l} β + b _{n, l} ) of the exponential function shown in the equation (19) with respect to the _{exponential function exp = f 1} (z) is set to the equation (67). ).

Also, shown in the equation (19), the approximation of the exponential function exp function _{g '1 (z) = 2α} (a n, l β + b n, l) of the exponential function exp _{= f} 1 (z) error rate for E' ₁ is defined as in equation (68).

FIG. 4 is a diagram showing the approximation error rate of the exponential function when the _{approximation function g 1 (z) is used.} The horizontal axis of the graph of FIG. 4 shows the value of z, and the vertical axis shows the approximate error rate.
The line L21 shows the approximate error rate when the number of divisions is n = 1. The line L22 shows the approximate error rate when the number of divisions is n = 2. The line L23 shows the approximate error rate when the number of divisions is n = 4.
As shown in FIG. 4, as the number of divisions n increases, the approximation error rate decreases. Even when the number of divisions n = 1, which has the largest approximation error rate, the maximum value of the approximation error rate is 7.59%, and the approximation can be performed with relatively high accuracy.

5, in the case of using an approximate function g _{'1 (z),} is a diagram showing an approximation error rate of the exponential function. The horizontal axis of the graph of FIG. 5 indicates the value of z, and the vertical axis indicates the approximate error rate.
The line L31 shows the approximate error rate when the number of divisions is n = 1. The line L32 shows the approximate error rate when the number of divisions is n = 2. The line L33 shows the approximate error rate when the number of divisions is n = 4.
As shown in FIG. 5, the approximation error rate decreases as the number of divisions n increases. Even when the number of divisions n = 1, which has the largest approximation error rate, the maximum value of the approximation error rate is 6.15%, and the approximation can be performed with relatively high accuracy.

6, the approximation function _g 1 (z), is a diagram illustrating a g _'1 (z) each approximation error rate. In Figure 6, approximation function _g 1 (z), the g _'1 (z), respectively, show the average value and the maximum value of the approximation error rate in each case the division number n = l, 2,4. Referring to FIG. 6, the approximation function g _{'1 (z),} but the average value of the approximation error rate than the approximation function g _{1 (z)} is larger, the maximum value is smaller. However, differences in the approximation function g _{1 (z)} and g _{'1 (z)} and the approximation error rate is relatively small value of about 1%.

Figure 7 is a diagram showing an example of the approximation of the exponential function according to the approximation function g _{'1 (z).} FIG. 7 shows an example in the case where the number of divisions n = 1. The horizontal axis of the graph of FIG. 7 indicates z, and the vertical axis indicates exp (z).
The line L41 shows the actual value of the exponential function exp (z). Line L42 shows the approximate value by the approximation function g _'1 (z).
As shown in FIG. 7, which can be approximated to an exponential function exp (z) by approximation function g _'1 (z).

Next, the approximation accuracy of the logarithmic function will be examined.
_{For the logarithmic function f 2} ( ⁽³⁾ O _c ) = log ₂ ⁽³⁾ O _c of _{the approximate function g 2} ( ⁽³⁾ O _c ) = m _c + n _c -1 of the logarithmic function shown in equation (59). The approximate error rate is defined as in Eq. (69).

FIG. 8 is a diagram showing the approximate error rate of the logarithmic function when the _{approximate function g 2 (z) is used.} The horizontal axis of the graph in Figure 8 shows the value of the ⁽³⁾ O _c, the vertical axis shows the approximation error rate.
The line L51 shows the approximate error rate when the number of divisions is n = 1. The line L52 shows the approximate error rate when the number of divisions is n = 2. Line L53 shows the approximate error rate when the number of divisions is n = 4.
Similar to the case of the approximation of the exponential function, in the case of the approximation of the logarithmic function, the approximation error rate becomes smaller as the number of divisions n increases.
⁽³⁾ for the _{O c} approaches 1 _log ^{2 (3)} _{O c} approaches 0, the approximation error rate gradually increases. Although the apparent approximation error rate is large _{, the approximation error is 0 on average even when the approximation error f 2} ( ⁽³⁾ O _c ) -g ₂ ( ⁽³⁾ O _c ) is the largest number of divisions n = 1. It is .058 ± 0.026, which can be approximated with relatively high accuracy.

FIG. 9 is a diagram showing an example of approximation of a logarithmic function by _{the approximation function g 2} ( ⁽³⁾ _Occ). FIG. 9 shows an example in the case where the number of divisions n = 1. The horizontal axis of the graph in Figure 9 shows the ⁽³⁾ _{O c,} the vertical axis represents the _log ^{2 (3)} _{O c.}
Line L61 shows the actual value of the logarithmic function _log ^{2 (3)} _{O c.} The line L62 shows an approximate value by the approximate function g ₂ ( ⁽³⁾ _Occ).
As shown in FIG. 9, the logarithmic function exp (log ₂ ⁽³⁾ O _c ) can be approximated _{by the approximation function g 2} ( ⁽³⁾ O _c).

Next, the experimental results of class identification based on the approximate mixed Gaussian model will be described. The neural network shown in FIG. 3 was implemented on an FPGA (Field-programmable Gate Array) to perform class identification and learning experiments.
On the hardware, the exponential function was calculated according to the following flow.
1-1. ⁽²⁾ Let the integer values of I _{c and m be α c and m,} and let the decimal values be β _{c and m} .
1-2. Add 1 to β _{c and m to shift α c and m} bits to the left.

The operation of the logarithmic function was performed as follows.
2-1. ⁽³⁾ When the _{O c} ≦ 1 2-2. Proceed to. _{⁽³⁾ O} c> 1 when 2-3. Proceed to.
2-2. ⁽³⁾ _Oc is doubled (1 bit left shift), and 2-1. Return to.
2-3. 2-2. The number of times of doubling (the number of times of 1-bit left shift) is defined as nc, and the calculation of equation (59) is performed.

The square root function can be calculated as follows.
3-1. When J> 2 3-2. Proceed to. When J ≦ 2, 3-3. Proceed to.
3-2. J is multiplied by 1/4 (shifted to the right by 2 bits), and 3-1. Return to.
3-3. 3-2. The number of times of 1/4 times (the number of times of right-shifting by 2 bits) is set to q, and the calculation of the equation (66) is performed.
From the above, each operation of the exponential function, the logarithmic function, and the square root function can be executed by the shift operation and the addition / subtraction on the hardware.

As a comparison target of the experiment, a neural network based on a mixed Gaussian model without the above approximation was constructed by software. The construction of the neural network by software was performed using a computer with a CPU clock frequency of 3.40 GHz (GHz) and a RAM (Random Access Memory) of 16.0 gigabytes (GB) and C language.
In the following, a configuration using FPGA of a neural network based on an approximate mixed Gaussian model will be referred to as an approximate GMM hard configuration. Further, a configuration by software of a neural network based on a mixed Gaussian model without approximation is referred to as a GMM software configuration.

Regarding the class identification rate, both the GMM software configuration and the approximate GMM hardware configuration showed a high identification rate of about 99, and no significant difference was observed.
Regarding the learning time, the GMM software configuration was 6.43 seconds, while the approximate GMM hardware configuration was 8.47 milliseconds, which was shortened to about 1/800.
As described above, each operation of the exponential function, the logarithmic function, and the square root function can be executed by the shift operation and the addition / subtraction, and in this respect, the hardware implementation is easy. Due to the simplification of hardware implementation and calculation, the learning time has been significantly reduced as described above.

As described above, the exponent separation unit 12 separates the exponent of 2 to the real power into an integer part and a decimal part of 0 or more and less than 1. The fractional part calculation unit 14 calculates the power of 2 for the decimal part of the exponent by linear approximation. The multiplication unit 15 multiplies the calculation result of the power of 2 for the integer part and the calculation result of the power of 2 for the fractional part by the fractional part calculation unit 14.
This allows the approximation calculator 10 to replace the power of 2 for the fractional part of the exponent with a simple linear operation. In this respect, according to the approximate calculation device 10, hardware mounting is easy, and it is expected that the calculation time can be shortened by simplifying the hardware mounting and calculation.

Further, the integer partial calculation unit 13 calculates a power of 2 for the integer portion of the exponent by a shift operation.
According to the approximate calculation device 10, hardware implementation is easy in that the power of 2 is performed by a shift operation on the integer part of the exponent, and the calculation time can be expected to be shortened by simplifying the hardware implementation and calculation.

Further, the fractional part calculation unit 14 calculates the power of 2 for the fractional part of the exponent by linear approximation for each section obtained by dividing the section of 0 or more and less than 1.
As a result, the approximation calculation device 10 can improve the accuracy of approximation by increasing the number of divisions when particularly high calculation accuracy is required. In addition, the power of 2 for the fractional part of the exponent can be replaced with a simple linear operation by linear approximation for each interval. In this respect, according to the approximate calculation device 10, hardware mounting is easy, and it is expected that the calculation time can be shortened by simplifying the hardware mounting and calculation.

Further, the fractional partial calculation unit 14 performs a straight line approximation with a line segment connecting both ends of a power of 2 in a section obtained by dividing a section of 0 or more and less than 1.
This allows the approximation calculator 10 to replace the power of 2 for the fractional part of the exponent with a simpler linear operation. In this respect, according to the approximate calculation device 10, hardware mounting is easy, and it is expected that the calculation time can be shortened by simplifying the hardware mounting and calculation.

Further, the bottom conversion unit 11 converts the Napier number to the real power of 2 to the real power of 2.
As a result, in the approximate calculation device 10, the exponential function based on the number of Napiers can be replaced with a simpler operation.

Further, the bottom conversion unit 11 converts the real power of the Napier number in the Bayesian identification based on the mixed Gaussian model into the real power of 2.
According to the approximate calculation device 10, an approximate mixed Gauss model that approximates the mixed Gauss model can be used. The approximate Gaussian model can be implemented with simpler operations than the Gaussian mixed model. In this respect, according to the approximate calculation device 10, it is easy to implement the approximate mixed Gaussian model in hardware, and it is expected that the calculation time can be shortened by simplifying the hardware implementation and calculation.

A program for realizing all or part of the functions of the calculation and control performed by the approximation computing device 10 is recorded on a recording medium that can be read by a computer or hardware, and the program recorded on the recording medium is recorded. Each part may be processed by loading it into a computer system or hardware and executing it. The term "computer system" as used herein includes hardware such as an OS and peripheral devices.
Further, the "computer system" includes a homepage providing environment (or a display environment) if a WWW system is used.
Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. Further, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, and may be a program for realizing the above-mentioned functions in combination with a program already recorded in the computer system.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like within a range not deviating from the gist of the present invention are also included.

10 Approximation calculation device 11 Bottom conversion unit 12 Exponential separation unit 13 Integer partial calculation unit 14 Fractional partial calculation unit 15 Multiplication unit 20 Neural network 21 Non-linear conversion unit 22 1st layer 23 2nd layer 24 3rd layer

Claims (7)

An exponent separation part that separates the exponent of 2 to the real power into an integer part and a fractional part of 0 or more and less than 1.
The power of 2 for the fractional part is divided by the number of the partial sections, which is the index value indicating the partial section including the fractional part among the partial sections in which the sections 0 or more and less than 1 are divided into a plurality of sections. A power of 2 as a power, a power of 2 as a value obtained by dividing -1 by the number, a linear coefficient based on the number, and a value obtained by dividing the index value by the number. A power of 2 as a power, a power of 2 as a power obtained by dividing -1 by the number, and a fractional part calculated by linear approximation using a linear approximation using a constant term based on the index value. Calculation department and
A multiplication unit that multiplies the calculation result of the power of 2 for the integer part and the calculation result of the power of 2 for the fractional part by the fractional part calculation unit.
Approximate calculator. The approximate calculation device according to claim 1, further comprising an integer partial calculation unit that calculates a power of 2 for the integer portion by a shift operation. The approximation calculation device according to claim 1 or 2, wherein the fractional partial calculation unit performs the linear approximation with a line segment connecting both ends of a power of 2 in a section obtained by dividing a section of 0 or more and less than 1. .. The approximate calculation device according to any one of claims 1 to 3, further comprising a bottom conversion unit that converts a real power of a Napier number into a real power of 2. The approximate calculation device according to claim 4, wherein the bottom conversion unit converts the real power of the Napier number in the Bayesian identification based on the mixed Gaussian model into the real power of 2. A programmable device has an exponent separation step that separates an exponent of 2 to the real power into an integer part and a fractional part greater than or equal to 0 and less than 1.
The device sets a power of 2 for the fractional part and an index value indicating a partial section including the fractional part among the partial sections in which a section of 0 or more and less than 1 is divided into a plurality of sections by the number of the partial sections. A power of 2 with the divided value as the power, a power of 2 with the value obtained by dividing -1 by the number, a linear coefficient based on the number, and the index value with the number. A linear approximation based on a linear equation using a power of 2 with the divided value as the power, a power of 2 with the value obtained by dividing -1 by the number, and a constant term based on the index value. Fractional partial calculation steps to calculate and
The apparatus comprising a multiplication step of multiplying the power of calculation result of 2, and a calculation result of the power of two for the fractional part in the fractional part calculating step for the integer part,
Approximate calculation method including. A programmable device with an exponent separation step that separates the exponent of 2 to the real power into an integer part and a fractional part greater than or equal to 0 and less than 1.
The power of 2 for the fractional part is divided by the number of the partial sections, which is the index value indicating the partial section including the fractional part among the partial sections in which the sections 0 or more and less than 1 are divided into a plurality of sections. A power of 2 as a power, a power of 2 as a value obtained by dividing -1 by the number, a linear coefficient based on the number, and a value obtained by dividing the index value by the number. A power of 2 as a power, a power of 2 as a power obtained by dividing -1 by the number, and a fractional part calculated by linear approximation using a linear approximation using a constant term based on the index value. Calculation steps and
A multiplication step that multiplies the calculation result of the power of 2 for the integer part and the calculation result of the power of 2 for the fractional part in the fractional part calculation step.
A program to execute.

Images