JPH10105531A - Dynamic quantizer using neural net - Google Patents

Abstract

PROBLEM TO BE SOLVED: To surely perform quantization through parallel processing at high speed and to simplify a mounting circuit by decreasing the number of wirings by limiting coupling loads corresponding to respective cells to one kind and feeding only the difference between the value of sum of products and an input parameter amount back to respective cells. SOLUTION: When the amount of sum of products between the outputs and weight of respective cells is more than the input parameter amount, the cell outputting '+1' is operated so as to transit the state of its output to '0' or '1' and when the amount of sum of products is less than the input parameter amount, the cell outputting '0' or '-1' is operated so as to turn its output to '+1'. Namely, a dynamic quantizing means is provided for dynamically repeating the state transition of an output vector from the cell so as to minimize the differential amount. Then, a dynamic quantizer is dynamically operated so that the difference between a representative quantized value obtained by the sum of products between the outputs and weight of respective cells, and the input parameter amount can be converged within the level of hysteresis width and it is enough to feed only this difference back to respective cells.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[Summary] With the development of digital processing technology, the importance of a quantizer for quantizing and encoding an analog signal is increasing. As a quantizer for converting an analog value to a quantized value, a quantizer using a comparator, a lookup table method, or the like is often used. However, in these methods, when the number of quantization steps is larger than the analog value, the circuit configuration becomes very complicated, and there is a problem that the speed is reduced. One of the causes of these problems is that the processing method of these quantization means is performed by sequential processing. As one of means for performing such a problem, that is, parallel processing instead of sequential processing, utilization of a neural network can be considered. As a quantizer using a neural network, J. J. There is an A / D converter proposed by Hopfield et al. Using an interconnected neural network and based on energy minimization theory. However, an A / D converter using this neural network cannot always perform optimal quantization due to generation of a minimum value of energy, and is incomplete when viewed as a quantizer. In addition, this A / D converter requires N × N wirings for the number of cells N, and there is a problem that mounting with an electronic circuit or the like becomes very difficult as the number of cells increases.
According to the present invention, the connection load from each cell is fixed, and the number of N × N wirings required in the conventional mutual connection type neural network is drastically reduced to N wirings. In particular, in the dynamic quantizer based on the neural network of the present invention, there is no minimum value of energy, which is a problem in the conventional interconnected neural network, but only a minimum value. For this reason, accurate quantization, which was impossible with an A / D converter using a conventional interconnected neural network, can be reliably performed in parallel.

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neurocomputing technique for artificially realizing an information processing process similar to the brain and nerves of an organism, and enables various quantizations. Furthermore, it has universality that can cope with various quantizations only by changing a weight matrix (referred to as a template), and enables conversion from various analog signals to digital signals. Such a quantizer is indispensable for digital processing technology, and the dynamic quantizer using the neural network of the present invention can perform the processing by parallel processing instead of sequential processing. By being able to perform parallel processing, the dynamic quantizer based on the neural network of the present invention has a very high quantization speed, and is very suitable for digital processing technology such as image processing that needs to process a large amount of data at once and at high speed. Useful for

[0003]

2. Description of the Related Art Conventionally, analog signal quantization techniques include an A / D converter using a comparator and a table reference method. However, changing the quantization step requires a change in the circuit configuration. As the number of steps increases, the circuit becomes complicated, and there is a problem that the processing speed is reduced because the processing is basically sequential. Further, when there are a plurality of output quantization numbers with respect to the input variable amount, the conventional A / D converter and the table reference method cannot cope. In order to solve these problems, there is an A / D converter using a neural network as a quantizer capable of parallel processing.

An A / D using an interconnected neural network is used as a quantizer using such a neural network.
There is a converter. The operation of an A / D converter using this interconnected neural network is governed by an energy function, and operates based on the principle of energy minimization. However, there is a problem that an input signal cannot be correctly quantized because there are many energy minimum values. In addition, in an interconnected neural network, the number of wirings is required to be N × N for the number N of cells, and when the mounting by an electronic circuit or the like is considered, the wiring becomes very complicated, and the realization is increased by the number of cells. Becomes very difficult. Further, since the number of product-sum operations increases exponentially, the calculation cost is also extremely required. When considering a quantizer for image processing, etc., correct quantization can be performed quickly and reliably, the quantization step, etc. can be easily changed, and more than one type of quantization It is important to realize a quantizer that can output the signal.

[0005]

SUMMARY OF THE INVENTION The present invention is concerned with a conventional quantizer using an interconnected neural network. (1) Correct quantization can be performed at high speed and reliably by parallel processing. (2) to have a generalization ability that can easily change the quantization step; (3) to allow a plurality of types of quantization results to be output depending on the case with respect to the input variable amount; (4) The purpose of the present invention is to solve the problem of reducing the number of wirings for simplifying a mounting circuit such as an electronic circuit. A using a conventional interconnected neural network
In the / D converter, the cells are completely coupled and the wiring becomes complicated. Therefore, in the present invention, the coupling load corresponding to each cell is set to 1
As a kind, a method of feeding back only the difference between the product sum value and the input variable amount to each cell is adopted. If this method is used, an A / D converter using an interconnected neural network for the number of cells N would require N × N wiring, but N
Only the book wiring is required, and the wiring is facilitated. In the dynamic quantizer using the neural network according to the present invention, the quantization step can be changed only by exchanging the weight matrix W, so that the quantization step and the quantization width can be easily changed, and have a high generalization ability. In addition, in the conventional A / D converter using an interconnected neural network, the generation of an erroneous quantization result due to the minimum value of the energy is a very serious problem, but in the present invention, the quantization corresponding to the input variable amount is performed. It has quantization means for performing dynamics reliably. Further, when there are a plurality of expected quantization results for the input variable amount, only one of them can be output in the related art, but in the present invention, an appropriate one is output according to the situation. It has a dynamic quantization means that makes it possible to

[0006]

According to the present invention, when the sum of products of the output and the weight of each cell is larger than the input variable,
The state of the output of the cell that outputs
1), and when the sum of products is smaller than the input variable amount, the output of the cell outputting 0 (or -1) makes a state transition and operates so as to become +1. By designing to operate in this manner, the difference between the sum of products of the output and weight of each cell and the input variable amount falls within the hysteresis width of the hysteresis characteristic of each cell, and the cell that outputs +1 and 0 ( Or, there is a dynamic quantization means in which both cells that output -1) do not have a possibility of making a state transition, that is, the output vector of the cell repeats a state transition dynamically so that the amount of difference is minimized. Furthermore, dynamic quantization means capable of controlling output of an arbitrary output vector when there is a plurality of output vectors that are not likely to make a state transition by controlling the initial value of an internal state variable of each cell. Having. It should be noted that the dynamic quantization means of the present invention can be applied whether the system is a continuous time system or a discrete time system, and when considering a continuous time system, quantization can be performed at high speed by parallel operation of each cell. It is possible. In the method of the present invention, the required sum of products is the same irrespective of the cell, so that the sum of products and the number of wires are drastically reduced as compared with the conventional A / D converter using an interconnected neural network. . In addition, the method of the present invention can cope with various quantizations only by changing the weight matrix (template), and realizes high-speed processing by parallel processing by using a neural network.

In the present invention, a dynamic operation is performed so that the difference between the input representative variable and the quantized representative value obtained by the product sum of the output and the weight of each cell converges within the hysteresis width. The quantization representative value is uniquely determined by the output vector, and only the difference between this value and the input variable amount needs to be fed back to each cell. For this reason, the input to each cell becomes uniform, and the number of wirings is drastically reduced. Unless the difference between the quantized representative value and the input variable amount falls within the hysteresis width, there is no possibility that the output of all cells will undergo a state transition. No quantization result is generated, and an appropriate quantization result is reliably and dynamically realized.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, embodiments of the present invention will be described with reference to the drawings. The target neural net can be roughly classified into a discrete time system and a continuous time system. Here, the operation of a dynamic quantizer using a continuous time system, that is, a neural network described by a differential equation will be described. A dynamic quantizer based on a target neural network can be described by a piecewise linear differential equation including the following hysteresis element.

[0009]

(Equation 1)

[0010] Here, x _i: internal state variables y _i: Output w _i: the connection weight output of i-th cell N: the number of cells u: Input T _h: threshold h (x _i) of the hysteresis Figure 1 The output is +1 (-
1) a time, the internal state variable x _i is the threshold -T
A piecewise linear binary hysteresis in which the output switches to -1 (+1) when _h ( _Th ) is hit. The equilibrium point p _i (y) of each cell corresponding to an arbitrary output vector y in the system represented by the equation (1) is as follows.

[0011]

(Equation 2)

Here, the stability of each cell is defined as follows. Definitions The balance point of each cell p _i (y), the cells that satisfy the _{p i (y) y i>} -T h (3) when the output was y _i is no possibility that the output is switched. Such cells are defined as stable.

On the other hand, when p _i (y) y _i ≦ −T _h (4) is satisfied, the output of the cell may be switched. Such cells are defined as stable.

When all the cells satisfy the condition (3),
The output vector cannot change. Such an output vector is defined as a stable output. When any cell satisfies (4), the output vector always changes.
Such an output vector is defined as an unstable output.

Based on the above definition, what kind of output vector becomes a stable output in the system represented by the equation (1) will be considered. Here, the equilibrium point p (y) corresponding to each output vector y is given as follows.

[0016]

(Equation 3)

Here, it should be noted that the position of the equilibrium point is determined only by the input and output vectors y and not by i. By definition, all cells must satisfy the condition (3) in order for the output vector to be a stable output. Therefore, what output vector becomes a stable output is considered. It is assumed that the range of the input u is normalized so as to satisfy | u | <ｗ | w _j |. For simplicity, w _i >
Assume that it is 0. FIG. 2 shows the internal state of the cell when u = 0. In FIG. 2, the column of y _i = −1 indicates the internal state of the cell whose output is −1, and the column of y _i = + 1 indicates the internal state of the cell whose output is +1. S is obtained by applying a connection weight to the output as represented by S = Σw _j y _j . S is | S | when a _{<T h, y i = -1} , stable with y _i = + 1 cells, the output vector because all the cells in a stable that is, a stable output. When the _{S ≧ + T h, y i} =
The equilibrium point of the cell of +1 does not exist on the branch of hysteresis,
The cell becomes an unstable cell, and the output vector becomes an unstable output.
At this time, the cell that becomes unstable is the cell of y _i = + 1, so that S is reduced by any one of the unstable cells undergoing a state transition.

[0018] When S ≦ -T _h, since the equilibrium of the cell of y _i = -1 is not present on the branches of the hysteresis, the output vector becomes unstable cell is unstable output. At this time, S increases when any of the unstable cells of y _i = −1 makes a state transition. I mean

[0019]

(Equation 4)

If the above condition is satisfied, the output vector y becomes a stable output. This is, in other words, | u−Σw _j y _j | <T _h
Only the output vector y that satisfies is that it becomes stable, by appropriately setting the T _h | u-
Only an output vector that minimizes Σw _j y _j | can be a stable output.

As described above, it has been clarified as to what conditions must be satisfied to make the output vector a stable output.
However, the coupling weight w _i and the hysteresis threshold T
Condition (6) depending on the size of the input value depending on the size of _h
May not exist. In order to enable appropriate quantization according to the magnitude of the input value, a stable output needs to be present according to an arbitrary input value u. Therefore all connection weights w _i such that there are stable output to the input value u, consider the threshold T _h of the hysteresis.

Assuming that the quantized value for the output vector is S = Σw _j y _j (7), since the output y _i takes only two values of ± 1, there are at most 2 ^N possible values of S. is there. Where S
Is represented as follows.

[0023] _{S 1 = min Σw j y j} S i> S i-1 (8) S k = max Σw j y j 1 ≦ k ≦ 2 N That, S _1, S ₂ from those values of S is _{small, ...,} expressed as S _k. This _Si is the quantization value for the input u. In order to output an appropriate quantization value for all inputs u, the quantization step width may be appropriately determined. Condition (6)
Accordingly, the quantization step width is determined by the threshold value of the hysteresis, and its magnitude is 2 | _Th |. Stable output for any input by setting the threshold value T _h of the hysteresis Therefore as follows always present.

The _{ΔS = max (S j + 1} -S j) T h = 0.5 ΔS (9) However, 1 ≦ j ≦ k-1,1 ≦ k ≦ 2 N.

The following is a specific example of quantization. [Example 1] Consider a case where N = 4 and w _j = 2 ^j-1 (j = 1 to N).

The threshold T _h of the hysteresis since it is at this time [Delta] S = 2 than (9), and T _h = 1.
FIG. 3 shows an example of quantization when parameters are set as described above.
Shown in In FIG. 3, the horizontal axis represents the input, and the vertical axis represents the quantization value.

In this case, there are 16 quantization values, and
There is only one stable output corresponding to quantization, and its output vector
Is equivalent to the input value expanded to 4 bits in binary.
Has become. [Example 2] N = 4, w_iConsider the case where = 1.
At this time, since ΔS = 2, T _h= 1. this
Fig. 4 shows an example of quantization when parameters are set as follows.
You. In this case, there are only five quantization values, and the quantization value
In some cases, there are a plurality of stable outputs. Especially for S = 0
There are six corresponding stable outputs. Output such that S = 0
6 types for inputs that produce stable output of force vector
Which of the stable outputs of the
It depends on the price. [Example 3] N = 4, w₁= 1, w_Two= 3, w_Three= 5,
w_Four= 7 is considered. In this case, the quantization value is
There are 15 types, and ΔS = 4. T from_h= 2 and
I do. Example of quantization when parameters are set in this way
Is shown in FIG. The quantization step width is equal to the hysteresis width.
It is even because it is more determined. In contrast, quantum
The conversion value is 4 steps from -16 to +16, but ± 12 exists.
Not even. In such a case, equation (9)
If the hysteresis threshold is determined according to
On the other hand, there are cases where there are two quantization values. Binary
Which of the quantized values converges depends on the internal state change.
Depends on the initial value of the number. Note that the coupling load w_jCombination of values
Depending on the input, a multi-valued multi-valued quantum
The output value is output as a stable output. Such characteristics are
It is a feature of this quantizer that is not available in
Which value converges to the initial value
More controllable.

The dynamics of the system represented by equation (1) is
Output y_iIs determined by the combination of possible values of^NIndividual
The N-dimensional linear equation of V (y) in the half space of
(X _i) As a system that switches by switching
Is:

[0029]

(Equation 5)

Here, V (y) ≡ ｛(x, y) | x _i y _i > −T _h }: half space The solution of the above equation is given by the following equation.

(X _i −p _i (y)) = (x _i (0) −p _i (y)) exp (−τ) (11) where x _i (0) is an initial value of an internal state variable. Since all the order constants are equal, the solution trajectory starting from an arbitrary initial value x (0) is on a half line in the half space V (y) connecting the initial value and the equilibrium point:

[0032]

(Equation 6)

Each space V (y) corresponds to each pattern and its plane
If at least one coordinate of the equilibrium point satisfies equation (4),
The pattern makes a state transition. Specifically, a certain coordinate is x_i=
-T _hy_iOutput y of the cell at the time_iBut
Invert. That is, the time when the pattern changes is (−T_hy_i-P_i(y)) = (x_i(0) -p_i(y)) exp (−τ_i(13) τ_iIs given by the minimum value for i.

Here, a set of initial values where two or more cells are simultaneously switched is 2 ^{N -1} in a 2 ^N dimensional space.
Since it is a dimension, the measure is 0. The internal state variable x _j of a cell other than the i-th cell that has transitioned at the time when the output vector has transitioned is given by the following equation.

[0035]

(Equation 7)

Using x _j obtained by the equation (14) as a new initial value, the calculation is repeated until all cells reach a stable state satisfying the equation (3). It is possible to know exactly whether the state will be a stable equilibrium state.

The cell satisfying the condition (4) becomes unstable and the output state may be switched. Since all the time constants are equal, the cell whose output switches among the unstable cells has the internal state variable closest to the threshold value.

The connection weights are all made uniform, and the input u is
Only cells are y_i= + 1, output of other cells is y_j= −
1 to be stable, and set the initial values of the internal state variables to all
X _k(0) <-T_hWhen given to satisfy
Sterisis threshold T_hClose to, that is, the internal state
Only the output of the cell corresponding to the maximum initial value of the state variables is y
_i= + 1 and functions as a maximum value detector. [Example 4] N = 4, w_i= 1, T_h= 1
Think. At this time, if u = -2, only one cell
y_i= + 1 and becomes stable.

Here, the initial value of the internal state variable is x ₁ (0) <x
_{If 2} (0) <x ₃ (0) <x ₄ (0) <− 1.0, the initial values of all outputs are y _i = −1, and all cells become unstable.
Among them, the one whose output switches is 4
The output of the fourth cell is switched, and the cell is y _{1
= Y 2 = y 3 = -1} , becomes y ₄ = + 1 becomes stable output. In this way, a large internal state variable can be detected.

By changing the size of the input u, it is possible to detect not only the maximum value but also the second largest and the third largest.

[0041]

The present invention is a dynamic quantizer based on a neural network using an element having a hysteresis characteristic in each cell, and almost solves the problems of the conventional A / D converter based on an interconnected neural network. It was done. That is, the dynamic quantizer based on the neural network of the present invention can be realized by an analog circuit, the circuit can be very easily configured, and parallel processing is possible. It also guarantees that appropriate quantization can be reliably performed on input values that were difficult to achieve with conventional interconnected neural networks. In addition, high-speed processing can be performed by performing parallel processing, and various quantizations can be handled simply by changing the connection weight, so that the generalization ability is excellent. Further, the present invention can be used not only as a quantizer but also as a maximum value (minimum value) detection circuit. As described above, the dynamic quantizer using the neural network according to the present invention can perform various quantizations, maximum value detection, and the like by the connection weight, and has a very high generalization ability.

[Brief description of the drawings]

FIG. 1 is a diagram showing binary hysteresis.

FIG. 2 is a diagram showing the state of each cell according to the size of S (the circles in the figure indicate the equilibrium point of each cell. If the equilibrium point exists on each branch of the hysteresis, its output is output). Is stable because there is no possibility of switching, and unstable when there is a possibility of switching).

FIG. 3 is a diagram illustrating an example of quantization when each connection weight is a power of 2, where the horizontal axis represents input and the vertical axis represents output. (N =
4, w _j = 2 ^j−1 , T _h = 1)

FIG. 4 is a diagram showing an example of quantization in a case where each connection weight is uniform, where the horizontal axis represents input and the vertical axis represents output. (N = 4, w _j
= 1, T _h = 1)

FIG. 5 is a diagram illustrating an example of quantization in a case where one input has two quantized output values. (N = 4, w ₁ = 1, w ₂
= 3, w ₃ = 5, w ₄ = 7, _Th = 2)

FIG. 6 is a diagram showing the present system.

Claims (9)

[Claims] 1. A dynamic quantization means for dynamically bringing a sum of products of a quantized output and a weight of a combined cell close to an input variable amount, and providing a self-coupling in each of the cells to obtain an input variable amount. A dynamic quantizer using a neural network, comprising a hysteresis means for setting the number of corresponding quantization levels to one or more. 2. The dynamic quantizing means, where u is an input variable, yi is a cell output value, and Wi is a weight for each cell output value, and the product sum of the output yi and the weight Wi and the input variable u 2. The dynamic quantizer according to claim 1, wherein a dynamic operation is performed so as to minimize the difference. 3. The weight W according to claim 2, wherein the number of quantization levels is
2. The method according to claim 1, wherein the control can be performed by the magnitude of i.
A dynamic quantizer using the described neural network. 4. When the magnitude of the weight Wi according to claim 2 is set to a power of 2, the input variable amount becomes 2 in the binary output of each cell.
2. The dynamic quantizer using a neural network according to claim 1, wherein the dynamic quantizer is in a form of a base number expansion. 5. The dynamic quantizer according to claim 1, wherein the signs of the weights Wi in claim 2 are all the same. 6. The dynamic quantizer according to claim 1, wherein the quantization width can be controlled by the hysteresis width of the hysteresis characteristic of each cell. 7. The dynamic quantizer according to claim 1, wherein the control of the hysteresis width according to claim 6 has a plurality of quantization results for a certain input. 8. The dynamic quantizer based on a neural network according to claim 1, further comprising means for selecting an equilibrium state that converges according to a state variable when the plurality of quantization results, that is, a plurality of equilibrium states exist in the system. 9. A dynamic quantizer based on a neural network according to claim 1, wherein said dynamic quantizer can be operated as a maximum value (or minimum value) selection circuit by the equilibrium state selection function according to claim 8.