CZ306533B6 - A hardware evaluator of the neural network with switching units - Google Patents

Abstract

The hardware evaluator is a device that is intended determining the value of the output variable to a given query, i.e. a group (vector) of input variables based on the already known previously acquired values characterizing the problem being processed. The values characterizing the problem being processed are stored in memory modules that are located in the device. The device, according to the invention, implements a sequence of decision-making and evaluation rules concerning the query, i.e. the input variables, in a manner which leads to achievement of the preset desired value of the output variable or the acceptable approximation to the required value of the output variable. The hardware evaluator determining the value of the output variable comprises more layers (9, 8, 10) and each layer consists of the weight selectors (4), the multiplication block (5), the summation block (6) and the selector of the control signal (7).

Description

Neural network hardware evaluator with switching units

Field of technology

The invention relates to a hardware evaluator of a neural network - a device which is intended to determine the value of an output quantity for a given query, i.e. a group (vector) of input quantities, based on already known, pre-obtained values characterizing the processed problem. The values characterizing the processed problem are stored in the device in memory modules. The device according to the invention implements a sequence of decision and evaluation rules over the query, i.e. the input variables, in a way that leads to reaching a predetermined output value or, at least from the point of view of the target application, an acceptable approach to the output value. Due to the strong connection between the individual steps of decision rules and physical implementers of these steps (on an electronic or optical basis) it is possible to achieve the speed of evaluation of the output quantity at the level of the fundamental frequency on which the physical implementers used.

Prior art

Determining the value of an output variable that controls, controls, or signals a complex fact is a common problem in automatic regulation and control. The output quantity can have different meanings. First, it may be a control variable that controls a device. Such a quantity usually assumes continuous values. In other cases, it can take only a few discrete values, or only two values. In these cases, the relevant device that generates the output quantity is referred to as a classifier, or separator, ie. input quantities (input states) are classified - classified - into several classes, or separates input states into two disjoint classes. The output variable is usually dependent on a number of input variables, in particular on the control deviation and a number of other conditioning variables. To create a relationship between the output variable and the input quantities, a number of devices are used, from simple mechanical or electromechanical, or hydraulic equipment, microprocessor systems, or computer controlled systems. All of these devices have limitations either in how complex the relationship between the input quantities and the output quantity they are able to realize, or in how quickly they can react to changes. In some applications, the requirements for the speed of response to changes in input variables are considerable. In addition, in many cases, the dependence between the input quantities and the output quantity is very complicated. As an example of the occurrence of both of these aspects, we can mention applications that filter measured natural phenomena that occur with a very high frequency in the tens of MHz (eg physical process detectors, massively parallel applications in image data evaluation, or data from phase spaces of image data, etc. .). Other applications of this type include monitoring and prediction of very fast processes in rotating electrical machines, internal combustion engines or turbines.

For the above reasons, processor systems have become more widely used in the field of automatic regulation and control with the advent of digital technology, both in the form of the use of standard computers and in the form of various "embeded" systems with specialized processors. However, despite the very high performance of today's processors operating at a fundamental frequency in the region around 4 GHz, for some applications requiring a very high number of input evaluations per unit time, the computational speed on generally designed processors is limiting for real use. In addition, many quick solutions used so far are not suitable in cases where the nature of the problem implies that it is necessary to process multidimensional signals, i.e. in fact a series of parallel running individual signals.

These limitations are addressed by the present invention, the essence of which is the implementation of the evaluation phase of a pre-learned neural network with switching units on a hardware platform dedicated to this purpose.

- 1 CZ 306533 B6

The essence of the invention

The problem of designing a device for efficient and very fast determination of the value of the output quantity for the vector of input values (e.g. measured parameters of a system) is solved by a hardware evaluator of a neural network with switching units according to the invention. The hardware evaluator creates a representation of the value of the output quantity, which is used to control or manage other devices or technological units. The processed data represent the dependence of the output quantity on many other quantities (ie the case where the output quantity is a function of many variables).

The essence of the invention lies in the fact that the hardware evaluator for determining the value of the output quantity consists of layers and each layer is formed by weight selectors, a multiplication block, a summing block and a control signal selector. The weight selector further consists of a switching block, a memory block and a transfer block. The layers are arranged so that the output of a given layer forms the input of the next layer. A vector of input values is fed to the inputs of the first layer. The output value of the last layer realizes the required approximation of the output quantity for the given vector of input values.

For time-efficient evaluation of the output quantity for a given vector of input values it is necessary to use a simple procedure of processing the passing signal on the one hand, on the other hand, for sufficient quality of determination of the output quantity, sophisticated complex signal processing procedures are suitable. These two requirements generally go against each other. The present invention solves this discrepancy by defining the next layer processing method by means of a control parameter during signal processing in a given layer. This procedure preserves the time and computational complexity of signal processing as it passes through the evaluator, but at the same time significantly expands the possible dependencies between output values and input value vectors, as it in turn atomizes a set of all assumed input vectors into many subgroups on which evaluation is performed independently.

The specific process of input signal processing by the hardware evaluator can be described as follows. The vector of input values of the hardware evaluator is denoted as (x _b x ₂ , ·· * n) · The number of layers of the hardware evaluator is denoted by the symbol K. Each layer represents a block that contains weight coefficients in _kjm , where k is the layer number, i takes values from 1 do n, am takes values from 1 to M _k , where M _k is the number of sets of weighting coefficients belonging to layer k. Threshold values (Pk, i, Pk) are assigned to each layer with index k, (except the last layer k = K). , 2,, Pfe.M _{fe +} iii) 0ally, each layer k (except for the first layer) is assigned a control parameter t _k /, which takes integer values in the range 1, ..., M _k .

_{The values (x b} x ₂ , ··· ·%) are fed to the input of the first layer · The output of the first layer are the control parameter t \ a «- element vector of weights (h'i _b w _b2 , ..., wj _n ), which are simultaneously connected to the input of the next second layer. In general, the output of the Λ-th layer (except for the last layer k = K) are the control parameter t _k and the vector of weights (w ^ i, w _k ^, ..., w ^ _n ), which are at the same time the input sequence (k + 1 ) —No layers. The output of the last K-th layer is only one quantity, which forms the evaluator's response to the inputs (x _b x ₂ , ... x „) of the first layer.

The input values in layer k are processed as follows. According to the value of the control parameter t _k \, the corresponding weight parameter ^v k, i, tk-! Is selected for each input value of the balance w _k. and with this selected weight parameter the value of the weight wk i is multiplied. We denote this product as the weight ^Wk < ⁱ ( ^{= v} k, i, tk-i ' ^w k- ~ i, í). Subsequently, the sum of S _k values of the weights w _kl is calculated over all i = 1, ..., n, and this sum S _k is compared with the threshold values _{m kj} , 7 = 1, ..., M _k . i + 1 of the layer k. Based on this comparison, the control parameter (k + 1) -layer is determined, as such an integer z, for which the value of the sum S _{k is} in the interval (p _k , z, p _k , _z · and). It is assumed here that the values of P _kA and Pk, M _{k + í} + i meet the requirement that for all possible assumed vectors of input values (x _h x ₂ , ... Xn) and all parameters used in _khm , k = 1, ..., K, i = 1, ..., n, m = \, ..., M _k .i + 1, the calculated values of S _{k were} included in the interval (Pk, i, Pk, M _{k + 1} + l}).

The processing of input weights in the last layer K takes place in a similar way as in the previous layers, but in this last layer the control parameter t _K is not defined. The output of the last layer is only the value y = S _K , which is equal to the sum of the values of the weights i = 1, ..., n. This value is the required approximation of the output quantity.

The procedure described above is implemented using individual memory, multiplication, transmission, switching and addition blocks. The implementation of these blocks can be based on different physical principles. The electronic principle represents various implementations, especially digital, as well as electronic analog and non-electronic principles or. in combination with electronic, e.g.

• use of analog electronic components especially for multiplication and addition operations. In principle, the entire system can be implemented in analog, • optical and optoelectronic principles for the implementation of some blocks and possibly the entire device.

Explanation of drawings

Figures 1 to 6 show an overall diagram of the arrangement of the device of the invention, according to the individual layers and functional units. The description of individual schemes is as follows:

• Giant. 1 shows an overall hardware evaluator arrangement that consists of K layers, including first and last layers. (Individual layers have a very similar function. The vector of input values (xi, ..., x „) gradually passes through layers with indices k = 1 to k = K, while each layer is characterized by weight coefficients in _kdm and threshold values p _kj (except first and last, the first layer has only weight coefficients V], and threshold values p _{x <m} , the last Xth layer has only weight coefficient parameters The number of threshold values p _k j differs for different layer indices k, threshold index j in layer k has values from 1 to M _{k + X} + 1. The output y of the last layer is the output value of the whole hardware evaluator.) • Fig. 2 represents an alternative implementation of the hardware evaluator, which is based on multiple use of the implementation of one layer of the hardware evaluator. This recursive evaluation method requires additional memory blocks, which contain sets of values of weighting coefficients in _kim and threshold values _kJ , which are dynamically transferred to the used layers during the evaluation. y. The outputs of the k-th layer are delayed for the time necessary to readjust the memory blocks containing the weight coefficients and threshold values of the subsequent (k + 1) -th layer. The advantage of this is that the hardware evaluator is less demanding on the number of multiplication and summation blocks, as well as weight selectors. This is advantageous for the case of processing high-dimensional vectors. The disadvantage is the lower speed of finding the response to the input vector.) • Fig. 3 shows the processing of the input values of the hardware evaluator by the first (input) layer. (For simplicity, this scheme is given for a fixed number of components of the input value vector (x _h ..., x _n ) equal to 2, with the fact that the extension to another number of input values is obvious. The input values are multiplied digitally or analogously in this layer. the respective values of the weighting coefficients V], jav, ₂ j, the sum of such weighted input values is compared with the threshold values p _Xm and on the basis of this comparison the control parameter t _x is defined, which is together with the values of the weights wi i (= jj. xj a (= Vj, 2,1 · * ₂ ) passed for processing in the next second layer.) • Fig. 4 shows the processing of the input values of the hardware evaluator in the inner layer, ie not in the first (input) or in the last (output) layer. (For simplicity, this scheme is again given for a fixed number of components of the vector of input values (x _b ..., x _n ) equal to 2. The input values of the weights w ^ ii and i _i2 are multiplied digitally or analogously in this layer by the respective values of the weight coefficients v _{kx m} and in _k2m , where the index m is used of that set of weighting coefficients ^v k, \, m is determined by the value of the control parameter t _k 1 generated by the previous layer. The sum of these weighted input values of the Λ-th layer is compared with the threshold values p _km and on the basis of. 3 CZ 306533 B6 control parameter t _k is defined in this comparison, which together with the values w _k , aw _{k2 is} passed for processing in the next (k + 1) layer.) • Fig. 5 shows the processing of the input values of the hardware evaluator in the last, i.e. which layer. (For simplicity, this scheme is again given for a fixed number of components of the input value vector (x _b ..., x „) equal to 2. The input values of the scales are digitally or analogously multiplied in this layer by the respective values of weight coefficients in _{K] m} and v _K2m , where the index m of the used set of weighting coefficients is determined by the value of the control parameter t _K j generated by the previous (K - 1) layer.The sum of weighted input values of the last K of that layer, which is denoted by the symbol y (= S _K ), forms the output value hardware evaluator for inputs (x _b ..., x „).) • Fig. 6 describes a detailed diagram of a weight selector in layer k for the z-input variable X, the hardware evaluator. (The input of the weight selector is control parameter 4 iz from the previous layer of the hardware evaluator. Based on the value 4 i, the switching block determines which weight coefficient in _kqm of the weight coefficients stored in the memory block will be used to multiply the input weight w _{k υ of the} layer k.)

Examples of embodiments of the invention

The implementation of the hardware evaluator (see Fig. 1) consists of at least input 9 and output layer 10, or input 9, output layer 10 and at least one inner layer 8. The output values of the first layer form the input values of the next layer. Except for the first layer, the input values of the layer are equal to the output values of the previous layer.

The input (first) layer 9 (see Fig. 3) is formed by memory blocks 2, which contain individual weight coefficients in _U) for input values x „, then by multiplication block 5, summation block 6 and control signal selector 7. The input of the first layer are values x _} , x ₂ , ..., x _n . The output values of the first layer are the values of the weights wi i, wi _{> 2} , ..., w _ln , and the control parameter t \ for selecting the weighting coefficients in the second layer.

The inner k-th layer 8 (see Fig. 4) is formed by weight selectors 4, which contain individual weight coefficients in _{k iJ} , then by a multiplication block 5, a summing block 6, and a selector of the control signal 7. The input of the inner k-th layer are the values of the weights w _k ij, w _k , ₂ , ·, w _k - _t „and the control parameter t _k i. The output values of the inner kth layer are the values of the weights w _kb , w _{k: 2} , ..., w _k , _n , and control parameter t _k for selecting weighting coefficients in the next layer.

The output (last, K-ta) layer 10 (see Fig. 5) is formed by weight selectors 4, which contain individual weight coefficients in _KjJ , then by multiplication block 5 and summation block 6. The input of the last K-th layer are the values of weights w _{K Ί} i, w _K ] ₂ , w _K control parameter ^ _i. The output value of the last K-th layer is the only value y = S _K , calculated by the summation block, which represents the response of the hardware evaluator to the input values x ₂ , x „

The multiplication block 5 in the k-th layer has input values equal to the values of the input weights w _k . ij, wi „layers ka further by weight coefficients which were selected by weight selectors

4. The output of the multiplication block are the values of the products of the corresponding input weights and the selected weighting coefficients, '> ^w ki, n' ^v k, n, t _k - ' _í ·

Summation block 6 has input values equal to the values of the products ’“ 't-i.n’t'MJt-i.

The output of the summation blocks the value S _k equal to the sum of its input values.

The selector of the control signal 7 has as input the value of the sum S _k . The output of the control signal selector in the k-th layer is an integer t _k , for which S _k & ^ Pk, t _k iPk, t _k + i}, ie the value of S _k lies between G- _{t and t k} . i-is the threshold value _lk aP _{/ k} i (possibly equal to the control para thus selected

-4 CZ 306533 B6 meter tk specifies a set of weighting coefficients in _k . ij _ft k to be used in the (k + 1) layer (either inner 8 or output 10).

The weight selector 4 (see Fig. 6) consists of a switching block 1, a memory block 2, and a transfer block 3. Each weight selector in the eighth layer has one input value, and this is the control parameter tk- \ z (k - 1 ) —No layers. The output value of the weight selector is the value ⁱⁿ kGk-i, where i is the order of placement of the weight selector in layer k, which corresponds to the yellow input value x, in the first layer 9. The output value of the weight selector is one of the input values of the subsequent multiplication block 5. .

For a priori determined number n of input values of the first layer 9 of the hardware evaluator, where n is equal to the number of variables describing the solved separation or approximation problem, the number of used layers K of the hardware evaluator, as well as all its parameters, ie specifically the number of threshold values , weighting coefficients in _kd \, v _kj , Mk, and threshold values p _k ^, ..., Pk, Mk <], where k = 1, ..., K, i = 1, ..., n, obtained during the learning phase for the hardware evaluator, which takes place independently of the structure of the hardware evaluator which is the subject of the present invention. From the point of view of the implementation of the invention, we consider the internal parameters of the hardware evaluator as predetermined values, obtained independently only on the basis of knowledge of the given input variables Xi, x ₂ ,, and their required responses, which the hardware evaluator aims to quantify.

The construction of the hardware evaluator can be realized by means of individual layers 9, 8, 10 of the hardware evaluator, which are connected in series (see Fig. 1).

A functionally identical but implementationally different implementation consists in creating only one layer, which operates in a cyclic mode, in which the outputs from this one layer are used as inputs to the same layer (see Fig. 2). This structure is supplemented by a counter 11 of the signal passage through the layer and before each passage the values of weight coefficients in _kdm , ··, Vk, i, Mk and threshold values pk _m , ... are additionally written to the memory blocks of the weight selector and to the control signal selector. , Pk, Mk, corresponding to the passage through the layer k, by means of a parametric block 12 consisting of the maximum possible number of weight selectors 4 in the individual layers. Obviously, this way of implementing a hardware evaluator requires lower demands on the quantity, design and manufacture of the respective electronic circuits, but at the cost of a lower computing speed.

Industrial applicability

The device according to the invention can form a component of systems for separation, i.e. for separating input quantities representing the desired state, from equally arranged and very similar input quantities representing the undesired state, at high speed. One possible application is the use of the invention in event measurement systems in physical experiments, where the device can serve as a filter for separating undesirable or already known phenomena from the phenomena observed by the experiment and only records of such phenomena make sense to further store and process. Another area is the processing of continuously generated image data in the analysis of extremely fast phenomena, such as lightning, explosions, or shock waves. Other applications are, for example, monitoring and prediction of very fast processes in rotating electrical machines, internal combustion engines, turbines. In all these cases, the advantage of the device according to the invention is applied in the processing of multidimensional signals, i.e. in fact a series of individual signals running in parallel at the same time.

Claims (2)

PATENT CLAIMS A hardware neural network evaluator with switching units, comprising electronic, optical, electro-optical or opto-electronic elements, consisting of an input (9), an output (10), and at least one inner layer (8), characterized in that the output values of the first layer form the input values of the next layer and, except for the first layer, the input values of the layer are equal to the output values of the previous layer, with the input, i.e. the first, layer (9) consisting of memory blocks (2) containing individual weight coefficients. in _ul for the input values x _b , then of the multiplication block (5), the summing block (6), and the control signal selector (7), with the input of the first layer being the values x ₂ , ... x _n and the output values of the first layers (9) are the values of the weighting coefficients wij, m \ ₂ ,, wi „and the control parameter t \ for selecting the weighting coefficients in the second layer; the inner k-th layer (8) consists of weight selectors (4), which contain individual weight coefficients in _klJ , then a multiplication block (5), a summation block (6), and a control signal selector (7), with the proviso that the input of the inner k-th layer are the values of the weights Wi,?,, Wi, «from the previous layer and the control parameter t _k . _b and the output values of the inner k-th layer are the values of the weights w _k \, w _ki 2, ..., w _kn , and the control parameter t _k for selecting the weights in the next layer; the output, ie the last K-th, layer (10) consists of weight selectors (4), which contain individual weight coefficients in _KilJ , then the multiplication block (5) and the summation block (6), with the input of the last K -th layers are the values of weights w _K ij, w _K i ₂ ,, Wk i. «from the previous layer and the control parameter t _K i, and the output value of the last K-th layer is the value S _K , calculated by the summation block (6), and which represents the response of the hardware evaluator to the input values x _} , x ₂ ,., x _n , with the multiplication block (5) in the k-th layer having input values equal to the values of the output weights w _k μ, ..., w _k i „from the previous layer k- \ and further weight coefficients ^v k, i, tk-1>'''> ^v k, n, t _k _ _lf which were selected by weight selectors (4), with the output of the multiplication block are the values of the products of the corresponding weighting coefficients and input weights, i '> ^v k, n, tk i - ^ fc-ι, η; the summation block (6) has input values equal to the values of the products of the corresponding input weights and weighting coefficients ^w k-iA ' ⁱⁿ k, i, tk ~ i>'> ^w ki, n · Vk, n, tk-i, with the proviso that that the output of the summing block is a value of Sk equal to the sum of its input values; the control signal selector (7) has as input the value of the sum Sk and the output of the control signal selector in the k-th layer is the integer tk for which Sk C applies (pk, tk, Pk, tk + i \ ie that the value of Sk lies between tk and tk + i by the threshold value Ph ^& Ptk + i or is equal to the threshold value Pt _k + \ and the number t _k thus chosen specifies the set of weighting coefficients Vk + i, i, t _k used in the subsequent k + 1 layer, ie. inner weight (8) or output layer (10), the weight selector (4) consists of a switching block (1), a memory block (2), and a transfer block (3), with the weight selector in the k-th layer having one input value, and this is the control parameter t _k । of the £ -l layer, and the output value of the weight selector is the value of the weight coefficient Vkď, t _k ^ where i is the order of placement of the weight selector in layer k, and the output value of the weight selector is one of the input values of the subsequent multiplication block (5). Hardware evaluator according to claim 1, characterized in that it consists of one inner layer (8), a counter block (11) and a parameter block (12), which consists of weight selectors (4), with the output of the counter block ( 11) is connected to the weight selectors (4) in such a way that it always activates only one of them, with the counter block (11) counting the current cycle number of the hardware evaluator calculation.