RU2656990C1 - System and method for artificial neural network invariant to transferring - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: group of inventions refers to artificial neural networks and can be used to process and recognize signals, such as images, video or sound. Method comprises the steps of: inputting data to the current layer of the trained neural network, processing the input data to obtain output data, and if the number of the current layer of the neural network is less than N, go to the next layer of the neural network, if the number is N, extract the output data. Processing step comprises applying an operation of transferring the input data in one of the predetermined directions, calculating a linear weighted sum of the input data, and adding the shifted input data and the weighted sum calculation result to obtain the output data, the processing being applied to each predetermined direction, or applying an operation of transferring the input data in one or more predetermined directions to obtain shifted input data in each of the predetermined directions, calculating a linear weighted sum of the shifted input data in each of the predetermined directions, and summing the input data and the weighted sum calculation results for each of the predetermined directions to obtain the output data.

EFFECT: reducing the consumption of computing resources while maintaining a high degree of recognition accuracy.

10 cl, 14 dwg

Description

FIELD OF THE INVENTION

This invention relates to the field of machine learning, namely to artificial invariant networks that are invariant to shear. In particular, the present invention relates to the processing and recognition of signals, such as images, video or sound, using such artificial neural networks.

BACKGROUND

Currently, neural networks are one of the effective approaches for recognition, regression, prediction and / or classification of signals. In particular, a convolutional neural network (CNN) is typically used to process and recognize image data, video, sound (e.g., speech) and 3D data (e.g., medical data). CNN learns and develops the necessary hierarchy of attributes (in other words, convolution kernels) during the learning process based on an approximate labeled data set for which the correct recognition answers are set. In other words, during the training process, the neural network selects weights for convolution kernels, and on the first layers, weights correspond to some common features inherent in all signals of a given type, for example, images. For images, such signs may be, for example, lines at different angles. The number of layers in a neural network determines the depth of the neural network. Going deeper, i.e. to subsequent layers, the network begins to highlight more complex features, and the convolution kernel weights become more specific to specific given classes or to a particular regression task. The last layer of the convolutional neural network performs classification or regression, depending on the statement of the problem, based on information received from previous layers. The selection of weighting factors for convolution kernels, for example, is carried out in accordance with the method of back propagation of error.

Thus, CNN can recognize data (for example, classify an object in the image) on the basis of the mentioned kernels and produce as the output signal one most probable object class name (Top-1 class) or several most probable class names, for example, 5 most probable names classes (Top 5 classes). To compare the quality of classification, as a rule, the metrics “Top-1 error” and “Top-5 error” are used. When using the “Top-1 error” metric, the classification of the input signal by the neural network is considered correct if the correct answer matches the Top-1 class. When using the “Top-5 error” metric, the classification of a signal by a neural network is considered correct if the correct answer is in the Top-5 classes issued by the network.

In recent years, the depth of neural networks used has increased, leading to greater accuracy with respect to the correct answers for various problems of machine pattern recognition, while the size of convolution kernels has decreased. In particular, since 2010, the ILSVRC (ImageNet Large Scale Visual Recognition Challenge - Image Recognition Competition in the ImageNet dataset) contest has been held, in which it was found that in 2010 the classification error by the Top-1 metric using well-known shallow networks was 28.2%, and in 2011 - 25.8%. In 2012, the depth of the network used (AlexNet) increased to 8 layers, and the error was 16.4%; this result was improved in 2013 with an error of 11.7%. In 2014, the depth of the neural network used was already 19 layers, which allowed the recognition of objects with an error of only 7.3% (VGG network). In the same year, another well-known network was developed - GoogleNet, the depth of which was 22 layers, and the error was 6.7%. The best result was achieved in 2015 using a ResNet network with a depth of 152 layers and a convolution kernel size of 3 × 3, and the error was only 3.57%. However, the use of such deep convolutional neural networks requires a high consumption of computing resources, namely a large amount of memory, since the repeated use of convolution operations is time-consuming and time-consuming.

One of the well-known solutions describing a parallel convolutional neural network is the solution disclosed, for example, in patent document WO 2014/105865 A1 (Google Inc., “System and method for parallelizing convolutional neural networks” - System and method for parallel convolutional neural networks ) Known CNN is implemented by many convolutional neural networks, each of which is located on the corresponding processing node. Each CNN has many layers. A subset of the layers is interconnected between the processing nodes so that the activation signals move further along the nodes. A disadvantage of the known solution is the high consumption of computing resources.

In the document US 20140180986 A1 (Google Inc., "System and method for addressing overfitting in a neural network" - a system for training a neural network). The switch is associated with feature detectors in at least some layers of the neural network. For each training example, a switch selectively disables each of the feature detectors in accordance with a predetermined probability. Then, the weights of each training example are normalized to apply the neural network to the test data. However, when implementing this known solution, the amount of memory consumed is also large due to the presence of a huge number of parameters on the network.

Thus, the problem of the existing level of technology is that the learning process is a difficult mathematical and computational task, which requires significant hardware resources, time and labor resources.

The objective of the present invention is to provide a faster and simpler architecture of a neural network for processing images, video or sound, in particular, it is important to design a simple and efficient neural network, while minimizing the number of hyperparameters (for example, the number of layers, the size of the nuclei) that must be set to a person when setting up a neural network architecture.

SUMMARY OF THE INVENTION

This problem is solved by the method and system, which are described in the independent claims. Additional embodiments of the present invention are presented in the dependent claims.

According to the present invention, an effective way is proposed in which shear and linear weighted sum operations are used instead of the convolution operation, thereby avoiding layers with laborious convolution operations, which leads to lower computational costs while maintaining high recognition accuracy, while the signal elements are transmitted still considered in context, i.e. taking into account some neighborhood of the element under consideration. This advantage is achieved by reducing the linear weighted sum operation to the well-known generalized matrix multiplication (GEMM) algorithm, and the use of additional memory is not required. GEMM operations are optimized for the GPU / CPU, resulting in less computational resources.

According to a first aspect of the claimed group of inventions, a method for processing signals using a neural network having N layers, comprising the steps of:

- submit the input data to the current layer of the trained neural network, and the input data is a set of numerical values of the signal for processing;

- process the submitted input data to obtain output data; and

- if the number of the current layer of the neural network is less than N, go to the next layer of the neural network and repeat the steps of the method using the obtained output as input, and

- if the number of the current layer of the neural network is N, output data is output;

moreover, the processing step comprises applying the input data shift operation in one of the given directions to obtain shifted input data, calculating the linear weighted sum of the input data and summing the shifted input data and the result of calculating the weighted sum to obtain the output data, and this processing is applied to each given direction , or applying an input data shift operation in one or more predetermined directions to obtain shifted input data in each of one or olee predetermined directions, calculating a linear weighted sum of shifted input data in each of the one or more predetermined directions and summing the results of calculating a weighted sum for each of the one or more predetermined directions and input data to obtain output data.

In this case, the calculation of the linear weighted sum reduces to the operation of generalized multiplication of two matrices (GEMM).

The method further comprises preliminary steps in which:

- receive an approximate labeled data set for training;

- train the neural network based on the obtained approximate labeled data set by determining the corresponding weighting factors.

The signal to be processed can be an image, and the given direction can be a direction in the direction of one of the adjacent neighboring pixels, while the neighboring pixels are determined solely by the topology of the data (for example, 1D (sound), 2D (image) and 3D (video), etc.).

The output is the name of the signal, the classification of the signal, or a list of the most probable names of the signal, or another signal sought, in the case of a regression problem.

According to another aspect of the claimed group of inventions, a system for processing signals using a neural network having N layers, comprising:

- a receiving device, configured to receive input data, which is a set of numerical values of the signal for processing;

- a memory configured to store received input data;

- a processing device configured to read the input data from the memory and perform the steps of the signal processing method according to the first aspect, the processing device further configured to write output data output by the processing device to the memory.

Said processing device is a central processing unit (CPU) and / or a graphics processing unit (GPU).

The technical effects of the present invention are as follows:

- simplicity: reducing the number of hyperparameters that need to be configured when developing a new neural network for a specific task;

- speed: lower time costs and less memory consumption for the CPU and GPU due to the direct implementation of the GEMM instead of the convolution operation without additional preparatory resource-consuming operations, as will be described later;

- accuracy: has the same degree of accuracy as the convolutional neural networks currently used;

- mobile processing (in a mobile device): without the need for additional hardware or cloud support.

Thus, the technical result achieved through the use of this group of inventions is to reduce the consumption of computing resources when recognizing signals using a neural network while maintaining a high degree of recognition accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features and advantages of the present invention will become apparent after reading the following description and viewing the accompanying drawings, in which:

FIG. 1 illustrates a schematic representation of a unit network block in accordance with a known convolutional neural network;

FIG. 2a illustrates a schematic illustration of one embodiment of a unit network block in accordance with an embodiment of the invention;

FIG. 2b illustrates a schematic illustration of another embodiment of a unit network block in accordance with an embodiment of the invention;

FIG. 3 illustrates the reduction of computations to multiplication of two-dimensional matrices;

FIG. 4a illustrates a schematic representation of the application of two parallel shear operations in different directions to the simultaneous application of shear operations in these directions in accordance with an embodiment of the invention;

FIG. 4b illustrates an exemplary representation of two directions with respect to the image in question;

FIG. 5a illustrates a schematic illustration of a unit block of a network when considering a set of sequential images while applying three-way shift operations in accordance with an embodiment of the invention;

FIG. 5b illustrates an exemplary representation of three directions with respect to the considered set of consecutive images;

FIG. 6a illustrates a graph comparing the accuracy of the learning outcomes of the proposed neural network with a ResNet neural network based on the CIFAR-10 data set;

FIG. 6b illustrates a graph comparing the accuracy of the learning results of the proposed neural network with a ResNet neural network based on the CIFAR-100 data set;

FIG. 6c illustrates a graph comparing the accuracy of the learning outcomes of the proposed neural network with a ResNet neural network based on an ImageNet data set;

FIG. 7a illustrates test results based on the CIFAR-10 dataset for the known ResNet neural network and the proposed neural network;

FIG. 7b illustrates test results based on the CIFAR-100 dataset for the well-known ResNet neural network and the proposed neural network;

FIG. 7c illustrates test results based on the ImageNet dataset for the well-known ResNet neural network and the proposed neural network.

DETAILED DESCRIPTION OF THE INVENTION

Various embodiments of the present invention are described in further detail below with reference to the drawings. However, the present invention can be embodied in many other forms and should not be construed as being limited by any particular structure or function described in the following description. Based on the present description, a person skilled in the art will understand that the scope of legal protection of the present invention covers any embodiment of the present invention disclosed herein, regardless of whether it is implemented independently or in combination with any other embodiment of the present invention. For example, a system may be implemented or the method may be practiced using any number of embodiments set forth herein. In addition, it should be understood that any embodiment of the present invention disclosed herein may be embodied using one or more of the claims.

The word “exemplary” is used herein to mean “serving as an example or illustration”. Any embodiment described herein as “exemplary” need not be construed as being preferred or taking precedence over other embodiments.

The following describes an embodiment of the present invention by the example of image processing, however, the claimed invention is also applicable to processing signals of a different type, for example, video, sound (speech), 3D medical data. By input in this document can be meant any data expressed as a matrix of numbers that make up the image, a series of numbers that make up the sound stream; multidimensional arrays of numbers that make up the video stream or medical data, etc. For example, in a grayscale black and white image, the numerical value of a pixel is expressed from 0 (black pixel) to 255 (white pixel). In a color image, for example, each pixel has 3 channels: B (blue), G (green) and R (red), each of which is expressed in the range from 0 to 255, etc.

By definition, convolution is a mathematical operation applied to two functions, generating a third function, which can sometimes be considered as a modified version of the first of them. In this case, the second function is called the convolution kernel. The essence of the convolution operation (or filtering with the core) is that each fragment of the processed image is multiplied by a matrix (convolution core) elementwise, and the result is summed and written to the same position of each element of the output image. The main property of this filtering is that, when the signal is properly normalized, the output value in each pixel is greater, the more the image fragment in the vicinity of the pixel resembles the core itself. Thus, an image convoluted with a certain core will give us another image of the same size, each pixel of which will indicate the degree of similarity of the image fragment to the convolution core, i.e. have a value from 0 (a pixel and its neighborhood, i.e. a fragment of an image, do not look like a convolution core) to 1 (a pixel and its neighborhood coincide with a convolution core). Such another image is called a feature map. To increase the efficiency of the network architecture with respect to the time and memory spent in known convolutional neural networks, small cores are used. Moreover, with a decrease in the size of the core in such networks, the network architecture becomes simpler, but much deeper to achieve greater accuracy, as was shown in the disclosure of the prior art. Currently, as a rule, 3 × 3 convolution kernels are used, or 1 × 3 and 3 × 1 convolutional kernels are used sequentially - separable convolution. Separable convolution reduces the number of operations required to complete the task. According to the present invention, the laborious convolution operation is not applied, and the operation used can be called degenerate convolution using a 1 × 1 kernel. At the same time, the local nature of data processing is preserved, since the connectivity of the processed signal is still taken into account. In other words, pairwise coupling of neighboring pixels is taken into account.

As a rule, the convolution operation is rather slow, and the kernel size is an additional parameter that must be set when designing the network for a specific task, due to the need to consider both the pixel and some neighborhood around it. In particular, when considering a traditional network with a convolution kernel size of 3 × 3, it is necessary to consider the current pixel, as well as 8 neighboring pixels, including those located diagonally, and 9 corresponding weighting factors.

For the well-known neural network AlexNet, when processing the image, the convolution operations in total take up most of the GPU and CPU work time, in particular 95% of the time for the GPU and 90% of the time for the CPU is spent on the convolution operations in the network layers. In this case, the activation operation (ReLU), which is a function used to display input data nonlinearly with the aim of improving the generalizing ability, and a pixel pooling operation that reduces the dimension of the generated feature maps, essentially storing only significant information, do not require large the amount of time or memory to implement.

In FIG. 1 is a schematic representation of a unit block in accordance with a traditional neural network, the application of which will be repeated N times, i.e. so many times what is the depth of the network. According to FIG. 1 in a traditional neural network, the input data I = {I _c } is an image of size PxQ, where P is the image width, Q is its height, and c = 1 ... C is the channel number, two processing paths pass, while the input data is matrix of numerical values of the image. In particular, considering the color image channels B (blue), G (green) and R (red), when considering 1 pixel, 3 numeric channel values are considered, etc. Moreover, the input data is essentially a set of C cards of features of size PxQ. On the first path to the input data, the activation function (ReLU) is applied, after which the input data undergoes a convolution operation, and along the second path, the input data passes unchanged. The size of the core depends on the number of C channels at the input, since each core is also multi-channel (the number of channels in the core coincides with the number of channels at the input). Then, the results of both paths are added up to obtain the output data O = {O _k }, which are essentially a set of K characteristic cards of size PxQ. Further, the passage of the two aforementioned paths is initiated anew with feature maps supplied to the mentioned block as input data, etc. However, it should be noted that the activation operation can also be applied after the convolution operation, and after the summation operation, or not at all, which remains at the discretion of the engineer when performing a specific task. In the example of FIG. 1, a 3 × 3 convolution kernel is applied; however, this figure also shows a region including the current pixel and its vicinity. The crosses in this area mark those pixel positions whose consideration is necessary to obtain information about the current pixel with its neighborhood on one layer of the neural network - only 9 parameters. When applying each convolution kernel, the output produces 1 image of size PxQ - the size of the original image; Thus, when applying K cores, the matrix PxQxK will be obtained, where K is the number of channels in the output image (the number of channels at the output is equal to the number of cores). For example, when using 10 cores, we obtain data with a size equal to P x Q x 10, because 10 results of different convolutions will be obtained.

, где w_c ^k - весовые коэффициенты, а k=1…K - номер ядра, который по существу соответствует номеру соответствующего канала на выходе) для применения обобщенного перемножения матриц (GEMM) вместо операций свертки. По второму пути входные данные проходят без изменения, но со сдвигом в одном из направлений x_i, где i=1…4-4 параметра (весовые коэффициенты, представляющие связь с каждым из 4 соседних пикселей только по горизонтали и вертикали). Операция сдвига в настоящем документе понимается в значении логического сдвига элементов набора числовых значений в заданном направлении. В частности, при применении операции сдвига к входным данным в направлении x_i числовые значения в матрице сдвигаются в этом направлении на одну позицию, а освободившиеся от числовых значений позиции заполняются нулями, что в данном случае аналогично умножению упомянутой матрицы числовых значений изображения на соответствующую матрицу сдвига. Получившийся в результате применения операции сдвига набор числовых значений в дальнейшем называется сдвинутыми входными данными.In FIG. 2a and 2b schematically depict two variants of a unit block in accordance with the neural network disclosed in the present invention, the application of which will also be repeated N times, which corresponds to the depth of the network. According to the proposed neural network, the input data I = {I _c } received using the corresponding receiving device also goes through two processing paths, where c is the channel number. According to the first embodiment shown in FIG. 2a, the activation function (ReLU) is applied on the first path to the input data, after which the input data undergoes the operation of calculating the linear weighted sum (

, where w _c ^k are weights and k = 1 ... K is the kernel number, which essentially corresponds to the number of the corresponding channel at the output) for applying generalized matrix multiplication (GEMM) instead of convolution operations. On the second path, the input data passes unchanged, but with a shift in one of the directions x _i , where i = 1 ... 4-4 parameters (weighting factors representing the relationship with each of 4 neighboring pixels only horizontally and vertically). The shift operation in this document is understood in the value of the logical shift of the elements of a set of numerical values in a given direction. In particular, when applying the shift operation to the input data in the _xi direction, the numerical values in the matrix are shifted in this direction by one position, and the positions freed from the numerical values are filled with zeros, which in this case is similar to multiplying the mentioned matrix of numerical values of the image by the corresponding shift matrix . The resulting set of numerical values resulting from the shift operation is hereinafter referred to as shifted input data.

Then, the results of both paths are added up to obtain the output data O = {O _k }, which are feature maps. Further, the passage of the two above-mentioned paths is initiated anew with feature maps already as input, etc.

). По второму пути входные данные проходят без изменения. Затем результаты обоих путей складываются для получения выходных данных O={O_k}, и прохождение двух вышеупомянутых путей инициируется заново с выходными данными уже в качестве входных данных и т.д. Согласно настоящему раскрытию операция активации также может быть применена и после операции свертки, и после операции суммирования или не применена вовсе, что остается на усмотрение инженера при выполнении конкретной поставленной задачи.According to the second embodiment shown in FIG. 2b, the activation function (ReLU) is applied on the first path to the input data, after which the input data undergoes a shift operation in one of the directions x _i , where i = 1 ... 4. Next, on the first path, the operation of calculating the linear weighted sum (

) On the second path, the input data passes unchanged. Then the results of both paths are added to obtain the output O = {O _k }, and the passage of the two paths mentioned above is re-initiated with the output as input, etc. According to the present disclosure, the activation operation can also be applied after the convolution operation, and after the summation operation or not at all, which is left to the discretion of the engineer when performing a specific task.

These figures also depict a region that includes the current pixel and its neighborhood, on which crosses indicate the connection positions, the consideration of which is necessary to obtain information about the current pixel with its neighborhood - the above 4 parameters corresponding to the weight coefficients assigned to the marked positions - connections. The current pixel in the area in the figures is marked with a dot. Moreover, on one layer of the neural network one image area is considered, i.e. a pixel in communication (aggregate) with a neighboring pixel. Thus, to obtain information about the current pixel with its vicinity, a simple processing of 4 parameters in 4 layers of a neural network is considered instead of the previously known complex processing of 9 parameters in one layer. The applied shift operations and linear weighted sum calculations are simpler and require less computational costs than the convolution operation, and fewer parameters for analyzing the same neighborhood of the pixel under consideration provide the need for less memory, which additionally makes the application of the claimed method on mobile devices more affordable.

As mentioned above, a pooling operation can also be used in a neural network. Traditionally, the subsampling operation is used to reduce the dimension of the data in order to obtain the maximum possible information from the available data at different scales, i.e. the need for subsampling depends on the specific application. Thus, the subsampling operation can be applied on each layer of the neural network, at any stage of this layer, or not applied at all.

In the present invention, as already mentioned above, it is not a pixel that is considered, but the pixel's relationship with a neighboring pixel in one of four directions, which is expressed in the fact that one weighting coefficient is assigned not to one pixel, as is traditionally done, but to two neighboring ones pixels. However, it should be noted that diagonal pixels are also taken into account according to the claimed invention, since at the same time as evaluating the relationship of the pixel, for example, with its neighboring top pixel, its relation to the pixel adjacent to its upper side, etc. . Thus, as indicated above, in a known convolutional neural network, a pixel and the area around this pixel can be analyzed by considering one layer and 9 parameters corresponding to custom weighting factors (see Fig. 1), while in the present invention the area will also be analyzed, but by considering 4 network layers and 4 parameters, 1 parameter for each layer (see Fig. 2a and Fig. 2b).

The convolution operation is, in general, equivalent to the generalized matrix multiplication (GEMM), in particular, the multiplication of three-dimensional matrices, which is performed many times in known convolutional neural networks. However, in traditional approaches, in order to be able to use GEMM, it is necessary to appropriately transform the data, rearrange it and copy it into memory, which leads to an increase in resource consumption. The software implementation of such an operation includes a cycle with 6 investment levels, since the output is a three-dimensional result (a pixel with 3 coordinates k, p, q and the calculation of the sum over three dimensions c, r, s). The classic convolution formula that can be written for the three-dimensional case will look like this:

where w is a set of K convolution kernels of size CxRxS, C is the number of channels in the nuclei (corresponds to the above number of channels at the input, i.e., feature maps in I), R is the width of the convolution kernels, S is the height of the convolution kernels, I is the set from C cards of signs of size PxQ (input data), O is a set of K cards of signs of size PxQ (output data), k is the number of cards of signs of output data (equal to the number of convolution kernels), p = 1, ..., P and q = 1 , ..., Q are the coordinates of the current pixel of the output image, c = 1, ..., C is the card number of the signs of the input data (corresponds to the channel number), r = 1, ..., R and s = 1, ..., S are local to coordinates in the vicinity of the current pixel, g is the function that controls the pixel pitch of the input data (for example, through one, after two, etc.), u and v are the step sizes in pixels vertically and horizontally. Such software calculation is not effective, since it is necessary to carry out either sequential summation or parallelize the process by creating a copy, which is undesirable to fill the memory. In this regard, the traditional approach is very complex and resource-intensive.

The claimed technical solution is similar to the application of linear weighted sum, because core sizes R = 1, S = 1, i.e. virtually eliminating the convolution operation as such. The present invention is reduced to the multiplication of two two-dimensional matrices (directly GEMM):

сводится к одному перемножению двух матриц (см. Фиг. 3). Осуществление доступа к памяти и считывание необходимой информации, хранящейся в ней, выполняется с помощью соответствующего устройства обработки, которое сконфигурировано с возможностью выполнять все вышеупомянутые операции, необходимые для обработки принятых сигналов. Такое устройство обработки может представлять собой, например, центральный процессор (CPU) и/или графический процессор (GPU). При этом отпадает необходимость предварительной обработки входных данных перед применением GEMM.Despite the fact that I represents a three-dimensional matrix, it is stored linearly in the memory, and it can be accessed in the memory as a two-dimensional one, which is clearly shown in FIG. 3. Moreover, w is the matrix of weighting coefficients of one layer of the neural network, each row in which is a vector of linear weighted sum coefficients (analogous to the convolution kernel of size 1 × 1xC). Thus, performing k operations

 reduced to one multiplication of two matrices (see Fig. 3). Access to the memory and reading of the necessary information stored in it is performed using the corresponding processing device, which is configured to perform all of the above operations necessary for processing the received signals. Such a processing device may be, for example, a central processing unit (CPU) and / or a graphics processing unit (GPU). This eliminates the need for pre-processing of input data before applying GEMM.

According to another embodiment, the simultaneous use of shear operations in two or more different directions is proposed. The application of shear operations in two directions at once is shown in FIG. 4a in the form of two parallel paths described above with a shift in two different directions - x ₁ and x ₂ , the results of which are added together and with the input data without change. The output data thus obtained are then again fed to the input of the presented unit block as input data, etc. When applying this other embodiment, no complications occur, because with this approach, the number of cores is divided by the number of different directions; Thus, the number of parameters in the network layer remains unchanged.

For clarity, in FIG. 4b, an exemplary representation of the directions x ₁ and x ₂ with respect to the image in question is illustrated. Of course, the shift operation can be applied simultaneously in any two possible directions from four directions x ₁ , x ₂ , x ₃ (corresponds to the direction - x ₁ ) and x ₄ (corresponds to the direction - x ₂ ).

In FIG. 5a is a schematic illustration of a unit block in accordance with the claimed neural network when considering a set of sequential images, according to which at the same time, shear operations are applied simultaneously in essentially three directions x ₁ , x ₂ and x ₃ , and in the present embodiment, the direction x ₃ along essentially corresponds to the "time axis", i.e. a given sequence of images in a set, in particular, a sequence of frames in a video. An exemplary representation of the indicated three directions with respect to the considered set of consecutive images is shown in FIG. 5 B. The use of such a unit block will also be repeated N times, which corresponds to the depth of the network. As illustrated in FIG. 5a, this embodiment is similar to the above-described simultaneous use of shear operations on the example of applying shear operations simultaneously in two directions, i.e. in parallel, the three paths described above are applied with a shift in three different directions - x ₁ , x ₂ and x ₃ , the results of which are added together and with the input data without change. The output data thus obtained are then again fed to the input of the presented unit block as input data, etc. Thus, one layer of the claimed neural network according to this embodiment processes the entire array of images in a sequence (in particular, frames in a video) at the same time and transfers all output data to the next layer as input.

In FIG. 6a, 6b and 6c show graphs comparing the accuracy of the learning results of the claimed neural network with the ResNet neural network based on the data sets CIFAR-10, CIFAR-100 and ImageNet, respectively. The CIFAR-10 dataset is a dataset containing many images from 10 different classes. CIFAR-100 Dataset - A dataset containing many images from 100 different classes. ImageNet is the largest data set of 1000 categories containing more than 1.2 million images. The ratio of learning process iterations to the percentage of error for a ResNet neural network is known, for example, from the source “Identity Mappings in Deep Residual Networks” (Kaiming He et al.). Based on the comparison results, we can conclude that the accuracy value of the proposed method is not inferior or approximately equal to the accuracy value of the results in accordance with the traditional convolutional neural network ResNet (a comparison of the values for the metrics “Top-1 error” and “Top-5 error” is given ) The results for the Top-1 error metric are shown on the graphs with solid lines, and the results for the Top-5 error metrics are shown with dashed lines, while the results for a known convolutional neural network (with a convolution kernel size of 3 × 3) are executed in black, and the results for the claimed network (using shift operations and calculating the linear weighted sum) are gray. It should also be noted that when considering the learning results on the CIFAR-10 set for the well-known ResNet network, 110 layers and 1.15 million parameters were considered, and when considering the learning results on this set for the declared neural network, 20 layers and 1 million parameters were considered. The CIFAR-100 set for the well-known ResNet network examined 164 layers and 1.7 million parameters, and for the claimed neural network, 38 layers and 2 million parameters were considered. On the ImageNet set, 18 layers and 12 million parameters were considered for the well-known ResNet network, and 32 layers and 13 million parameters were considered for the declared neural network. As graphically illustrated, the claimed neural network is not inferior to the known neural network in accuracy, having similar values of the percentage of error during all iterations of the learning process.

In FIG. 7a, 7b, and 7c show brief tables of test results in the “Top-1 Error” metric for the well-known ResNet network and the declared neural network based on the CIFAR-10, CIFAR-100, and ImageNet data sets, respectively. In particular, the error of the ResNet network based on the CIFAR-10 data set is 6.19%, while the error of the real network based on the same data set is 6.28%; the error of the ResNet network based on the CIFAR-100 data set is 24.83%, while the error of the real network based on the same data set is 25.04%; and the error of the ResNet network based on the ImageNet data set is 29.96%, while the error of the real network based on the same data set is 30.67%. These test results also confirm that the claimed neural network is not inferior in accuracy to the well-known ResNet network.

It should be noted that in relation to other well-known neural networks, for example, such as NIN, DSN, FitNet, Highway, ELU, original-ResNet, stoc-depth, pre-act-ResNet, etc., similar comparison results were obtained, confirming that the claimed the neural network is not inferior or approximately equal in accuracy in comparison with the data of known neural networks, the results of which are disclosed, for example, in the article “Wide residual networks” (Sergey Zagoruyko, Nikos Komodakis, revised June 14, 2017), in particular, table 5 on page 8 of this article. It should be noted that networks with a slightly higher percentage of accuracy (i.e., a lower percentage of error) use a much larger number of parameters.

In addition, it should be noted that to detect the type of network or verify the application of a particular network, knowing its parameters, you can impose on the image a specially designed noise signal. The image will not change by eye for a person, however, the recognition result will become incorrect, because the neural network will be misled. The specific incorrect recognition result is known to the developer of the constructed noise signal, which allows him to detect the use of a specific neural network for which this noise signal was designed. Methods for constructing such signals are known from the prior art and are disclosed, for example, in the information source: rigIntriguing properties of neural networksʺ, Christian Szegedy et al., ArXiv, 2013.

The claimed invention can find application in the field of self-driving vehicles (for recognizing images and videos, in particular for detecting pedestrians, traffic signs and vehicles), video conferences (for video processing, in particular for compressing and decrypting videos, “improving” video by completing parts), medicine (ultrasound, in particular, damage detection, improving image quality by removing noise), security of mobile devices (to identify benefits Vatel, in particular, recognition of the iris, facial or fingerprint).

Specialists in the art should understand that the shown embodiments are exemplary and, as necessary, can be adjusted to achieve greater efficiency in a particular application, unless otherwise specified in the description. Mention of elements of the system in the singular does not exclude many of these elements, unless explicitly stated otherwise.

Although exemplary embodiments of the invention are shown in the present description, it should be understood that various changes and modifications can be made without departing from the scope of protection of the present invention defined by the attached claims.

Claims (23)

1. A method of processing signals using a neural network having N layers, comprising stages in which: - submit the input data to the current layer of the trained neural network, and the input data is a set of numerical values of the signal for processing; - process the submitted input data to obtain output data; and - if the number of the current layer of the neural network is less than N, go to the next layer of the neural network and repeat the steps of the method using the obtained output as input, and - if the number of the current layer of the neural network is N, output data is output; wherein the processing step comprises: applying the input data shift operation in one of the given directions to obtain shifted input data, calculating the linear weighted sum of the input data and summing the shifted input data and the result of calculating the weighted sum to obtain the output data, and this processing is applied to each given direction, or applying an input data shift operation in one or more specified directions to obtain shifted input data in each of one or more specified directions, calculating a linear weighted sum of shifted input data in each of one or more specified directions, and summing the input data and the results of calculating the weighted sum each of one or more specified directions to obtain output data. 2. The method according to claim 1, wherein the calculation of the linear weighted sum is reduced to the operation of generalized multiplication of two matrices (GEMM). 3. The method according to claim 1, wherein the method further comprises preliminary steps in which: - receive an approximate labeled data set for training; - train the neural network based on the obtained approximate labeled data set by determining the corresponding weighting factors. 4. The method according to claim 1, additionally containing stages, which apply the operation of activation and / or subsampling of pixels in any predetermined order. 5. The method according to claim 1, wherein the signal is an image, and predetermined directions are directions toward each of adjacent adjacent pixels. 6. The method according to claim 1, wherein the output is the name of the signal. 7. The method according to claim 1, wherein the output is a signal classification. 8. The method according to claim 1, wherein the output is a list of the most likely signal names. 9. A system for processing signals using a neural network having N layers, containing: - a receiving device, configured to receive input data, which is a set of numerical values of the signal for processing; - a memory configured to store received input data; - a processing device configured to read the input data from the memory and perform the steps of the signal processing method according to claim 1, moreover, the processing device is further configured to write output data output by the processing device to the memory. 10. The system for processing signals using the neural network according to claim 9, in which the processing device is a Central processing unit (CPU) and / or graphic processor (GPU).

Images