JP6844565B2 - Neural network device and program - Google Patents

Description

The present invention relates to neural network devices and programs.

Conventionally, a one-dimensional convolutional neural network has been proposed in which series data in which data elements are arranged in one dimension is used as input data, and features of the input data are extracted by finding a pattern in the input data. For example, Non-Patent Documents 1 and 2 disclose a one-dimensional convolutional neural network that uses a character string representing a protein as series data as input data and predicts the secondary structure of the protein from the character string.

Further, in Non-Patent Document 3, an error function between two series data is calculated by a method called Dynamic Time Warping, and the error function is partially differentiated to obtain the error function. It is disclosed to make an evaluation.

Z. Li and Y. Yu. "Protein Secondary Structure Prediction Using Cascaded Convolutional and Recurrent Neural Networks" In Proc. IJCAI’16, pp. 2560-2567. S. Wang, J. Peng, J. Ma, and J. Xu. "Protein Secondary Structure Prediction Using Deep Convolutional Neural Fields" Scientific reports, Vol. 6, No. 18962, 2016. M. Cuturi, M. Blondel, "Soft-DTW: a Differentiable Loss Function for Time-Series" In Proc. ICML’17

In the one-dimensional convolutional neural network, the comparison between the two series data was performed by the Position-wise Comparion as shown in FIG. 7 (a). Position-wise Comparion is a method of sequentially comparing the i-th data elements of the series data 1 and the series data 2.

However, according to Position-wise Comparion, when a data element is inserted or deleted in at least one of the two series data, the similarity between the two series data that are originally similar may be judged to be low. For example, as shown in FIG. 7B, the series data 1 and the series data 2 are originally similar series data, but the character "L" which is the second disturbance element of the series data 1 is inserted. .. In this case, according to the Position-wise Comparion as shown in FIG. 7A, since the 2nd to 4th characters are different between the series data 1 and the series data 2, the series data 1 and the series data 2 are different. The similarity between the data and the data is low and the data is erroneously determined.

An object of the present invention is the similarity of two series data even when a data element is inserted or deleted in at least one of two originally similar series data between series data in which a plurality of data elements are lined up. It is an object of the present invention to provide the neural network apparatus which can detect more accurately.

The present invention is a neural network device in which input data, which is series data in which a plurality of data elements are arranged, is input, and is partial input data which is at least a part of the input data and series data in which a plurality of data elements are arranged. A similarity calculation unit that calculates the similarity of series data with a filter, wherein at least one of the partial input data and the filter deletes at least one of the data elements, and the partial input data and the said. The neural network apparatus relates to a similarity calculation unit that calculates the series data similarity based on the similarity between the filters and the series data similarity between the partial input data and the filter. It is a neural network device characterized by including a learning unit for learning parameters.

Desirably, the similarity calculation unit at least a process of adding each data element included in the partial input data to the comparison input data element string and a process of adding each data element included in the filter to the comparison filter element string. The process of executing one to calculate the intermediate similarity, which is the similarity between the comparison input data element string and the comparison filter element string, is sequentially performed from the beginning to the end of the partial input data and the sequence of the filters. The series data similarity is calculated by repeating the process, and in each intermediate similarity calculation process, the data element is added to both the comparison input data element string and the comparison filter element string, and the data element is added to the comparison filter element string. The case where the data element is added to the comparison filter element column without being added to the comparison input data element column, and the case where the data element is added to the comparison input data element column and not added to the comparison filter element column. Among the similarities between the above and the comparison filter element sequence, the largest similarity is calculated as the intermediate similarity.

Desirably, the similarity calculation unit considers the deletion cost, which is the cost required to delete the data element when the data element is not added to either the comparison input data element string or the comparison filter element string. , The intermediate similarity is calculated.

Desirably, the learning unit learns the parameters related to the neural network device based on the partial input data, the filter, and the partial differential with respect to the series data similarity with respect to the deletion cost.

Further, the present invention is a neural network device in which input data, which is series data in which a plurality of input data elements are arranged, is input to a computer, and a partial input data which is at least a part of the input data and a plurality of data elements. A similarity calculation unit that calculates the similarity of series data with a filter that is series data in which is arranged, and when at least one of the data elements is deleted in at least one of the partial input data and the filter. Based on the similarity between the partial input data and the filter, the similarity calculation unit that calculates the series data similarity, and the series data similarity between the partial input data and the filter. , A program characterized by functioning as a neural network device including a learning unit for learning parameters related to the neural network device.

According to the present invention, even when a data element is inserted or deleted in at least one of two originally similar series data between series data in which a plurality of data elements are lined up, the similarity of the two series data It is possible to provide a neural network device that can detect the above more accurately.

It is a block diagram of the neural network apparatus which concerns on this embodiment. It is a figure which shows the example of the input data and a filter. It is a calculation algorithm of series data similarity. It is a figure which shows the content of the matrix S used in the calculation algorithm of the sequence data similarity. It is a calculation algorithm of the matrix R necessary for the partial differential calculation regarding the similarity of series data. It is a calculation algorithm of the matrix P necessary for the partial differential calculation regarding the similarity of series data. It is a figure for demonstrating Position-wise Comparison.

Hereinafter, embodiments of the present invention will be described.

FIG. 1 shows a schematic configuration diagram of the neural network device 10 according to the present embodiment. The neural network device 10 in the present embodiment is a server computer, but the neural network device 10 may be any device as long as it exhibits the functions described below.

The neural network device 10 uses a neural network 14 described later to input data which is series data in which a plurality of data elements are arranged and a filter which is series data in which a plurality of data elements are arranged (as described later, this is a neural network). While calculating the series data similarity, which is the similarity with (one of the 14 parameters), the parameters related to the neural network 14 are learned or the characteristics of the input data are extracted.

In the present embodiment, the neural network device 10 uses a character string representing a protein as input data, and by inputting the character string into the neural network 14, a characteristic portion (character array) included in the character string representing the protein. ), The secondary structure of the protein is predicted. Here, a protein is represented by a one-dimensional sequence of 20 kinds of amino acids, and each amino acid is represented by a letter such as "A", "R", or "N". The secondary structure of a protein is a three-dimensional structure composed of several to 20 amino acids, and can be classified into eight types such as α-helix and β-sheet.

Specifically, the input data of the neural network device 10 is
x = x ₁ , x ₂ , ... x _T
It can be expressed as. Note that T is the length (number of characters) of the input data. Each x _i is either a character representing the 20 amino acids. The neural network 14 outputs the predicted value of the label (label indicating the secondary structure of the protein) at each position of the input data. Here, the predicted value of the label at each position of the input data is
y = y ₁ , y ₂ , ... y _T
It is represented by.

The storage unit 12 includes, for example, a hard disk, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. As shown in FIG. 1, the neural network 14 is stored in the storage unit 12. The substance of the neural network 14 is a processing execution program for processing various parameters (including the above-mentioned filter or the deletion cost G described later) related to the neural network 14 and input data. Therefore, the storage of the neural network 14 in the storage unit 12 means that the various parameters and the processing execution program are stored in the storage unit 12.

The communication unit 16 includes, for example, a network adapter and the like. The communication unit 16 exhibits a function of communicating with another device via a communication line such as a LAN (Local Area Network) or the Internet. For example, the communication unit 16 transmits the result information indicating the processing result in the neural network device 10 to another device.

The control unit 18 includes, for example, a CPU (Central Processing Unit) and the like, and controls each unit of the neural network device 10. In particular, the control unit 18 also functions as the similarity calculation unit 20, the prediction unit 22, and the learning unit 24, as shown in FIG. 1, by executing the processing program of the neural network 14 stored in the storage unit 12. To do.

The similarity calculation unit 20 calculates the series data similarity, which is the similarity between the partial input data, which is at least a part of the input data, and the above-mentioned filter. In the present embodiment, since the input data is a character string (amino acid string) representing a protein, the partial input data is a part (or may be the whole) of the character string. The filter is also a character string (amino acid string). As a filter, a character string to be extracted from the input data is set. A plurality of filters may be provided.

The prediction unit 22 predicts the label of each data element of the input data based on the series data similarity between the partial input data and the filter calculated by the similarity calculation unit 20. In the present embodiment, the prediction unit 22 predicts the secondary structure of the protein for each character (amino acid) contained in the character string representing the protein.

The learning unit 24 learns each parameter related to the neural network 14 based on the series data similarity between the partial input data calculated by the similarity calculation unit 20 and the filter.

Details of the processing of the similarity calculation unit 20, the prediction unit 22, and the learning unit 24 will be described later.

The outline of the configuration of the neural network device 10 is as described above. When the neural network device 10 is a computer other than a server (for example, a personal computer), it is configured to include, for example, a keyboard or a mouse in addition to the above-mentioned configuration requirements, and a user's instruction is input to the neural network device 10. It may include an input unit for the purpose of operation, a display unit including a liquid crystal panel and the like, and a display unit for displaying the processing result of the control unit 18.

Hereinafter, the details of the processing of the similarity calculation unit 20 and the prediction unit 22 will be described.

In the present embodiment, the character string of the protein as the input data is a character string composed of 20 kinds of characters. The similarity calculation unit 20 represents each of the 20 types of characters as a d-dimensional vector (feature vector). As a result, the input data which is a character string is converted into a d-dimensional vector string. The conversion to the d-dimensional may be executed by a predetermined method.

For example, as shown in FIG. 2 (a), the input data "A, L, C, K, M, V, ··· " is, d-dimensional vector sequence _{"e 1, e 2, e 3} , e ₄ , e ₅ , e ₆ , ..., e _T "is converted.

Similarly, in the present embodiment, the filter of the neural network 14 is also a character string composed of 20 kinds of characters. Similar to the input data, the similarity calculation unit 20 uses a filter in which 20 types of characters are represented by d-dimensional vectors (feature vectors).

For example, as shown in FIG. 2 (b), "A, C, K, G, ..." Is a d-dimensional vector sequence "W (1), W (2), W (3), W ( 4), ..., W (M) ”is used. Note that M is the length (number of characters) of the filter.

In this embodiment, a plurality of filters are prepared. Each filter is represented _{by W 1} , W ₂ , ... W _J. J is the number of filters.

で表される。 In the present embodiment, a window having a width N centered on the position i of the input data is considered, and the character string included in the window is the partial input data. Therefore, the partial input data is

It is represented by.

The similarity calculation unit 20 calculates the sequence data similarity Sim (X, W) between the partial input data X and the filter W. As described above, since a plurality of filters are prepared, the similarity calculation unit 20 has a plurality of series data similarity Z ^(j) _{between the partial input data X and each filter W j (j is 1 to J).} = Sim (X, W _j ) (1 ≦ j ≦ J) is calculated. The series data similarity is a real value. Sim (X, W) will be described later.

The calculated plurality of series data similarity Z ^(j) = Sim (X, W _j ) (1 ≦ j ≦ J) can be regarded as a J-dimensional vector. Further, since the partial input data X is centered on the position i in the input data, the J-dimensional vector Z _i is calculated for each i (i is 1 to N), and the J-dimensional vector sequence Z _i ^(j) is formed. Will be. The J-dimensional vector sequence is Z _i ^(j) = Z ₁ ^(j) , Z ₂ ^(j) , ... Z _T ^(j).
Is. Here, as each filter W _j , a filter common to each position i is used.

By the above processing, the input data x ₁ , x ₂ , ... X _T are converted into the J-dimensional vector sequences Z ₁ , Z ₂ , ... Z _T.

_{The prediction unit 22 sets the label y i} of each position (each character) i of the input data based on the plurality of series data similarity (J-dimensional vector sequence) Z _i ^{(j) calculated for each position i of the input data.} Predict. Various methods may be used to predict the label y _i , but in the present embodiment, the prediction is made using a multi-layered fully connected neural network having _{a plurality of series data similarity Zi} ^{(j) as inputs.} Alternatively, using a recurrent neural network, after converting a plurality of series-data-similarity Z _i to ^_(j) into a new series data Z _'i, may be predicted using total binding neural network described above.

The fully connected neural network or the recurrent neural network may be stored in the storage unit 12 (included in the neural network 14), or may be prepared separately. Further, an embodiment in which the prediction unit 22 and the above-mentioned fully connected neural network or recurrent neural network are provided in a device other than the neural network device 10 can also be adopted.

Hereinafter, a detailed calculation method of Sim (X, W) by the similarity calculation unit 20 will be described. As described above, the partial input data X is a d-dimensional vector sequence of length N, and the filter W (here, focus on one filter W) is a d-dimensional vector sequence of length M, Sim (X, W). ) Is the sequence data similarity between the partial input data X and the filter W. Hereinafter, X (t) will be the t-th vector of the partial input data X, and W (s) will be the s-th vector of the filter W.

FIG. 3 shows a sequence data similarity calculation algorithm 30. The calculation algorithm 30 shown in FIG. 3 is an algorithm that performs recursive calculation on the matrix S in the (N + 1) row (M + 1) column. Specifically, the procedure is such that the values of each component of the matrix S are calculated from the upper left (1 row and 1 column) to the lower right (N rows and M columns).

The matrix S is shown in FIG. S (t, s), which is the value of each component of the matrix S, represents the degree of similarity between the partial input data X (1: t) and the filter W (1: s). Note that X (1: t) is a vector sequence (data element sequence) from the first to the tth of the partial input data, and W (1: s) is a vector from the first to the sth of the filter W. It is a column.

Explaining the outline of the calculation algorithm 30, the similarity calculation unit 20 adds each vector (each data element) included in the partial input data X to the comparison input vector string (comparison input data element string), and the filter W. Perform at least one of the processes of adding each vector included in to the comparison filter vector string (comparison filter element string) to calculate the intermediate similarity, which is the similarity between the comparison input vector string and the comparison filter vector string. The processing is sequentially repeated from the beginning to the end of the arrangement of the data elements in the partial input data X and the filter W, and the intermediate similarity finally obtained is defined as Sim (X, W).

Hereinafter, the details of the calculation algorithm 30 shown in FIG. 3 will be described with reference to FIG.

In the block 32 of the calculation algorithm 30, a process of initializing each component of the 0th row of the matrix S is performed. Specifically, the value −sG is input to the s (s is 0 to M) column in the 0th row of the matrix S. Here, G is a value representing the cost required to delete one vector (data element) of the partial input data X or the filter W. G is a parameter of the neural network 14 to be learned by the learning unit 24.

In the block 34 of the calculation algorithm 30, a process of initializing each component in the 0th column of the matrix S is performed. Specifically, the value −tG is input to the t (t is 1 to N) column in the 0th column of the matrix S.

In the block 36 of the calculation algorithm 30, a process of calculating S (t, s), which is the value of the remaining components of the matrix S, is performed. Specifically, as shown in block 36, the element that takes the maximum value among the three elements is calculated as S (t, s) by the max function. The three elements are S (t-1, s-1) + VecSim (X (t), W (s)), as the first element.
As the second element, S (t-1, s) -G,
As the third element, S (t, s-1) -G
Is.

Here, the first to third elements described above will be described by taking the case of calculating the component 40 in the third row and third column of the matrix S as an example. As described above, the partial input data X is X (1), X (2), X (3), ..., X (N).
And
Filter W is W (1), W (2), W (3), ..., W (M)
Is.

Among the first elements, the first term represents the degree of similarity between the partial input data X (1: t-1) and the filter W (1: s-1). In this example, it is the degree of similarity between the vector sequences of X (1) and X (2) and the vector sequences of W (1) and W (2).

The second term of the first element, VecSim (X (t), W (s)), is a function for calculating the degree of similarity between X (t) and W (s). For example, VecSim may be a function that calculates the inner product of the vector X (t) and the vector W (s). In this example, VecSim (X (t), W (s)) represents the degree of similarity between the vector X (3) and the vector W (3).

That is, the first element has the degree of similarity between the vector sequences of X (1) and X (2) and the vector sequences of W (1) and W (2), and the vector X (3) and the vector W (3). It is the sum of the similarity with. This is a comparison input vector sequence in which a new vector X (3) is added to the vector sequences of X (1) and X (2), and a new vector W in the vector sequences of W (1) and W (2). (3) corresponds to the degree of similarity with the added comparison filter vector sequence. The first element corresponds to the case where the vector is not deleted in both the partial input data X and the filter W.

The first term of the second element represents the degree of similarity between the partial input data X (1: t-1) and the filter W (1: s). In this example, the degree of similarity between the vector sequences of X (1) and X (2) and the vector sequences of W (1), W (2), and W (3). This is a comparison input vector sequence in which a new vector X (3) is not added to the vector sequences of X (1) and X (2), and a new vector in the vector sequence of W (1) and W (2). W (3) is the degree of similarity with the added comparison filter vector sequence. The second element corresponds to the case where one vector of the filter W is deleted (skipped), that is, it is not considered for the calculation of the series data similarity.

The second term of the second element, -G, is the cost required to delete one vector of the filter W, as described above. As described above, in the second element, since one vector of the filter W is deleted, the value obtained by subtracting the deletion cost is calculated.

The first term of the third element represents the degree of similarity between the partial input data X (1: t) and the filter W (1: s-1). In this example, it is the similarity between the vector sequences of X (1), X (2), and X (3) and the vector sequences of W (1) and W (2). This is a comparison input vector sequence in which a new vector X (3) is added to the vector sequences of X (1) and X (2), and a new vector W in the vector sequences of W (1) and W (2). (3) is the degree of similarity with the comparison filter vector sequence to which it was not added. The third element corresponds to the case where one vector of the partial input data X is deleted (skipped), that is, it is not considered for the calculation of the series data similarity.

The second term of the third element, -G, is the cost required to delete one vector of the partial input data X, as described above. As described above, in the third element, since one vector of the partial input data X is deleted, the value obtained by subtracting the deletion cost is calculated.

By the max function, the element having the maximum value among the first to third elements is calculated as the intermediate similarity S (t, s). In other words, the similarity calculation unit 20 adds the vector X (t) to the comparison input vector sequence and adds the vector W (s) to the comparison filter vector in the calculation process of the intermediate similarity S (t, s). When added to the column, when the vector X (t) is not added to the comparison input vector column and the vector W (s) is added to the comparison filter vector column, and when the vector X (t) is added to the comparison input vector column. When the vector W (s) is not added to the comparison filter vector string The largest similarity between each comparison input vector string and the comparison filter vector string is calculated as the intermediate similarity S (t, s). To do.

本実施形態では、ソフトマックス関数が用いられる。 Although the max function is used in the calculation algorithm 30, other functions may be used as long as the maximum value among the first to third elements is taken. For example, a differentiable softmax function may be used instead of the max function. The softmax function is expressed by the following equation.

In this embodiment, the softmax function is used.

By the processing of block 36, the value of each component of the matrix S (see FIG. 4) is calculated, and finally S (N, M), which is the value of the component 42 of N rows and M columns, is calculated. In block 38 of the calculation algorithm 30, the S (N, M) is output as a series data similarity Sim (X, W) between the partial input data X and the filter W.

In the calculation method of Sim (X, W) in the present embodiment, the lengths of the partial input data X and the filter W are the same (N = M), and the deletion cost G is sufficiently large (G → ∞). ), It matches the inner product, which is a standard vector similarity function. That is, the method of calculating Sim (X, W) in the present embodiment is a generalization of a convolutional neural network in which the inner product is a similarity function.

As described above, in the present embodiment, the similarity calculation unit 20 deletes at least one vector (X (t) or W (s)) in at least one of the partial input data X and the filter W. Based on the similarity between the partial input data X and the filter W, the series data similarity between the partial input data X and the filter W is calculated.

According to the present embodiment, even when a data element is inserted or deleted in at least one of the partial input data X and the filter W, the partial input is accurately performed in consideration of the inserted or deleted data element. The degree of similarity between the data X and the filter W can be calculated. As a result, the neural network 14 can predict or learn labels more preferably than before.

In particular, in the character string representing a protein, a certain character in the character string may be deleted or newly inserted in the process of evolution. Therefore, it can be said that the neural network 14 according to the present embodiment is particularly suitable for predicting the secondary structure of a protein. That is, according to the neural network 14, a more natural similarity with the filter is calculated in consideration of the missing or inserted characters (amino acids), and by extension, the prediction accuracy of the secondary structure of the protein is improved.

Regarding the prediction result of the secondary structure of the protein, in the conventional one-dimensional convolutional neural network that was fully trained, the prediction accuracy for the training data was 44.19% and the prediction accuracy for the unknown data was 42.05%. However, in the neural network 14 according to the present embodiment that has been sufficiently learned, the prediction accuracy for the training data is 44.93%, and the prediction accuracy for the unknown data is 42.97%.

Hereinafter, the details of the processing of the learning unit 24 will be described.

The learning of the parameters related to the neural network 14 is performed based on the input data and the learning data whose characteristics of the input data are known. In the present embodiment, since the input data is a character string representing a protein, the character string and data in which the label (secondary structure of the protein) for each character is known are used as training data.

The learning unit 24 learns (updates) each parameter related to the neural network 14 so as to fit the learning data. Specifically, the loss function is differentiated with respect to the parameter, and the parameter is updated in the direction in which the loss becomes smaller. As a specific method of learning, for example, there is a stochastic gradient descent method, but other methods may be used.

In the present embodiment, the learning unit 24 sets Sim (X, W) to each parameter related to the neural network 14 (that is, partial input data X, based on the partial differential with respect to the partial input data X, the filter W, and the deletion cost G). The filter W and the deletion cost G) are learned. According to this, the loss function can be partially differentiated with respect to each parameter of the neural network 14 by the chain rule of partial differentiation, and each parameter can be updated in the direction of reducing the loss.

In order to calculate the partial differential with respect to the partial input data X of Sim (X, W), the filter W, and the deletion cost G, the matrix S used to calculate Sim (X, W) (see FIG. 4). , And a new matrix R and matrix P are needed.

を意味する。 The calculation algorithm 50 for the matrix R is shown in FIG. In the block 52 of the calculation algorithm 50, s decreases by 1 from M to 1, and t decreases by 1 from N to 1. That is, contrary to the calculation of the matrix S, the calculation of the value of each component of the matrix R is performed from the lower right to the upper left of the matrix R. The value R (t, s) of each component of the matrix R calculated in block 52 is

Means.

を意味する。 Further, the calculation algorithm 60 of the matrix P is shown in FIG. In block 62 of the calculation algorithm 60, s increases by 1 from 1 to M, and t increases by 1 from 1 to N. That is, in the calculation of the matrix P, the calculation of the value of each component of the matrix P is performed from the upper left to the lower right of the matrix P. The return value P (N, M) of the calculation algorithm 60 is

Means.

ただし、

である。 From the calculation result of the calculation algorithm 50, the partial differential with respect to the partial input data X of Sim (X, W), the filter W, and the deletion cost G can be calculated as follows.

However,

Is.

のケースを考えると

となる。 Here, when the similarity can be expressed by the inner product, that is,

Considering the case of

Will be.

Although the embodiments according to the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the spirit of the present invention.

For example, in the above embodiment, a character string representing a protein is used as the input data, but other series data may be used as the input data.

For example, a base sequence may be used as input data, and a neural network device 10 may be used to predict the function of a gene. Further, voice recognition may be performed using the neural network device 10 using the voice waveform as input data. Alternatively, the character string may be used as input data, and the neural network device 10 may be used to perform natural language processing on the character string.

10 Neural network device, 12 Storage unit, 14 Neural network, 16 Communication unit, 18 Control unit, 20 Similarity calculation unit, 22 Prediction unit, 24 Learning unit.

Claims (5)

A neural network device in which input data, which is series data in which multiple data elements are arranged, is input.
A similarity calculation unit that calculates the similarity of series data between a partial input data that is at least a part of the input data and a filter that is series data in which a plurality of data elements are arranged, and is the partial input data and the filter. A similarity calculation unit that calculates the series data similarity based on the similarity between the partial input data and the filter when at least one of the data elements is deleted in at least one of the above.
A learning unit that learns parameters related to the neural network device based on the series data similarity between the partial input data and the filter.
A neural network device characterized by comprising. The similarity calculation unit executes at least one of a process of adding each data element included in the partial input data to the comparison input data element string and a process of adding each data element included in the filter to the comparison filter element string. Then, the process of calculating the intermediate similarity, which is the similarity between the comparison input data element string and the comparison filter element string, is sequentially repeated from the beginning to the end in the partial input data and the sequence of the filters. When the series data similarity is calculated and the data element is added to both the comparison input data element string and the comparison filter element string in each intermediate similarity calculation process, the data element is added to the comparison input data. When adding to the comparison filter element column without adding to the element string, and when adding the data element to the comparison input data element column and not adding to the comparison filter element column, the comparison input data element string and the comparison respectively. Among the similarities with the filter element sequence, the largest similarity is calculated as the intermediate similarity.
The neural network apparatus according to claim 1. The similarity calculation unit considers the deletion cost, which is the cost required to delete the data element when the data element is not added to either the comparison input data element string or the comparison filter element string, and considers the intermediate. Calculate similarity,
The neural network apparatus according to claim 2. The learning unit learns the parameters related to the neural network device based on the partial input data, the filter, and the partial differential with respect to the series data similarity with respect to the deletion cost.
The neural network apparatus according to claim 3. Computer,
A neural network device in which input data, which is series data in which a plurality of input data elements are arranged, is input.
A similarity calculation unit that calculates the similarity of series data between a partial input data that is at least a part of the input data and a filter that is series data in which a plurality of data elements are arranged, and is the partial input data and the filter. A similarity calculation unit that calculates the series data similarity based on the similarity between the partial input data and the filter when at least one of the data elements is deleted in at least one of the above.
A learning unit that learns parameters related to the neural network device based on the series data similarity between the partial input data and the filter.
A program characterized by functioning as a neural network device.

Images