JP7017533B2 - Classification device, learning device, classification method and program - Google Patents

Description

The present invention relates to a device for classifying machine translation and human translation.

Traditionally, machine translation has been used by attackers to create malicious documents such as spam emails. Therefore, by determining whether the document is machine-translated or human-translated, it is possible to detect a malicious document with a high probability and reduce the security risk. ..
For example, Non-Patent Documents 1 to 8 propose a method for classifying machine translation and human translation.

Chae, J.M. , Nenkova, A. : Predicting the fluency of text with fluency structural features: case studies of machine translation and human-writing text. In: EACL, pp. 139-147 (2009). Li, Y. , Wang, R. , Zhao, H. : A machine learning method to distinguish machine translation from human translation. In: PACLIC, pp. 354-360 (2015). Arase, Y. , Zhou, M.D. : Machine translation detection from monolingual web-ext. In: ACL (1). pp. 1597-1607 (2013). Aharoni, R.M. , Koppel, M.D. , Goldberg, Y. : Automatic detection of machine translateddext and translation quality estimation. In: ACL (2014). Nguyen-Son, H.M. Q. , Echizen, I. : Transmitting computer-generated textures fluenchyand noise features. In: International Conference of the Pacific Association for Com-Putational Linguistics. pp. 288-300. Springer (2017). Nguyen-Son, H.M. Q. , Thieu, N.M. D. T. , Nguyen, H. et al. H. , Yamagishi, J.M. , Zen, I. E. : Iden-tifying computer-generated textured statistical analysis. In: Asia-Pacific Signaland Information Processing Information Processing Annual Summit and Conference (APSIPAASC), 2017. pp. 1504-1511. IEEE (2017). Labbe, C.I. , Labbe, D. : Duplicate and publications in the scientific literature: how many SCIgen papers in computer science? In: Scientometrics 94 (1), 379-396 (2013). Nguyen-Son, H.M. Q. , Thieu, N.M. D. T. , Nguyen, H. et al. H. , Yamagishi, J.M. , Echizen, I. : Generated computer-generated paragraphs using coherence and fluency facials. In: Proceedings PACLIC (2018).

However, the conventional classification method largely depends on the quality of machine translation, and although it is possible to classify both when the quality of machine translation is significantly inferior to that of human translation, due to the recent improvement in machine translation performance. , The classification accuracy was low.

An object of the present invention is to provide a classification device, a learning device, a classification method and a program capable of accurately classifying machine translation and human translation.

The classification device according to the present invention has a tagging unit that divides each of the document data into words and assigns a part of speech tag to each of the words, and for each of the document data, for each of the words, for each of the words. A distance calculation unit that calculates the distance between the defined word vectors, and an extraction unit that extracts the minimum value in the distance group for the same word for each combination of the document data and the part of speech tag. For each combination of the document data and the part of speech tag, the feature amount calculation unit that calculates the minimum value of the statistics in the group as the feature amount, and for each of the document data, the said for the combination of the part of speech tag. It is provided with a classification unit that inputs a feature amount and outputs a classification result by a model trained by document data in which a classification of machine translation or human translation is labeled in advance.

The statistic may include at least one of mean or variance.

The learning device according to the present invention has a tagging unit that divides each of the document data into words and assigns a part of speech tag to each of the words, and for each of the document data, for each of the words, for each of the words. A distance calculation unit that calculates the distance between the defined word vectors, and an extraction unit that extracts the minimum value in the distance group for the same word for each combination of the document data and the part of speech tag. For each combination of the document data and the part of speech tag, the feature amount calculation unit that calculates the minimum value of the statistics in the group as the feature amount, and for each of the document data, the said for the combination of the part of speech tag. It is provided with a learning unit that takes a feature amount as an input and generates a model that has learned a pre-labeled division of machine translation or human translation.

The learning unit may generate a plurality of the models by a plurality of learning algorithms and select the model having the highest output accuracy of the division.

In the classification method according to the present invention, each of the document data is divided into words, and a tagging step of assigning a part of speech tag to each of the words, and for each of the document data, for each of the words, for each of the words. A distance calculation step for calculating the distance between the defined word vectors, an extraction step for extracting the minimum value in the distance group for the same word for each combination of the document data and the part of speech tag, and an extraction step. For each combination of the document data and the part of speech tag, a feature amount calculation step for calculating the minimum value of the statistics in the group as a feature amount, and for each of the document data, the said for the combination of the part of speech tags. The computer executes a classification step of outputting the classification result by a model trained by the document data in which the feature amount is input and the classification of machine translation or human translation is labeled in advance.

The classification program according to the present invention is for operating a computer as the classification device.

The learning program according to the present invention is for making a computer function as the learning device.

According to the present invention, machine translation and human translation can be accurately classified.

It is a figure which shows the functional structure of the classification apparatus which concerns on embodiment. It is a figure which illustrates the kind of the part of speech tag which concerns on embodiment. It is a figure which illustrates the procedure which the part-of-speech tag is attached to the word which constitutes the document data which concerns on embodiment. It is a figure which shows the specific example of the calculation unit of the distance which concerns on embodiment. It is a figure which shows the procedure which extracts the minimum value of the distance between words which concerns on embodiment. It is a figure which illustrates the document data which concerns on embodiment, and the distance data between words for each part of speech pair. It is a figure which illustrates the feature amount for each document data which concerns on embodiment.

Hereinafter, an example of the embodiment of the present invention will be described.
FIG. 1 is a diagram showing a functional configuration of the classification device 1 according to the present embodiment.
The classification device 1 as a classification device and a learning device is an information processing device (computer) such as a server device or a personal computer, and includes a control unit 10 and a storage unit 20, as well as various data input / output devices and communication devices. ..

The control unit 10 is a part that controls the entire classification device 1, and realizes each function in the present embodiment by appropriately reading and executing various programs stored in the storage unit 20. The control unit 10 may be a CPU.

The storage unit 20 is a storage area for various programs and various data for making the hardware group function as the classification device 1, and may be a ROM, RAM, flash memory, hard disk (HDD), or the like. Specifically, the storage unit 20 stores the dictionary data 21, the corpus 22, the learning model 23, and the like, in addition to the program for causing the control unit 10 to execute each function of the present embodiment.

The control unit 10 includes an input unit 11, a tagging unit 12, a distance calculation unit 13, an extraction unit 14, a feature amount calculation unit 15, a learning unit 16, and a classification unit 17. The control unit 10 generates a learning model 23 that classifies machine translation and human translation by these functional units, and the new document data is either machine-translated or human-translated. The result is output by classifying into.

The input unit 11 accepts the input of the training data of the learning model 23 or the document data (text) as the classification target by the learning model 23.

The tagging unit 12 divides each of the input document data into words, and assigns a part-of-speech tag to each of these words.
Existing morphological analysis methods can be used to add part-speech tags. At this time, the dictionary data 21 in which the part of speech corresponding to the language such as Japanese or English is defined is referred to.
The dictionary data 21 may be stored in a device other than the classification device 1, or may be stored in, for example, a public database accessible via the Internet or the like.

FIG. 2 is a diagram illustrating the types of part of speech tags according to the present embodiment.
Here, 45 kinds of part-of-speech (POS tag) given to the decomposed words and their meanings are shown by taking the case of processing English document data as an example.

FIG. 3 is a diagram illustrating a procedure in which a part-of-speech tag is added to a word constituting the document data according to the present embodiment.
For example, the nouns "school", "morning", "bag" and the like included in the document 1 are given the part of speech tag "NN", and the verbs "go", "walk" and the like are given the part of speech tag "VB". It has been granted.
As described above, the tagging unit 12 assigns one of the above-mentioned 45 types of part-speech tags to each of the plurality of input document data for all the words constituting the document data.

The distance calculation unit 13 calculates the distance between the word vectors defined for each word for each word combination for each document data.
Multidimensional (eg, 300-dimensional) word vectors that are unique to each word are data that are trained on the basis of large datasets and whose relative closeness between words indicates semantic similarity or relevance. be. The word vector is stored in the corpus 22 in advance.
The corpus 22 may be stored in a device other than the classification device 1, or may be stored in, for example, a public database accessible via the Internet or the like.

The distance calculation unit 13 calculates the Euclidean distance d by, for example, the following formula. Here, p and q represent two words, and p _i and q _i represent the i (1 ≦ i ≦ n) th element of the n-dimensional word vector.

FIG. 4 is a diagram showing a specific example of the distance calculation unit according to the present embodiment.
The distance calculation unit 13 includes "school" and "morning", "school" and "bag", and "morning" and "bag" as combinations (part of speech pairs) of nouns (NN) and nouns included in the document 1. As described above, the combinations of the two words are extracted in order, and the distance d of these combinations is calculated.
Similarly, for example, when there are 45 types of part of speech, the distance d between words is calculated for each document for each of the 1035 part of speech pairs.

The extraction unit 14 extracts the minimum value in the group of distances from other words related to the same word for each document data and part of speech pair.
For example, for the part-speech pair "NN-NN" in Document 1, a group of distances between "school" and other words (such as "morning" and "bag"), and a group of distances between "morning" and other words. , The minimum distance is extracted from each of a plurality of groups, such as a group of distances between "bag" and other words.

FIG. 5 is a diagram showing a procedure for extracting the minimum value of the distance between words according to the present embodiment.
In the part of speech pair 1 (NN-NN) of document 1, the minimum value of 0.6 is extracted from the distance group "6.3, 4.6, 2.8, 0.6, 9.2" for a certain word. Will be done. Further, the minimum value 2.1 is extracted from the group "3.9, 6.5, 2.1, 5.8, 4.6" of the distances related to another word.
In this way, one or more minimum value data is extracted for each document data and part of speech pair, and the same processing is performed for all part of speech pairs and document data.

FIG. 6 is a diagram illustrating document data and distance data between words for each part of speech pair according to the present embodiment.
In this example, the minimum value data extracted by the extraction unit 14 is described as a matrix of document data and part of speech pairs. As described above, one or more minimum value data are arranged in each element of the matrix.
Further, when the document data is training data for learning, each document data is given a classification label of machine translation or human translation.

The feature amount calculation unit 15 calculates the statistic in the minimum value group as the feature amount for each document data and part of speech pair.
The statistic may include, for example, at least one of the average or variance calculated by the following formula. Here, _ai (1 ≦ i ≦ n) indicates the i-th of the n minimum value data included in the element (group) of the matrix.

FIG. 7 is a diagram illustrating the feature amount for each document data according to the present embodiment.
In this example, two feature quantities, average and variance, are calculated for 1035 part-speech pairs for each document data.

The learning unit 16 inputs a feature amount for a part of speech pair for each document data, and generates a learning model 23 that has learned a pre-labeled machine translation or human translation division.
The method for generating the learning model 23 may be appropriately selected from various learning algorithms such as logistic regression, a linear classifier, a support vector machine by a stochastic gradient descent method, and a support vector machine by a sequential minimum problem optimization method.
Further, the learning unit 16 may generate a plurality of learning models 23 by a plurality of learning algorithms, and in this case, the learning model 23 having the highest output accuracy may be selected.

When the document data to be classified is input, the classification unit 17 inputs the feature amount (for example, average and variance) for the part of speech pair calculated by the feature amount calculation unit 15 described above, and classifies by the learning model 23. Output the result.

According to the present embodiment, the classification device 1 attaches a part of speech tag to a word constituting the document data, and calculates the distance between the words for each part of speech pair. Then, the classification device 1 generates a learning model 23 based on a known classification of machine translation or human translation by inputting a feature amount obtained by statistically processing the distance between words.
Compared to machine translation, human translation tends to use words with high similarity or relevance consistently not only in a single sentence but also in multiple sentences in a document. The classification device 1 can express such a difference in tendency by a feature amount based on the distance between words, and can generate an appropriate learning model 23.
Therefore, the classification device 1 can accurately classify machine translation and human translation. As a result, it becomes possible to discriminate document data that the user does not want, such as spam mail, with high accuracy.

Further, the classification device 1 can classify the document data accurately by simple calculation by using at least one of the average or the variance regarding the minimum value of the distance for each word as the feature amount.
Further, since the classification device 1 generates the learning model 23 using a plurality of learning algorithms and selects the one with the highest accuracy, the document data can be classified with higher accuracy.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. Moreover, the effects described in the above-described embodiments are merely a list of the most suitable effects resulting from the present invention, and the effects according to the present invention are not limited to those described in the embodiments.

The learning method and the classification method by the classification device 1 are realized by software. When realized by software, the programs that make up this software are installed in the information processing device (computer). Further, these programs may be recorded on a removable medium such as a CD-ROM and distributed to the user, or may be distributed by being downloaded to the user's computer via a network. Further, these programs may be provided to the user's computer as a Web service via a network without being downloaded.

1 Classification device (learning device)
10 Control unit 11 Input unit 12 Tagging unit 13 Distance calculation unit 14 Extraction unit 15 Feature quantity calculation unit 16 Learning unit 17 Classification unit 20 Storage unit 23 Learning model

Claims (7)

A tagging unit that divides each of the document data into words and assigns a part-of-speech tag to each of the words.
For each of the document data, for the combination of the words, a distance calculation unit that calculates the distance between the word vectors defined for each of the words, and a distance calculation unit.
An extraction unit that extracts the minimum value in the distance group for the same word for each combination of the document data and the part of speech tag.
A feature amount calculation unit that calculates a statistic of the minimum value in the group as a feature amount for each combination of the document data and the part of speech tag.
For each of the document data, a classification unit that outputs the classification result by a model trained by the document data in which the feature amount for the combination of the part of speech tags is input and the classification of machine translation or human translation is labeled in advance, and the classification unit. A classification device equipped with. The classification device according to claim 1, wherein the statistic comprises at least one of mean and variance. A tagging unit that divides each of the document data into words and assigns a part-of-speech tag to each of the words.
For each of the document data, for the combination of the words, a distance calculation unit that calculates the distance between the word vectors defined for each of the words, and a distance calculation unit.
An extraction unit that extracts the minimum value in the distance group for the same word for each combination of the document data and the part of speech tag.
A feature amount calculation unit that calculates a statistic of the minimum value in the group as a feature amount for each combination of the document data and the part of speech tag.
A learning device including a learning unit that inputs the feature amount for the combination of the part of speech tags for each document data and generates a model that learns a pre-labeled machine translation or human translation classification. The learning device according to claim 3, wherein the learning unit generates a plurality of the models by a plurality of learning algorithms, and selects a model having the highest output accuracy in the category. A tagging step that divides each of the document data into words and assigns a part-of-speech tag to each of the words.
For each of the document data, for the combination of the words, a distance calculation step for calculating the distance between the word vectors defined for each of the words, and a distance calculation step.
An extraction step for extracting the minimum value in the distance group for the same word for each combination of the document data and the part of speech tag.
For each combination of the document data and the part of speech tag, a feature amount calculation step of calculating the statistic of the minimum value in the group as a feature amount, and
For each of the document data, the classification step of outputting the classification result by the model trained by the document data in which the feature amount for the combination of the part of speech tags is input and the classification of machine translation or human translation is labeled in advance, and the classification step. The classification method that the computer performs. A classification program for operating a computer as the classification device according to claim 1 or 2. A learning program for operating a computer as the learning device according to claim 3 or 4.

Images