KR102334018B1 - Apparatus and method for validating self-propagated unethical text - Google Patents

Abstract

The present invention is a text acquisition unit that acquires a plurality of self-multiplying texts generated in a self-replicating method using a learning text that has been verified in advance for ethical or unethical and labeled learning text, the self-replicating text is authorized, A dictionary-based discriminating unit that determines the unethical nature of the self-multiplying text by searching for words that are similar to the acquired profanity registered in the profanity dictionary at a predetermined level or more. A learning model-based discriminating unit that extracts sentence feature vectors according to the pre-learned pattern estimation method to determine the unethics of self-replicated text, searches the learning text most similar to the self-replicated text, and self-reports according to the label of the searched training text By combining the results of discriminating the unethics of self-propagated texts determined in each of the text-based discriminating unit and dictionary-based discriminating unit, the learning model-based discriminating unit and the original text-based discriminating unit for discriminating the unethics of the augmented text, It is possible to provide an apparatus and method for verifying text validity including a determination result comparison unit for obtaining a final determination result.

Description

Apparatus and method for validating self-propagated unethical text

The present invention relates to an apparatus and method for verifying text, and to an apparatus and method for validating self-propagated unethical text.

Although the current online environment provides various communication means to many users, various profanity and unethical language are frequently used due to online anonymity. Accordingly, online service companies are making efforts to filter out profanity or unethical language, but there is a limitation in that the filtering is not easy as profanity and unethical language are also transformed into various forms and used.

In recent years, attempts have been made to detect profanity or unethical language using an unethical text detection device composed of an artificial neural network. However, in order to use an artificial neural network, learning must precede it, and a large amount of text for learning is required for learning. Here, the training data is text that has been previously verified and labeled whether profanity or unethical language is included.

In the past, since the text was directly verified by a human and used as the training text, there was a limitation in that it was very difficult to obtain a sufficient amount of training text to train the artificial neural network. In order to overcome this difficulty in acquiring texts for learning, a large amount of text for learning is used using an unethical text self-proliferation device that implements a pre-learned artificial neural network to self-proliferate a small amount of text for learning whose ethics or unethical has been verified in advance. It has been proposed to obtain By self-proliferating the learning text using an artificial neural network, it is possible to easily acquire a large amount of learning text from a small amount of learning text, and to acquire learning texts containing various modified forms of profanity or unethical language. .

However, it is necessary to determine whether the self-propagated learning data is also accurately labeled. If the self-propagated learning data is incorrectly labeled, the learning of the unethical text detection device is performed inaccurately, which causes a problem in that text containing profanity or unethical language cannot be properly filtered.

Korean Patent Publication No. 10-2019-0108958 (published on September 25, 2019)

An object of the present invention is to provide an apparatus and method for validating text that can determine whether self-propagated unethical text for learning is valid.

Another object of the present invention is to provide an apparatus and method for validating self-propagated text that can verify the effectiveness of an unethical text self-proliferation apparatus for generating unethical text for learning.

To achieve the above object, an apparatus for validating the validity of self-propagated text according to an embodiment of the present invention is a plurality of self-propagated texts generated in a self-propagation method using a labeled learning text that is ethical or unethical in advance. a text acquisition unit to acquire; a dictionary-based discriminating unit that receives permission from the self-multiplying text and searches for words similar to the profanity registered in the profanity dictionary acquired in advance from the approved self-proliferated text at a predetermined level or more to discriminate the unethical nature of the self-proliferated text; a learning model-based discriminating unit that receives permission and vectorizes the self-multiplying text in word units, and extracts sentence feature vectors from the vectorized words according to a pre-learned pattern estimation method to determine the unethics of the self-multiplying text; a text-based determination unit that searches for a learning text most similar to the self-multiplying text, and determines the unethicality of the self-multiplying text according to a label of the searched learning text; And the dictionary-based discrimination unit, the learning model-based discrimination unit, and the original text-based discriminant determined by each of the unethical determination results of the self-multiplying text by combining the determination to obtain a final determination result for the self-multiplying text A result comparison section is included.

The dictionary-based determining unit performs an N-gram similarity analysis for each word of the profanity registered in the profanity dictionary and the self-multiplying text, and determines whether the profane word is included in the self-multiplying text, and determines that the profane word is included. If determined, the self-replicating text can be determined to be unethical.

The learning model-based determining unit includes: a vector conversion unit for obtaining a plurality of word vectors by embedding and vectorizing each word of the self-multiplying text; a sentence feature extractor configured to acquire the sentence feature vector by accumulating and extracting features of the plurality of word vectors according to a pre-learned pattern estimation method; and a feature classifier configured to classify the sentence feature vector according to a pre-learned pattern classification scheme to discriminate unethical behavior of the self-multiplying text.

The sentence feature extractor may be implemented as a Long Short Term Memory (LSTM).

The determination result comparison unit obtains the final determination result by applying a majority rule to the determination result of the self-multiplying text unethical determined by each of the dictionary-based determination unit, the learning model-based determination unit, and the text-based determination unit can do.

The determination result comparison unit assigns different weights predetermined for each to the determination result of the unethical self-multiplying text determined by each of the dictionary-based determination unit, the learning model-based determination unit, and the text-based determination unit, A result to which a higher weight of ethical or unethical is assigned according to the assigned weight may be obtained as the final determination result.

The device for verifying the validity of the self-multiplying text compares the final determination result with the label labeled as ethical or unethical at the time of generating the self-multiplying text, and determines that the label of the self-multiplying text is valid, Otherwise, it may further include a label comparison unit that determines that it is invalid.

The label comparator may calculate the reliability of the self-multiplying text according to a result of determining the validity of the labels of a plurality of self-multiplying texts.

The self-propagated text validation apparatus removes additional components from the self-proliferated text obtained by the text acquisition unit, and divides it into sentences by the dictionary-based determination unit, the learning model-based determination unit, and the original text-based determination unit It may further include a pre-processing unit for transmitting to each unit.

In a method for validating self-propagated text according to another embodiment of the present invention for achieving the above object, a plurality of self-propagated texts generated in a self-propagated method using a labeled learning text with ethical or unethical verification in advance Self-propagation text acquisition step to acquire; A dictionary-based determination step of determining the unethics of the self-multiplying text by searching for words similar to the profanity registered in the profanity dictionary acquired in advance at a predetermined level or more in the self-multiplying text; The self-propagated text is received and vectorized in word units, a sentence feature vector is extracted from the vectorized word using a learning model in which a pattern estimation method has been previously learned, and the unethical nature of the self-propagated text is determined based on the extracted sentence characteristics. a learning model-based determination step to determine; a text-based determination step of searching for a learning text most similar to the self-multiplying text, and discriminating the unethicality of the self-multiplying text according to a label of the searched learning text; And by combining the results of determining the unethics of the self-propagated text determined in each of the dictionary-based determination step, the learning model-based determination step, and the text-based determination step, a final determination result for the self-multiplying text is obtained. a determination step.

Accordingly, the apparatus and method for validating self-propagated text according to an embodiment of the present invention verifies the label of a large amount of labeled learning text generated by the self-proliferation method, thereby accurately verifying the validity of the training text obtained by the self-propagation method. can be verified. Therefore, it is possible to greatly increase the reliability of the training text for learning a detection device that detects profanity or unethical language by being implemented as an artificial neural network.

1 shows a schematic structure of a self-propagated text validation apparatus according to an embodiment of the present invention.
FIG. 2 shows a detailed configuration of the learning model-based determining unit of FIG. 1 .
3 shows a self-propagated text validation method according to an embodiment of the present invention.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the practice of the present invention, reference should be made to the accompanying drawings illustrating preferred embodiments of the present invention and the contents described in the accompanying drawings.

Hereinafter, the present invention will be described in detail by describing preferred embodiments of the present invention with reference to the accompanying drawings. However, the present invention may be embodied in various different forms, and is not limited to the described embodiments. In addition, in order to clearly describe the present invention, parts irrelevant to the description are omitted, and the same reference numerals in the drawings indicate the same members.

Throughout the specification, when a part "includes" a certain component, it does not exclude other components, unless otherwise stated, meaning that other components may be further included. In addition, terms such as "...unit", "...group", "module", and "block" described in the specification mean a unit that processes at least one function or operation, which is hardware, software, or hardware. and a combination of software.

1 shows a schematic structure of a self-propagated text validation apparatus according to an embodiment of the present invention, and FIG. 2 shows a detailed configuration of the learning model-based discrimination unit of FIG. 1 .

Referring to FIG. 1 , the self-propagated text validation apparatus according to the present embodiment includes a text obtaining unit 100 , a preprocessing unit 200 , a dictionary-based determining unit 300 , a learning model-based determining unit 400 , and an original text. It may include a base determination unit 500 , a determination result comparison unit 600 , and a label comparison unit 700 .

The text acquisition unit 100 acquires a plurality of self-multiplying texts generated in a self-replicating method based on the labeled learning texts whose ethics or unethics have been verified in advance. Here, the self-propagated text is a plurality of texts generated by a pre-trained self-replicating device with the approval of the training text to generate a large amount of training data using a small number of training texts, similar to the input training text, ethical or unethical is the labeled text. At this time, if a plurality of sentences are included in the text, ethics or unethical may be labeled in units of each sentence, and the self-replicating device generates self-replicating text labeled as ethics from the learning text labeled as unethical according to the learned method. Generate or generate self-replicating text labeled as unethical from training texts labeled as ethics.

That is, the text acquisition unit 100 acquires a plurality of self-multiplying texts generated by the self-proliferating device, and may be implemented as a storage device or a database for storing the plurality of self-proliferating texts generated by the self-proliferating device.

Also, the text acquisition unit 100 may store the original text learning text used to generate the self-multiplying text together with the self-multiplying text.

The pre-processing unit 200 receives the self-propagated text obtained from the text obtaining unit 100 and performs a predetermined pre-processing operation. In this case, the pre-processing unit 200 may be configured to receive the label together with the self-proliferated text, but may be configured to receive only the text excluding the label.

The preprocessor 200 adds special characters, URLs, and SNS-specified characteristic characters (# hash tag, @ mention), etc., which are the remaining components in addition to the sentence components constituting the sentence, such as characters, spaces, punctuation marks, etc. in the self-multiplying text Remove all components. This is to exclude additional components other than the sentence components constituting the sentence, such as characters, punctuation marks, and spaces, from detection because it is unlikely that they will be used as profane or unethical words.

However, the self-proliferation text generated by the self-proliferation device may be generated so that additional components are not included, and in this case, the process of removing all of the additional components may be omitted.

And the pre-processing unit 200 divides each sentence included in the text from which the additional components are removed, and delivers it to each of the dictionary-based determining unit 300 , the learning model-based determining unit 400 , and the original text-based determining unit 500 . do.

The dictionary-based discriminating unit 300, the learning model-based discriminating unit 400, and the text-based discriminating unit 500 each determine the ethics or unethics of self-replicated texts in different designated methods.

First, the dictionary-based determining unit 300 analyzes whether or not profanity is included in the sentence applied by the pre-processing unit 200 in consideration of the fact that a sentence containing profanity is highly likely to be an hate sentence or an unethical sentence to be ethical or unethical. Determine whether it is ethical or not.

The dictionary-based determining unit 300 may analyze whether profanity is included in a sentence by using, for example, a profanity dictionary. Here, the profanity dictionary can be obtained by using an already published profanity dictionary or prepared in advance. have. The profanity dictionary has already been written and published, and the method of generating the profanity dictionary is also a known technique, so it will not be described in detail here.

At this time, the dictionary-based determining unit 300 does not search only profanity words that completely match so as to respond to profane words that are modified in various ways, but N-grams (N-grams) for each word of the profanity registered in the profanity dictionary and the approved sentence. ) by performing a similarity analysis, it is possible to determine whether profanity is included in each sentence. For example, the dictionary-based determining unit 300 compares each word listed in the profanity dictionary and the words included in the approved sentence to determine whether to correspond based on the number of matching characters, and according to the determination result, the corresponding sentence is unethical. You can determine whether it is a sentence or not.

The learning model-based determining unit 400 embeds and vectorizes each word of the applied sentence, receives the vectorized word to extract sentence features, and determines whether the sentence is an unethical sentence based on the extracted sentence feature vector to determine

Referring to FIG. 2 , the learning model-based determining unit 400 may include a vector converting unit 410 , a sentence feature extracting unit 420 , and a feature classifying unit 430 .

The vector conversion unit 410 obtains a plurality of word vectors by embedding and vectorizing each word included in the applied sentence. The vector conversion unit 410 may convert each word in a sentence into a word vector by using a pre-learned embedding model. The vector conversion unit 410 may convert a word into a word vector by using an embedding model published to convert a word into a word vector, such as Word2Vec or fastText.

The sentence feature extracting unit 420 obtains a sentence feature vector by receiving a word vector from the vector converting unit 410 and accumulating and extracting features of the applied word vector.

The sentence feature extractor 420 may be implemented as an artificial neural network in which a pattern estimation method has been previously learned, and in particular, may be implemented as a Long Short Term Memory (LSTM). The LSTM is a neural network with an improved structure so that the Recurrent Neural Network (RNN) can reflect the long-term characteristics, and the characteristics of the previously extracted word vectors are accumulated and reflected in the word vectors that are input later, thereby providing sentence characteristics. It has the advantage of being easy to obtain.

In addition, the feature classifying unit 430 receives the sentence feature vector obtained from the sentence feature extracting unit 420, classifies the sentence feature vector according to a pre-learned pattern classification method, and determines ethics or unethics. The feature classifier 430 may be implemented as a fully connected layer of an artificial neural network and may discriminate ethics or unethics by binary classification of sentence feature vectors.

On the other hand, the original text-based determining unit 500 receives a sentence from the preprocessor 200, compares the approved sentence with the original learning text used to generate the self-multiplying text in the self-proliferation apparatus, and finds the most similar original learning text. Explore. And according to the label of the searched original learning text, the sentence is judged as ethical or unethical. Here, the text-based determining unit 500 may also perform N-gram similarity analysis to determine the most similar original text learning text.

The determination result comparison unit 600 is applied based on the result of each of the dictionary-based determination unit 300, the learning model-based determination unit 400, and the text-based determination unit 500 determining whether the self-multiplying text is unethical. Finally, the ethical or unethical of the self-propagated text is determined.

Here, the determination result comparison unit 600 is simply based on the determination result of each of the dictionary-based determination unit 300, the learning model-based determination unit 400, and the text-based determination unit 500, based on the majority rule based on the ethics of self-propagated text. Or you can identify unethical.

However, in some cases, the determination result comparison unit 600 weights different weights to the respective determination results of the dictionary-based determination unit 300, the learning model-based determination unit 400, and the text-based determination unit 500 to self-multiply text. It is also possible to determine the ethical or unethical of That is, different weights may be weighted to the respective determination results of the dictionary-based determination unit 300 , the learning model-based determination unit 400 , and the text-based determination unit 500 according to the importance of the predetermined determination result.

For example, in the case of the dictionary-based determination unit 300, since the ethics or unethics of the self-multiplying text are determined based on the already verified profanity dictionary, the learning model-based determination unit 400 or the original text-based determination unit 500 After weighting a higher weight compared to that, in the determination result of the weighted dictionary-based determining unit 300, the learning model-based determining unit 400, and the text-based determining unit 500, the higher weight of ethical or unethical is You can choose the given result.

On the other hand, the label comparison unit 700 compares the result finally determined by the determination result comparison unit 600 with the label of the corresponding self-multiplying text stored in the text obtaining unit 100 and, if the same, the label of the self-multiplying text is valid. If it is not the same, it is judged as invalid.

The label comparator 700 may calculate the reliability of the self-multiplying text generated by the self-multiplying apparatus by accumulating determination results for labels of a plurality of self-multiplying texts. As an example, the reliability of the self-multiplying text may be calculated as a ratio of the self-multiplying text determined to be valid in the entire self-multiplying text.

3 shows a self-propagated text validation method according to an embodiment of the present invention.

1 and 2, the self-propagated text validation method of FIG. 3 will be described. First, a plurality of self-propagated texts generated in a self-proliferating manner based on labeled learning texts with ethical or unethical validation in advance. to obtain (S11). Here, the label can be labeled on a per-sentence basis within the self-propagating text.

Then, a predetermined pre-processing operation such as removing additional components and dividing the obtained self-multiplying text into sentence units is performed (S12).

When the self-replicating text is preprocessed, the ethics or unethical of the self-replicating text is determined in a different specified manner for the preprocessed self-replicating text.

First, use a dictionary of profanity to determine ethics or unethics. In the case of using the profanity dictionary, first, a word similar to the profanity described in the profanity dictionary at a predetermined level or higher is searched for in each sentence of the self-multiplying text ( S13 ). And based on the search result, it determines the ethics or unethics of the sentence (S14). That is, if a word determined to be similar to profanity is searched for, it is determined as unethical, and if it is not searched, it can be determined as ethical.

On the other hand, based on the pre-trained learning model, the ethics or unethics of self-replicating texts are discriminated.

To this end, first, a plurality of word vectors are obtained by vectorizing the words included in each sentence of the self-multiplying text using a pre-learned embedding model (S15). Then, a plurality of obtained word vectors are input to an artificial neural network in which a pattern estimation method has been previously learned to obtain a sentence feature vector representing the characteristics of the sentence (S16). Here, the artificial neural network may be implemented as an LSTM.

When the sentence feature vector is obtained, the sentence feature vector is classified according to the pre-learned pattern classification method to determine ethics or unethical behavior (S17).

Also, by using the original learning text used to generate the self-replicating text, the ethics or unethical of the self-replicating text is determined.

That is, the original learning text most similar to the authorized self-multiplying text among the original learning texts is searched (S18). When the most similar original learning text is searched, the sentence is determined as ethical or unethical according to the label of the searched original learning text (S19).

Thereafter, the determination result using the profanity dictionary, the determination result based on the learning model, and the determination result using the original learning text are combined in a predetermined manner to obtain the final determination result of ethics or unethics for the self-replicating text (S20). Here, the final determination result can be determined according to the principle of majority voting, or by assigning a predetermined weight to each determination result, the higher weight among ethical or unethical can be obtained as the final determination result.

When the final determination result is obtained, the obtained final determination result and the label of the self-multiplying text are compared to determine the validity of the self-multiplying text (S21). Then, by accumulating the results of validity determination for a plurality of self-multiplying texts, the reliability of the self-multiplying texts generated by the self-multiplying method is calculated (S22).

The method according to the present invention may be implemented as a computer program stored in a medium for execution by a computer. Here, the computer-readable medium may be any available medium that can be accessed by a computer, and may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, and read dedicated memory), RAM (Random Access Memory), CD (Compact Disk)-ROM, DVD (Digital Video Disk)-ROM, magnetic tape, floppy disk, optical data storage, and the like.

Although the present invention has been described with reference to the embodiment shown in the drawings, which is only exemplary, those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom.

Accordingly, the true technical protection scope of the present invention should be defined by the technical spirit of the appended claims.

100: text acquisition unit 200: pre-processing unit
300: dictionary-based discriminator 400: learning model-based discriminator
410: vector conversion unit 420: sentence feature extraction unit
430: feature classification unit 500: text-based discrimination unit
600: determination result comparison unit 700: label comparison unit

Claims (17)

a text acquisition unit for acquiring a plurality of self-multiplying texts generated in a self-replicating method using the labeled learning texts whose ethics or unethics are verified in advance;
a dictionary-based discriminating unit that receives permission from the self-multiplying text and searches for words similar to the profanity registered in the profanity dictionary acquired in advance from the approved self-proliferated text at a predetermined level or more to discriminate the unethical nature of the self-proliferated text;
a learning model-based discriminating unit that receives permission and vectorizes the self-multiplying text in word units, and extracts sentence feature vectors from the vectorized words according to a pre-learned pattern estimation method to determine the unethics of the self-multiplying text;
a text-based determination unit that searches for a learning text most similar to the self-multiplying text, and determines the unethicality of the self-multiplying text according to a label of the searched learning text; and
A determination result of obtaining a final determination result for the self-multiplying text by combining the results of determining the unethical of the self-multiplying text determined by each of the dictionary-based discriminating unit, the learning model-based discriminating unit, and the original text-based discriminating unit including a comparator,
each of the generated self-replicating texts is labeled as ethical or unethical;
A label comparison unit that compares the label labeled as ethical or unethical when generating the self-multiplying text and the final determination result and determines that the label of the self-multiplying text is valid, and determines that the label of the self-multiplying text is not valid if it is not the same Further comprising self-propagated text validation device.
The method of claim 1, wherein the dictionary-based determining unit
An N-gram similarity analysis is performed on the profanity registered in the dictionary of profanity and each word of the self-multiplying text, and it is determined whether or not profanity is included in the self-multiplying text, and when it is determined that profanity is included, the self A self-propagated text validation device that discriminates the propagated text as unethical. The method of claim 1, wherein the learning model-based determining unit
a vector converter for obtaining a plurality of word vectors by embedding and vectorizing each word of the self-multiplying text;
a sentence feature extractor configured to acquire the sentence feature vector by accumulating and extracting features of the plurality of word vectors according to a pre-learned pattern estimation method; and
Validation apparatus for self-propagated text, comprising: a feature classifier configured to classify the sentence feature vector according to a pre-learned pattern classification scheme to discriminate unethical behavior of the self-multiplying text. The method of claim 3, wherein the sentence feature extraction unit
A self-propagated text validation device implemented with LSTM (Long Short Term Memory). The method of claim 1, wherein the determination result comparison unit
Self-propagated text that obtains the final determination result by applying the majority rule to the result of determining the unethical of the self-multiplying text determined by each of the dictionary-based discriminating unit, the learning model-based discriminating unit, and the original text-based discriminating unit 's validation device. The method of claim 1, wherein the determination result comparison unit
Allocating different predetermined weights to the results of determining the unethics of the self-multiplying texts determined by each of the dictionary-based determining unit, the learning model-based determining unit, and the text-based determining unit, and according to the assigned weights A self-propagated text validation device that obtains a result assigned with a higher weight among ethical or unethical as the final determination result. delete The method of claim 1, wherein the label comparison unit
A validation device for self-multiplying texts that calculates the reliability of self-multiplying texts according to the validation results for labels of a plurality of self-multiplying texts. According to claim 1, wherein the self-propagated text validation device is
Further comprising a preprocessing unit that removes additional components from the self-multiplying text obtained in the text acquisition unit, divides them into sentence units, and transmits them to each of the dictionary-based determining unit, the learning model-based determining unit, and the original text-based determining unit Self-propagated text validation device. A method for validating self-propagated text, which is performed in an apparatus for validating self-propagated text, the method comprising:
A self-multiplying text acquisition step of acquiring a plurality of self-replicating texts generated by a self-replicating method using the text for learning that is ethical or unethical and labeled in advance;
A dictionary-based determination step of determining the unethics of the self-multiplying text by searching for words similar to the profanity registered in the profanity dictionary acquired in advance at a predetermined level or more in the self-multiplying text;
The self-propagated text is received and vectorized in word units, a sentence feature vector is extracted from the vectorized word using a learning model in which a pattern estimation method has been previously learned, and the unethical nature of the self-propagated text is determined based on the extracted sentence characteristics. a learning model-based determination step to determine;
a text-based determination step of searching for a learning text most similar to the self-multiplying text, and discriminating the unethicality of the self-multiplying text according to a label of the searched learning text; and
Final determination to obtain a final determination result for the self-multiplying text by combining the results of the determination of the unethical of the self-propagated text determined in each of the dictionary-based determination step, the learning model-based determination step, and the text-based determination step comprising steps,
each of the generated self-replicating texts is labeled as ethical or unethical;
A label comparison step of comparing the label labeled as ethical or unethical when generating the self-multiplying text with the final determination result and determining that the label of the self-multiplying text is valid if it is the same, and determining that the label of the self-multiplying text is invalid if not identical Validation method of self-propagated text further comprising a.
11. The method of claim 10, wherein the dictionary-based determining step
performing N-gram similarity analysis on the profanity registered in the dictionary of profanity and each word of the self-multiplying text, and determining whether the self-multiplying text includes the profanity; and
If it is determined that profanity is included, a method for validating the self-propagated text comprising the step of determining the self-proliferated text as unethical. The method of claim 10, wherein the learning model-based determination step
obtaining a plurality of word vectors by embedding and vectorizing each word of the self-multiplying text;
obtaining the sentence feature vector by accumulating and extracting features of the plurality of word vectors using a learning model in which a pattern estimation method has been previously learned; and
Classifying the sentence feature vector according to a pre-learned pattern classification method, the method of validating the self-propagated text comprising the step of discriminating the unethical of the self-propagated text. 13. The method of claim 12, wherein the learning model is
Validation method of self-propagated text implemented with LSTM (Long Short Term Memory). 11. The method of claim 10, wherein the final determination step
Self-propagated text to obtain the final determination result by applying the majority rule to the determination result of the unethical self-multiplying text determined in each of the dictionary-based determination step, the learning model-based determination step, and the text-based determination step of validation methods. 11. The method of claim 10, wherein the final determination step
allocating different predetermined weights to the results of determining the unethics of the self-multiplying text determined in each of the dictionary-based determination step, the learning model-based determination step, and the text-based determination step; and
A method for validating self-propagated texts in which a result to which a higher weight of ethical or unethical is assigned according to the assigned weight is obtained as the final determination result. delete 11. The method of claim 10, wherein the validation method of the self-propagated text is
The validation method of the self-multiplied text further comprising a reliability calculation step of calculating the reliability of the self-multiplying text according to a result of the validation of a plurality of self-multiplying text labels.

Images