KR20210112955A - Swearwords detection system based on hangul jamo similarity and method of detecting the swearwords - Google Patents

Abstract

An expletive detection system includes a preprocessing unit, an initial, middle, and final consonants separation unit, an initial consonant distance calculation unit, a middle consonant calculation unit, a final consonant distance calculation unit, and an expletive determination unit. The preprocessing unit separates an input phrase into words. The initial, middle, and final consonants separation unit separates syllables in the word of the input phrase into initial, middle, and final consonants. The initial consonant distance calculation unit calculates an initial consonant distance by comparing the initial consonant of the syllable of the input phrase with the initial consonant of the syllable of an expletive sample. The middle consonant distance calculation unit calculates a middle consonant distance by comparing the middle consonant of the syllable of the input phrase with the middle consonant of the syllable of the expletive sample. The final consonant distance calculation unit calculates a final consonant distance by comparing the final consonant of the syllable of the input phrase with the final consonant of the syllable of the expletive sample. The expletive determination unit compares the average distance of the initial consonant distance, the middle consonant distance, and the final consonant distance with a threshold to determine whether the input phrase includes an expletive. Therefore, it is possible to greatly improve the accuracy of expletive detection.

Description

System and method for detecting profanity based on similarity between Korean alphabets

The present invention relates to a system and method for detecting profanity based on the similarity between Korean alphabets, and more specifically, to divide an input phrase into word words, and divide a syllable within the word into a leading consonant, a middle vowel, and a final consonant to compare with a sample of profanity. To a system and method for detecting profanity based on

Many celebrities are suffering a lot from reckless malicious comments. Recently, some celebrities have taken their own lives, unable to overcome the depression caused by malicious comments and rumors. In addition, various people, including non-celebrities, influencers, and the general public, are becoming victims of malicious comments. This is a situation in which access to the Internet is improved and it is increasingly becoming a serious social problem.

Accordingly, various community sites use a profanity filtering system, but if the profanity does not completely match, filtering is not possible in most cases. Users use this weakness to bypass the existing profanity filtering program, replacing the content they are trying to convey with similar letters and using modified profanity. There is a problem that profanity is not filtered despite the fact that such modified profanity is included. Therefore, the operator suffers from the inconvenience of having to directly read and filter the reported sentences.

It is an object of the present invention to provide a profanity detection system based on the similarity between Hangul letters and letters, in which an input phrase is divided into word words, and syllables within the word word are divided into a leading consonant, a middle consonant, and a final consonant to improve accuracy by comparing with profanity samples.

Another object of the present invention is to provide a method for detecting profanity using a profanity detection system based on the similarity between Korean alphabets.

A profanity detection system according to an embodiment for realizing the object of the present invention includes a preprocessing unit, a leading consonant, a neutral, a final consonant separation unit, a leading consonant distance calculation unit, a neutral distance calculation unit, a final consonant distance calculation unit and a profanity determination unit . The preprocessor separates the input phrase into words. The leading, neutral, and final separating unit separates syllables in the word of the input phrase into a leading, neutral, and final consonant. The leading consonant distance calculating unit calculates a leading consonant distance by comparing the beginning of the syllable of the input phrase with the beginning of the syllable of the profanity sample. The neutral distance calculator calculates a neutral distance by comparing the neutrality of the syllable of the input phrase with the neutrality of the syllable of the profanity sample. The closing distance calculating unit calculates a closing distance by comparing the ending of the syllable of the input phrase with the ending of the syllable of the profanity sample. The profanity determining unit compares the average distance of the initial distance, the neutral distance, and the final distance with a threshold to determine whether the input phrase includes a profanity.

In an embodiment of the present invention, the preprocessor may remove alphabetic characters, numbers, and special characters from the word word of the input phrase.

In one embodiment of the present invention, when the number of syllables of the word syllable of the input phrase is greater than the number of syllables of the profanity sample, the n-gram setting for separating the word word of the input phrase to match the number of syllables of the profanity sample It may include more wealth.

In an embodiment of the present invention, the starting distance calculating unit may calculate the starting distance by using a leading similarity matrix including information on the similarity between the leading consonants of Hangul. The leading consonant similarity matrix may have at least three values according to the similarity between the leading consonants.

In one embodiment of the present invention, when the leading consonant of the syllable of the input phrase matches the leading consonant of the syllable of the profanity sample, the leading consonant similarity matrix may have a value of zero. When the leading consonant of the syllable of the input phrase and the leading consonant of the syllable of the profanity sample do not match, the leading consonant similarity matrix may have a value greater than or equal to 0 and less than or equal to 1 according to the similarity between the leading consonants. . As the similarity between the leading consonants is lower, the leading consonant similarity matrix may have a value close to 1.

In an embodiment of the present invention, the neutral distance calculator may calculate the neutral distance using a neutral similarity matrix including information on the similarity between neutrals in Korean. The neutral similarity matrix may have at least three values according to the similarity between the neutrals.

In an embodiment of the present invention, when the neutrality of the syllable of the input phrase and the neutrality of the syllable of the profanity sample match, the neutral similarity matrix may have a value of 0. If the neutrality of the syllable of the input phrase and the neutrality of the syllable of the profanity sample do not match, the neutral similarity matrix may have a value greater than or equal to 0 and less than or equal to 1 according to the similarity between the neutrals. . As the similarity between the neutrals is lower, the neutral similarity matrix may have a value close to 1.

In an embodiment of the present invention, the final consonant distance calculator may calculate the final consonant distance by using a final consonant similarity matrix including information on the similarity between the consonants of Hangul. The species similarity matrix may have at least three or more values according to the similarity between the species.

In an embodiment of the present invention, when the finality of the syllable of the input phrase matches the finality of the syllable of the profanity sample, the longitudinal similarity matrix may have a value of zero. If the finality of the syllable of the input phrase and the finality of the syllable of the profanity sample do not match, the finality similarity matrix may have a value greater than or equal to 0 and less than or equal to 1 according to the similarity between the finalities. . As the similarity between the species is lower, the matrix similarity of the species may have a value close to 1.

In one embodiment of the present invention, it may further include a threshold setting unit for setting the threshold. The threshold setting unit sets a value that maximizes recall of the class 1 as the threshold when the class including the profanity is class 0 and the class not including the profanity is class 1 can

In an embodiment of the present invention, when the input phrase includes the profanity, the profanity determining unit K-NN (K-Nearest) to determine which profanity sample the input phrase corresponds to among a plurality of profanity samples Neighbors) method can be used. Here, K = 1 may be.

According to an embodiment of the present invention for realizing another object of the present invention, a method for detecting profanity includes dividing an input phrase into word words, separating syllables within the word of the input phrase into a leading consonant, a middle consonant, and a final consonant, and the input Comparing the leading consonant of the syllable of the phrase to the leading of the syllable of a profanity sample, calculating a consonant distance by comparing the syllable neutral of the input phrase with the syllable neutral of the profanity sample; calculating a closing distance by comparing the ending of the syllable of the input phrase with the ending of the syllable of the profanity sample; and determining whether the phrase contains profanity.

In one embodiment of the present invention, when the number of syllables of the word syllable of the input phrase is greater than the number of syllables of the profanity sample, the method further includes separating the word word of the input phrase to match the number of syllables of the profanity sample can do.

According to the system and method for detecting profanity based on the similarity between Hangeul letters and letters according to the present invention, an input phrase is divided into word words, syllables within the word word are divided into a leading consonant, a middle consonant, and a final consonant, and a consonant distance reflecting the pronunciation similarity of Hangul, a consonant distance, and a Profanity can be detected using the finality distance.

Accordingly, not only the profanity matching the profanity sample, but also the profanity modified from the profanity sample can be detected, thereby greatly improving the accuracy of profanity detection.

1 is a block diagram illustrating a system for detecting profanity according to an embodiment of the present invention.
FIG. 2 is a conceptual diagram illustrating an operation of the preprocessor of FIG. 1 .
FIG. 3 is a diagram illustrating a leading similarity matrix used in the starting distance calculating unit of FIG. 1 .
4 is a diagram illustrating a neutral similarity matrix used in the neutral distance calculator of FIG. 2 .
5 is a diagram illustrating a longitudinal similarity matrix used in the longitudinal distance calculating unit of FIG. 2 .
6 is a table showing an example of profanity detection accuracy according to a threshold set in the threshold setting unit of FIG. 1 .
7 is a table showing an example of profanity detection accuracy according to a threshold set in the threshold setting unit of FIG. 1 .

With respect to the embodiments of the present invention disclosed in the text, specific structural or functional descriptions are only exemplified for the purpose of describing the embodiments of the present invention, and the embodiments of the present invention may be embodied in various forms. It should not be construed as being limited to the embodiments described in .

Since the present invention can have various changes and can have various forms, specific embodiments are illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to the specific disclosed form, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms may be used for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle. Other expressions describing the relationship between elements, such as "between" and "immediately between" or "neighboring to" and "directly adjacent to", should be interpreted similarly.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof is present, and includes one or more other features or numbers, It should be understood that the possibility of the presence or addition of steps, operations, components, parts or combinations thereof is not precluded in advance.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as meanings consistent with the context of the related art, and unless explicitly defined in the present application, they are not to be interpreted in an ideal or excessively formal meaning. .

On the other hand, when a certain embodiment can be implemented differently, functions or operations specified in a specific block may occur in a different order from that specified in the flowchart. For example, two consecutive blocks may be performed substantially simultaneously, or the blocks may be performed in reverse according to a related function or operation.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

1 is a block diagram illustrating a system for detecting profanity according to an embodiment of the present invention. FIG. 2 is a conceptual diagram illustrating an operation of the preprocessor 100 of FIG. 1 .

1 and 2, the profanity detection system based on the similarity between the Korean alphabets includes a preprocessor 100, a leading consonant, a neutral, and a final consonant separating unit 300, a starting distance calculating unit 400, and a neutral distance calculating unit 500. ), a final distance calculation unit 600 and a profanity determination unit 800 . For example, the profanity detection system may further include an n-gram setting unit 200 . The profanity detection system may further include a threshold setting unit 700 .

The profanity detection system may be implemented through electronic devices and software applications such as mobile phones, smart phones, tablets, notebook computers, and desktops.

The profanity detection system compares the input phrase with a plurality of samples of profanity stored in a database to determine whether the input phrase includes profanity or not.

The preprocessor 100 separates the input phrase into words. Referring to FIG. 2 , the input phrase is “Thinking and living or a fool”. For example, the profanity samples include "fuck", "fuck" and "assassin."

The pre-processing unit 100 may separate the input phrase, “thinking and living, or not” into “thinking”, “doing”, “living”, and “bysin”, respectively.

Words of the input phrase may be compared with the profanity samples, respectively. The profanity detection system does not judge "thinking" as profane because "thinking" does not match and is not similar to the profanity samples. The profanity detection system does not judge "do" as a profanity because "and" does not match and is not similar to the profanity samples. The profanity detection system does not judge "living" as a profanity because "I'm living" does not match and is not similar to the profanity samples. On the other hand, since "Bongsin" matches one of the profanity samples, the profanity detection system determines "Byongsin" as a profanity. In this way, the preprocessor 100 may separate the input phrase into each word.

The preprocessor 100 may remove English letters, numbers, and special characters from the word word of the input phrase. For example, when "Byung 22222 Sin" is input, "Byung lll Sin" is input, or "Byeong!!! Sin" is input, the preprocessor 100 is an English letter in the word of the input phrase, Since numbers and special characters are removed, the accuracy of detecting the profanity can be improved by converting all of the "Byung22222Shin", "ByungllllShin", and "Byeong!!!Sin" into "Byeongsin".

When the number of syllables of the word of the input phrase is greater than the number of syllables of the profanity sample, the n-gram setting unit 200 may separate the word of the input phrase to match the number of syllables of the profanity sample.

For example, when the input phrase is "fuck you guys", if you compare the input phrase "fuck guys" with "fuck you" of the profanity sample, the similarity is calculated to be low due to the difference in word length, so the input phrase It can be judged that it does not contain this profane language.

Accordingly, the n-gram setting unit 200 may set the n-gram to be 2, which is the number of syllables of the profanity sample, since the number of syllables of “fuck” as the profanity sample is 2. When the n-gram is set to 2, the input phrase “fuck you guys” may be divided into subsets of “fuck”, “bare guys”, “gnomes”, “theys”, and “yes”, respectively.

The profanity detection system compares the subsets (“Fuck”, “Balnom”, “Norm”, “Eyes”, “Ine”) separated by the n-gram setting unit 200 with the profanity samples, respectively. Therefore, it can be determined that the input phrase (“these fuckers”) includes profanity.

In this embodiment, the profanity detection system separates the syllables in the word of the input phrase into a leading consonant, a neutral and a final consonant, and separates the syllables of the profanity sample into a leading consonant, a neutral and a final consonant, Neutrals and finalities are compared with each other.

The leading, neutral, and final consonant separating unit 300 separates syllables in the word of the input phrase into leading, neutral, and final consonants.

The leading consonant distance calculating unit 400 compares the leading consonant of the syllable of the input phrase with the consonant of the syllable of the profanity sample to calculate the leading consonant distance.

The neutral distance calculator 500 calculates a neutral distance by comparing the neutrality of the syllable of the input phrase with the neutrality of the syllable of the profanity sample.

The final distance calculating unit 600 calculates a final distance by comparing the final syllable of the input phrase with the final syllable of the profanity sample.

The profanity determining unit 800 compares the average distance of the starting distance, the neutral distance, and the final distance with a threshold to determine whether the input phrase includes a profanity.

FIG. 3 is a diagram illustrating a leading similarity matrix used in the leading distance calculating unit 400 of FIG. 1 .

Referring to FIG. 3 , the leading consonant distance calculator 400 may calculate the leading consonant distance using a consonant similarity matrix including information on the similarity between consonants of Hangul.

The leading consonant similarity matrix may have at least three values according to the similarity between the leading consonants. For example, when the leading consonant of the syllable of the input phrase matches the leading consonant of the syllable of the profanity sample, the leading consonant similarity matrix may have a value of zero. When the leading consonant of the syllable of the input phrase and the leading consonant of the syllable of the profanity sample do not match, the leading consonant similarity matrix may have a value greater than or equal to 0 and less than or equal to 1 according to the similarity between the leading consonants. . As the similarity between the leading consonants is lower, the leading consonant similarity matrix may be set to have a value close to 1.

For example, in FIG. 3, "a" is the same as "a", and thus the value of "a"-"a" has 0. In view of the purpose and pronunciation of profanity, "ㄲ" is very similar to "a", so the value of "a"-"ㄲ" is set to 0. In terms of the purpose and pronunciation for transforming profanity, "ㅋ" is very similar to "ㄱ", so the value of "ㄱ"-"ㅋ" is set to 0. On the other hand, "a", "b", "c", "ㄹ", and "ㅁ" have very low similarity to each other, so "a"-"b", "a"-"c", "a"- The values of "ㄹ" and "a"-"ㅁ" were set to 1.

For example, in FIG. 3, "s" is the same as "s", and thus the value of "s" - "s" has 0. In terms of the purpose and pronunciation of the profanity, "ㅆ" is very similar to "ㅅ", so the value of "ㅅ"-"ㅆ" is set to 0. In terms of the purpose of transforming profanity and pronunciation, "s" is somewhat similar to "h", so the value of "s"-"h" is set to 0.3. On the other hand, "ㅅ" and "a", "b", "c", "ㄹ", etc. have very low similarity to each other. The values of "c", "s"-"d" were set to 1.

4 is a diagram illustrating a neutral similarity matrix used in the neutral distance calculator 500 of FIG. 1 .

Referring to FIG. 4 , the neutral distance calculator 500 may calculate the neutral distance by using a neutral similarity matrix including information on the similarity between neutrals in Korean.

The neutral similarity matrix may have at least three values according to the similarity between the neutrals. For example, if the neutrality of the syllable of the input phrase matches the neutrality of the syllable of the profanity sample, the neutral similarity matrix may have a value of zero. If the neutrality of the syllable of the input phrase and the neutrality of the syllable of the profanity sample do not match, the neutral similarity matrix may have a value greater than or equal to 0 and less than or equal to 1 according to the similarity between the neutrals. . As the similarity between the neutrals is lower, the neutral similarity matrix may be set to have a value close to 1.

For example, in FIG. 4 , "a" is the same as "a", and thus the value of "a"-"a" has 0. In view of the purpose and pronunciation of profanity for transforming profanity, "a" is very similar to "ㅑ", so the value of "a"-"ㅑ" is set to 0.1. In terms of the purpose and pronunciation of the profanity, "a" is slightly similar to "ㅓ", so the value of "a"-"ㅓ" was set to 0.7. On the other hand, "a", "ㅗ", "TT", "ㅡ", and "ㅣ" have very low similarity to each other, so "a"-"ㅗ", "a"-"TT", "a"- The values of "ㅡ", "A"-"ㅣ" were set to 1.

FIG. 5 is a diagram illustrating a longitudinal similarity matrix used in the longitudinal distance calculating unit 600 of FIG. 1 .

Referring to FIG. 5 , the final consonant distance calculator 600 may calculate the final consonant distance by using a final consonant similarity matrix including information on the similarity between the final consonants of Hangul.

The species similarity matrix may have at least three or more values according to the similarity between the species. For example, if the finality of the syllable of the input phrase matches the finality of the syllable of the profanity sample, the finality similarity matrix may have a value of zero. If the finality of the syllable of the input phrase and the finality of the syllable of the profanity sample do not match, the finality similarity matrix may have a value greater than or equal to 0 and less than or equal to 1 according to the similarity between the finalities. . As the similarity between the species is lower, the matrix similarity may be set to have a value close to 1.

The final consonant of FIG. 5 is similar to the initial consonant in that it is a consonant, but is significantly different from the initial consonant in terms of pronunciation.

For example, in FIG. 5, "c" is the same as "c", and thus the value of "c"-"c" has 0. In terms of the purpose and pronunciation for transforming profanity, "c" is somewhat similar to "ㅅ", so the value of "c"-"ㅅ" is set to 0.3. In terms of the purpose and pronunciation of the profanity, "c" is somewhat similar to "t", so the value of "c"-"t" was set to 0.3. On the other hand, "c" and "a", "b", "ㄹ", and "ㅁ" have very low similarity to each other, so "c"-"a", "c"-"b", "c"- The value of "d", "c"-"ㅁ" was set to 1.

For example, the longitudinal similarity matrix of FIG. 5 may include information (“none”) for a case in which there is no support. For example, the longitudinal similarity matrix of FIG. 5 may include information on the overlapping of ㄺ, ㄻ, ㄼ, ㄽ, ㅄ, and the like.

As described above, the profanity determining unit 800 may determine whether the input phrase includes a profanity by comparing the average distance of the initial consonant distance, the neutral distance, and the final consonant distance with a threshold.

For example, if the word of the input phrase is "geek" and the profanity sample is "fuck", the first syllable leading distance of "a" and "ㅆ" is 1, and the first syllable neutral of "ㅣ" and "ㅣ" The distance is 0, the closing distance of the first syllable of "none" and "none" is 0, the starting distance of the second syllable of "b" and "b" is 0, and the second syllable neutral distance of "a" and "a" is 0 , and the final distance between the second syllable of "d" and "d" is 0. Since the average distance between the input phrase and the profanity sample is 1/6, it is 0.167. If the threshold is 0.1, "geeky" is not considered profanity.

For example, if the word of the input phrase is "?first?" and the profanity sample is "fuck", the first syllable leading distance of "ㅅ" and "ㅆ" is 0, and the first syllable distance of "ㅢ" and "ㅣ" is The syllable neutral distance is 0.1, the first syllable ending distance of "none" and "none" is 0, the second syllable neutral distance of "b" and "b" is 0, and the second syllable neutral of "a" and "a" The distance is 0, and the final distance of the second syllable of "d" and "d" is 0. Since the average distance between the input phrase and the profanity sample is 0.1/6, it becomes 0.0167. If the threshold is 0.1, "?first?" is judged as profane.

When the input phrase includes the profanity, the profanity determining unit 800 determines which profanity sample among a plurality of profanity samples corresponds to a K-Nearest Neighbors (K-NN) method (K). nearest neighbor method) can be used.

Here, when K is designated as a large number, K=1 may be set because it can be classified as another profanity even though there is a closest profanity. That is, the profanity determiner 800 may classify the profanity using the 1-NN method.

6 is a table showing an example of profanity detection accuracy according to a threshold set in the threshold setting unit of FIG. 1 . 7 is a table showing an example of profanity detection accuracy according to a threshold set in the threshold setting unit of FIG. 1 .

1 to 7 , if the average distance between the word of the input phrase and the initial, middle, and final consonant of the sample of profanity is less than or equal to the threshold, it is determined that the word of the input phrase includes the profanity can be Conversely, if the average distance between the word of the input phrase and the leading, neutral, and final consonant of the profanity sample is greater than the threshold, it may be determined that the word of the input phrase does not include the profanity.

When comparing words with words, the distance can be increased by one when additions, deletions, or substitutions occur. In this embodiment, since each of the initial, neutral, and final is compared, if any one of the initial, neutral, and final is converted into a dissimilar one, the distance is 1/3. However, as shown in the initial consonant similarity matrix, the neutral similarity matrix, and the final consonant similarity matrix, if any one of the initial consonant, neutral, and final consonant is converted to a relatively similar one, the value in the matrix has a value less than 1, so the distance It can be less than 1/3.

For example, the threshold may be set to be less than 0.5. 6 shows accuracy, precision, and recall when the thresholds are 0.2, 0.3, 0.4, and 0.5. The data set used in FIG. 6 may include 747 input phrases (Class 1) including 709 input phrases without profanity (Class 0) including profanity. The proportion of profanity in the data set is 51.3%, and it is a data set generated with a significantly higher rate of profanity comments compared to general comments.

Accuracy varies depending on the threshold, but the average was 79% and the maximum was 80.15%. The profanity detection system of this embodiment showed significantly better accuracy than the 48% accuracy of the conventional system that simply filters only the list of profanity in the database.

However, even if it is erroneously determined that a sentence without profanity contains profanity, since detection of profanity is more important, the threshold setting unit 700 determines that the class including the profane word is Class 0 and the class does not include the profane word. When is Class 1, a value that maximizes the recall of Class 1 may be set as the threshold. In FIG. 6 , the class 1 recall rate is 0.89, and 0.4, which is the maximum, may be set as the threshold.

7 shows the accuracy, precision, and recall ( recall) is shown.

In FIG. 7, the precision of Class 0 was 96%, and the recall of Class 1 was 93%. That is, the profanity detection system of the present embodiment detects most of the actual profane words, and shows a performance of only 4% of the misclassified general words.

Here, a case where the correct answer is True and the system predicts True, b when the correct answer is True and the system predicts False, b when the correct answer is False and the system predicts False, and if the correct answer is False, the system Assuming that false prediction is d, the precision is a/(a+c), the recall is a/(a+b), and the accuracy is (a+) d)/(a+b+c+d).

According to this embodiment, it is possible to divide an input phrase into word words, divide syllables within the word into a leading consonant, a middle consonant, and a final consonant, and detect profanity using the consonant distance, the middle distance, and the final consonant distance reflecting the pronunciation similarity of Hangul.

Accordingly, not only the profanity matching the profanity sample, but also the profanity modified from the profanity sample can be detected, thereby greatly improving the accuracy of profanity detection.

The present invention can be applied to electronic devices such as mobile phones, smart phones, tablets, notebook computers, and desktops.

Although the above has been described with reference to the preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention as set forth in the following claims. you will understand that you can

Claims (13)

a preprocessor that separates input phrases into words;
a leading, neutral, and final separating unit for separating syllables in the word of the input phrase into a leading, neutral, and final consonant;
a leading consonant distance calculating unit for calculating a leading consonant distance by comparing a consonant of the syllable of the input phrase with a consonant of a syllable of a profanity sample;
a neutral distance calculator for calculating a neutral distance by comparing the neutrality of the syllable of the input phrase with the neutrality of the syllable of the profanity sample;
a finality distance calculation unit for calculating a finality distance by comparing a finality of the syllable of the input phrase with a finality of the syllable of the profanity sample; and
and a profanity determining unit configured to determine whether the input phrase includes profanity by comparing an average distance of the starting distance, the neutral distance, and the final distance with a threshold. The profanity detection system of claim 1, wherein the preprocessor removes alphabetic characters, numbers, and special characters from the word word of the input phrase. The method of claim 1 , further comprising an n-gram setting unit configured to separate the words of the input phrase to match the number of syllables of the profanity sample when the number of syllables of the word of the input phrase is greater than the number of syllables of the profanity sample. Profanity detection system, characterized in that. The method of claim 1, wherein the starting distance calculating unit calculates the starting distance by using a starting similarity matrix including information on the similarity between the leading vowels of Hangul;
The profanity similarity matrix has at least three values according to the degree of similarity between the initial consonants. 5. The method of claim 4, wherein if the leading consonant of the syllable of the input phrase coincides with the leading consonant of the syllable of the profanity sample, the consonant similarity matrix has a value of 0;
If the leading consonant of the syllable of the input phrase is inconsistent with the leading consonant of the syllable of the profanity sample, the leading consonant similarity matrix has a value greater than or equal to 0 and less than or equal to 1 according to the similarity between the leading consonants,
The profanity detection system, characterized in that, as the similarity between the initial consonants is lower, the initial consonant similarity matrix has a value close to 1. 5. The method of claim 4, wherein the neutral distance calculator calculates the neutral distance by using a neutral similarity matrix including information on the similarity between neutrals in Hangul;
The profanity detection system, characterized in that the neutral similarity matrix has at least three values according to the similarity between the neutrals. 7. The method of claim 6, wherein the neutral similarity matrix has a value of 0 if the neutrality of the syllable of the input phrase and the neutrality of the syllable of the profanity sample match;
If the neutrality of the syllable of the input phrase and the neutrality of the syllable of the profanity sample do not match, the neutral similarity matrix has a value greater than or equal to 0 and less than or equal to 1 according to the similarity between the neutrals,
The profanity detection system of claim 1, wherein the neutral similarity matrix has a value close to 1 as the similarity between the neutrals is lower. The method of claim 6, wherein the final consonant distance calculation unit calculates the final consonant distance using a final consonant similarity matrix including information on the similarity between the final consonants of Hangul;
The profanity similarity matrix has at least three values according to the degree of similarity between the subspecies. 9. The method of claim 8, wherein if the finality of the syllable of the input phrase matches the finality of the syllable of the profanity sample, the longitudinal similarity matrix has a value of 0;
If the finality of the syllable of the input phrase and the finality of the syllable of the profanity sample do not match, the finality similarity matrix has a value greater than or equal to 0 and less than or equal to 1 according to the similarity between the finalities,
The profanity detection system, characterized in that the lower the degree of similarity between the last names, the closer the value of the last word similarity matrix is to 1. According to claim 1, further comprising a threshold setting unit for setting the threshold,
The threshold setting unit sets a value that maximizes recall of the class 1 as the threshold when the class including the profanity is class 0 and the class not including the profanity is class 1 Profanity detection system, characterized in that. The K-Nearest Neighbors (K-NN) method of claim 1, wherein the profanity determining unit determines which profanity sample from among a plurality of profanity samples the input phrase corresponds to when the input phrase includes the profanity. use ,
A profanity detection system, characterized in that K = 1. separating the input phrase into words;
separating the syllables in the word of the input phrase into a leading consonant, a middle consonant, and a final consonant;
comparing the beginning of the syllable of the input phrase with the beginning of the syllable of a profanity sample and calculating a leading distance;
calculating a neutral distance by comparing the neutrality of the syllable of the input phrase with the neutrality of the syllable of the profanity sample;
comparing the ending of the syllable of the input phrase with the ending of the syllable of the profanity sample and calculating a final distance; and
and determining whether the input phrase includes profanity by comparing an average distance of the starting distance, the neutral distance, and the final distance with a threshold. 13. The method of claim 12, further comprising, when the number of syllables of the word of the input phrase is greater than the number of syllables of the profanity sample, separating the word of the input phrase to match the number of syllables of the profanity sample. A method of detecting profanity.

Images