KR102500115B1 - Spam message blocking apparatus and method - Google Patents

Abstract

Exemplary embodiments of the present invention relate to a spam message blocking device and method for improving spam message blocking efficiency by preventing blocking errors that occur when spam message blocking occurs through a learning (deep learning) based method. An apparatus for blocking spam messages according to an aspect includes: a first classification unit for classifying an input message as spam or non-spam through a deep learning-based spam filter learned based on learning data (deep learning); a calculation unit for calculating a similarity between the message classified as spam through the deep learning-based spam filter and the learning data; and a second classification unit for re-classifying the spam-classified message as spam or non-spam based on the calculated similarity.

Description

Spam message blocking apparatus and method {Spam message blocking apparatus and method}

The present invention relates to a spam message blocking technology, and more particularly, to a spam message blocking device and method for improving spam message blocking efficiency by reducing blocking errors that occur when spam message blocking occurs through a learning (deep learning) based method. it's about

Generally, spammers use e-mails or text messages to maliciously achieve their desired goals. The spread of the Internet in the early days mainly caused the problem of spam e-mail, but the recent increase in usage of smartphones has dramatically increased the amount of spam e-mail as well as spam text messages.

In order to block such spam (text) messages, a method of blocking if it matches a registered spam character/phrase based on a spam character data set has been proposed and used. Since the amount is huge and various patterns are continuously added, it is difficult for people to extract and respond to phrases that become spam.

So, recently, the efficiency of spam blocking has been improved by using a learning (deep learning)-based classification model that learns and classifies character patterns and features. This learning-based spam blocking method is When spam probability unexpectedly rises, error detection occurs. 2) Error detection occurs when learning is based on the characteristics of surrounding general words, not spam phrases. Problems such as the occurrence of errors were found due to operational problems.

Registered Patent Publication No. 10-2001375 (2019.07.12.)

The present invention is to solve the above-mentioned conventional problems, the object of which is to improve spam message blocking efficiency by preventing blocking errors that occur when spam message blocking occurs through a learning (deep learning) based method, spam message blocking device and to provide a method.

In order to achieve the above object, an apparatus for blocking spam messages according to an aspect of the present invention includes a device for classifying an input message as spam or non-spam through a deep learning-based spam filter learned based on learning data (deep learning) 1 classification unit; a calculation unit for calculating a similarity between the message classified as spam through the deep learning-based spam filter and the learning data; and a second classifier for re-classifying the spam-classified message as spam or non-spam based on the calculated similarity, and classifies the input message as spam or non-spam based on the pre-registered spam phrase. It may further include a pre-processing unit for classifying, and the non-spam message classified through the pre-processing unit may be provided as an input message to the first classifying unit.

The second classification unit may re-classify the corresponding message classified as spam by the first classification unit into a non-spam message when the similarity value calculated by the calculation unit is less than a predetermined reference value.

The step 2 classification unit may re-classify the message classified as spam by the first classification unit as a spam message when the similarity value calculated by the calculation unit is equal to or greater than a predetermined reference value.

The second classifier classifies the corresponding message classified as spam by the first classifier when similar data in the training data corresponding to the corresponding message classified as spam by the first classifier has contradictory labeling of spam and non-spam. The message can be reclassified as a non-spam message.

The calculator calculates the first average value of the first similarity (hereinafter, the average value of spam similarity) with each of the plurality of spam data in the training data for the message classified as spam through the deep learning-based spam filter and the first average value of the plurality of non-spam data. 2 An average value of similarity (hereinafter referred to as average non-spam similarity value) may be obtained, and a difference between the average similarity value of spam and the average value of non-spam similarity (hereinafter referred to as final score value) may be calculated as the similarity value.

The average similarity value of the spam or non-spam may be an average value of the top n pieces that are equal to or greater than a set minimum value among the calculated plurality of first or second similarities, respectively.

The calculator may calculate similarity through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding-based similarity comparison method, and the search similarity comparison method may include a BM25 algorithm. .

In order to achieve the above object, a spam message blocking method according to an aspect of the present invention includes: (a) classifying an input message as spam or non-spam through a deep learning-based spam filter learned (deep learning) based on learning data; steps to do; (b) calculating a similarity between a message classified as spam through the deep learning-based spam filter and the training data; and (c) re-classifying the spam-classified message as spam or non-spam based on the calculated similarity, and (d) spam the input message based on the pre-registered spam phrase. Alternatively, a pre-processing step for classifying as non-spam may be further included, and the non-spam message classified through the pre-processing step may be provided as the input message of step (a).

In step (c), when the similarity value calculated in step (b) is less than a predetermined reference value, the message classified as spam in step (a) may be reclassified as a non-spam message.

In the step (c), when the similarity value calculated in the step (b) is greater than or equal to a predetermined reference value, the message classified as spam in the step (a) may be reclassified as a spam message.

In the step (c), if the similar data in the learning data corresponding to the message classified as spam in step (a) has contradictory labeling of spam and non-spam, the message classified as spam in step (a) The message can be reclassified as a non-spam message.

In the step (b), for the messages classified as spam through the deep learning-based spam filter, the average value of the first similarity with each of the plurality of spam data in the training data (hereinafter, the average value of the spam similarity) and each of the plurality of non-spam data A second average similarity value (hereinafter referred to as average non-spam similarity value) may be obtained, and a difference between the average similarity value of spam and the average similarity value of non-spam (hereinafter referred to as final score value) may be calculated as the similarity.

The spam or non-spam average similarity value may be calculated as an average value for the top n pieces that are equal to or greater than a set minimum value among the calculated plurality of first or second similarities, respectively.

The step (b) may calculate similarity through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding-based similarity comparison method, and the search similarity comparison method includes a BM25 algorithm can do.

According to another aspect of the present invention in order to achieve the above object, a computer-readable recording medium recording a program for executing the spam message blocking method in a computer may be provided.

According to another aspect of the present invention in order to achieve the above object, an application stored in a computer-readable recording medium may be provided to execute the spam message blocking method in combination with hardware.

According to another aspect of the present invention in order to achieve the above object, a computer program stored in a computer-readable recording medium may be provided in order to execute the spam message blocking method on a computer.

As described above, according to various aspects of the present invention, there is an effect of improving spam message blocking efficiency by preventing blocking errors that occur when spam message blocking occurs through a learning (deep learning) based method.

In other words, according to the conventional learning-based spam blocking method, 1) when the test data is not in the existing training data, the probability of spam unexpectedly increases, and an error cutoff occurs, 2) When the spam phrase is learned with the surrounding general word characteristics, an error cutout occurs, 3) Problems such as error detection occur due to operational problems in which spam criteria are changed in real time (due to customer complaints and replacement of classified personnel). There is an effect of improving the blocking efficiency of spam messages.

1 is a block diagram of an apparatus for blocking spam messages according to an exemplary embodiment of the present invention;
2 is a flowchart of a method for blocking spam messages according to an exemplary embodiment of the present invention;
3 is a detailed flowchart of the similarity calculation step of FIG. 2;
4 is a detailed flowchart of the second classification step of FIG. 2;
5 is a graph showing a spam false positive prevention ratio according to an exemplary embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In adding reference numerals to the components of each drawing, the same components should have the same numerals as much as possible, even if they are displayed on different drawings. In addition, when it is determined that a detailed description of a known configuration or function related to an embodiment of the present invention may obscure the gist of the present invention, the detailed description will be omitted.

1 is a block diagram of a spam message blocking device according to an exemplary embodiment of the present invention. As shown in the figure, the spam message blocking device 10 includes a pre-processing unit 11 and a first classification unit 13 , a similarity calculation unit 15, and a second classification unit 17.

The pre-processing unit 11 is for classifying the input message as spam or non-spam based on pre-registered spam phrases. For example, if the input message matches a pre-registered spam phrase, the input message is classified as spam. It can be classified and blocked, otherwise it can be classified as non-spam and provided as an input to normal processing, that is, the first classification unit 13.

The first classification unit 13 is for classifying an input message as spam or non-spam through a deep learning-based spam filter learned (or referred to as deep learning) based on learning data. For example, the pre-processing unit 11 ), input the message that is classified as non-spam and output, and classify the input message as spam or non-spam through a deep learning-based spam filter, then output the message classified as non-spam and classify it as spam The message may be provided as an input to the similarity calculating unit 15 and the second classifying unit 17.

The similarity calculation unit 15 is for calculating the similarity between the spam message classified as spam through the deep learning-based spam filter of the first classification unit 13 and input, and the training data used when learning the deep learning-based spam filter. , For example, similarity may be calculated through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding-based similarity comparison method.

In this embodiment, similarity can be calculated through the following BM25 algorithm, which is widely used as one of search similarity comparison methods.

f(qi, D) is the frequency of qi in document D, and |D| is the length of document D. Also, avgdl is the average length of documents. k1, b means free parameters. Usually k1 is in [1.2, 2.0] and b is chosen as 0.75.

The calculation unit 15 calculates the first similarity (BM25 score) between the first message classified as spam through the deep learning-based spam filter and each of the plurality of spam data in the training data, and calculates the calculated plurality (top n). Calculate the average value of the first similarity (hereinafter, the average value of spam similarity).

In addition, a second similarity (BM25 score) between the first message and the learning data and each of the plurality of non-spam data is calculated, and the average value of the calculated plurality (top n) second similarities (hereinafter, the average value of the non-spam similarity) ) is calculated.

In addition, a difference value obtained by subtracting the average similarity value of non-spam from the average similarity value of spam (hereinafter, the final score value) can be calculated and used as the final similarity value.

final score value = spam similarity average value ?? Nonspam similarity average

The spam or non-spam average similarity value may be calculated as an average value for the top n pieces that are equal to or greater than a set minimum value among the plurality of first or second similarities calculated respectively.

The second classification unit 17 is for finding and reclassifying the spam messages classified as normal if there is an ospam message (that is, a message that is not spam but classified as spam) among the spam messages classified by the first classification unit 13. 1. Spam messages can be reduced by re-classifying messages classified as spam by the classification unit 13 into spam or non-spam based on the similarity calculated by the similarity calculation unit 15.

The second classification unit 17, for example, when the similarity value (ie, the final score value) calculated by the calculation unit 15 for the message classified as spam by the first classification unit 13 is less than a preset reference value, The spam message may be reclassified as non-spam message.

As another example, the second classification unit 17 calculates the similarity value (i.e., the final score value) from the calculation unit 15 for the message classified as spam by the first classification unit 13 if it is greater than or equal to a preset reference value, The spam message can be reclassified as spam message again.

As another example, the second classification unit 17 may, in the case where similar data (based on similarity) in the learning data corresponding to the corresponding message classified as spam by the first classification unit 13, have contradictory labeling of spam and non-spam , the corresponding spam message can be reclassified as a non-spam message.

As shown in the example shown in FIG. 5, the standard value of the second classification unit 17 of the spam message blocking device 10 is set to 5, and as a result of the re-classification test, the conventional method using only the deep learning-based spam filter for the same input (i.e., not re-classifying), the number of false cuts was 81. In contrast, as a result of re-classification through the second classification unit 17 of the spam message blocking device 10, the number of false cuts decreased to two, It decreased by about 97% compared to the previous one.

FIG. 2 is a flow chart of a spam message blocking method according to an exemplary embodiment of the present invention, and since it is applied to the spam message device 10 of FIG. 1, the operation of the device 10 will be described in parallel.

First, when a message is input to the spam message blocking device 10, in the pre-processing step (S210), the input message and pre-registered spam messages (including words/phrases/sentences, etc.) are compared through the pre-processing unit 11, and if they match, spam It is classified and blocked, and if it does not match, it is classified as non-spam and provided as an input to the first classification unit 13 (S210).

Subsequently, in the first classification step (S220), the first classification unit 13 inputs the message classified as non-spam through the pre-processing unit 11 in step S210 and outputs the message, and converts the input message to a deep learning-based spam filter After classifying them as spam or non-spam, messages classified as non-spam are normally output, and messages classified as spam are provided as inputs to the similarity calculation unit 15 and the second classification unit 17 (S220). .

Subsequently, in the similarity calculation step (S230), the similarity calculation unit 15 learns the spam message classified as spam through the deep learning-based spam filter of the first classification unit 13 in step S220 and the input spam message and the deep learning-based spam filter. The similarity between the learning data used in the test can be calculated through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding-based similarity comparison method. In this embodiment, one of the search similarity comparison methods is used. The similarity is calculated through the widely used BM25 algorithm (S230).

Finally, in the second classification step (S240), the second classification unit 17 converts the messages classified as spam in the first classification unit 13 in the first classification step (S220) to the calculation unit in the similarity calculation step (S230). It is reclassified as spam or non-spam based on the similarity calculated in (15), and among the spam messages classified in the first classification unit 13 in the first classification step (S220), the spam message (that is, spam that is not spam) If there is a message incorrectly classified as ), it is filtered out and reclassified as normal, thereby reducing the number of spam messages (S240).

FIG. 3 is a flowchart illustrating an example of a detailed configuration of the similarity calculation step ( S230 ) of FIG. 2 .

First, the first similarity (BM25 score) between the first message classified as spam through the deep learning-based spam filter and each of the plurality of spam data in the training data is calculated using the BM25 algorithm, and the calculated plurality (top n ) of the first similarity (hereinafter referred to as spam average similarity) is calculated (S231).

Subsequently, a second similarity (BM25 score) between the first message and the learning data and each of the plurality of non-spam data is calculated using the BM25 algorithm, and the average value of the calculated plurality (top n) second similarities (below , non-spam similarity average value) is calculated (S233).

Finally, a difference value obtained by subtracting the average non-spam similarity value of step S233 from the average similarity value of spam calculated in step S231 (hereinafter, the final score value) is calculated as the final similarity value to be used in S240 (S235).

The average similarity value of spam in step S231 and the average value of similarity value of non-spam in step S233 may be calculated as average values of the top n values that are equal to or greater than a preset minimum value among the plurality of first similarities and the plurality of second similarities, respectively.

FIG. 4 is a flowchart illustrating an example of a detailed configuration of the second classification step ( S240 ) of FIG. 2 .

First, the second classification unit 17 determines whether the similarity value (ie, the final score value) calculated by the calculation unit 15 for the corresponding message classified as spam by the first classification unit 13 is less than a predetermined reference value. It is determined whether or not (S241).

As a result of the determination in step S241, if it is determined that the corresponding similarity value is less than the preset reference value, the corresponding message classified as spam in the first classification unit 13 is reclassified as non-spam (S243), and as a result of determination in step S241, the corresponding similarity value is If it is determined that the message is equal to or greater than the preset reference value, the message classified as spam by the first classification unit 13 is reclassified as spam and processed (S245).

Meanwhile, in the second classification step (S240), the second classification unit 17 identifies spam and non-spam to similar data (based on similarity) in the learning data corresponding to the corresponding message classified as spam by the first classification unit 13. In the case of inconsistent labeling, the spam message may be reclassified as non-spam message.

According to the methods of FIGS. 2 to 4 described above, as in the example shown in FIG. 5, as a result of the re-classification test by setting the reference value to 5 in the second classification step (S240) of the spam message blocking method, the same input message In the case of following the conventional method using only deep learning-based spam filters (i.e., not re-classifying), the number of false cuts was 81, and in contrast, re-classification through the second classification step (S240) of the spam message blocking method As a result, the number of error cutoffs was reduced to two, which was about 97% less than before.

According to the present invention described above, in the case of the conventional learning-based spam blocking method, 1) when the test data is not included in the existing training data, the probability of spam unexpectedly increases, resulting in an error cutoff, 2) learning with the characteristics of surrounding common words rather than spam phrases. 3) Occurrence of errors due to operational problems in which the standards for spam are changed in real time (due to customer complaints and replacement of classified personnel). The efficiency of spam message blocking can be improved by preventing the error-cutting problem. An example of this can be described as follows.

1) When the test data is not in the existing training data, the spam probability unexpectedly rises and errors occur:

For example, if the conventional learning-based spam blocking method is followed for an input message such as "[Sent from Web] Daewangam Agujjim, all menus will be delivered. Delivery fee is free, daily business is normal, be careful of Corona 19", spam similar to learning data / There is no non-spam phrase, but the result is that the spam probability rises to 99% without knowing the cause. This seems to be an OOD (Out-Of-Distribution) problem that mainly occurs in deep learning. However, on the other hand, in contrast, if it is not in the learning data through the post-processor that compares the degree of similarity in the learning data through the apparatus and method of the present invention described above, the final score of similarity is lowered, and if it is lower than the threshold, it is classified as non-spam. This can be solved by reclassifying.

2) Problems that occur when learning is based on the characteristics of surrounding general words rather than spam phrases:

For example, an input message such as “You can work on a simple interior tomorrow. Regarding input messages such as "from the 1st ~ (membership registration required)", according to the conventional learning-based spam blocking method, "Jang" and "Anniversary Thanks Event" are not spam phrases, but spam containing these words in the learning data On the other hand, in the case of the device and method of the present invention, errors can be prevented by reclassifying spam or not through similarity comparison of entire texts, not just some phrases.

In addition, since the similarity comparison is performed for whole characters rather than for some phrases, it is possible to block smishing characters that come repeatedly while not blocking general characters.

[Table 1]

For example, if the deep learning model that has learned the phrases in Table 1 misdiagnoses "Send to the Internet because my phone is not available" as spam, according to the apparatus and method of the present invention, some matches with the training data, but smishing text and text Since the total similarity is low, error judgment can be prevented.

3) Occurrence of errors due to operational problems in which spam standards are changed in real time (customer complaints and replacement of classified persons) Problem:

For example, if spam/non-spam comes and goes for similar data, according to the apparatus and method of the present invention, the overall similarity score is lowered, so it can be reclassified as non-spam to prevent false detection.

On the other hand, according to the method for blocking spam messages described above, a computer-readable recording medium having a program for executing the corresponding method in a computer can be implemented.

On the other hand, according to the above-described spam message blocking method, an application stored in a computer-readable recording medium may be implemented in order to execute the corresponding method in combination with hardware.

On the other hand, according to the method for blocking spam messages described above, a computer program stored in a computer-readable recording medium may be implemented in order to execute the corresponding method on a computer.

For example, as described above, the spam message blocking method according to an exemplary embodiment of the present invention is a computer-readable recording medium containing program instructions for performing various computer-implemented operations or an application stored on such a recording medium. can be implemented The computer readable recording medium may include program instructions, local data files, local data structures, etc. alone or in combination. The recording medium may be specially designed and configured for the embodiment of the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and ROMs, RAMs, flash memories, and the like. A hardware device specially configured to store and execute the same program instructions is included. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

10: spam message blocking device
11: pre-processing unit
13: first classification unit
15: similarity calculator
17: second classification unit

Claims (21)

A first classification unit for classifying an input message as spam or non-spam through a deep learning-based spam filter learned based on learning data (deep learning);
a calculation unit for calculating a similarity between the message classified as spam through the deep learning-based spam filter and the learning data; and
A second classification unit for re-classifying the spam classified as spam or non-spam based on the calculated similarity; and
When the second classification unit has contradictory labeling of spam and non-spam in a plurality of similar data having a similarity value of a predetermined value or more in the training data corresponding to the corresponding message classified as spam by the first classification unit ( That is, when similar data labeled as spam and similar data labeled as non-spam are mixed), spam message blocking characterized in that the message classified as spam in the first classification unit is reclassified as a non-spam message. Device. According to claim 1,
Wherein the second classification unit reclassifies the message classified as spam by the first classification unit as a non-spam message when the similarity value calculated by the calculation unit is less than a predetermined reference value. According to claim 1,
Wherein the second classification unit reclassifies the corresponding message classified as spam by the first classification unit as a spam message when the similarity value calculated by the calculation unit is equal to or greater than a predetermined reference value. delete According to claim 1,
The calculation unit calculates the first average value of similarity (hereinafter referred to as the average value of spam similarity) with each of the plurality of spam data in the training data for the message classified as spam through the deep learning-based spam filter and the value of each of the plurality of non-spam data. Spam message blocking device characterized by obtaining a second average similarity value (hereinafter, non-spam similarity average value), and calculating a difference between the spam similarity average value and the non-spam similarity average value (hereinafter, a final score value) as the similarity value . According to claim 5,
The spam message blocking device, characterized in that the average value of the spam or non-spam similarity is an average value for the top n pieces that are equal to or greater than a set minimum value among the calculated plurality of first or second similarities, respectively. According to claim 1,
Wherein the calculator calculates the similarity through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding based similarity comparison method. According to claim 7,
The spam message blocking device, characterized in that the search similarity comparison method comprises a BM25 algorithm. According to claim 1,
Further comprising a pre-processing unit for classifying an input message as spam or non-spam based on a pre-registered spam phrase, and providing the non-spam message classified through the pre-processing unit as an input message to the first classification unit. Spam message blocking device. (a) classifying an input message as spam or non-spam through a deep learning-based spam filter learned based on learning data (deep learning);
(b) calculating a similarity between a message classified as spam through the deep learning-based spam filter and the learning data; and
(c) re-classifying the spam-classified message as spam or non-spam based on the calculated similarity;
In the step (c), in the learning data corresponding to the corresponding message classified as spam in step (a), a plurality of similar data having a similarity value equal to or greater than a predetermined value are labeled spam and non-spam inconsistently. (i.e., when similar data labeled as spam and similar data labeled as non-spam are mixed), spam characterized in that the message classified as spam in step (a) is reclassified as a non-spam message. How to block messages. According to claim 10,
In step (c), when the similarity value calculated in step (b) is less than a predetermined reference value, the corresponding message classified as spam in step (a) is reclassified as a non-spam message. method. According to claim 10,
In step (c), if the similarity value calculated in step (b) is equal to or greater than a predetermined reference value, the spam message classified as spam in step (a) is reclassified as a spam message. . delete According to claim 10,
In the step (b), for the messages classified as spam through the deep learning-based spam filter, the average value of the first similarity with each of the plurality of spam data in the training data (hereinafter, the average value of the spam similarity) and the plurality of non-spam data Spam characterized in that the average value of the second similarity (hereinafter, hereinafter, the average similarity value of non-spam) is obtained, and the difference between the average value of the similarity of the spam and the average value of the non-spam similarity (hereinafter, the final score value) is calculated as the similarity. How to block messages. According to claim 14,
The spam message blocking method, characterized in that the average value of the spam or non-spam similarity is an average value for the top n pieces that are equal to or greater than a set minimum value among the calculated plurality of first or second similarities, respectively. According to claim 10,
Wherein step (b) calculates the similarity through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding-based similarity comparison method. According to claim 16,
The spam message blocking method, characterized in that the search similarity comparison method comprises a BM25 algorithm. According to claim 10,
(d) further comprising a pre-processing step for classifying the input message as spam or non-spam based on the pre-registered spam phrase, and providing the non-spam message classified through the pre-processing step as the input message of step (a) Spam message blocking method characterized in that for. A computer-readable recording medium storing a program for executing the spam message blocking method according to any one of claims 10 to 12 and 14 to 18 in a computer. A computer-readable recording medium for executing the spam message blocking method according to any one of claims 10 to 12 and 14 to 18 in combination with hardware. A computer program stored in a computer-readable recording medium in order to execute the spam message blocking method according to any one of claims 10 to 12 and 14 to 18 in a computer.

Images