KR20220052156A - Spam message blocking apparatus and method - Google Patents

Abstract

An embodiment of the present invention relates to an apparatus and a method for blocking spam messages, which can improve the efficiency of blocking spam messages by preventing blocking errors that occur when blocking the spam messages through a learning (deep learning)-based manner. According to an embodiment of the present invention, an apparatus for blocking spam messages comprises: a first classification unit for classifying input messages into spam or non-spam messages through a deep learning-based spam filter trained based on training data; a calculation unit for calculating a similarity between messages, classified as spam messages through the deep learning-based spam filter, and the training data; and a second classification unit for re-classifying the messages, classified as spam messages, into spam or non-spam messages based on the calculated similarity.

Description

Spam message blocking apparatus and method

The present invention relates to a spam message blocking technology, and more particularly, to a spam message blocking apparatus and method for improving spam message blocking efficiency by reducing blocking errors that occur when blocking spam messages through a learning (deep learning)-based method it's about

In general, spammers use e-mails or text messages to maliciously achieve their desired purpose. The spread of the Internet in the early days mainly caused the problem of spam e-mail, but the recent increase in smartphone usage has led to a sharp increase in the amount of spam e-mail as well as spam text messages.

In order to block these spam (text) messages, a method of blocking if they match the registered spam texts/phrases based on the spam text data set has been proposed and used. Because the amount is huge and various patterns are continuously added, it is difficult for a person to extract and respond to phrases that become spam one by one.

So, recently, spam blocking efficiency has been improved by using a learning (deep learning)-based classification model that learns and classifies character patterns and features. When the probability of spam rises unexpectedly, an error stage occurs, 2) an error stage occurs when it is learned from the characteristics of surrounding general words rather than a spam phrase, 3) when the standard of spam is changed in real time (due to customer complaints and replacement of classifiers, etc.) Problems such as the occurrence of error stages were discovered as a problem of operability.

Registered Patent Publication No. 10-2001375 (2019.07.12.)

The present invention is to solve the above-mentioned conventional problems, the purpose of which is to prevent a blocking error that occurs when blocking spam messages through a learning (deep learning)-based method to improve spam message blocking efficiency, spam message blocking device and to provide a method.

In order to achieve the above object, an apparatus for blocking spam messages according to an aspect of the present invention is a method for classifying an input message as spam or non-spam through a deep learning-based spam filter learned (deep learning) based on learning data. 1 classification; a calculation unit for calculating a similarity between the message classified as spam and the training data through the deep learning-based spam filter; and a second classification unit for reclassifying the message classified as spam as spam or non-spam based on the calculated similarity degree; It may further include a pre-processing unit for classifying into , and may provide the non-spam message classified through the pre-processing unit as an input message of the first classification unit.

When the similarity value calculated by the calculator is less than a preset reference value, the second classifier may reclassify the message classified as spam by the first classifier as a non-spam message.

The second classifying unit may reclassify the message classified as spam by the first classifying unit as a spam message when the similarity value calculated by the calculating unit is equal to or greater than a preset reference value.

When the second classification unit has contradictory labeling of spam and non-spam in similar data in the training data corresponding to the message classified as spam by the first classification unit, the corresponding message classified as spam by the first classification unit A message can be reclassified as a non-spam message.

The calculation unit provides an average value of a first similarity with each of a plurality of spam data in the training data for a message classified as spam through the deep learning-based spam filter (hereinafter referred to as an average value of spam similarity) and a product with each of a plurality of non-spam data An average value of the two similarities (hereinafter referred to as an average non-spam similarity value) may be obtained, and a difference value (hereinafter, a final score value) between the average value of the non-spam similarity from the average value of the spam similarity may be calculated as the similarity.

The average value of the spam or non-spam similarity may be an average value of the top n items that are equal to or greater than a set minimum value among the plurality of calculated first or second similarities, respectively.

The calculator may calculate the similarity through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding-based similarity comparison method, and the search similarity comparison method may include a BM25 algorithm. .

In order to achieve the above object, the spam message blocking method according to an aspect of the present invention is (a) classifying an input message as spam or non-spam through a deep learning-based spam filter learned (deep learning) based on learning data steps to do; (b) calculating a similarity between the message classified as spam and the training data through the deep learning-based spam filter; and (c) reclassifying the message classified as spam as spam or non-spam based on the calculated similarity; Alternatively, the method may further include a pre-processing step for classifying as non-spam, and the non-spam message classified through the pre-processing step may be provided as the input message of step (a).

In step (c), when the similarity value calculated in step (b) is less than a preset reference value, the message classified as spam in step (a) may be reclassified as a non-spam message.

In step (c), when the similarity value calculated in step (b) is equal to or greater than a preset reference value, the message classified as spam in step (a) may be reclassified as a spam message.

In step (c), if there is contradictory labeling of spam and non-spam in similar data in the training data corresponding to the corresponding message classified as spam in step (a), it is classified as spam in step (a). The corresponding message can be reclassified as a non-spam message.

In step (b), the average value of the first similarity with each of the plurality of spam data in the training data (hereinafter, the average value of the spam similarity) and the plurality of non-spam data for a message classified as spam through the deep learning-based spam filter An average value of the second degree of similarity (hereinafter referred to as the average value of similarity to non-spam) may be obtained, and a difference value (hereinafter, a final score value) between the average value of similarity to non-spam from the average value of similarity to spam may be calculated as the degree of similarity.

The average value of the spam or non-spam similarity may be calculated as an average value of the top n items that are equal to or greater than a set minimum value among the plurality of calculated first or second similarities, respectively.

In step (b), the similarity may be calculated through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding-based similarity comparison method, and the search similarity comparison method includes a BM25 algorithm can do.

According to another aspect of the present invention to achieve the above object, there may be provided a computer-readable recording medium in which a program for executing the spam message blocking method in a computer is recorded.

According to another aspect of the present invention in order to achieve the above object, there may be provided an application stored in a computer-readable recording medium in order to execute the spam message blocking method in combination with hardware.

According to another aspect of the present invention in order to achieve the above object, there may be provided a computer program stored in a computer-readable recording medium in order to execute the spam message blocking method in a computer.

As described above, according to various aspects of the present invention, there is an effect of improving the blocking efficiency of spam messages by preventing a blocking error that occurs when blocking spam messages through a learning (deep learning)-based method.

In other words, according to the conventional learning-based spam blocking method, 1) when the test data is not in the existing training data, the probability of spam increases unexpectedly, resulting in an error stage; 2) when it is learned from the characteristics of surrounding general words rather than spam phrases, an error stage occurs; 3) Problems such as error stages occur due to operational problems in which the standards of spam are changed in real time (due to customer complaints and replacement of classifiers). It has the effect of improving the blocking efficiency of spam messages.

1 is a block diagram of an apparatus for blocking spam messages according to an exemplary embodiment of the present invention;
2 is a flowchart of a method for blocking spam messages according to an exemplary embodiment of the present invention;
3 is a detailed flowchart of the similarity calculation step of FIG. 2;
4 is a detailed flowchart of the second classification step of FIG. 2;
5 is a graph showing a spam error block prevention ratio according to an exemplary embodiment of the present invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In adding reference numerals to the components of each drawing, the same components are to have the same reference numerals as much as possible even though they are indicated in different drawings. In addition, when it is determined that a detailed description of a known configuration or function related to the embodiment of the present invention may obscure the gist of the present invention, the detailed description thereof will be omitted.

1 is a block diagram of an apparatus for blocking spam messages according to an exemplary embodiment of the present invention. As shown in the figure, the apparatus for blocking spam messages 10 includes a preprocessor 11 and a first classifier 13 , a similarity calculating unit 15 , and a second classifying unit 17 .

The preprocessor 11 is for classifying the input message as spam or non-spam based on the pre-registered spam phrase. For example, if the input message matches the pre-registered spam phrase, the input message is classified as spam. It can be classified and blocked, otherwise classified as non-spam and provided as an input to normal processing, that is, the first classification unit 13 .

The first classification unit 13 is for classifying the input message as spam or non-spam through a deep learning-based spam filter learned (or referred to as deep learning) based on the learning data, for example, the preprocessor 11 ), input the message classified as non-spam, classify the input message as spam or non-spam through a deep learning-based spam filter, then output the message classified as non-spam normally and classify it as spam The received message may be provided as an input to the similarity calculating unit 15 and the second classifying unit 17 .

The similarity calculating unit 15 is for calculating the similarity between the input spam message classified as spam through the deep learning-based spam filter of the first classification unit 13 and the training data used when the deep learning-based spam filter is trained. , for example, the similarity may be calculated through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding-based similarity comparison method.

In this embodiment, the similarity can be calculated through the following BM25 algorithm, which is often used as one of the search similarity comparison methods.

f(qi, D) is the frequency of qi in document D and |D| is the length of document D. Also, avgdl is the average length of documents. k1 and b mean free parameters. Mainly k1 is in [1.2, 2.0] and b is chosen to be 0.75.

The calculator 15 calculates a first similarity (BM25 score) between a first message classified as spam and a plurality of spam data in the training data through a deep learning-based spam filter, and the calculated plurality (top n) Calculate the average value of the first similarity of (hereinafter referred to as the average value of spam similarity).

In addition, a second degree of similarity (BM25 score) between the first message and each of the plurality of non-spam data within the training data is calculated, and the average value of the calculated second similarities of the plurality (top n pieces) (hereinafter referred to as the non-spam similarity average value) is calculated. ) is calculated.

In addition, a difference value obtained by subtracting the average non-spam similarity value from the spam similarity average value (hereinafter, the final score value) may be calculated and used as the final similarity value.

Final score value = average spam similarity ?? Average value of non-spam similarity

The average value of the spam or non-spam similarity may be calculated as an average value of the top n pieces that are equal to or greater than a set minimum value among a plurality of calculated first or second similarities, respectively.

The second classification unit 17 is to find and reclassify as normal if there is an spam message (that is, a message classified as spam but not spam) among the spam messages classified by the first classification unit 13. 1 The message classified as spam by the classifying unit 13 may be reclassified as spam or non-spam based on the similarity calculated by the similarity calculating unit 15 to reduce spam messages.

The second classification unit 17, for example, when the similarity value (that is, the final score value) calculated by the calculator 15 for the corresponding message classified as spam by the first classification unit 13 is less than a preset reference value, The spam message may be reclassified as a non-spam message.

As another example, when the similarity value (that is, the final score value) calculated by the calculator 15 for the message classified as spam by the first classifier 13 is greater than or equal to a preset reference value, The spam message can be reclassified as a spam message again.

As another example, when the second classification unit 17 has contradictory labeling of spam and non-spam in similar data (based on similarity) in the training data corresponding to the message classified as spam by the first classification unit 13 , it is possible to reclassify the spam message as a non-spam message.

As in the example shown in FIG. 5 , the standard value of the second classification unit 17 of the spam message blocking device 10 is set to 5 and the result of the reclassification test, the existing method using only a deep learning-based spam filter for the same input According to (that is, if not reclassified), the number of error stages was 81, and in contrast, as a result of reclassification through the second classification unit 17 of the spam message blocking device 10, the number of error stages decreased to two. It is reduced by about 97% compared to the previous one.

2 is a flowchart of a spam message blocking method according to an exemplary embodiment of the present invention. Since it is applied to the spam message device 10 of FIG. 1 , the operation of the device 10 will be described in parallel.

First, when a message is input to the spam message blocking device 10, in the pre-processing step (S210), the pre-processing unit 11 compares the input message with the pre-registered spam message (including a word/phrase/sentence, etc.) and matches the spam It is classified as and blocked, and if it does not match, it is classified as non-spam and provided as an input of the first classification unit 13 (S210).

Next, in the first classification step (S220), the first classification unit 13 inputs a message that is classified as non-spam through the preprocessor 11 in step S210 and outputs the message, and uses the input message as a deep learning-based spam filter. After classifying the message as spam or non-spam through .

Next, in the similarity calculation step (S230), the similarity calculation unit 15 is classified as spam through the deep learning-based spam filter of the first classification unit 13 in step S220, and the input spam message and the deep learning-based spam filter are learned. The degree of similarity between the training data used at the time can be calculated through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding-based similarity comparison method. In this embodiment, one of the search similarity comparison methods The similarity is calculated through the widely used BM25 algorithm (S230).

Finally, in the second classification step (S240), the second classification unit 17 calculates the message classified as spam in the first classification unit 13 in the first classification step (S220) in the similarity calculation step (S230). Reclassified as spam or non-spam based on the degree of similarity calculated in (15), and among spam messages classified in the first classification unit 13 of the first classification step (S220), spam messages (that is, spam that is not spam) If there is a message misclassified as , it reduces the spam message by filtering it and reclassifying it as normal (S240).

3 is a flowchart illustrating an example of a detailed configuration of the step of calculating the similarity of FIG. 2 ( S230 ).

First, the first similarity (BM25 score) between the first message classified as spam through the deep learning-based spam filter and each of the plurality of spam data in the training data is calculated using the BM25 algorithm, and the calculated plurality (top n) ), an average value of the first similarity (hereinafter, spam similarity average value) is calculated (S231).

Next, a second similarity (BM25 score) between the first message and each of the training data and each of the plurality of non-spam data is calculated using the BM25 algorithm, and the average value of the calculated plurality (top n pieces) of the second similarity (hereinafter referred to as the average value) , a non-spam similarity average value) is calculated (S233).

Finally, a difference value (hereinafter, the final score value) obtained by subtracting the average non-spam similarity value of step S233 from the average spam similarity calculated in step S231 is calculated as the final similarity value to be used in S240 (S235).

The average value of the spam similarity of step S231 and the average value of the non-spam similarity of step S233 may be calculated as the average value of the top n pieces that are equal to or greater than a preset minimum value among the plurality of first similarities and the plurality of second similarities, respectively.

4 is a flowchart illustrating an example of a detailed configuration of the second classification step ( S240 ) of FIG. 2 .

First, the second classification unit 17 determines whether the similarity value (that is, the final score value) calculated by the calculator 15 for the message classified as spam by the first classification unit 13 is less than a preset reference value. It is determined whether or not (S241).

If it is determined in step S241 that the similarity value is less than the preset reference value, the first classifier 13 reclassifies the message classified as spam as non-spam (S243), and as a result of the determination in step S241, the similarity value is If it is determined that it is greater than or equal to the preset reference value, the message classified as spam in the first classification unit 13 is reclassified as spam and processed (S245).

On the other hand, in the second classification step (S240), the second classification unit 17 divides spam and non-spam into (similarity-based) similar data in the training data corresponding to the message classified as spam by the first classification unit 13. In the case of inconsistent labeling, the spam message may be reclassified as a non-spam message.

According to the method of FIGS. 2 to 4 described above, as in the example shown in FIG. 5 , the reclassification test was performed by setting the reference value of the second classification step ( S240 ) of the spam message blocking method to 5. As a result, the same input message In the case of following the existing method using only deep learning-based spam filters (that is, when not reclassifying), the number of error stages was 81, and in contrast, reclassification through the second classification step (S240) of the spam message blocking method As a result, the number of error stages was reduced to two, which was reduced by about 97% compared to the previous one.

According to the present invention described above, according to the conventional learning-based spam blocking method, 1) when the test data is not in the existing training data, the probability of spam increases unexpectedly, resulting in an error stage, 2) learning with the characteristics of surrounding general words rather than spam phrases 3) Errors occur due to operational problems in which the standard of spam is changed in real time (due to customer complaints and replacement of classifiers). By preventing the error edge problem, the blocking efficiency of spam messages can be improved, and an example of this can be described as follows.

1) When the test data is not in the existing training data, the probability of spam increases unexpectedly, and an error stage occurs:

For example, if you follow the conventional learning-based spam blocking method for input messages such as "[Send from Web] All menu items will be delivered. Delivery fee is free. We are open daily, be careful of Corona 19." There is no non-spam phrase, but the probability of spam rises to 99% without knowing the cause, which appears to be an OOD (Out-Of-Distribution) problem that occurs mainly in deep learning. On the other hand, if it is not the case, in contrast, if there is no similarity in the learning data through the post-processor that compares whether it is in the learning data through the apparatus and method of the present invention, the final score of the similarity is lowered, and if it is lower than the threshold, it is classified as non-spam. This can be solved by reclassifying.

2) A problem that occurs when an error occurs when learning is done with the characteristics of surrounding general words rather than spam phrases:

For example, an input message such as "You can work on a simple interior design tomorrow. Ilsan Workers 010 6810 9000" or "Hello^^ This is Pet Hotel 1st Avenue. We are holding a 4th anniversary thank you event. ?'Hotel 1 night full'? March For input messages such as “From the 1st of the year to (member registration required)”, if you follow the conventional learning-based spam blocking method, “Jean” and “Anniversary Thanks Event” are not spam phrases, but spam containing these words in the learning data While there is a lot of data and an error stage occurs by focusing on a specific phrase, according to the apparatus and method of the present invention, it is possible to prevent an error stage by reclassifying whether it is spam or not by comparing the similarity of all characters rather than some phrases.

In addition, since similarity comparison of all characters, not just some phrases, is made, it is possible to prevent repeated smishing characters while not blocking regular characters.

[Table 1]

For example, if the deep learning model that learned the phrases in Table 1 mistakenly rejects "the phone doesn't work, so send it to the Internet" as spam, according to the device and method of the present invention, it partially matches the training data, but smishing characters and texts Since the overall similarity is low, error stages can be prevented.

3) Error stage (replacement of customer complaints and classification personnel) due to operational issues in which the standard of spam is changed in real time:

For example, when spam/non-spam comes and goes with respect to similar data, according to the apparatus and method of the present invention, since the overall similarity score is lowered, it is reclassified as non-spam and an error stage can be prevented.

Meanwhile, according to the above-described spam message blocking method, it is possible to implement a computer-readable recording medium in which a program for executing the method in a computer is recorded.

On the other hand, according to the above-described spam message blocking method, it is possible to implement an application stored in a computer-readable recording medium in order to execute the method in combination with hardware.

On the other hand, according to the above-described spam message blocking method, it is possible to implement a computer program stored in a computer-readable recording medium in order to execute the method in a computer.

For example, as described above, the spam message blocking method according to an exemplary embodiment of the present invention is a computer-readable recording medium including program instructions for performing various computer-implemented operations or an application stored in the recording medium. can be implemented. The computer-readable recording medium may include program instructions, local data files, local data structures, and the like alone or in combination. The recording medium may be specially designed and configured for the embodiment of the present invention, or may be known and used by a person skilled in the art of computer software. Examples of the computer-readable recording medium include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and ROMs, RAMs, flash memories, and the like. Hardware devices specially configured to store and execute the same program instructions are included. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those generated by a compiler.

The above description is merely illustrative of the technical spirit of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those skilled in the art to which the present invention pertains. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

10: Spam Message Blocker
11: preprocessor
13: first classification unit
15: similarity calculator
17: second classification unit

Claims (21)

A first classification unit for classifying an input message as spam or non-spam through a deep learning-based spam filter learned (deep learning) based on the learning data;
a calculation unit for calculating a similarity between the message classified as spam and the training data through the deep learning-based spam filter; and
a second classification unit for reclassifying the message classified as spam as spam or non-spam based on the calculated similarity;
A spam message blocking device that includes. The method of claim 1,
The second classifying unit reclassifies the message classified as spam by the first classifying unit as a non-spam message when the similarity value calculated by the calculating unit is less than a preset reference value. The method of claim 1,
The second classifying unit reclassifies the message classified as spam by the first classifying unit as a spam message when the similarity value calculated by the calculating unit is equal to or greater than a preset reference value. According to claim 1,
When the second classification unit has contradictory labeling of spam and non-spam in similar data in the training data corresponding to the message classified as spam by the first classification unit, the corresponding message classified as spam by the first classification unit A spam message blocking device, characterized in that the message is reclassified as a non-spam message. According to claim 1,
The calculation unit is, for a message classified as spam through the deep learning-based spam filter, an average value of a first similarity with each of a plurality of spam data in the training data (hereinafter referred to as an average value of spam similarity) and each of a plurality of non-spam data An apparatus for blocking spam messages, characterized in that the average value of the second similarity (hereinafter referred to as the non-spam similarity average value) is obtained, and a difference value (hereinafter, the final score value) between the average value of the spam similarity level and the average value of the non-spam similarity level is calculated as the similarity level. . 6. The method of claim 5,
The spam or non-spam similarity average value is an average value of the top n items that are equal to or greater than a set minimum value among the plurality of calculated first or second similarities, respectively. According to claim 1,
wherein the calculator calculates the similarity through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding-based similarity comparison method. 8. The method of claim 7,
The spam message blocking device, characterized in that the search similarity comparison method includes a BM25 algorithm. The method of claim 1,
Further comprising a pre-processing unit for classifying an input message as spam or non-spam based on the pre-registered spam phrase, characterized in that for providing the non-spam message classified through the pre-processing unit as an input message of the first classification unit Spam message blocking device. (a) classifying the input message as spam or non-spam through a deep learning-based spam filter learned (deep learning) based on the learning data;
(b) calculating a similarity between the message classified as spam and the training data through the deep learning-based spam filter; and
(c) reclassifying the message classified as spam as spam or non-spam based on the calculated similarity;
How to block spam messages, including. 11. The method of claim 10,
In step (c), when the similarity value calculated in step (b) is less than a preset reference value, blocking spam messages, characterized in that the message classified as spam in step (a) is reclassified as a non-spam message Way. 11. The method of claim 10,
In step (c), when the similarity value calculated in step (b) is equal to or greater than a preset reference value, the spam message blocking method, characterized in that the message classified as spam in step (a) is reclassified as a spam message . 11. The method of claim 10,
In step (c), if there is inconsistent labeling of spam and non-spam in similar data in the training data corresponding to the corresponding message classified as spam in step (a), it is classified as spam in step (a). A method for blocking spam messages, characterized in that the message is reclassified as a non-spam message. 11. The method of claim 10,
In the step (b), the average value of the first similarity with each of the plurality of spam data in the training data (hereinafter, the average value of the spam similarity) and the plurality of non-spam data for a message classified as spam through the deep learning-based spam filter Spam, characterized in that the average value of the second similarity with each other (hereinafter, the average value of similarity to non-spam) is obtained, and the difference between the average value of similarity to non-spam from the average value of spam similarity (hereinafter, the final score value) is calculated as the similarity. How to block messages. 15. The method of claim 14,
The spam or non-spam similarity average value is an average value of the top n pieces that are equal to or greater than a set minimum value among the plurality of calculated first or second similarities, respectively. 11. The method of claim 10,
The step (b) is a method for blocking spam messages, characterized in that the similarity is calculated through at least one of a search similarity comparison method, a string similarity comparison method, and a sentence embedding-based similarity comparison method. 17. The method of claim 16,
The search similarity comparison method is a spam message blocking method, characterized in that it includes a BM25 algorithm. 11. The method of claim 10,
(d) further comprising a pre-processing step for classifying the input message as spam or non-spam based on the pre-registered spam phrase, and providing the non-spam message classified through the pre-processing step as the input message of step (a) A method for blocking spam messages, characterized in that for 19. A computer-readable recording medium in which a program for executing the spam message blocking method of any one of claims 10 to 18 in a computer is recorded. 19. A computer-readable recording medium in order to execute the spam message blocking method of any one of claims 10 to 18 in combination with hardware. A computer program stored in a computer-readable recording medium in order to execute the method for blocking spam messages according to any one of claims 10 to 18 in a computer.

Images