KR100902475B1 - Method and System for Deciding Spam Document - Google Patents

Abstract

Spam document evaluation method according to an embodiment of the present invention that can be applied to a plurality of spam detection techniques to increase the reproducibility and accuracy of the spam document determination system, extracting predetermined information from a predetermined document; And determining whether the document is spam by applying a plurality of spam detection techniques to the extracted information.

The determining whether or not the spam may include: detecting whether the document is spam by applying the plurality of detection techniques to the extracted information; Assigning each document a predetermined spam index assigned to each spam detection technique whenever the document is detected as spam by the respective spam detection technique; And determining that the document is spam when the sum of the spam indices is equal to or greater than a reference value.

Spam, Filtering, Documents, Bayesian, Recall, Accuracy

Description

Spam Document Determination System and Method {Method and System for Deciding Spam Document}

1 is a schematic block diagram of a spam document determination system according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a method of determining whether a predetermined document is spam by using the spam document determination system shown in FIG. 1.

3 is a graph showing the accuracy and reproducibility of each spam detection technique calculated by applying the spam detection techniques used by the spam document detection unit of FIG.

FIG. 4 is a diagram showing a proportion of documents that are determined to be spam by spam detection techniques among documents that the spam document determination system shown in FIG. 1 is mistaken for spam.

<Description of the symbols for the main parts of the drawings>

10: spam document determination system 12: information extraction unit

14: spam judgment 16: word reassembly

18: index word extraction unit 20: spam document detection unit

22: spam document determination unit 24: evaluation unit

26: spam index readjustment unit

The present invention relates to a spam document determination system, and more particularly, to a spam document determination system and method that can increase the reproducibility and accuracy of the spam document determination system.

Recently, due to the development of the Internet service and the search service, the search request through the search portal is rapidly increasing, and as a result, the provision of search results corresponding to the search request is directly increasing. Recently, there has been an increase in attempts to exploit spam, such as advertisements, on top of search results, and as a result, a spam document determination system that can determine whether a document included in such a search result is a spam document. Research is actively underway.

The performance of such a spam document determination system is measured by precision and recall. Here, the accuracy rate means the probability that the document determined to be spam as a result of spam filtering is the actual spam document, and the reproducibility rate means the probability of the document determined as the actual spam among the documents to be determined as the spam document.

If the system finds as many spam documents as possible, the reproducibility improves, and most of the documents judged as spam improve the accuracy. However, since these two measures have an exclusive relationship with each other in the same system, when the accuracy is increased in the same system, the reproducibility is lowered, and when the reproducibility is increased, the accuracy is lowered.

Meanwhile, in determining whether a predetermined document is a spam document by using the above-described spam document determination system, words were extracted from the predetermined document and spam was determined using the extracted words, and spam document producers exploited this point. Therefore, special characters are inserted into words that can be determined as spam documents, or some of these words are connected to each other. Thus, a document produced by such a method cannot be judged as a spam document.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problem, and it is a technical object of the present invention to provide a spam document evaluation system and a method for applying a plurality of spam detection techniques to increase the reproducibility of the spam document determination system.

In addition, according to the present invention, a spam document evaluation system and method capable of re-adjusting the spam index given to each spam detection technique by continuously monitoring the spam document determination result in order to increase both the reproducibility and the accuracy rate of the spam document determination system. It is another technical problem to provide.

In addition, the present invention is to solve the above-mentioned other problems, to provide a spam document determination system and method that can further perform word reassembly or index word extraction in the process of extracting a word from a predetermined document another technical It is a task.

Spam document determination method according to an aspect of the present invention for achieving the above object comprises the steps of: extracting predetermined information from a predetermined document; And determining whether the document is spam by applying a plurality of spam detection techniques to the extracted information.

The determining whether or not the spam may include: detecting whether the document is spam by applying the plurality of detection techniques to the extracted information; Assigning each document a predetermined spam index assigned to each spam detection technique each time the document is detected as spam by the respective spam detection method; And determining that the document is spam when the sum of the spam indices is equal to or greater than a reference value.

In one embodiment, the information extracted from the document in the predetermined information extraction step is a word or link information. If the extracted word includes a special character, the word is removed after the special character is removed from the word. When the word is assembled or the extracted word is longer than a reference value, index word extraction may be performed to separate the word into a plurality of words.

On the other hand, the spam document determination method, the step of calculating the correct rate of the respective spam detection techniques based on the spam determination result; And adjusting the spam indexes of the spam detection techniques according to the accuracy rate of each spam detection technique.

Spam document determination system according to another aspect of the present invention for achieving the above object is an information extraction unit for extracting predetermined information from a predetermined document; And a spam determination unit that determines whether the document is spam by applying a plurality of spam detection techniques to the extracted information.

Here, the spam determination unit detects whether or not the document is spam by applying the plurality of spam detection techniques to the extracted information, and when the document is detected as spam, a predetermined number assigned to each spam detection technique. Spam document detection unit for assigning a spam index to the document; And a spam document determination unit for calculating a sum of the spam indexes assigned to the document and determining the document as spam when the sum of the spam indexes is equal to or greater than a reference value.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic block diagram of a spam document determination system according to an embodiment of the present invention. As illustrated, the spam document determination system includes an information extraction unit 12 and a spam determination unit 14.

The information extracting unit 12 extracts predetermined information from a document. Here, the document means a general web page, but is not limited thereto, and any type of document that can be displayed on a computer will correspond to this. The information extracting unit 12 extracts predetermined information that can determine whether or not spam from such documents. In one embodiment, the information extracted from the documents includes a link including a specific word or a URL of a specific web page. Information is applicable.

The information extracting unit 12 includes a word extracting unit 16, a word reassembling unit 18, and an index word extracting unit 20 as shown. The word extractor 16 extracts a word or link information to be used for spam detection from the document. When extracting a word from the document, the word extractor 16 extracts the word based on a space unit of the document.

When the word extracted by the word extracting unit 16 includes the special characters, the word reassembly unit 18 reassembles the extracted words after removing the special characters from the extracted words. For example, when the word extracting unit 16 extracts a word containing a special character such as "study / child / gold / loan / export", the special word "/" is removed from the extracted word and the extracted word is extracted. Will be reassembled into words like "student loans".

The index word extracting unit 20 separates the extracted word into a plurality of words by performing index word extraction on the extracted word when the length of the word extracted by the word extracting unit 16 is longer than a reference value. For example, when a word such as "student loan" is extracted by the word extracting unit 16, index words are extracted for the extracted word to "student money", "loan", "credit", "housing" "Into multiple words, such as".

The word reassembly unit 18 and the index word extracting unit 20 may include a special character in a word that allows a specific document to be determined to be spam, or may include a special character in the word so that the specific document is not classified as spam. Since it is a component for preventing, inclusion of the word reassembly 18 and the index word extraction unit 20 may be optional. In addition, when the length of the word reassembled by the word reassembly unit 18 is longer than the reference value, the index word extraction may be performed again on the reassembled word.

The spam determination unit 14 determines whether the corresponding document is a spam document by applying a plurality of spam detection techniques to the word or link information extracted by the information extraction unit 12. As illustrated, the spam document detection unit (22) and the spam document determination unit 24.

The spam document detector 22 detects whether a document is spam by applying a plurality of spam detection techniques to the extracted word or link information, and each spam detection technique detects each document as spam. The spam index assigned to the technique is assigned to the document.

In one embodiment, a plurality of detection techniques used by the spam document detection unit 22 may be a statistic based technique, a pattern matching technique, a Bayesian technique, or the like.

The statistics-based technique calculates statistical information from predetermined information such as extracted words or links, and analyzes the calculated statistical information to detect whether a corresponding document is spam. Here, the statistical information includes the repetition frequency of the extracted information, the overlapping ratio of the extracted information, the length of the extracted information, the frequency of creating the actual document, and the like. The actual document writing frequency is statistical information that can be used when the actual writing time is known. When a large amount of documents are published for a short time, the statistical information can be used to detect whether the document is spam.

The pattern matching technique is a technique for classifying a document containing a word of a predetermined pattern as a spam document. Since learning is not required, even if an emergency is required, the pattern matching technique may determine whether the document is spammed. For example, a link to a web page containing malicious code or a specific web page for advertising purposes is recognized as a pattern, or a non-normal word such as "card geno" is recognized as a pattern and the pattern is extracted from the document. If extracted, the document is detected as a spam document.

The Bayesian technique is derived from Bayes' Theorem and detects spam documents through learning using probability models. These Bayesian classification techniques are largely divided into Naive-Bayesian and Paul Graham's Bayesian.

First, the naive Bayesian classification technique is used when the probability that the document is spam P (S | D) is greater than the probability that the document is normal, P (N | D), that is, P (S | D)> P (N | D). Is detected as spam. Here, the probability that the document is spam P (S | D) and the probability P (N | D) that is normal is calculated by the following equations (1) and (2).

According to Equations 1 and 2, if the following Equation 3 is satisfied, the document is classified as spam.

Here, the document can be regarded as a set of words (w = w1, w2, ..., wn) included in the document, and P (D | S) It can be expressed as Equation 4 below.

That is, the naive Bayesian technique detects whether the document is spam by using Equations 1 to 4 as described above.

Next, Paul Graham's Bayesian classification technique is an improvement on the Naive Bayesian technique described above. First, when a specific word w exists in a document as described in Equation 5, the probability value P (S | w) of the document is spam. Calculate.

Then, the difference d between the probability value calculated as described in Equation 6 and the predetermined reference value is calculated, and the combined probability (7) is used for only n probability values having a large value d among the calculated probability values. Combinational Probability is calculated and the document is detected as spam if the calculated Combination Probability is equal to or greater than the threshold.

은 n 번째 단어가 스팸일 확률값을 나타낸다.here,

Denotes the probability that the nth word is spam.

In one embodiment, since the calculated probability values have a value between 0 and 1, the reference value is determined as 0.5, which is the median value, and 15 of the calculated probability values are larger than d (that is, n = 15). The combined probability can be calculated using only the probability value of.

In other words, if the probability is calculated based on all words appearing in the document, the more the normal words are included in the document, the lower the spam detection ability. Detecting whether or not the document is spam by determining whether the word is spam only.

The spam document detection unit 22 applies the plurality of spam detection techniques described above to extracted words or link information, and each spam detection technique detects the document as spam, and the spam index given to each spam detection technique. Will be assigned to the document. For example, if the spam index assigned to the statistics-based method is 5, the spam index assigned to the pattern matching method is 10, and the spam index assigned to the Bayesian method is defined as 20, the document is spammed by the statistics-based method. If it is detected as, assign 5 to the document, if the document is detected as spam also by pattern matching method, then assign 10 to the document again, and if the document is detected as spam also by Bayesian method 20 is allocated.

Here, the spam index is defined in advance for each spam detection technique, and the value of the spam index may be defined equally for all spam detection techniques, but may be defined differently according to the performance of each spam detection technique.

The spam document determination unit 24 calculates the sum of spam indices assigned to the document by the spam document detection unit 22, and determines that the document is spam when the sum of the assigned spam indices is equal to or greater than a reference value. For example, in the above example, if the document is detected as spam by all three techniques, the sum of spam indexes of the document is 35, and if the document is detected as spam only by pattern matching and Bayesian techniques, the spam of the document is detected. The sum of the indexes is 30, and if the spam is detected only by Bayesian techniques, the spam index of the document is 20. If the sum of the summed spam index is more than the reference value spam document determination unit 20 is to determine the document as spam.

In one embodiment, the spam document determination system 10 further includes an evaluation unit 26 and a spam index readjustment unit 28 to improve the performance of the spam document determination system 10. Feedback can be performed based on the judgment result by

The evaluation unit 26 calculates the accuracy rate of each spam detection technique used by the spam document detection unit 22 based on the determination result generated by the spam determination unit 20. In one embodiment, the accuracy rate is calculated through the probability that the document detected as spam by the spam detection technique is determined as the actual spam document.

The spam index readjustment unit 28 readjusts the spam indexes of the spam detection methods according to the accuracy rate of each spam detection method calculated by the evaluator 26. Specifically, the spam index is readjusted by weighting each spam detection technique in the order of high accuracy.

In the above-described embodiment, the spam index is readjusted according to the accuracy rate of each spam detection technique. In the modified embodiment, the spam index may be readjusted by evaluating the reproducibility or error rate of each spam detection technique.

A method of determining whether a corresponding document is a spam document using the above-described spam document detection system will be described in detail with reference to FIG. 2.

First, predetermined information to be used for spam detection is extracted from the document (step 100). Here, the predetermined information means a word or link information included in the document as described above. In one embodiment, when the extracted word includes the special character (step 102), a process of reassembling the word after removing the special character from the word is performed (step 104). When the length is longer than the reference value (step 106), the index word may be extracted to the word to separate the word into a plurality of words (step 108). It has already been described above that the word reassembly step or the index word extraction step may be selectively performed, and that the index word extraction process may be performed again when the length of the word reassembled is greater than or equal to the reference value.

Subsequently, by applying a plurality of spam detection techniques to the extracted information, the system detects whether the document is spam and assigns the spam index assigned to each spam detection technique to the document whenever the document is detected as a spam document. (Step 110). That is, if the document is detected as a spam document only by one of the spam detection techniques, only the spam index of the spam detection technique is assigned to the document, and if the document is detected as a spam document by all the spam detection techniques, The spam index of each spam detection technique is assigned to the document.

In one embodiment, a spam detection technique used to detect whether a document is spam is a statistical based technique, a pattern matching technique, a Bayesian technique, and the like. A detailed description of these methods is provided in the spam document detection unit 20. Since it has already been described in the description, a detailed description thereof will be omitted.

Thereafter, the sum of spam indexes assigned to the document is calculated to determine whether the document is spam (step 120). Specifically, if the sum of spam indices is greater than or equal to the reference value, the document is determined as a spam document, otherwise it is not determined as spam.

In one embodiment, the spam document determination method may further include a feedback process to improve the performance of the system, which is as follows.

After calculating the accuracy rate for each spam detection technique based on the spam document determination result (step 130), the spam index of the spam detection technique is readjusted by assigning a weight to each spam detection technique in the order of high accuracy. Step 140). Through this feedback process, the spam index of high-performance spam detection technique is raised and the spam index of low-performance spam detection is relatively lowered to improve the system performance. In a modified embodiment, the spam index may be readjusted by evaluating not only the accuracy of each spam detection technique but also the recall or error rate of each spam detection technique.

3 is a view showing the accuracy and reproducibility of each spam detection techniques calculated by applying the above-described spam detection techniques to the documents posted on a predetermined blog, Figure 4 is a document misjudged by spam spam determination system of the present invention Each spam detection technique shows the proportion of documents that are determined to be spam. As shown in FIG. 3, in the case of the statistical-based method based on the appearance frequency of the extracted information, the accuracy rate and the reproducibility rate were high, but 88.7% of the documents judged to be spam based on the frequency of appearance of the total misjudged documents were 88.7%. It can be seen that the log ratio is too high.In the case of the statistics-based method based on the overlapping rate, the accuracy rate is similar to that based on the frequency of occurrence, but the percentage of documents judged to be spam based on the overlapping rate among all the miscalculated documents is the frequency of occurrence. It can be seen that it is somewhat lower.

In the case of the pattern matching technique, both accuracy and recall rate is 90%, and the ratio of the document mismatched by the pattern matching technique is 38%. In the Bayesian technique, the accuracy rate is higher than that of the pattern matching method, but the recall is about 55%, which is significantly lower than the pattern matching method. For this reason, first, in case of spam documents such as advertisements, it is important that the spam authors newly appear each time and the words used in the spam documents change frequently, so that they continuously learn using the latest data. This is because it is based only on the documents posted on existing blogs, not the latest training data.

Second, most existing spam documents are written only in text, but recent spam documents are created by inserting images or linking to normal documents to prevent them from being recognized as spam. There are many documents that are difficult to detect with Bayesian techniques for detection. In the case of Bayesian technique, the percentage of documents which Bayesian technique judged as spam among all misjudged documents is quite low as 9%, and it can be seen that it can be a powerful spam detection technique if continuous learning is performed.

As described above, the performance of the spam document determination system is improved by continuously adjusting the spam index of each spam detection technique by using performance indicators such as the accuracy rate of each spam detection technique generated based on the spam determination result.

The above-mentioned spam document determination method may be implemented in a program form that can be executed using various computer means. In this case, a program for performing the spam document determination method may be a hard disk, a CD-ROM, a DVD, a ROM. Data is stored in a computer-readable recording medium, such as, RAM, or flash memory.

Those skilled in the art to which the present invention pertains will understand that the present invention can be implemented in other specific forms without changing the technical spirit or essential features. For example, in the above-described embodiment, it has been described that the spam detection technique includes a statistics-based technique, a pattern matching technique, or a Bayesian technique. However, any method capable of detecting spam may be applied. For example, the K-Nearest Neighbor technique, which finds documents similar to the evaluation document and adopts the classification of the document as the classification of the evaluation document, or defines a tree node as a branching point indicating a condition, and reaches the leaf node along each condition. There may be a decision tree technique that allows the classification of documents to be determined. In addition, a neural network technique or a technique using support vector machines (SVMs) may be included.

Therefore, it is to be understood that the embodiments described above are exemplary in all respects and not restrictive. The scope of the present invention is shown by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

As described above, according to the present invention, it is possible to increase the reproducibility of the spam document evaluation system by applying a plurality of spam detection techniques to detect spam of a predetermined document.

In addition, according to the present invention, the spam index of each spam detection technique can be assigned differently according to the accuracy rate of the plurality of spam detection techniques by continuously monitoring the spam document determination result, thereby increasing the accuracy rate together with the reproducibility of the spam document determination system. Also, since each spam detection technique is given a spam index, even if a new spam detection technique appears, it can be applied directly to the system by applying only the spam index to the spam detection technique, thereby increasing the expandability of the system.

In addition, according to the present invention, in the process of extracting a word from a predetermined document, word reassembly or index word extraction is additionally performed, thereby effectively blocking the intention of spam document producers to avoid being classified as spam documents. have.

Claims (19)

As a method of determining spam documents using a system for determining whether a document is spam, Extracting word or link information from the document; When the extracted word contains a special character, after removing the special character from the word and reassembling the word, or if the extracted word is longer than the reference value index index extraction to separate the word into a plurality of words step; Determining whether the document is spam by applying a plurality of spam detection techniques to the extracted information; Determining whether the document is spam by applying the plurality of spam detection techniques to the extracted information; Assigning the document a predetermined spam index assigned to each spam detection technique whenever the document is determined to be spam by each spam detection technique of the plurality of spam detection techniques; Determining the document as spam when the sum of the spam indexes is equal to or greater than a reference value; Calculating an accuracy rate of each spam detection technique based on the spam determination result; And Readjusting the spam index of each spam detection scheme according to the accuracy rate of each spam detection technique Spam document determination method comprising a. delete delete The method of claim 1, wherein the readjustment of the spam index, Spam document determination method characterized in that the weighting of the respective spam detection techniques in the order of the calculated accuracy is high. delete delete delete The method of claim 1, wherein the plurality of spam detection techniques include a statistics based technique, a pattern matching technique, or a Bayesian technique. The method of claim 8, wherein the statistics-based technique determines whether the document is spam by using the number of repetitions of the extracted information, the overlapping ratio of the extracted information, the length of the extracted information, or the frequency of actual writing of the document. Spam document determination method, characterized in that judging. 10. The method of claim 8, wherein the Bayesian technique uses one of a Naive-Bayesian technique and a Paul Graham's Bayesian technique. The method of claim 1, wherein the determining of the spam of the document comprises: Calculating probability values that the document including the extracted information is a spam document; Calculating difference values between the probability values and a predetermined reference value; And Determining whether or not the document is spam using the calculated difference values Spam document determination method comprising a. The method of claim 11, Calculating a combined probability value using n probability values according to the order of the difference values among the probability values; And determining that the document is spam when the combination probability is greater than or equal to a threshold. 13. A recording medium having recorded thereon a computer readable program for performing the method according to any one of claims 1, 4 and 8-12. A system for determining whether a document is spam. A word extracting unit which extracts word or link information from the document; A word reassembling unit for reassembling the extracted word after removing the special character when the word extracted by the word extracting unit includes the special character; An index word extracting unit for dividing the word into a plurality of words through index word extraction when the length of the word extracted by the word extracting unit is longer than a reference value; It is determined whether or not the document is spam by applying the plurality of spam detection techniques to the extracted information. Spam document detection unit for assigning a spam index to the document; A spam document determination unit for calculating a sum of the spam indices assigned to the document and determining the document as spam when the sum of the spam indices is equal to or greater than a reference value; An evaluator configured to calculate an accuracy rate of each spam detection technique based on a spam determination result determined by the spam document determiner; And A spam index readjustment unit which readjusts the spam index of each spam detection method according to the accuracy rate of each spam detection method calculated by the evaluator; Spam document determination system comprising a. delete delete delete delete delete

Images