JP2007265368A - Spam eliminating method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively eliminate a spam message or the like including an unauthorized link. <P>SOLUTION: A method of processing transmission messages in message distribution service, e-mail service or electronic bulletin board service comprises processes for extracting a link from received message contents, determining whether the link is a link destination registered in a black list when the link is included in the message contents, determining the appearance frequency of the link destination when the link is a link destination not registered in the black list, analyzing the page contents of the link destination to determine whether words and phrases registered in a forbidden word list are included when the appearance frequency is a predetermined value or more, and canceling the corresponding transmission message when the link destination is registered in the black list and when the words and phrases registered in the forbidden word list are included in the page contents of the link destination. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a spam removal method that removes spam messages, spam mails, spam posts, spam trackbacks, and the like that contain illegal links.

  A message delivery service using the Internet (a service for delivering a message to a specific party in real time by an application called a messenger), an e-mail service, an electronic bulletin board service, and the like are provided, and the user's usage is high.

  By the way, the other party's member ID, e-mail address, bulletin board ID, trackback URL (Uniform Resource Locator), etc. are obtained by some method, or automatically generated by appropriately combining alphanumeric characters, etc., and adult sites and harmful blogs (blog: It is a simple name for Web Log, and it is a problem that so-called spam messages, spam mails, spam posts, spam trackbacks, etc. including URL links to diary-like sites) are sent to the server in large quantities. ing. Trackback is a function that associates with other blog articles. A system with a trackback function issues a trackback URL for an article that accepts a trackback, and refers to that article (parent article). If you write an article (child article) on your blog, you can send a trackback PING to the trackback URL of the parent article, so that you can A URL link or the like is posted.

  Conventionally, in order to deal with such a case, for example, in a message distribution server, link destinations included in a message are totaled in real time, and an extremely high frequency is automatically discarded as a spam message. .

On the other hand, in Patent Document 1, in an e-mail system, a mail including a SPAM determination word and a mail with the same source IP (Internet Protocol) address exceeding a predetermined flow rate are determined to be spam mails, and distribution is rejected. Technology is disclosed.
JP 2005-208780 A

  As described above, in the conventional message delivery server, the method of automatically discarding a message with extremely high frequency of links included in a message as a spam message is discarded only with high frequency. Even if the target normal message is discarded, the message is not delivered to the other party who wants to convey it, and there is a problem that the serviceability deteriorates.

  On the other hand, the technique described in Patent Document 1 has the same problem as that described above in that mail with the same source IP address exceeding a predetermined flow rate (corresponding to the above-described frequency) is determined as spam mail. Have. In addition, it has been added that e-mails containing SPAM judgment words are determined to be spam mails, but in general, e-mails that contain links to adult sites, etc. have a small number of characters and the possibility of intentionally selecting words The probability of being a SPAM determination word is not so high.

  The present invention has been proposed in view of the above-described conventional problems, and the object of the present invention is to effectively remove spam messages, spam mails, spam posts, spam trackbacks, and the like containing illegal links. It is to provide a spam removal method.

  In order to solve the above problems, according to the present invention, there is provided a processing method for a transmission message in a message distribution service, an e-mail service, an electronic bulletin board service, or the like, as described in claim 1. A step of extracting a link from the message content, a step of determining whether or not a link destination is registered in the black list when the link is included in the message content, and a link not registered in the black list. A step of determining the appearance frequency of the link destination if it is the destination, and a phrase registered in the prohibited word list by analyzing the page contents of the link destination if the appearance frequency is a predetermined value or more Determining whether or not the link destination is registered in the black list, and the phrase registered in the prohibited word list If included in the page content of the link destination, spam removal method and a discarding step of transmitting a message corresponding is the gist.

  Further, as described in claim 2, in the spam removal method according to claim 1, when the message content does not include a link, the appearance frequency is not equal to or higher than a predetermined value, and the prohibited word If the list does not include a registered phrase, a process for delivering, transmitting, or posting the corresponding transmission message as a normal process can be provided.

  Further, as described in claim 3, in the spam removal method according to claim 1 or 2, the link destination including a word / phrase registered in the prohibited word list is changed to the black address. A step of additionally registering in the list can be provided.

  Further, as described in claim 4, in the spam removal method according to any one of claims 1 to 3, when a link is included in the message content, the link destination in a plurality of transmission messages to be processed The step of listing the appearance frequency of the link to can be provided.

  Further, as described in claim 5, in the spam removal method according to any one of claims 1 to 4, the target service sets a proxy trackback URL for all articles to be displayed. This is an electronic bulletin board service that issues and receives trackback PING addressed to the proxy trackback URL of an article, displays an article on the sending blog in association with the corresponding parent article based on the received trackback PING, and spams all articles It can be determined whether or not.

  Further, as described in claim 6, a server for processing a transmission message in a message delivery service, an e-mail service, an electronic bulletin board service or the like, and means for extracting a link from the received message content; When a link is included in the message content, a means for determining whether or not the link destination is registered in the black list, and when the link destination is not registered in the black list, the link destination appears. Means for determining the frequency, means for analyzing whether or not the word registered in the prohibited word list is included by analyzing the contents of the linked page when the appearance frequency is a predetermined value or more, and the black If the link destination is registered in the list, and the phrase registered in the prohibited word list is included in the page contents of the link destination If there are, it may be configured as and a means for discarding the corresponding transmission message server.

  According to a seventh aspect of the present invention, there is provided a program for processing a transmission message in a message distribution service, an e-mail service, an electronic bulletin board service or the like, and the computer extracts a link from the received message content. Means, if the message content contains a link, means for determining whether the link destination is registered in the black list, and if the link destination is not registered in the black list, the link Means for determining the appearance frequency of the destination, and means for determining whether or not the word registered in the prohibited word list is included by analyzing the page contents of the link destination when the appearance frequency is equal to or higher than a predetermined value. When the link destination is registered in the black list, and the phrase registered in the prohibited word list is the link destination If included in the page content may be configured as a program to function as a means for discarding the corresponding transmission message.

  According to the spam removal method of the present invention, it is possible to effectively remove spam messages, spam mails, spam posts, spam trackbacks, and the like including illegal links.

  Hereinafter, preferred embodiments of the present invention will be described.

  FIG. 1 is a diagram showing a configuration example of a message delivery system according to an embodiment of the present invention. Although the message distribution service will be described as an example, the present invention can be similarly applied to an electronic mail service, an electronic bulletin board service, and the like.

  In FIG. 1, a message distribution system includes a message management server 1 connected to a network 2 such as the Internet, a PC (Personal Computer), a PDA (Personal Digital Assistant), a mobile phone, etc. that can access the network 2 by wire or wirelessly. A plurality of clients 3 are included. When a message is transmitted from a certain client 3 specifying the member ID of the other party, the message is delivered to the client 3 of the other party via the message management server 1.

  FIG. 2 is a diagram illustrating a configuration example of the message management server 1.

  In FIG. 2, the message management server 1 includes a communication control unit 101 that controls communication with the network 2, a message buffer unit 102 that temporarily stores messages received by the communication control unit 101, and a predetermined condition. And a message transfer unit 103 that transfers the message stored in the message buffer unit 102 to the other party.

  The message management server 1 also extracts a message receiving unit 104 that receives messages stored in the message buffer unit 102 in units of one message, and extracts link destination information such as a URL from the body of the message received by the message receiving unit 104. And an appearance frequency listing unit 106 that registers the appearance frequency of the link destination extracted by the link extraction unit 105 in the listing table 107 and manages the listing (in order of appearance frequency). The listing table 107 stores information about a plurality of link destinations with a pair of link destination information and the number of appearances.

  Further, the message management server 1 includes a black list collating unit 108 for collating whether the link destination extracted by the link extracting unit 105 is registered in the black list 109, and the black list collating unit 108 registering the link destination in the black list 109. An appearance frequency determining unit 110 that acquires the appearance frequency from the appearance frequency listing unit 106 for each link destination and determines whether or not a predetermined value is exceeded is provided. In the black list 109, a plurality of link destination information extracted from past spam messages and the like are registered.

  The message management server 1 also includes a link destination page access unit 111 that performs page access by making a page request to a link destination whose appearance frequency exceeds a predetermined value in the appearance frequency determination unit 110, and HTML ( Hypertext Markup Language) is used to analyze the contents described in the page analysis unit 112 to extract words and phrases, and prohibited word matching to check whether the words extracted in the page analysis unit 112 are registered in the prohibited word list 114 Unit 113 and a message discard unit 115 that discards the message stored in the message buffer unit 102 under a predetermined condition. In the forbidden word list 114, a plurality of obscene words and the like that are likely to be used on adult sites and the like are registered.

  FIG. 3 is a flowchart showing a processing example when receiving a message in the message management server 1.

  In FIG. 3, when the message reception unit 104 receives a message (step S1), the link extraction unit 105 extracts a link destination from the message body (step S2). Then, it is determined whether or not the message includes a link (step S3). When the message does not include a link (No in step S3), the message transfer unit 103 stores the target message stored in the message buffer unit 102 as a normal process. Is transferred via the communication control unit 101 (step S4), and the process is terminated (step S16).

  If the message includes a link (Yes in step S3), the link extraction unit 105 registers the appearance frequency listing unit 106 to be a listing target (step S5), and the appearance frequency listing unit 106 lists the listing table 107. Update. That is, the appearance frequency listing unit 106 registers the link destination at the lowest position of the listing table 107 as the number of appearances for the first link destination, and increments the number of appearances of the corresponding link destination in the case of multiple times. The appearance frequency is grasped according to the number of times.

  Next, the blacklist collating unit 108 collates the link destination with the blacklist 109 (step S6), and determines whether or not it is registered in the blacklist 109 (step S7). If the message is registered in the black list 109 (Yes in step S7), the message discard unit 115 discards the target message stored in the message buffer unit 102 (step S15), and the process ends (step S16). .

  If not registered in the black list 109 (No in step S7), the appearance frequency determination unit 110 refers to the appearance frequency of the target link destination from the appearance frequency listing unit 106 (step S8) and is within the upper predetermined order. Whether or not (step S9). Here, when it is not in the upper predetermined order (No in step S9), the message transfer unit 103 transfers the target message stored in the message buffer unit 102 as a normal process via the communication control unit 101 (step S4), and the process Is finished (step S16).

  In addition, when it is within the upper predetermined order (Yes in Step S9), the link destination page access unit 111 performs page access by making a page request to the link destination (Step S10), and the page analysis unit 112 The contents of the accessed page are analyzed and words are extracted (step S11).

  Next, the prohibited word collating unit 113 collates the extracted word with the prohibited word list 114 (step S12), and determines whether it is registered in the prohibited word list 114 (step S13). Here, when not registered in the prohibited word list 114 (No in step S13), the message transfer unit 103 transfers the target message stored in the message buffer unit 102 as a normal process via the communication control unit 101 (step S13). S4), the process ends (step S16).

  When the prohibited word list 114 is registered in the prohibited word list 114 (Yes in step S13), the prohibited word collating unit 113 registers the link destination including the word in the black list 109 (step S14) and the message discarding unit 115. Thus, the target message stored in the message buffer unit 102 is discarded (step S15), and the process is terminated (step S16).

  In this way, because the received message controls whether to forward or discard the message depending on the presence or absence of a link, the presence or absence of a blacklist, the appearance frequency of a link destination, the presence or absence of a prohibited word, a large number of people It is possible to effectively remove a spam message and the like without unjustly discarding a normal message targeted at.

  Next, an embodiment in which the present invention is applied to a bulletin board system will be described.

  FIG. 4 is a diagram showing an example of the bulletin board system. In the bulletin board server 10, the blog articles A and C that have transmitted the track back PING to the track back URL (proxy track back URL) of the main theme are displayed as child articles of the main theme. Blog article B that sent trackback PING to the trackback URL of article A is displayed as a child article (grandchild article of the main theme) of blog article A, and blog articles D and E that sent trackback PING to the trackback URL of blog article C A blog article F that is displayed as a child article of the blog article C (a grandchild article of the main theme) and that sends a trackback PING to the trackback URL of the blog article E is displayed as a child article of the blog article E (a grandchild article of the main theme) Indicates the state. In addition to the parent article, proxy trackback URLs are issued for all articles, and the articles that have received the trackback PING are hierarchically displayed as child articles of the article. Parent article, child article, grandchild article ... and multiple generations of trackback reception display are possible, and display can be performed without impairing the continuity and integrity of articles developed from one main theme.

  FIG. 5 is a diagram showing a configuration example of the bulletin board system. The bulletin board server 10 is connected to a network 2 such as the Internet, and the network 2 includes clients 31, 32... And other blog systems 41, 42.・ And are connected.

  The bulletin board server 10 accepts write data to the bulletin board, accepts trackback PING, outputs bulletin board display data, inputs / outputs data to / from the data storage unit 12, and saves write data to the bulletin board, etc. Access to the data storage unit 12 that performs the processing, the proxy trackback URL issuing unit 13 that issues the proxy trackback URL to the article on the bulletin board, and the blog system of the original article that has transmitted the trackback PING to the article on the bulletin board The trackback URL extracting unit 14 for extracting the trackback URL of the original article and the trackback URL of the original article stored when the trackback PING is received for the article of another blog posted on the bulletin board by the trackback. Trackback P against Whether the trackback PING proxy transmission unit 15 that transmits NG and the article written by the clients 31, 32, etc. to the bulletin board server 10 or the original article tracked back by the blog system 41, 42, etc. is spam And a spam removing unit 16 for removing the spam if it is spam.

  The internal configuration of the spam removal unit 16 includes the message reception unit 104, the link extraction unit 105, the appearance frequency listing unit 106, the listing table 107, the black list matching unit 108, the black list 109, and the appearance frequency determination unit 110 shown in FIG. , The linked page access unit 111, the page analysis unit 112, the prohibited word collating unit 113, the prohibited word list 114, and the message discarding unit 115. The data storage unit 12 in FIG. 5 corresponds to the message buffer unit 102 in FIG. 2, and the data processing unit 11 in FIG. 5 corresponds to the communication control unit 101 and message transfer unit 103 in FIG. , Except that the message (article) is displayed without forwarding the message).

  FIG. 6 is a diagram showing an example of the structure of stored data recorded in the data storage unit 12 of the bulletin board server 10. As stored data, an “article ID” field for identifying an article in the bulletin board server 10, a “parent article ID” field for identifying a parent article when a parent article exists in the article, a written article or a blog article An “article data” field in which the summary is recorded, a “trackback URL” field in which the trackback URL of the original article when the article is tracked back from another blog system, and a proxy issued for the article A “proxy trackback URL” field in which the trackback URL is recorded is provided.

  FIG. 7 is a flowchart showing an example of processing at the time of article writing in the bulletin board server 10.

  In FIG. 7, when the data processing unit 11 receives article data from a client (31, 32, etc.) (step S101), the data processing unit 11 stores the article data in the data storage unit 12 (step S102). Specifically, the article data is stored with a unique article ID in the data structure of FIG. Further, when it is written as a child article for a specific parent article, it is stored in the data storage unit 12 in association with the parent article. Specifically, in the data structure of FIG. 6, the article ID of the parent article is set in the “parent article ID” field to associate with the parent article.

  Returning to FIG. 7, the spam removal unit 16 determines whether or not the written article is spam (step S103). If it is determined that the article is spam (Yes in step S104), the data storage unit 12 reads the article data. Discard it (step S105) and end the process. The determination of whether or not it is spam is the same as the processing shown in FIG. 3 (steps S2, S3, S5 to S14).

  Returning to FIG. 7, when it is determined that it is not spam (No in step S104), the proxy trackback URL issuing unit 13 issues a proxy trackback URL to the written article (step S106), and the proxy trackback URL is set as article data. And stored in the data storage unit 12 (step S107). Specifically, in the data structure of FIG. 6, the issued proxy trackback URL is set in the “proxy trackback URL” field of the corresponding record.

  Returning to FIG. 7, the data processing unit 11 hierarchically displays the latest articles according to the data stored in the data storage unit 12 (step S108).

  Thus, if an article written on the bulletin board is spam, it can be effectively removed.

  FIG. 8 is a flowchart showing an example of processing at the time of trackback reception in the bulletin board server 10.

In FIG. 8, when the data processing unit 11 receives the trackback PING from the blog system (41, 42, etc.) (step S201), the data processing unit 11 stores the article data in the data storage unit 12 in association with the parent article (step S202). FIG. 9 is a diagram showing an example of data included in the trackback PING, including Title (blog article title), Excerpt (blog article summary), url (blog article permalink), and blog_name (blog name). These are recorded in the “article data” field of the data structure of FIG. Permalink is a URI (Uniform) that does not change even if the article content is updated.
Resource Indicator), and one URI exists in one article.

  Returning to FIG. 8, the spam removal unit 16 determines whether or not the written article is spam (step S203). If it is determined that the article is spam (Yes in step S204), the data storage unit 12 reads the article data. Discard it (step S205) and end the process. The determination of whether or not it is spam is the same as the processing shown in FIG. 3 (steps S2, S3, S5 to S14). The links included in the message (article) include a URL link to the original article that has been tracked back and a URL link included in the summary of the article.

Returning to FIG. 8, when it is determined that the message is not spam (No in step S204), the trackback URL extraction unit 14 accesses the blog system that has transmitted the trackback PING and extracts the trackback URL of the original article (step S206). That is, by accessing the url shown in FIG. 9, the HTML data of the original article as shown in FIG. 10 is acquired, and the RDF (Resource
Description Framework) A trackback URL (“http://xxxx.jp/yyyyyy/tb/1234567890” in this example) described in the element type name “trackback: ping” is extracted from the data.

  Returning to FIG. 8, the proxy trackback URL issuing unit 13 issues a proxy trackback URL for the article that has been tracked back (step S207), and associates the trackback URL of the original article that has already been extracted and the issued proxy trackback URL with the article data. The data is stored in the data storage unit 12 (step S208). Specifically, in the data structure of FIG. 6, the trackback URL of the original article is set in the “trackback URL” field of the corresponding record, and the issued proxy trackback URL is set in the “proxy trackback URL” field.

  Returning to FIG. 8, when the parent article to be tracked back is a blog article (Yes in step S209), the trackback PING proxy transmission unit 15 tracks back the trackback URL of the original article stored in the data storage unit 12. PING is transmitted (step S210). This is because, when a user sends a trackback PING to a blog article posted by a trackback on the bulletin board server 10, the user should send the trackback PING to the blog system with the original blog article. Since it is troublesome to transmit the trackback PING, the bulletin board server 10 automatically substitutes it.

  Then, the data processing unit 11 hierarchically displays the latest articles according to the data stored in the data storage unit 12 (step S211).

  Thus, when the articles tracked back on the bulletin board are spam, not only the parent articles but also the child articles can be effectively removed.

  The present invention has been described above by the preferred embodiments of the present invention. While the invention has been described with reference to specific embodiments, various modifications and changes may be made to the embodiments without departing from the broad spirit and scope of the invention as defined in the claims. Obviously you can. In other words, the present invention should not be construed as being limited by the details of the specific examples and the accompanying drawings.

It is a figure which shows the structural example of the message delivery system concerning one Embodiment of this invention. It is a figure which shows the structural example of a message management server. It is a flowchart which shows the process example at the time of message reception. It is a figure which shows the example of a bulletin board system. It is a figure which shows the structural example of the bulletin board system concerning other embodiment of this invention. It is a figure which shows the example of a structure of preserve | saved data. It is a flowchart which shows the process example at the time of article writing. It is a flowchart which shows the process example at the time of trackback reception. It is a figure which shows the example of the data contained in track back PING. It is a figure which shows the example of the RDF data contained in HTML data.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Message management server 101 Communication control part 102 Message buffer part 103 Message transfer part 104 Message reception part 105 Link extraction part 106 Appearance frequency listing part 107 Listing table 108 Blacklist collation part 109 Blacklist 110 Appearance frequency judgment part 111 Link destination page access Unit 112 page analysis unit 113 prohibited word collating unit 114 prohibited word list 115 message discarding unit 10 bulletin board server 11 data processing unit 12 data storage unit 13 proxy trackback URL issuing unit 14 trackback URL extracting unit 15 trackback PING proxy transmission unit 16 spam removing unit 2 Network 3, 31, 32 Client 41, 42 Blog system

Claims (7)

A method of processing a transmission message in a message delivery service, an e-mail service, or an electronic bulletin board service,
Extracting a link from the received message content;
A step of determining whether or not a link destination is registered in a black list when a link is included in the message content;
A step of determining an appearance frequency of the link destination when the link destination is not registered in the black list;
When the appearance frequency is a predetermined value or more, analyzing the link destination page content and determining whether or not a word or phrase registered in the prohibited word list is included;
And a step of discarding a corresponding transmission message when a link destination is registered in the black list, and when a phrase registered in the prohibited word list is included in the page content of the link destination. Spam removal method characterized by this. The spam removal method according to claim 1,
When the message content does not include a link, when the appearance frequency is not equal to or higher than a predetermined value, and when a word or phrase registered in the prohibited word list is not included, a corresponding transmission message is delivered as a normal process. A spam removal method comprising a step of transmitting or posting. In the spam removal method according to any one of claims 1 and 2,
A spam removing method comprising a step of additionally registering a link destination including a phrase registered in the prohibited word list in the black list. In the spam removal method according to any one of claims 1 to 3,
A spam removal method comprising a step of listing the frequency of appearance of a link to a link destination in a plurality of transmission messages to be processed when a link is included in the message content. In the spam removal method according to any one of claims 1 to 4,
The target service issues a proxy trackback URL for all articles to be displayed, accepts a trackback PING addressed to the proxy trackback URL of the article, and responds to an article on the transmitting blog based on the received trackback PING. An electronic bulletin board service that displays in association with the parent article,
A spam removal method characterized by determining whether or not all articles are spam. A server for processing a transmission message in a message distribution service, an e-mail service, or an electronic bulletin board service,
Means for extracting links from the received message content;
Means for determining whether a link destination is registered in the blacklist when a link is included in the message content;
Means for determining the appearance frequency of the link destination when the link destination is not registered in the blacklist;
Means for analyzing whether or not the word registered in the prohibited word list is included by analyzing the contents of the linked page when the appearance frequency is a predetermined value or more;
Means for discarding a corresponding transmission message when a link destination is registered in the black list, and when a phrase registered in the prohibited word list is included in the page content of the link destination; A server characterized by A program for processing a transmission message in a message delivery service, an e-mail service, or an electronic bulletin board service,
Computer
Means for extracting links from the received message content;
Means for determining whether a link destination is registered in the blacklist when a link is included in the message content;
Means for determining the appearance frequency of the link destination when the link destination is not registered in the blacklist;
Means for analyzing whether or not the word registered in the prohibited word list is included by analyzing the contents of the linked page when the appearance frequency is a predetermined value or more;
When the link destination is registered in the black list, and when the phrase registered in the prohibited word list is included in the page content of the link destination, it is made to function as a means for discarding the corresponding transmission message. A featured program.

Images