JP5775508B2 - Spam account extraction apparatus and spam account extraction method - Google Patents

Description

  The present invention relates to an extraction apparatus and an extraction method for extracting a spam account.

  Conventionally, a communication tool such as a miniblog (microblog), which is a kind of blog, is used in a service for posting a document to a server on a network such as the Internet. Users post documents that write various things such as daily events and feelings. The posted document becomes stream data and is analyzed and utilized for various uses in many fields such as business and entertainment as reflecting public opinion.

  As an example of the use of stream data, there is a service that analyzes the appearance frequency of words appearing in stream data, uses a word whose appearance frequency is rising as a topic word, and extracts the topic word.

  Due to their characteristics, topical words are often placed in conspicuous positions on a mini-blog site or a linked site, so the number of browsing tends to increase. Therefore, paying attention to this trend, spam postings including one-sided advertisements to unspecified many people such as links that lead to discussions about topic words and lead to their own site are rampant.

  As a prior art for dealing with these spam posts, a method for eliminating spam mails and spam mails (for example, see Patent Document 1) and a method for identifying spammers (accounts for sending spam) using machine learning techniques (for example, And non-patent document 1).

  As a method of eliminating spam mails and spam mails, a keyword that serves as a criterion for rejecting transmission of mail transmitted from the transmission side to the reception side is managed by the server. Then, when the registered keyword is included in the mail body to be transmitted from the transmission side to the reception side, spam mail and spam mail are not distributed to the reception side by discarding the mail.

  A method of identifying a spammer using a machine learning method is a method that pays attention to a feature that a spammer on a spam transmission side forms an unfair link. That is, first, a fictitious user account is created, and then a spammer is determined from the accounts that have transmitted the link formation application to the fictitious user account. Further, by using a machine learning method, the spammers are classified from the behaviors that are seen in the account determined to be a spammer.

JP 2004-133508 A

K. Lee, J.M. Cavellee, and S.C. Webb. “Uncovering social spammers: Social honeypots + machine learning”, SIGIR, 2010.

  However, since the method described in Patent Document 1 determines whether or not the mail is a spam mail based on the keyword included in the mail text, the keyword registered in the mail text is included. If it is not, it is difficult to determine even an email targeted at a topic word. For this reason, if the registered keyword is not included in the posted document, this document may not be determined as a spam document.

  Moreover, since the method described in the said nonpatent literature 1 uses the detailed information regarding the action of each account for determination of a spam account, it is difficult to make all accounts into object. Furthermore, in this method, when trying to improve the accuracy of spam determination, there is a possibility that an erroneous determination (false negative) may occur if the user is not a spammer even though he is actually a spammer.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to narrow down documents to be determined as spam or not and efficiently extract spam accounts from simple features.

The spam account extracting apparatus according to the present invention is a topic that has a particular topic of words that appear in the stream data composed of documents , and that appears frequently in a service for posting documents to a server on a network. A topic word collection unit that collects as a topic word, a search unit that searches for a document that includes at least one topic word with respect to a posted document, and a document that is searched by the search unit includes a certain number of topic words or more If, obtain Preparations and spam documents and determining spam document judging section documents, based on the frequency of spam document is posted, an account posting spam documents spam account and determining spam account determination unit.

According to the spam account extraction device of the present invention, topic words that are specific topics are collected by the topic word collection unit, and at least one word collected by the topic word collection unit from the documents posted by the search unit. Documents including the above are retrieved. After that, the spam document determination unit determines whether the posted document is a spam document. If the posted document is a spam document, the spam account determination unit determines that the account that posted the document is a spam account. Is done. For this reason, the document to be searched is narrowed down based on the topic word, and on the other hand, it is determined whether the narrowed document is a spam document. An account that posts a document determined to be a spam document a certain number of times or more is determined as a spam account. Therefore, it becomes possible to extract spam accounts efficiently and exhaustively. Furthermore, it is possible to determine a document including a certain number of topic words of a plurality of types in the same document as a spam document. For this reason, it becomes possible to extract a spam account more efficiently.

  The spam account extraction device further includes an inappropriate URL determination unit that determines a URL as an inappropriate URL when the URL included in the document includes a link to an inappropriate site, and the spam document determination unit includes: If the URL is determined to be inappropriate, the document may be determined as the spam document.

  With this configuration, a document that includes a link to an inappropriate site in the document can be determined as a spam document. For this reason, it becomes possible to extract a spam account more efficiently.

  In the spam account extracting device, the inappropriate URL determination unit may determine that the URL is an inappropriate URL when the URL or the URL to which the URL is redirected includes a predefined character string.

  If comprised in this way, an inappropriate URL determination part will determine beforehand from URL that it is an inappropriate site, such as an affiliate or an adult site, for example. For this reason, since a document including an inappropriate URL can be determined as a spam document, a spam account can be extracted more efficiently.

  In the spam account extracting device, the inappropriate URL determination unit may determine that the URL is an inappropriate URL when the URL or the document to which the URL to which the URL is redirected includes a predefined word. Good.

  With this configuration, the inappropriate URL determination unit determines that the transition source URL is an inappropriate URL when the URL included in the document or the document to which the URL redirect destination URL transitions includes an inappropriate word. To do. For this reason, since a document including an inappropriate URL can be determined as a spam document, a spam account can be extracted more efficiently.

  In the spam account extraction device, the inappropriate URL determination unit may include a link from a document including a plurality of unrelated topic words when the document in which the URL or the URL to which the URL is redirected changes. The URL may be determined as an inappropriate URL.

  With this configuration, the inappropriate URL determination unit determines the transition source URL when the document in which the URL or the URL to which the URL is redirected has a link from a document including a plurality of unrelated topic words. It is determined as an inappropriate URL. For this reason, since a document including an inappropriate URL can be determined as a spam document, a spam account can be extracted more efficiently.

  In the spam account extracting apparatus, the inappropriate URL determination unit may determine that the URL or the URL to which the URL is redirected does not include the topic word used in the document. May be determined.

  With this configuration, the inappropriate URL determination unit determines that the transition source URL is inappropriate when the URL or the document to which the URL to which the URL is redirected does not include a topic word used in the transition source document. The URL is determined. For this reason, since a document including an inappropriate URL can be determined as a spam document, a spam account can be extracted more efficiently.

  In the spam account extraction device, the inappropriate URL determination unit may determine that an URL is an inappropriate URL if the document to which the URL or the URL to which the URL is redirected includes a plurality of unrelated topic words. You may judge.

  If comprised in this way, an inappropriate URL determination part will determine URL as inappropriate URL, when the document to which URL or URL of the redirect destination of URL contains a plurality of unrelated topic words. For this reason, since a document including an inappropriate URL can be determined as a spam document, a spam account can be extracted more efficiently.

In addition, the spam account extracting device described above includes a friend list acquisition unit that acquires a list of accounts that can view documents posted by the first account, and a friend list of the first account acquired by the friend list acquisition unit. When, further comprising a and account inter similarity calculation unit for calculating a list of accounts that are viewable documents account identified as spam accounts posted, the similarity by spam account determination unit, spam account determination unit May determine the spam account based on the similarity.

If comprised in this way, a friend list acquisition part will acquire the list of the accounts which can browse the document which the 1st account contributed. The inter-account similarity calculation unit can browse a list of accounts that can be viewed by the account determined as a spam account by the spam account determination unit and a document posted by the first account. Calculate the similarity between the list of accounts . For this reason, if the spam account is extracted, it is possible to first account to determine whether spam account efficiently. Therefore, it becomes possible to extract spam accounts efficiently and exhaustively.

  Further, the friend list acquisition unit extracts a spam account extraction unit that extracts, from the friend list of the first account, accounts whose similarity exceeds a certain threshold based on the similarity between accounts calculated by the inter-account similarity calculation unit. May be included.

  If comprised in this way, the spam account extraction part can extract an account with high similarity with a spam account from the list of accounts which the friend list acquisition part acquired based on the posting frequency of a document. Therefore, it becomes possible to extract spam accounts more accurately, efficiently and comprehensively.

  In addition, the spam account extraction device includes a document index generation unit that generates a document index based on a determination result of the spam account, and a browsing unit that allows browsing only from accounts that are not spam accounts. Further, it may be provided.

  If comprised in this way, a document index production | generation part will produce | generate the index of a document based on the determination result which a spam account determination part determines, and a browsing means will be able to browse only the posting from the account which is not a spam account. . For this reason, the browsing means can prevent the posting of the spam account from being displayed in the document search result based on the document index.

  By the way, the invention related to the spam account extraction apparatus described above can be obtained as an invention of a spam account extraction method, and has the same operations and effects. The spam account extraction method can be described as follows.

Spam account extraction method according to the present invention, a topic word collecting step of collecting a word occurrence frequency of the word appearing in the stream data composed of a document is increased as the topic word has become a particular topic, post A search step for searching for a document including at least one topic word from the retrieved document, and a spam document determination for determining a document as a spam document when the document searched by the search step includes a certain number of topic words or more a method, based on the frequency of spam document is posted, it constructed example Preparations and spam account determination step of determining an account and spam account to post spam documents, the.

  According to the present invention, it is possible to narrow down documents to be judged as spam or not and efficiently extract a spam account from simple features.

It is a functional block diagram of the spam account extraction device concerning a 1st embodiment. It is a flowchart which extracts the spam account in the spam account extraction device concerning a 1st embodiment. It is a flowchart which determines the spam document in the spam account extraction device concerning a 1st embodiment. It is a flowchart which determines the spam account in the spam account extraction device concerning a 1st embodiment. It is a functional block diagram of the spam account extraction device concerning a 2nd embodiment. It is a functional block diagram of the inappropriate URL determination part with which the spam account extraction device concerning a 2nd embodiment is provided. It is a flowchart which determines inappropriate URL in the spam account extraction apparatus which concerns on 2nd Embodiment. It is a functional block diagram of the spam account extraction device concerning a 3rd embodiment. It is a flowchart which extracts the spam account in the spam account extraction device concerning a 3rd embodiment. It is a functional block diagram of the spam account extraction device concerning a 4th embodiment. It is a flowchart which controls document browsing in the spam account extraction device concerning a 4th embodiment. It is the schematic which shows the structure of a document index storage part. It is the schematic which shows the structure of a document index storage part. It is the schematic which shows the structure of an inappropriate URL storage part. It is the schematic which shows the structure of a spam document storage part. It is the schematic which shows the structure of a spam user storage part. It is the schematic which shows the structure of an inappropriate word storage part. It is the schematic which shows the structure of a reference origin topic word storage part. It is the schematic which shows the structure of a determination result storage part. It is a figure which shows an example of the hardware constitutions of a spam account extraction apparatus.

  Hereinafter, embodiments of the present invention will be specifically described with reference to the accompanying drawings. However, the present invention is not limited to the following embodiments. In addition, the same code | symbol is attached | subjected to the element which is the same or it corresponds, and the overlapping description is abbreviate | omitted.

(First embodiment)
FIG. 1 is a functional block diagram of the spam account extraction apparatus according to the first embodiment. The spam account extraction device 1 is preferably applied to a device that posts a document to a server on a network such as the Internet.

  As shown in FIG. 1, the spam account extraction apparatus 1 of the present embodiment mainly includes a determination unit 10 </ b> A and a storage unit 20. The determination unit 10A includes a topic word collection unit 11, a search unit 12, a URL expansion unit 13, a spam document determination unit 14, and a spam account determination unit 15. The storage unit 20 includes a document index storage unit 21, an inappropriate URL storage unit 22, a spam document storage unit 23, and a spam user storage unit 24. Further, the determination unit 10 </ b> A may include a document index generation unit 16. The function of each part is outlined below.

The topic word collection unit 11 collects a list of topic words currently being discussed. The search unit 12 searches for a document including at least one topic word W and collects a set of documents including the topic word W. The URL developing unit 13 once accesses the URL in the text, and accesses the redirect destination again while being redirected. Spam document determination unit 14, each document was collected to determine whether it is spam submissions. The spam account determination unit 15 determines an account that posts a spam document as a spam account based on the frequency with which the spam document is posted. The document index generation unit 16 registers the spam account extraction result in the index so that spam posts are not displayed in the search result.

  In the present embodiment, two types of spam-type posts are assumed. One is posting by an account that automatically posts a list of topic words, search suddenly rising words, etc. periodically, and is hereinafter referred to as trend-type posting. The trend-type posting has a feature that a plurality of topic words are included in the same document because the topic words are ranked and displayed. Although the trend-type posting is not strictly a spam document, it is a document that is unnecessary from the viewpoint of extracting the discussion on the topic word targeted by the present invention, and is therefore classified as spam in this embodiment.

  Another type of spam-type post is a post containing a link to a specific URL defined in advance in addition to a topic word, and is hereinafter referred to as a URL-type post. The URL type posting is basically for the purpose of guiding to its own site or affiliate program. For this purpose, two types have been confirmed. One is a product sales site that has posted a PR of a product that contains a topic word as a keyword, and the other is that it is completely irrelevant if it actually transitions to a link destination while pretending to contain a data source related to the topic word. It is guided to the transition destination. URL-type postings are also seen in the comment field of blog sites and are similar to the existing spam, but it is difficult to discriminate from the text information of the document itself, the URL is shortened, and the amount of documents is Due to its enormous volume, it is practically difficult to investigate all documents.

  As shown in FIGS. 12 and 13, the document index storage unit 21 includes a document list 21a and a transposed index 21b. As shown in FIG. 12, the document list 21a of the document index storage unit 21 stores a document ID, a posting date, a user ID, and a text in association with each other. Further, as shown in FIG. 13, the transposed index 21b of the document index storage unit 21 stores a word and a document ID in association with each other. For this reason, it is possible to easily search for which document ID is used to manage each word.

  As shown in FIG. 14, the inappropriate URL storage unit 22 stores a list of URLs determined to be inappropriate.

  The spam document storage unit 23 stores the document ID and the account ID that made the posting for the document determined to be spam. In the spam document storage unit 23, as shown in FIG. 15, document ID, posting date / time, user ID, topic word, URL, URL type flag, and trend type flag are recorded in a one-to-one correspondence.

  The spam user storage unit 24 determines an account having a certain number or more of trend-type or URL-type posts as a spam account, and stores a determination result. As shown in FIG. 16, the spam user storage unit 24 stores a user ID determined to be a spam account.

  FIG. 20 shows an example of the hardware configuration of the spam account extraction apparatus 1. The spam account extracting device 1 stores, as a hardware configuration, a CPU 1A, a RAM 1B, a ROM 1C, an input unit 1D including a keyboard, a voice recognition device for voice input, and the like, and a storage medium M inserted at a predetermined position. A reading unit 1E that reads the read data and programs, a communication unit 1F that communicates with the outside, an auxiliary storage unit 1G, and a display unit 1H that displays an image including a spam determination result or a document search result. Prepare. The function of each functional block of the spam account extracting apparatus 1 described above is realized by causing the RAM 1B or the like to read a program or data and executing the program under the control of the CPU 1A. Note that the hardware configuration of the spam account extraction apparatus in the second to fourth embodiments described later is the same as the hardware configuration of FIG.

  Next, a flow for extracting a spam account in the first embodiment will be described with reference to FIG. FIG. 2 is a flowchart for extracting a spam account in the spam account extraction apparatus 1.

  As shown in FIG. 2, first, in step S <b> 11, the topic word collection unit 11 collects a topic word W that is currently a topic. Here, the topic word W may be generated from a time-series transition of information stored in the document index storage unit 21, or an external miniblog analysis service, a search keyword rapid increase ranking, or the like may be used.

  Next, in step S12, the search unit 12 uses the topic word W collected by the topic word collection unit 11 to search for a document including at least one topic word W, and collects a set of documents including the topic word W. To do. The number of documents searched by the search unit 12 is not limited as long as it includes an arbitrary one of the collected topic words W. Further, since there is no need to consider the suitability of a set of documents including the topic word W, the search unit 12 may search in the order of arrival. Further, the purpose is to comprehensively collect posts including the topic word W after each topic word W is released, but it is difficult to know the date and time when each word was released. The flow may be executed periodically, or only documents posted after a predetermined time may be collected.

  In step S13, the spam document determination unit 14 determines whether each collected document is a spam post. The spam document determination unit 14 determines a document as a spam document when the topic word W included in the document searched by the search unit 12 is not related to a specific topic. The flow for determining a document as spam will be described later with reference to FIG.

  In step S14, the spam account determination unit 15 determines that the account that posts the spam document is a spam account based on the frequency with which the spam document is posted. The spam account determination unit 15 determines an account that posts a document determined as a spam document by the spam document determination unit 14 as a spam account. A flow for determining a spam account for posting a spam document will be described later with reference to FIG.

  Furthermore, in the present embodiment, as shown in FIG. 1, a document index generation unit 16 may be provided. If comprised in this way, the document index production | generation part 16 will register the extraction result of a spam account to an index, and will prevent the posting of spam from being displayed on a search result.

  As shown in FIG. 2, in step S15, the document index generation unit 16 may register the spam account extraction result in the index (indexer) so that spam posts are not displayed in the search result.

  Here, with reference to FIG. 3, a flow for determining a document as spam in this embodiment will be described. FIG. 3 is a flowchart showing a flow of determining a spam document.

  The document spam determination in this embodiment (S13 in FIG. 2) follows the flow shown below based on information stored in the document index storage unit 21 and the inappropriate URL storage unit 22 by the spam document determination unit 14. Done.

  First, as shown in FIG. 3, in step S21, the spam document determination unit 14 determines whether or not a URL is included in the document. If the post includes a URL in the document, the URL is determined in step S22. The expansion unit 13 expands the URL.

  In general, in a mini blog service, when a posting including a URL is performed, a URL shortening service is often used together in order to simplify the relationship and display of the number of posted characters. The URL shortening service is a service that replaces an original URL with another shorter URL, and is provided by a plurality of businesses. The shortened URL is realized by HTTP redirection. For this reason, in the present embodiment, the URL in the text may be accessed once, the redirect destination may be accessed again while being redirected, and the URL may be updated each time.

  Next, in step S23, the spam document determination unit 14 determines whether or not a certain number of topic words are included in the same document (for example, five or more), and a certain number or more of topic words are included in the same document ( For example, in a case where five or more are included, the spam document determination unit 14 determines that the document to be determined is a spam document of the trend type posting in step S24.

  In step S25, the spam document determination unit 14 determines whether or not at least a part of the URL included in the document or the URL in the blacklist in which the redirect destination URL is predefined matches. If at least a part of the URL included in the document or the URL of the redirect destination matches the URL in the predefined black list, in step S26, the spam document determination unit 14 The document is determined to be a URL-type posted spam document.

  Further, when the URL or the URL to which the URL is redirected includes a predefined character string, the spam document determination unit 14 may determine that the determination target document is a spam document. Then, the spam document determination unit 14 records the URL determined to be inappropriate in the inappropriate URL storage unit 22. The URLs recorded by the spam document determination unit 14 may be recorded in the inappropriate URL storage unit 22 as a list. Alternatively, a transition destination URL such as an affiliate of an adult site or a product sales site may be registered in advance in the inappropriate URL storage unit 22.

  Then, the document determined to be spam is stored in the spam document storage unit 23 with the document ID and the account ID that made the posting. A single document may be classified into both a trend type and a URL type.

  Here, a flow for determining a spam account in the present embodiment will be described with reference to FIG. FIG. 4 is a flowchart showing a flow for determining a spam account.

  The spam account determination (S14 in FIG. 2) in the present embodiment is performed along the flow shown below based on the information stored in the spam document storage unit 23 by the spam account determination unit 15.

  First, as shown in FIG. 4, in step S31, the spam account determination unit 15 extracts the latest d days from the information stored in the spam document storage unit 23, and in step S32, the number of unique documents. For each account ID. Also, d days may be set to 7 days, for example.

  In the first embodiment, the spam account determination unit 15 may include document counting means (not shown) for counting the number of unique documents for each account ID. In step S32, the document counting means may count the number of unique documents.

  Next, in step S33, the spam account determination unit 15 makes a determination for each account based on the total number of posts. In step S34, the account having a certain number of trend-type or URL-type posts is determined as a spam account. The determination result is stored in the spam user storage unit 24.

  In step S35, the spam account determination unit 15 determines a spam account.

  The series of processes described above may be performed periodically, and the spam account (spam user) information may be constantly updated. Also, even if it is determined that the account is spam, it may be determined that the account is not a spam account if spam posting is not performed for d days. Alternatively, if spam posting is not performed for d days even if it is determined to be a spam account, the account may be set to be released from the list stored in the spam user storage unit 24.

  If comprised in this way, the topic word W used as the specific topic will be collected by the topic word collection part 11, and one or more words which at least the topic word collection part 11 collected among the documents posted by the search part 12 will be collected. The containing document is searched. The spam document determination unit 14 can determine whether the posted document is a spam document. Furthermore, if the posted document is a spam document, the spam account determination unit 15 can determine that the account that posted the document is a spam account.

  Therefore, it is possible to narrow down the documents to be searched based on the topic word W and determine whether the narrowed document is a spam document. In addition, a document including a certain number of plural types of topic words in the same document can be determined as a spam document. Furthermore, an account that posts a document determined to be a spam document a certain number of times or more can be determined as a spam account.

  For this reason, it becomes possible to extract spam accounts efficiently and exhaustively.

  In the first embodiment described above, the document index generation unit 16 is not an indispensable configuration for efficiently and comprehensively extracting spam accounts. In the first embodiment, by providing the document index generation unit 16, it is possible to process a post from an account determined to be a spam count so as not to be displayed in a search result. In the first embodiment described above, the determination of the spam account is not limited to the method described above.

(Second Embodiment)
FIG. 5 is a functional block diagram of the spam account extraction apparatus according to the second embodiment. The spam account extraction device 2 according to the second embodiment further includes an inappropriate URL determination unit 17 in the spam account extraction device 1 according to the first embodiment.

  FIG. 6 shows a functional block diagram of the inappropriate URL determination unit 17. The inappropriate URL determination unit 17 includes a determination unit 100 and a storage unit 200. The determination unit 100 includes a URL development unit 101, a web page acquisition unit 102, and a web page analysis unit 103. The storage unit 200 includes a storage unit 200 including an inappropriate word storage unit 201, a reference source topic word storage unit 202, and a determination result storage unit 203.

  The URL development unit 101 develops a shortened URL. The web page acquisition unit 102 acquires the text of the transition destination web page. The Web page analysis unit 103 checks whether or not inappropriate words stored in the reference source topic word storage unit 202 and the inappropriate word storage unit 201 appear in the acquired Web page body.

  As shown in FIG. 17, the inappropriate word storage unit 201 stores inappropriate words registered in advance. Further, the inappropriate word list 201a may be updated sequentially, or words may be stored as needed by learning. As shown in FIG. 18, the reference source topic word storage unit 202 stores the topic word W included in the transition source document in association with the posting date and URL. As shown in FIG. 19, the determination result storage unit 203 stores URLs determined to be inappropriate URLs.

  The inappropriate URL determination unit 17 determines that a URL corresponding to any one of the following conditions is an inappropriate URL and stores the URL in the determination result storage unit 203. Condition 1 includes an inappropriate word at the transition destination, Condition 2 includes no reference source topic word at the transition destination, Condition 3 refers to more than a certain topic word W, Condition 4 includes the transition destination This is a case where a plurality of unrelated topic words are included. At this time, the number of topic words W above a certain value may be five or more, for example.

  FIG. 7 is a flow for determining an inappropriate URL. First, in step S41, the URL development unit 101 develops a shortened URL. Next, in step S <b> 42, the determination unit 100 checks whether the expanded URL obtained by expanding the shortened URL is already registered in the determination result storage unit 203.

  If the URL is not already registered, in step S43, the determination unit 100 stores the URL and reference source topic word pair in the reference source topic word storage unit 202, and then the Web page acquisition unit 102 determines the transition destination Web page. Get the body of.

  In step S <b> 44, the determination unit 100 determines whether or not an inappropriate word stored in the reference source topic word storage unit 202 and the inappropriate word storage unit 201 appears in the Web page text acquired by the Web page analysis unit 103. To determine.

  Next, in step S <b> 47, the inappropriate URL determination unit 17 determines that the URL corresponding to any one of the following three conditions is an inappropriate URL and stores it in the determination result storage unit 203. Condition 1 is a case where an inappropriate word is included in the transition destination (step S44). Condition 2 is a case where the reference topic word is not included in the transition destination (step S45). Condition 3 is a case where reference is made from more than a certain number of topic words (for example, five or more), and condition 4 is a case where a plurality of unrelated topic words are included in the transition destination (step S48) (step S46). ).

  In the present embodiment, the determination in step S45 assumes that the transition destination does not include an inappropriate word, and that the topic word of the reference source is not closely related. It is determined that For this reason, even if a document including a URL that is not predefined is posted, it is possible to detect as a spam user.

  Further, in the present embodiment, the determination in step S46 is that the transition destination does not include an inappropriate word, and even if the transition destination includes a reference source topic word, a certain number or more of references It is determined that the URL referenced from the beginning is an inappropriate URL. For this reason, even if a document including a URL that is not predefined is posted, it is possible to detect as a spam user.

  Furthermore, in this embodiment, the determination in step S48 does not include an inappropriate word at the transition destination, the topic word of the reference source is included at the transition destination, and is referenced from a certain number of reference sources or more. Even if the URL is not set, if the transition destination includes a plurality of unrelated topic words, the URL is determined to be inappropriate. For this reason, even if a document including a URL that is not predefined is posted, it can be detected as a spam user.

  Further, the document index generation unit 16 refers to the determination result storage unit 203 in addition to the inappropriate URL storage unit 22, and determines whether to display or not when generating the document index.

  With this configuration, a document that includes a link to an inappropriate site in the document can be determined as a spam document. Further, the inappropriate URL determination unit 17 can determine in advance from the URL that the site is an inappropriate site such as an affiliate or an adult site, for example. The inappropriate URL determination unit 17 may determine that the transition source URL is an inappropriate URL if the URL included in the document or the URL redirect destination URL includes an inappropriate word. it can. Then, the inappropriate URL determination unit 17 determines the transition source URL as an inappropriate URL when the document in which the URL or the URL to which the URL is redirected has a link from a document including a plurality of unrelated topic words. Can be determined. Furthermore, the inappropriate URL determination unit 17 determines that the URL is an inappropriate URL even when the URL or the URL redirected document includes a plurality of unrelated topic words. it can.

  For this reason, since a document including an inappropriate URL can be determined as a spam document, a spam account can be extracted more efficiently.

(Third embodiment)
FIG. 8 is a functional block diagram of the spam account extraction apparatus according to the third embodiment. The spam account extraction device 3 according to the third embodiment further includes a friend list acquisition unit 18, a spam account extraction unit 181, and an inter-account similarity calculation unit 19 in addition to the spam account extraction device 2 according to the second embodiment.

  In general, friendships in the miniblog service include a bidirectional relationship that is established by one side approving an application from one side, and a state in which one user can unilaterally view the other user's post (hereinafter referred to as the other user's post). , Called follow). In consideration of the ease of understanding the explanation, in this embodiment, a user who is in a follow relationship is described as a user who is in a friend relationship.

  The friend list acquisition unit 18 acquires a list of friends of the user account stored in the spam user storage unit 24. The spam account extraction unit 181 extracts an account with a high similarity as a spam account from a list of other accounts related to the first account. The similarity calculation part 19 between accounts calculates the similarity of each spam account using a friend relationship. The storage unit 20 stores an account whose similarity exceeds a certain threshold in the spam user storage unit 24 as a spam user.

  FIG. 9 is a flow for extracting a spam account in the spam account extraction apparatus 3. That is, it is a flowchart showing a flow of processing for determining a spam account from a friend relationship.

  As shown in FIG. 9, in step S <b> 51, the friend list acquisition unit 18 acquires a friend list of an account determined to be a spam account from the spam user storage unit 24. In step S52, the spam account extraction unit 181 extracts an account having a high appearance frequency, that is, an account having many spam accounts as friends from the list of user accounts acquired by the friend list acquisition unit 18.

  Next, in step S53, the friend list acquisition unit 18 acquires the friend list of the user account extracted by the spam account extraction unit 181 in step S52.

Further, in step S54, the inter-account similarity calculation unit 19 calculates the similarity between the account extracted by the spam account extraction unit 181 in step S52 and the friend account of the account. In addition, the storage unit 20 stores an account whose similarity exceeds a certain threshold in the spam user storage unit 24 as a spam account. For example, a Jaccard coefficient may be used as the similarity index. Using the Jaccard coefficient, the similarity between the two accounts X and Y is evaluated by the following equation (1).

  Here, | X∩Y | is the number of accounts that X and Y have as friends in common, and | X∪Y | is the total number of accounts that have at least one of X and Y as friends.

  Alternatively, as a similarity index, an index representing another similarity such as a cosine distance or a Simpson coefficient may be used.

  If comprised in this way, when a 1st account is determined to be a spam account, another spam account can be extracted efficiently. That is, it is possible to determine a user account determined to be spam from a friend relationship on the miniblog service. Therefore, it becomes possible to extract spam accounts efficiently and exhaustively.

  In the above-described embodiment, the user who follows the direction in the friendship has been described as the friendship. However, the friendship in the present embodiment is not limited to the case of following, but is a two-way case. Also good. In the above-described embodiment, one user may own and manage one or more accounts.

(Fourth embodiment)
FIG. 10 is a functional block diagram of the spam account extraction device 4 according to the fourth embodiment. The spam account extraction device 4 according to the fourth embodiment further includes a document browsing control unit 30 in addition to the spam account extraction device 3 according to the third embodiment.

  The document browsing control unit 30 excludes browsing of documents that are determined to be posted from the spam account based on the document index generated by the document index generating unit 16.

  FIG. 11 is a flow for determining document browsing in the document browsing control unit 30.

  First, as shown in FIG. 11, in step S61, the document browsing control unit 30 acquires an index of a document.

  Next, in step S <b> 62, the document browsing control unit 30 acquires the search result for the spam account from the spam user storage unit 24. In step S63, the document browsing control unit 30 determines whether the document search result is a document posted from a spam account. In step S65, the document browsing control unit 30 determines that browsing is not possible when the search result is determined to be a document posted from the spam count. On the other hand, in step S66, when it is determined that the search result is not a document posted from the spam count, the document browsing control unit 30 determines that the document can be browsed.

  If comprised in this way, based on the index of the document which the document index production | generation part 16 produced based on the determination result of the spam account, the document browsing control part 30 will be able to browse only the posting from the account which is not a spam account. be able to. For this reason, the browsing means (not shown) can prevent the posting of the spam account from being displayed in the search result of the document displayed by the display unit 1H based on the document index.

  Each embodiment mentioned above shows an example of the spam account extraction device and spam account extraction method concerning the present invention, and is not restricted to the device and method concerning an embodiment, but changes, or others It may be applied to.

  1, 2, 3, 4 ... Spam account extraction device, 10A, 10B, 10C, 10D ... determination unit, 11 ... topic word collection unit, 12 ... search unit, 13 ... URL expansion unit, 14 ... spam document determination unit, 15 ... Spam account determination unit, 16 ... Document index generation unit, 17 ... Inappropriate URL determination unit, 18 ... Friend list acquisition unit, 19 ... Inter-account similarity calculation unit, 20 ... Storage unit, 21 ... Document index storage unit, 21a ... Document list, 21b ... Transposed index, 22 ... Inappropriate URL storage unit, 23 ... Spam document storage unit, 24 ... Spam user storage unit, 30 ... Document browsing control unit, 100 ... Determination unit, 101 ... URL development unit, 102 ... Web page acquisition unit, 103 ... Web page analysis unit, 181 ... Spam account extraction unit, 200 ... Storage unit, 201 ... Inappropriate word storage unit, 202 ... Reference source Title word storage unit, 203 ... judgment result storage unit, W ... topic words.

Claims (11)

In a service for posting a document to a server on a network, a topic word collection that collects, as a topic word that is a specific topic, the word having an increased appearance frequency among words appearing in stream data composed of the document And
A search unit relative posted the document, searches for at least one comprising said document the topic word,
A spam document determination unit that determines that the document is a spam document if the document searched by the search unit includes a certain number of the topic words ;
A spam account determination unit that determines, based on the frequency with which the spam document is posted, an account that posts the spam document as a spam account;
Bei El, spam account extraction device. The spam account extraction device further includes an inappropriate URL determination unit that determines that the URL is an inappropriate URL when the URL included in the document includes a link to an inappropriate site,
The spam document determination unit determines the document as the spam document when the URL is determined as the inappropriate URL.
The spam account extraction device according to claim 1. The inappropriate URL determination unit determines the URL as the inappropriate URL when the URL or a URL to which the URL is redirected includes a predefined character string.
The spam account extraction device according to claim 2. The inappropriate URL determination unit determines that the URL is an inappropriate URL when a document to which the URL or a URL to which the URL is redirected includes a predefined word,
The spam account extraction device according to claim 2 or 3. The inappropriate URL determination unit determines that the URL or the URL to which the URL is redirected includes a link from a document including a plurality of unrelated topic words. To determine,
The spam account extraction device according to any one of claims 2 to 4. The inappropriate URL determination unit determines that the URL is the inappropriate URL when the URL or the document to which the URL redirect destination transitions does not include the topic word used in the document.
The spam account extraction device according to any one of claims 2 to 5. The inappropriate URL determination unit determines the URL as the inappropriate URL when the URL or the document to which the URL redirect destination includes transition includes a plurality of unrelated topic words.
The spam account extraction device according to any one of claims 2 to 6. A friend list acquisition unit that acquires a list of accounts that can view documents posted by the first account;
The degree of similarity between the friend list of the first account acquired by the friend list acquisition unit and the list of accounts that can view documents posted by the account determined to be a spam account by the spam account determination unit. A calculation unit for calculating similarity between accounts;
Further comprising
The spam account determination unit determines the spam account based on the similarity;
The spam account extraction device according to any one of claims 1 to 7. The friend list acquisition unit
A spam account extraction unit that extracts, from the friend list of the first account, accounts whose similarity exceeds a certain threshold based on the similarity between accounts calculated by the inter-account similarity calculation unit. 9. The spam account extraction device according to 8. A document index generation unit that generates an index of the document based on a determination result determined by the spam account determination unit;
A document browsing control unit that controls browsing of documents posted from the spam account;
The spam account extraction device according to any one of claims 1 to 8, further comprising: A spam account extraction method executed by a spam account extraction device,
In a service for posting a document to a server on a network, a topic word collection that collects, as a topic word that is a specific topic, the word having an increased appearance frequency among words appearing in stream data composed of the document Steps,
A search step of relative posted the document, searches for at least one comprising said document the topic word,
A spam document determination step of determining , when the document searched by the search step includes a certain number of the topic words or more, the document as a spam document;
A spam account determination step of determining an account that posts the spam document as a spam account based on the frequency with which the spam document is posted;
Bei El, spam account extraction methods.

Images