JP6447680B2 - Information processing apparatus, information processing method, and program - Google Patents

Description

  The present invention relates to a technique for classifying a contributor who has posted a sentence.

  In recent years, with the development of social media, information transmission from the public has been increasing. A typical social media is called microblogging, which has a short text and high anonymity. Because of these characteristics, it is considered that microblog users are more likely to speak their real intentions in real time, so the content posted on microblogs is used for marketing and reputation assessment.

  When analyzing microblogs and utilizing them for marketing, etc., it is important to improve the accuracy of analysis. For that purpose, it is necessary to analyze after classifying according to the content of the remarks and the position of the contributor.

  For example, if you want to analyze microblogging comments about a product, categorize whether each comment is for advertising purposes from the media or for sharing the feeling of use by users of the product. It is necessary to analyze above.

  As an example of classification, Patent Document 1 discloses a system for classifying a poster based on the characteristics of the poster.

JP 2012-212286 A

  As described above, it is necessary to improve the accuracy of classifying posters, but there are many unspecified posters, and it is not easy to classify posters accurately. The technique disclosed in Patent Document 1 uses only the characteristics of the poster for classification of the poster, and does not consider the content of the posted text, which is insufficient to increase the accuracy of the classification. is there.

  Moreover, although patent document 1 sets the rule divided into a specific category in advance, this rule is fixed and cannot set the useful knowledge and knowledge which a user has.

  Accordingly, the present invention has been made to solve the above-described problems, and an object of the present invention is to provide a mechanism capable of appropriately classifying a poster who has posted an article on a microblog or the like.

  The present invention includes dictionary data creation means for creating dictionary data based on the content posted by a poster classified into each category according to a user instruction, rule setting means for accepting setting of a rule for classifying a poster, Based on the dictionary data created by the dictionary data creation means, the ratio of the posters classified into each category with respect to the sum of the number of classified posters, and the rules set by the rule setting means, unclassified An information processing apparatus comprising classification means for classifying a poster into each category.

  Further, the present invention provides a dictionary data creation step in which dictionary data creation means of an information processing device creates dictionary data based on the content posted by a poster classified into each category according to a user's instruction; A rule setting step in which the rule setting means accepts the setting of a rule for classifying a poster; the dictionary means created by the dictionary data creation step in the classification means of the information processing device; and the number of classified posters A classification step of classifying uncategorized contributors into each category based on the ratio of the posters classified into each category with respect to the sum of the total and the rules set in the rule setting step Information processing method.

  In addition, the present invention is a program executed in an information processing device, the dictionary data creation for creating the dictionary data based on the content posted by a contributor classified into each category according to a user instruction Means, rule setting means for accepting setting of a rule for classifying a poster, dictionary data created by the dictionary data creating means, and a poster classified into each category with respect to the total number of classified posters Based on the ratio and the rule set by the rule setting means, it is made to function as a classification means for classifying unclassified contributors into each category.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to provide the structure which can classify the poster who contributed to the microblog etc. with high precision.

It is a figure which shows the structure of the contributor classification | category system in embodiment of this invention. It is a figure which shows an example of the screen which registers the rule for classifying into a specific category at the user side in embodiment of this invention. It is a figure which shows an example of the screen displayed when column "setting" is set to "confirmation" in FIG. 2 in embodiment of this invention. It is a figure which shows an example of the rule which can be set by the user side in embodiment of this invention. It is a figure which shows an example of the screen which produces the dictionary table in embodiment of this invention. It is a figure which shows an example of the table which registers the posting text in embodiment of this invention. It is a figure which shows an example of the table which registers the data of the contributor in embodiment of this invention. It is a figure which shows an example of the table which registers the rule registered by the user in embodiment of this invention. It is a figure which shows an example of the table which registers the category classified in embodiment of this invention. It is a figure which shows an example of the table of the dictionary used when classifying into the category in embodiment of this invention. It is a flowchart explaining the "dictionary creation process" in the embodiment of the present invention. It is the flowchart explaining the "undecided poster classification process" in the embodiment of the present invention. It is a flowchart explaining "Contributor determination at the time of rule update and deletion" in the embodiment of the present invention. It is a figure which shows an example of the table which matched category ID and category name in embodiment of this invention. It is a figure which shows an example of the dictionary table in embodiment of this invention. It is a figure which shows an example of the rule registration data table in embodiment of this invention. It is a flowchart which shows an example of a process when there is a change in contributor data in the embodiment of the present invention. 2 is a diagram illustrating an example of a hardware configuration of an information processing apparatus 101. FIG. It is a figure which shows an example of the warning screen displayed when the selected rule is not an appropriate rule. It is a figure which shows an example of the screen which displays the warning that there exists a possibility of misclassification, when a decision rule is set. It is a figure which shows an example of a warning screen when the category set by the user is not a category with the highest weight.

  FIG. 1 is a diagram showing a configuration of a contributor classification system according to an embodiment of the present invention. The poster classification system includes a user rule setting unit, a teacher data registration unit, a dictionary creation unit, a rule weight setting unit, a posted sentence decomposition unit, a category classification probability calculation unit, and a determination unit.

  Further, a rule registration data table, a category master table, a contributor data table, a posted sentence data table, and a dictionary table are provided.

  The user rule setting unit has a function of registering a rule set by the user. The user sets a rule via the rule setting screen shown as an example in FIG.

  FIG. 2 is a diagram illustrating an example of the rule setting screen. In this embodiment, a case will be described in which a poster is classified into three categories (“BOT”, “news”, and “general user”). Here, “BOT” refers to a program programmed to automatically post for the purpose of guiding to a specific site. “News” refers to media such as news sites. The “general user” refers to a contributor other than BOT and news. In this embodiment, a case will be described in which a poster is classified into these three categories. Therefore, three tuples are registered in the category master table shown in FIG.

  In FIG. 2, items “rule 201” and “setting 206” accept settings from the user.

  The rule 201 has a pull-down, and accepts selection of any of the rules displayed by the pull-down from the user. The rules displayed in the pull-down are registered in the table shown in FIG. These rules are registered by the user via a rule registration screen (not shown). Further, it is assumed that a logical product can be used for the rule.

  Here, the rule is a condition relating to information not related to the posted text (posted content) as shown in FIG. For example, a self-introduction sentence, the number of posts, and information on a link person. In this way, by creating a rule based on information unrelated to the posted content, information other than the posted text can be used as an index for category classification. In this way, categorization using information other than posted text can achieve more appropriate categorization than when using only posted content or using only poster information. it can.

  When the rule 201 is selected, the number of hits 202 is calculated and displayed among the contributors whose teacher data flag is 1, corresponding to the selected rule. In addition, for the BOT 203, the news 204, and the general user 205, the number and ratio (ratio) of the contributors classified into each category among the contributors whose teacher data flag is 1 and corresponding to the selected rule. Are calculated and displayed.

  Thus, by displaying the number and ratio of contributors, the user can set rules while referring to these numerical values, so that it is possible to set appropriate rules.

  Further, the ratio calculated here is registered as a “weight” in the rule registration data table shown in FIG. For example, when Rule 1 has a ratio (90%, 10%, 0%) as shown in FIG. 2, integer values 90, 10, and 0 are registered in each category as weights.

  Here, for example, when the calculated ratio (weight) is a value such as 30, 30, 40, the rule is inappropriate as a rule for determining which category the poster should be classified into. I can say that. That is, since such a rule is not a rule representing the characteristics of the posters belonging to each category, appropriate classification cannot be performed using such a rule. Therefore, in such a case, the user is warned that the rule is not appropriate.

  An example of a screen for warning that the rule is not appropriate is shown in FIG.

  In determining whether or not the rule is appropriate, for example, a criterion such as whether or not there is a category having a ratio (weight) exceeding 80 (appropriate if there is, or inappropriate if not) is registered in advance. It is possible to determine.

  The setting 206 accepts a selection of “automatic” or “confirmed” by a radio button from the user. “Automatic” means that the determination unit uses the probability calculated by the category classification probability calculation unit to make a determination (a rule for which automatic is set is referred to as an “automatic rule”). Further, “determined” means that the determination unit makes a determination using only the rule (a rule for which determination is set is referred to as a “determined rule”).

  In a rule where there is no category with a weight of 100, if “determined” is selected, the user is warned that misclassification will be included. That is, in the case of the final rule, the posters corresponding to the rule are always classified into the categories set on the screen shown in FIG. Therefore, even though there is a probability (possibility) of belonging to another category, it is judged that it belongs to the set category without considering it, so a warning that the possibility of misclassification remains is issued. Therefore, it is necessary to give attention to the user.

  When “Confirm” is selected, a confirmation rule registration screen shown in FIG. 3 is displayed, and a setting as to which category a contributor corresponding to the rule belongs is accepted. In the example of FIG. 3, a poster who falls under rule 3 is determined to belong to the category “general user”. Further, a warning may be issued when the category set by the user is not the category with the highest weight. For example, a warning may be issued when “BOT” is selected on the screen of FIG. 3 even though the weights are “BOT” 10, “news” 10, and “general user” 80. Such warnings can prevent erroneous rule settings. An example of a warning screen indicating the possibility of misclassification is shown in FIG.

  FIG. 21 shows an example of a warning screen when the category set by the user is not the category with the highest weight.

  A contributor with a teacher flag of 1 is a contributor who has been categorized by the user by the processing of the flowchart shown in FIG. That is, it is the contributor whose category is determined by the user's visual confirmation.

  The teacher data registration unit has a function of registering teacher data. An example of a screen used when registering teacher data is shown in FIG.

  In FIG. 5, a contributor ID (501) randomly extracted from the contributor data table and a posted sentence (502) posted by the contributor are displayed. The user confirms the posted text, determines which category the poster belongs to, and registers.

  In the category column (503), the category name registered in the category master table (FIG. 9) is displayed, and selection is accepted by a radio button.

  In addition, by clicking the poster ID, it is possible to display data registered in the poster data table of the poster. Also, by clicking on the posted text, it is possible to newly collect and display the past posted text of the poster. In addition, when a past posted sentence cannot be acquired, another posted sentence of the poster is displayed from the posted sentence data table.

  When the user presses the registration button (504), the poster category is registered. When registered, the selected category ID is registered in the attribute “category ID” of the poster data table (FIG. 7) of the poster ID. Also, “1” is registered in the attribute “teacher data flag” and used as teacher data.

  By pressing the batch registration button (505), it is possible to register the category of the poster displayed on the screen shown in FIG.

  The dictionary data creation unit has a function of creating a dictionary table (FIG. 10) based on teacher data. The dictionary data creation unit obtains a posted sentence with a poster ID whose attribute “teacher data flag” in the poster data table (FIG. 7) is “1” from the posted text data table (FIG. 6). Then, the acquired posted sentence data is subjected to morphological analysis, and the word obtained by the morphological analysis is registered in the attribute “word” of the dictionary table (FIG. 10). In addition, the number of occurrences of the word in the posted text of the category is registered in the attribute “number of occurrences of word”. Also, the category ID of the poster who included the word in the attribute “category ID” is registered. Also, the total number of posters by category is registered in the attribute “number of posters by category”.

  Here, when a URL (Uniform Resource Locator) is included in the posted text, the host name is registered as a word. If the URL is a shortened URL, the user actually accesses the site indicated by the shortened URL and acquires the actual URL. Then, the host name of the acquired URL is registered as a word.

  The shortened URL is a URL that represents an actual URL with a smaller number of characters. Various services provide services that convert actual URLs into shortened URLs, and even if the actual URLs are the same, they may be converted into different character strings depending on the services to be converted. Therefore, it is necessary to actually access the site and obtain an actual URL. That is, it is necessary to access an actual site and acquire an actual URL in order to prevent registration as a different word even though the same content (same site) is actually shown.

  In addition, the host name is registered instead of registering all character strings of actual URLs as words because the site indicated by the URL is registered, not the content indicated by the URL.

  That is, when all the character strings of the URL are registered, it will register the temporary one (content of each site), but register the permanent one (site) by registering the host name. can do.

  For example, a contributor who should be classified into the “news” category is not “a contributor who posted an article on June 1, 2013 of an ABC newspaper” but a “poster who posted an article of an ABC newspaper”. As described above, since it can be said that it is appropriate to judge whether or not the posting is made for the permanent one, not the one for the temporary one, the host name is registered in the dictionary.

  Further, when all character strings of URLs are registered in the dictionary as words, the registered words are likely to become words that will not be used for future classification.

  That is, for example, when a URL indicating “product A” is registered in the dictionary, if the product A is sold out, it is highly likely that no one will post about the product A in the future. Therefore, it can be said that the URL registered in the dictionary will no longer be used. Similarly, with regard to news, it seems that fewer people will post about past articles.

  From the above viewpoint, in the present embodiment, the host name of the URL is registered in the dictionary.

  The rule weight calculation unit extracts a poster who applies to the rule registered in the rule registration data table shown in FIG. 8 from the poster data table in FIG. 7, calculates a weight for each category, and creates a poster data table. The function of registering the calculated weight is provided.

  The weight registered here is the sum of the weights of all the applicable rules.

  The posted sentence analysis unit has a function of extracting a posted sentence from the posted sentence data table (FIG. 6), performing morphological analysis, and decomposing it into words. Note that the morphological analysis can be realized by a known technique, and thus detailed description thereof is omitted here.

  The category classification probability calculation unit uses the weights calculated by the rule weight calculation unit, the words obtained by the posted sentence analysis unit, and the dictionary table shown in FIG. It has a function to calculate each probability. Specific processing will be described later.

  The determination unit has a function of comparing the probabilities calculated by the category classification probability calculation unit and classifying a poster into a category having the highest probability. Then, the contributor data table of FIG. 7 is updated.

  The posted text data table shown in FIG. 6 is a table in which posted text data collected from a microblog or the like is registered. In the posted text data table, a posted text ID for uniquely identifying a posted text, a poster ID for identifying a poster who posted the posted text, a posted text, a posting date and time, and a position indicating a posted location Information is registered.

  The contributor data table shown in FIG. 7 is a table in which contributor data collected from a microblog or the like is registered.

  In the contributor data table, the contributor ID for identifying the contributor, the registered name of the contributor, the category ID for identifying the category determined by the determination unit, the teacher data flag, and each calculated by the rule weight calculation unit The total category weight, date and time of data registration, and the self-introduction sentence of the poster are registered.

  The rule registration data table shown in FIG. 8 is a table in which rules set by the user via the screen shown in FIG. 2 are registered.

  It should be noted that “0” is registered in the deletion flag, and “1” is registered when a deletion instruction is given by the user via the screen of FIG.

  The category master table shown in FIG. 9 is a table for managing categories for classifying posters. In the category master table, category IDs and category names are registered in association with each other.

  The dictionary table shown in FIG. 10 is a table in which dictionary data used in the category classification probability calculation unit is registered. As described above, the data registered in the dictionary table is created and registered by the dictionary data creation unit.

  Next, dictionary data creation processing by the teacher data registration unit and the dictionary data creation unit will be described with reference to FIG.

  The process illustrated in FIG. 11 is a process in which the CPU 1801 of the information processing apparatus 101 reads and executes a predetermined control program.

  In step S <b> 101, the CPU 1801 of the information processing apparatus 101 acquires a posted sentence to be analyzed such as a microblog and information on a poster who posted the posted sentence from the Internet or the like.

  The acquired information is stored in a posted text data table (FIG. 6) or a poster data table (FIG. 7).

  In step S102, the CPU 1801 of the information processing apparatus 101 displays the teacher data creation screen shown in FIG. 5, and displays the data acquired in step S101. And the category classification | category with respect to each contributor is received from a user. When the registration button (504, 505) is pressed by the user, data including a category ID indicating the accepted category is registered in the contributor data table (FIG. 7), and the teacher data flag is registered as “1”.

  In step S103, the CPU 1801 of the information processing apparatus 101 acquires a posted message for each category. Specifically, a posted message by a poster who has the same category ID registered in step S102 is acquired from the posted message data table.

  In step S104, the CPU 1801 of the information processing apparatus 101 decomposes the posted sentence acquired in step S103 into words by morphological analysis.

  In step S105, the CPU 1801 of the information processing apparatus 101 registers the number of appearances of the word for each category and the number of contributors using the word in the dictionary table for each word obtained by the process of step S104. And the process of this flowchart is complete | finished.

  By the above processing, the poster is classified by the user's judgment, and dictionary data can be created from the result of the category classification. Based on the dictionary data created here, the category classification processing in FIG. 12 is executed.

  Next, with reference to FIG. 12, a process for classifying a poster who has not performed category determination using the dictionary data created by the process of FIG.

  The process illustrated in FIG. 12 is a process in which the CPU 1801 of the information processing apparatus 101 reads and executes a predetermined control program.

  In step S <b> 201, the CPU 1801 of the information processing apparatus 101 acquires a posted sentence of a poster who is a determination target (target to be classified) from a posted text data table.

  The posted text acquired from the posted text data table is a posted text by a contributor who is not categorized among the posted text stored in step S101 in FIG.

  In step S202, the CPU 1801 of the information processing apparatus 101 acquires the poster information (self-introduction text) of the judgment subject poster from the poster data table (FIG. 7).

In step S203, the CPU 1801 of the information processing apparatus 101 calculates prior probabilities for each category using data registered in the dictionary table.
The prior probability is calculated by the following formula.
Prior probability = total number of contributors by category / total number of contributors by category

  In step S204, the CPU 1801 of the information processing apparatus 101 acquires a rule registered by the user from the rule registration data table (FIG. 8).

  In step S205, the CPU 1801 of the information processing apparatus 101 determines whether or not the posting subject to be determined matches the determination rule among the rules acquired in step S204.

  If there is a confirmed rule to which the determination subject poster is applicable (step S205: YES), the process proceeds to step S211.

  In step S <b> 211, the CPU 1801 of the information processing apparatus 101 classifies the determination subject poster into a category corresponding to the confirmation rule, and ends the process. Here, when it classify | categorizes into a some category (when it corresponds to a some decision rule), it classify | categorizes into the category with the highest prior probability.

  If there is no confirmed rule to which the determination subject poster is applicable (step S205: NO), the process proceeds to step S206.

  In step S <b> 206, the CPU 1801 of the information processing apparatus 101 calculates the total weight of the automatic rule to which the determination subject poster corresponds, by category, and registers the calculated total category weight in the poster data table.

In step S207, the CPU 1801 of the information processing apparatus 101 updates the prior probability in order to increase the likelihood of the category having the highest total weight value for each category calculated in step S206.
An example of processing for updating the prior probability will be described below.

  For example, assume that there are three categories (BOT, news, general user), and the prior probabilities are A, B, and C, respectively. Of these, A is the highest value. On the other hand, it is assumed that the news category has the largest weight total value. In this case, since it is desired to increase the likelihood of the news category having the highest weight, the prior probability of the news category is updated from B to A. This makes it possible to increase the prior probability of news.

  In this way, by updating the prior probabilities in consideration of the weights calculated based on the rules, it is possible to perform category classification that considers not only the posted content but also the poster information. Further, by combining the prior probability with the maximum category, it is possible to perform category classification based on the posted content without taking the prior probability into consideration.

  In addition to this method, for example, a method of multiplying each prior probability by a weight ratio for each category is also conceivable. In the present invention, the specific method of the prior probability update process may be any method.

  In step S <b> 208, the CPU 1801 of the information processing apparatus 101 decomposes the posted text of the determination target poster into words by morphological analysis. The conditions of morphological analysis here (the dictionary used for analysis, the part of speech to be obtained, etc.) are the same as the conditions used when the dictionary table was created.

  Here, when a URL (Uniform Resource Locator) is included in the posted text, the host name is registered as a word. If the URL is a shortened URL, the site actually indicated by the shortened URL is accessed and the URL is actually acquired. Then, the host name of the acquired URL is registered as a word.

  The shortened URL is a URL that represents an actual URL with a smaller number of characters. Various services provide services for converting actual URLs into shortened URLs, and even the same URL is converted into a different character string depending on the service to be converted. Therefore, it is necessary to actually access the site and obtain an actual URL. That is, it is necessary to access an actual site and acquire a URL in order to prevent registration as a different word even though the same content is actually shown.

  In step S209, the CPU 1801 of the information processing apparatus 101 obtains likelihood for each word by category using the word obtained by the morphological analysis in step S208 and the dictionary table. Here, the likelihood for each word is obtained by the following calculation formula.

  Likelihood by word = number of words registered in dictionary table / total number of words contained in dictionary table by category

  In step S210, the CPU 1801 of the information processing apparatus 101 obtains the posterior probability for each category using the prior probability obtained in step S207 and the likelihood for each word obtained in step S209.

  In step S211, the CPU 1801 of the information processing apparatus 101 determines the category having the maximum posterior probability obtained in step S210 as the category of the poster as a determination target, and registers the category in the poster data table.

  When naive Bayes classification is used as in this embodiment, a zero frequency problem that the posterior probability becomes zero occurs when a word not registered in the dictionary table appears. This problem can be dealt with by using a technique such as Laplace smoothing.

  Next, processing that occurs when a new rule is added to the rule registration data table and when an existing rule is deleted will be described with reference to FIG.

  The process illustrated in FIG. 13 is a process in which the CPU 1801 of the information processing apparatus 101 reads and executes a predetermined control program.

  In step S301, the CPU 1801 of the information processing apparatus 101 extracts a newly added rule and a deleted rule. This is executed when the registration button is pressed or the delete button is pressed on the rule setting screen shown in FIG.

  In step S <b> 302, the CPU 1801 of the information processing apparatus 101 acquires a posted message and a poster data of a poster corresponding to a newly added or deleted rule from a posted text data table and a poster data table, respectively.

  In step S303, the CPU 1801 of the information processing apparatus 101 calculates a prior probability. The calculation process here is the same as the process in step S203.

  And the process of step S305-S313 is performed with respect to all the contributors acquired at step S302 (step S304).

  In step S <b> 305, the CPU 1801 of the information processing apparatus 101 determines whether or not the determination subject poster matches the determined rule among the rules registered in the rule registration data table.

  If there is a confirmation rule to which the determination subject poster is applicable (step S305: YES), the process proceeds to step S313.

  In step S313, the CPU 1801 of the information processing apparatus 101 determines that the determination subject poster is a category corresponding to the confirmation rule, and registers the category in the poster data table. Here, when it classify | categorizes into a some category (when it corresponds to a some decision rule), it classify | categorizes into the category with the highest prior probability.

  If there is no confirmed rule to which the determination subject poster is applicable (step S305: NO), the process proceeds to step S306.

  In step S306, the CPU 1801 of the information processing apparatus 101 acquires a category-specific weight from the poster data table.

  In step S307, when a rule is newly added, the CPU 1801 of the information processing apparatus 101 adds the weight of the rule for each category. On the other hand, when a rule is deleted, the weight of the rule is subtracted for each category. Then, the poster data table is updated with the added / subtracted weights.

  In step S308, the CPU 1801 of the information processing apparatus 101 determines whether or not the category having the largest weight has been changed by the processing in step S307.

  If the category with the largest weight has been changed (step S308: YES), the process proceeds to step S309.

  When the category having the largest weight has not been changed (step S308: NO), the process proceeds to the process for the next poster, and the processes of steps S305 to S313 are executed for the next poster.

  In step S309, the CPU 1801 of the information processing apparatus 101 updates the prior probability of the category with the highest weight. The process of step S309 is the same as the process of step S207 of FIG.

  Further, the processes in steps S310 to S313 are the same as the processes in steps S208 to S211 in FIG. 12, respectively, and thus description thereof is omitted here.

  Next, processing executed when there is a change in the poster data will be described with reference to FIG.

  The process illustrated in FIG. 17 is a process executed when the CPU 1801 of the information processing apparatus 101 reads a predetermined control program.

  In step S401, the CPU 1801 of the information processing apparatus 101 creates a dummy table having the same attribute value as that of the poster data table. Then, the processes of steps S403 to S419 are executed for all of the posters registered in the poster data table and whose teacher data flag is “0” (step S402).

  In step S403, the CPU 1801 of the information processing apparatus 101 acquires the latest poster data and registers it in the dummy table created in step S401. At this time, the attribute value registered in the poster data table is registered as the weight of the category ID and category N. “0” is registered for the teacher data flag.

  In step S404, the CPU 1801 of the information processing apparatus 101 compares the attribute value of the poster data table excluding the category ID and the weight of category N with the attribute value registered in the dummy table, and compares whether there is a difference. To do.

  If there is a difference (step S405: YES), the process proceeds to step S406.

  If there is no difference (step S405: NO), the process proceeds to the process for the next poster.

  If there is a difference, the data registered in the dummy table is determined for all rules (step S406).

  In step S407, the CPU 1801 of the information processing apparatus 101 determines whether the poster registered in the dummy table matches the confirmed rule among the rules registered in the rule registration data table.

  When there is a matching rule that matches (step S407: YES), the process proceeds to step S418.

  In step S418, the CPU 1801 of the information processing apparatus 101 registers the category set in the confirmation rule determined to match in step S407 in the dummy table.

  If there is no matching confirmation rule (step S407: NO), the process proceeds to step S408.

  In step S408, the CPU 1801 of the information processing apparatus 101 acquires the category-specific weight from the poster data table.

  In step S409, the CPU 1801 of the information processing apparatus 101 registers the weight acquired in step S408 in the dummy table.

  The above steps S407 to S409 are performed for all the rules (step S410).

  In step S411, the CPU 1801 of the information processing apparatus 101 determines whether the category having the highest weight has changed.

  If the category having the highest weight has changed (step S411: YES), the process proceeds to step S412.

  If the category with the highest weight has not changed (step S411: NO), the process proceeds to step S419.

  In step S412, the CPU 1801 of the information processing apparatus 101 calculates a prior probability.

  In step S413, the CPU 1801 of the information processing apparatus 101 updates with the prior probability calculated in step S412.

  In step S414, the CPU 1801 of the information processing apparatus 101 acquires the posted text of the determination target person.

  In step S415, the CPU 1801 of the information processing apparatus 101 decomposes the posted sentence acquired in step S414 into words by morphological analysis.

  In step S416, the CPU 1801 of the information processing apparatus 101 obtains likelihood for each word by category using the word obtained in step S415 and the dictionary table.

  In step S417, the CPU 1801 of the information processing apparatus 101 obtains the posterior probability for each category using the prior probability and the likelihood for each word.

  In step S418, the CPU 1801 of the information processing apparatus 101 identifies the category having the maximum posterior probability and registers the category in the dummy table.

  In step S419, the CPU 1801 of the information processing apparatus 101 overwrites the poster data table with the value registered in the dummy table.

  Through the above processing, when the poster information is changed, the poster can be classified again.

  Next, a process for classifying a poster who has not been subjected to category determination will be described using a specific example.

  Here, on the assumption that the tables shown in FIGS. 14 to 16 are prepared in advance, a specific process for actually classifying an undetermined poster (explained here as “A_news”) is shown in FIG. This will be described with reference to the flowchart.

  Note that the part-of-speech extracted at the time of morpheme analysis is a noun and an adjective as a condition for creating a dictionary table.

  First, the posted text of the poster subject to determination is acquired. Here, it is assumed that there is only one post sentence of the judgment subject contributor, “[Breaking News] Sale has started!”.

  Next, the posting information of the poster is acquired. Here, it is assumed that the self-introduction sentence of the poster subject to determination is “A post from A news site”.

  Next, the prior probability is calculated. From the dictionary table of FIG. 15, the prior probabilities of BOT, news, and general user are BOT: 70/100, news: 20/100, and general user: 10/100. Specifically, the number of contributors registered in the dictionary table is 100, of which the number of contributors whose category is BOT is 70, news is 20, and general users are 10.

  Next, check the applicable rules. Here, A_news corresponds to rule 2 (rule 2 is a rule that “the word“ news ”is included in the self-introduction sentence”). Therefore, each weight of rule 2 is registered in the contributor data table.

  Then, the prior probability is updated. Comparing the weights of the respective categories obtained by the above processing, it can be seen that the weight of the category B (news) is the largest (FIG. 16). Therefore, the prior probability of category B is updated from 20/100 to 70/100. Here, the prior probability of the category having the largest weight is updated by matching the prior probability with the largest prior probability.

  Next, morphological analysis is performed on the posted text and it is broken down into words. By extracting the nouns and adjectives from the A_news post, “[Breaking News] Sale!”, You get “Breaking News” and “Sale”.

  Next, the likelihood is calculated for each category. First, the likelihood of BOT is obtained. Looking at the dictionary table in FIG. 15, “breaking news” does not exist in the word of BOT (category ID: A), but “sale” exists. Therefore, the likelihood of BOT “breaking news” is 1/60, and the likelihood of “sale” is 10/60. The likelihood calculation is the number of occurrences of the word in the category / the total number of word appearances of the category. The reason why the likelihood of “breaking news” is not 0/60 is that the above-described technique called Laplace smoothing is used.

  Similarly, the likelihood of “breaking news” and “sale” of news is 20/60 and 1/60, respectively. The likelihoods of general users' “breaking news” and “sale” are 1/40 and 1/40, respectively.

  Next, the posterior probability is calculated. First, the posterior probability of BOT is obtained. The prior probability of BOT is 70/100, and the likelihood of words (“breaking news” and “sale”) is 1/60 and 10/60, respectively. Therefore, the posterior probability is (70/100 × 1/60) × (70/100 × 10/60) = 0.00136 (rounded to the sixth decimal place).

  Similarly, the posterior probability of news is (70/100 × 20/60) × (70/100 × 1/60) = 0.00272 using the updated prior probability. Further, the posterior probability of the general user is (10/100 × 1/40) × (10/100 × 1/40) = 0.00001.

  Next, the calculated posterior probabilities are compared and classified into categories. Comparing the posterior probabilities obtained by the above calculations, the posterior probability of news is the highest. Therefore, the poster “A_news” is classified into the news category.

  Finally, the news category ID “B” is registered in the attribute “category ID” of the table in which the attribute “poster ID” of the poster data table is A_news.

  FIG. 18 is a diagram illustrating a hardware configuration of the information processing apparatus 101.

  In FIG. 18, reference numeral 1801 denotes a CPU, which comprehensively controls each device and controller connected to the system bus 1804. Further, the ROM 1803 or the external memory 1811 is necessary for realizing a BIOS (Basic Input / Output System) or an operating system program (hereinafter referred to as an OS), which is a control program of the CPU 1801, and functions executed by each server or each PC. Various programs are stored.

  Reference numeral 1802 denotes a RAM that functions as a main memory, work area, and the like of the CPU 1801. The CPU 1801 implements various operations by loading a program or the like necessary for execution of processing from the ROM 1803 or the external memory 1811 to the RAM 1802 and executing the loaded program.

  An input controller 1805 controls input from the input device 1809 or the like. A video controller 1806 controls display on a display device 1810 such as a liquid crystal display. The display device is not limited to a liquid crystal display, and may be a CRT display or the like. These are used by clients as needed.

  Reference numeral 1807 denotes a memory controller which is connected via an adapter to a hard disk (HD), a flexible disk (FD), or a PCMCIA card slot for storing a boot program, various applications, font data, user files, editing files, various data, and the like. Controls access to an external memory 1811 such as a compact flash (registered trademark) memory.

  Reference numeral 1808 denotes a communication I / F controller which is connected to and communicates with an external device via a network, and executes communication control processing on the network. For example, communication using TCP / IP is possible.

  Note that the CPU 1801 can perform display on the display device 1810 by executing outline font rasterization processing on a display information area in the RAM 1802, for example. Further, the CPU 1801 enables a user instruction with a mouse cursor (not shown) on the display device 1810.

  Various programs operating on the hardware are recorded in the external memory 1811 and executed by the CPU 1801 by being loaded into the RAM 1802 as necessary.

  It should be noted that the configuration and contents of the various data described above are not limited to this, and it goes without saying that the various data and configurations are configured according to the application and purpose.

  Moreover, the program in this invention is a program which makes a computer perform the process of FIGS. 11-13, and FIG. The program in the present invention may be a program for each process in FIGS. 11 to 13 and FIG.

  As described above, a recording medium that records a program that implements the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.

  In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium recording the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, nonvolatile memory card, ROM, EEPROM, silicon A disk or the like can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program is actually It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the processing and the processing is included.

  Furthermore, after the program read from the recording medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is based on the instructions of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  The present invention may be applied to a system constituted by a plurality of devices or an apparatus constituted by a single device. Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can enjoy the effects of the present invention.

  Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or apparatus can enjoy the effects of the present invention. In addition, all the structures which combined each embodiment mentioned above and its modification are also included in this invention.

101 Information processing apparatus

Claims (7)

An information processing apparatus for storing teacher data in which a poster is associated with a category of the poster,
The author and receiving means for receiving a set of criteria to classify in each category,
Classification means for classifying uncategorized contributors into each category based on the breakdown of the categories of contributors that satisfy the conditions accepted by the accepting means among the contributors stored as the teacher data ;
An information processing apparatus comprising: The accepting means accepts a plurality of conditions for classifying a poster into each category,
The classifying means sets the breakdown of the category of the poster that satisfies the condition as the probability of the category of the poster that satisfies the condition, and classifies the unclassified poster into each category based on the probability in the plurality of conditions. The information processing apparatus according to claim 1.   A confirmation condition setting accepting means for accepting a setting for accepting the setting by the accepting means, that the poster is classified into categories only by satisfying the condition, and which category is classified when the condition is satisfied; ,
  Further comprising
  3. The information processing apparatus according to claim 1, wherein the classifying unit classifies a poster who satisfies a condition whose setting is received by the confirmation condition setting receiving unit, into a category set in the condition.   2. The apparatus according to claim 1, further comprising: a determination unit that determines whether or not the condition is appropriate based on a breakdown of a category of a poster who satisfies a condition received by the reception unit among the posters stored as the teacher data. 4. The information processing apparatus according to any one of items 1 to 3.   The information processing apparatus according to claim 4, further comprising a notification unit that notifies a determination result by the determination unit. An information processing method in an information processing apparatus for storing teacher data in which a poster is associated with a category of the poster,
An accepting step in which the accepting means of the information processing apparatus accepts the setting of conditions for classifying the poster into each category ;
The classification means of the information processing apparatus classifies uncategorized contributors into each category based on the breakdown of the categories of contributors that satisfy the conditions accepted by the accepting process among the contributors stored as the teacher data. A classification process;
An information processing method comprising: A program executable in an information processing apparatus for storing teacher data in which a poster and a category of the poster are associated with each other,
The information processing apparatus;
The author and receiving means for receiving a set of criteria to classify in each category,
The program for functioning as a classification | category means to classify | categorize an uncategorized contributor into each category based on the breakdown of the category of the contributor who satisfy | fills the conditions received by the said reception means among the contributors memorize | stored as the said teacher data .

Images