JP6937707B2 - Information processing equipment, information processing methods, and programs - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

Conventionally, when providing various contents to users, a service has been provided in which comments indicating opinions, impressions, etc. regarding the contents posted by a third party are provided according to the contents (see, for example, Patent Document 1). ..

Japanese Unexamined Patent Publication No. 2014-49094

Comments posted by third parties, such as those mentioned above, may include content that is not appropriate for public disclosure, such as slander or slander against a particular person. Such comments that slander or slander a specific person should not be made public, and it is preferable to delete them promptly. For this reason, the content provider may manually monitor and delete such inappropriate comments.

In addition, a method of automatically detecting inappropriate comments is performed using a fixed model in which inappropriate words and expressions are defined in advance. However, the expressions used for inappropriate comments are diversified and complicated, and there are many that cannot be detected by a simple model. For example, if an inappropriate expression is incorporated as part of a positive comment expression, or if an inappropriate expression is made due to jargon or the like, it may not be detected by the above model. In addition, since the expressions used for such inappropriate comments are generated one after another, it is not easy to manually monitor all of them, and the method using a fixed model is delayed in responding. There was a case that it ended up.

The present invention has been made in consideration of such circumstances, and an object of the present invention is to provide an information processing device, an information processing method, and a program capable of detecting inappropriate comments quickly and with high accuracy. It is one of.

One aspect of the present invention is a reception unit that receives a report indicating that the second content is inappropriate by a user who has viewed the second content, which is a comment posted on the first content, and the reception unit. It is an information processing apparatus including a first determination unit that determines whether or not the second content is inappropriate based on the report received by.

One aspect of the present invention has been made in consideration of such circumstances, and it is possible to detect inappropriate comments quickly and with high accuracy.

It is a figure which shows the use environment of the service providing apparatus 1 in embodiment. It is a figure explaining an example of the determination process performed by the 2nd determination unit 25 in embodiment. It is a figure which shows an example of the news page P1 generated by the service providing apparatus 1 in embodiment. It is a figure which shows an example of the report page P2 generated by the service providing apparatus 1 in embodiment. It is a figure which shows an example of the report information 28A stored in the report information storage unit 28 in embodiment. It is a flowchart which shows the flow of the 1st determination process of the inappropriate comment by the service providing apparatus 1 in embodiment. It is a figure which shows an example of the comment information 26B stored in the content information storage unit 26 in embodiment. It is a figure which shows an example of the user information 30A stored in the user information storage unit 30 in embodiment. It is a figure which shows an example of the comment information 26B stored in the content information storage unit 26 in embodiment. It is a figure which shows an example of the learning data in an embodiment. It is a flowchart which shows the flow of the 2nd determination process of the inappropriate comment by the service providing apparatus 1 in embodiment. It is a flowchart which shows the flow of the news page generation processing by the service providing apparatus 1 in embodiment. It is a figure which shows an example of the news page P1 generated by the service providing apparatus 1 in embodiment.

Hereinafter, the information processing apparatus, the information processing method, and the embodiment of the program of the present invention will be described with reference to the drawings. The information processing device is realized by one or more processors (computers). The information processing device determines whether or not the comment is an inappropriate comment based on the content reported by the user who viewed the comment on the content. If the information processing device determines that this comment is an inappropriate comment, it lowers the ranking of this comment or excludes it from the posting target.

FIG. 1 is a diagram showing a usage environment of the service providing device 1 (information processing device). The service providing device 1 is connected to one or more terminal devices T via the network NW, and communicates with each other via the network NW. The network NW includes, for example, WAN (Wide Area Network), LAN (Local Area Network), the Internet, a dedicated line, a wireless base station, a provider, and the like.

The service providing device 1 is referred to as, for example, a web server or an application server. The service providing device 1 provides content to be reproduced by a browser or an application program in response to a request from the terminal device T.

The content provided by the service providing device 1 includes, for example, a web page referenced by a browser and an application page referenced by an application program. Such web pages or app pages include any page that implements the comment posting feature. For example, a web page includes pages such as a news site, an auction site, and a question site. In addition, the content includes messages posted on social network services (SNS) such as miniblogs. In the following, a case where the content provided by the service providing device 1 is a web page referenced by a browser and the web page is a page of a news site will be described as an example.

The content provided by the service providing device 1 includes the main content (first content) and comments (second content) indicating opinions, impressions, etc. regarding the first content posted by a third party. .. Based on the content reported by the user who viewed the comment, the service providing device 1 includes a comment whose comment is not appropriate to be disclosed to the public, such as slander or slander against a specific person (hereinafter, "non-existent"). It is also called "appropriate comment"). The service providing device 1 lowers or deletes the posting order of inappropriate comments among the comments made on a certain content, and then provides the comments to the user.

The terminal device T is operated by a user who browses a page of a news site provided by the service providing device 1. The terminal device T is, for example, a personal computer, a mobile phone such as a smartphone, a tablet terminal, or a computer device such as a PDA (Personal Digital Assistant). The terminal device T realizes the following functions by operating a UA (User Agent) such as a browser or an application program. The terminal device T transmits a request for a news site page provided by the service providing device 1 based on the user's operation, and receives information about the news site page in response to the request from the service providing device 1. To display.

[Configuration of service providing device]
The service providing device 1 includes, for example, a communication unit 10 (reception unit), a content acquisition unit 12, a ranking determination unit 14, a page generation unit 16, a report information processing unit 18 (reception unit), and a first determination unit 20. A label giving unit 22, a learning unit 24, a second determination unit 25, a content information storage unit 26, a report information storage unit 28, a user information storage unit 30, and a model storage unit 32 are provided. Each functional unit included in the service providing device 1 may be distributed to a plurality of devices. For example, it may be realized by a device separate from the first determination unit 20, the labeling unit 22, the learning unit 24, and the other functional units. Further, for example, the first determination unit 20 and the learning unit 24 may be realized by a separate device.

The communication unit 10 communicates with the terminal device T or the like via the network NW. The communication unit 10 includes, for example, a communication interface such as a NIC.

The content acquisition unit 12 obtains the article information 26A (first content) of the news page requested by the user and the comment information 26B (second content) including one or more comments posted to the article information 26A. , Obtained from the content information storage unit 26.

The ranking determination unit 14 determines the posting order of one or more comments acquired by the content acquisition unit 12. The ranking determination unit 14 determines the ranking of comments based on, for example, the click rate of comments, the evaluation of comments, the expressions included in comments, and the like. For example, the ranking determination unit 14 determines the ranking of comments in descending order of click rate of comments. The ranking determination unit 14 determines the ranking of comments so as to lower the ranking or exclude the comments that are determined to be inappropriate comments.

The page generation unit 16 generates a news page including the article information 26A acquired by the content acquisition unit 12 and one or more comments included in the comment information 26B, and transmits the news page to the terminal device T. In this news page, the comments included in the comment information 26B are posted in the ranking determined by the ranking determination unit 14.

Further, when the page generation unit 16 receives a request for reporting an inappropriate comment by a user who has viewed the comment included in the news page via the communication unit 10, the page generation unit 16 generates a report page that accepts the input of the report content. It is transmitted to the terminal device T.

The report information processing unit 18 acquires the report content input by the user for the above report page and stores it in the report information storage unit 28.

The first determination unit 20 determines whether or not each of the comments included in the comment information 26B is an inappropriate comment based on the report information 28A stored in the report information storage unit 28.

Based on the determination result by the first determination unit 20, the label giving unit 22 indicates a label indicating an inappropriate comment for each of the comments included in the comment information 26B (hereinafter, referred to as an “inappropriate label”). Either a label indicating that the comment may be inappropriate (hereinafter referred to as "quasi-inappropriate label") or a label indicating that the comment is not inappropriate (hereinafter referred to as "appropriate label"). Label. The comment to which this label is attached becomes the learning data used for the learning of the learning unit 24.

The learning unit 24 performs machine learning using the data labeled by the labeling unit 22 (a set of a comment and a label) as learning data, and determines whether or not the comment is an inappropriate comment. To generate. The learning unit 24 stores the generated determination model 32A in the model storage unit 32.

The second determination unit 25 determines whether or not the comment is an inappropriate comment by using the determination model 32A generated by the learning unit 24. For example, when a new comment is input by the user to the comment input field of the page of the news site, the second determination unit 25 can determine whether or not this comment is an inappropriate comment. FIG. 2 is a diagram illustrating an example of determination processing performed by the second determination unit 25. The second determination unit 25 receives the comment information input by the user as the input information. Next, the second determination unit 25 determines whether or not the comment included in the input comment information is an inappropriate comment by using the determination model 32A stored in the model storage unit 32. Next, the second determination unit 25 outputs information regarding whether or not the comment is inappropriate as output information. The output information may be, for example, a probability value (inappropriate probability 0.8, etc.), or information indicating a positive example (appropriate), a negative example (inappropriate), or the like.

The learning unit 24 and the second determination unit 25 may perform processing using, for example, a support vector machine. Further, the learning unit 24 and the second determination unit 25 use a recurrent neural network (RNN), a multi-layered neural network (DNN), a convolutional neural network (CNN), and the like. The deep learning technology that was used may be adopted.

The content information storage unit 26 stores, for example, article information 26A indicating a news article and comment information 26B indicating a comment posted on a news article. The article information 26A may be updated as appropriate based on the operation of the person in charge of distributing the news article, or may be updated by batch processing performed at a predetermined timing such as daily. The comment information 26B may be appropriately updated based on the processing of posting a comment by the user on the news page.

The report information storage unit 28 stores, for example, report information 28A indicating the report content input by the user on the report page.

The user information storage unit 30 stores, for example, user information 30A including information indicating attributes for each user. The model storage unit 32 stores the determination model 32A generated by the learning unit 24.

Each functional unit of the service providing device 1 is realized, for example, by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). The service providing device 1 may include a plurality of processors for realizing each functional unit. In addition, some or all of these functional units are hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc. It may be realized by the circuit unit (including circuitry), or it may be realized by the cooperation of software and hardware.

Each storage unit of the service providing device 1 is, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), a flash memory, or a hybrid storage device in which a plurality of these is combined. Is realized by. Each storage unit of the service providing device 1 may be realized by a single device. Further, a part or all of each storage unit of the service providing device 1 may be an external device such as NAS or an external storage server that can be accessed by the service providing device 1.

[Acquisition process of report information]
Hereinafter, the processing of the service providing device 1 will be described. First, the process of acquiring the report information of inappropriate comments carried out by the service providing device 1 will be described. FIG. 3 is a diagram showing an example of a news page P1 generated by the service providing device 1 and displayed on the terminal device T. On the news page P1, for example, an article area R1 in which a news article is displayed, a first comment area R2 in which a first comment posted for this article is displayed, and a second comment are displayed. 2 Includes comment area R3. The news page P1 may be set with an area for displaying three or more comments.

In each of the first comment area R2 and the second comment area R3, a button B1 for receiving a report indicating that the comment by the user who viewed the comment is inappropriate and a button B2 for receiving a positive evaluation for the comment are received. ("Good" button) and button B3 ("bad" button) that accepts input of negative evaluation for comments are included. Each of the buttons B2 and B3 also displays the total number of times all users who viewed the comment pressed the button. This total number is incremented by 1 based on the user pressing button B2 or B3.

When the button B1 is pressed by the user operating the terminal device T, a report page for accepting the input of the report content by the user is displayed on the terminal device T. FIG. 4 is a diagram showing an example of a report page P2 displayed on the terminal device T when the button B1 of the second comment area R3 in FIG. 3 is pressed. The report page P2 accepts the specification of the type of inappropriate comment. Types of inappropriate comments include, for example, "sexism," "nationality discrimination," "racism," and the like. In the example shown in FIG. 4, the user selects a radio button set for each type of inappropriate comment to accept the designation of the type. In the example shown in FIG. 4, the case where "slandering / slander" is specified by the user is shown.

In addition, the report page P2 accepts the user's designation of inappropriate expressions in the comment. The example shown in FIG. 4 shows a case where the expression "I hate the voice" in the comment "I hate the voice of B. Never come out again." Is specified by the user. In addition, the report page P2 accepts free input (text input) such as a user's opinion on a comment as details of inappropriate contents.

By operating the terminal device T, the user inputs the above-mentioned report contents, and then presses the "send" button, so that the information regarding the inappropriate comment can be reported to the service providing device 1. The service providing device 1 stores the report content received from the terminal device T in the report information storage unit 28.

FIG. 5 is a diagram showing an example of the report information 28A stored in the report information storage unit 28. The report information 28A includes an "article ID" that identifies a news article, a "comment ID" that identifies a comment, and a "user ID" that identifies a user (reporter) who has reported an inappropriate comment. "Type" indicating the type of inappropriate comment specified by the user on page P2 of the above report, "inappropriate expression" indicating the inappropriate expression specified by the user, and inappropriate expression entered by the user. Contains one or more data associated with "inappropriate content details" indicating content details. As described above, the service providing device 1 can collect the report information regarding the inappropriate comment.

[Judgment processing of inappropriate comments (1st example)]
Hereinafter, an example of the inappropriate comment determination process performed by the service providing device 1 will be described. FIG. 6 is a flowchart showing the flow of the first determination process, which is a first example of the determination process of inappropriate comments by the service providing device 1. This first determination process may be performed by a batch process performed at a predetermined timing such as daily or weekly.

First, the service providing device 1 extracts one comment from the comment information 26B stored in the content information storage unit 26, and extracts the report information for the extracted comment from the report information storage unit 28 (S101).

FIG. 7 is a diagram showing an example of comment information 26B stored in the content information storage unit 26. For example, the service providing device 1 extracts the comment "I hate the voice of B. Never come out again" of the comment ID "C2" from the comment information 26B, and uses this comment ID "C2" as a search key in FIG. Report information is extracted from the indicated report information 28A.

Next, the service providing device 1 determines whether or not the number of reports for the extracted comments is equal to or greater than a predetermined threshold value (S103). It is assumed that comments for which a certain number of reports have been made are likely to be inappropriate comments. Therefore, the service providing device 1 makes a primary determination as to whether or not the comment is an inappropriate comment based on the number of reports. For example, it is determined whether or not the number of reports for the extracted comments is equal to or greater than the threshold value (for example, 10).

If the service providing device 1 determines that the number of reports for the extracted comment is not equal to or more than a predetermined threshold value, it cannot conclude that this comment is an inappropriate comment, and therefore assigns an "appropriate label" to this comment. (S105).

On the other hand, when it is determined that the number of reports for the extracted comments is equal to or greater than a predetermined threshold value, the service providing device 1 performs the following processing for determining the reliability of the report contents. If the report for a comment is made by some users who have a specific biased idea, opinion, etc., the reliability of this report may not be high. For example, if a comment on a political news article is reported by a user who has a different idea, opinion, etc. from this comment, even if the number of reports is sufficient, it is not always judged by the general public. It may not be an inappropriate comment. Therefore, the service providing device 1 refers to the distribution of the attributes of the user who made the report, determines whether or not the attributes are biased (S107), and extracts based on the determination result (degree of bias). Determine if the comment you made is inappropriate.

When the service providing device 1 determines that the distribution of the user's attributes is not biased, it determines that the report content for the comment is highly reliable, and assigns an "inappropriate label" to the comment (S109). .. On the other hand, when the service providing device 1 determines that the distribution of user attributes is biased, it determines that the reliability of the report content for the comment is low, and assigns a "quasi-inappropriate label" to this comment. (S111).

FIG. 8 is a diagram showing an example of the user information 30A stored in the user information storage unit 30. For example, the service providing device 1 acquires the user ID of the user who reported the extracted comment from the report information 28A shown in FIG. 5, and uses the acquired user ID as a search key to relate to the attribute of this user. Information is acquired from the user information 30A. Then, the service providing device 1 determines whether or not the distribution of the acquired user attributes is biased. For example, when the user's attributes are biased only to "male" and "30's", the service providing device 1 determines that the distribution of the user's attributes is biased, and trusts the content of the report for the comment. Judging that the character is low, a "quasi-inappropriate label" is given to this comment.

When determining the bias of the distribution of the user attributes, the service providing device 1 may generate a user vector (feature information) based on the user attributes, for example. The user vector may be vector data represented by a sparse vector in which the user's attributes are replaced with elements. Alternatively, the user vector may be generated based on the user's behavior history on the Web, the user's attribute information, and the like. The service providing device 1 may determine whether or not the distribution of the acquired user attributes is biased based on the inter-vector distance of the user vector.

The service providing device 1 may set the "quasi-inappropriate label" in more detail based on the degree of bias of the user's attributes as described above. For example, the service providing device 1 may be set so that the "quasi-inappropriate label" is represented by a numerical value from 0 to 1 based on the degree of bias of the user's attributes. The "quasi-inappropriate label" represented by this numerical value may be set to indicate, for example, that the closer it is to 1, the higher the possibility of being inappropriate.

Even if the service providing device 1 determines the above label by using the information on the number of times the "good" button B2 is pressed and the number of times the "bad" button B3 is pressed included in the comment information 26B. good. For example, the service providing device 1 may give an "inappropriate label" to a comment in which the number of times the "bad" button B3 is pressed is equal to or greater than a predetermined threshold value.

The service providing device 1 may store the information regarding the above determination result in the comment information 26B. FIG. 9 is a diagram showing an example of comment information 26B to which information regarding the determination result is added. In the example shown in FIG. 9, the labels "appropriate", "inappropriate", and "quasi-inappropriate" are added to each data.

The service providing device 1 repeatedly performs the above processes (S101 to S111) for each of the comments included in the comment information 26B.

Next, the service providing device 1 performs machine learning using the labeled comment (a set of the comment and the label) as training data, and generates the determination model 32A (S113). FIG. 10 is a diagram showing an example of learning data. FIG. 10 shows an example in which a set of a “comment” and a “label” extracted from the comment information 26B shown in FIG. 9 is used as learning data. The service providing device 1 stores the determination model 32A generated by performing machine learning using such learning data in the model storage unit 32. When a new comment is input by the user, the second determination unit 25 of the service providing device 1 uses the determination model 32A generated above and stored in the model storage unit 32, and the input comment is inappropriate. It can be determined whether or not it is a comment. As described above, the service providing device 1 ends the processing of this flowchart.

The service providing device 1 may learn the relationship between the inappropriate comment and the type of the inappropriate comment. Further, the service providing device 1 may use only the part of the "inappropriate expression" specified by the user as the learning data, instead of using the entire comment as the learning data. Further, the service providing device 1 may use only the comments with high reliability of the report content as the learning data instead of using all the comments as the learning data. Further, the service providing device 1 may incorporate the text of the news article into the learning data.

[Judgment processing of inappropriate comments (second example)]
Hereinafter, another example of the inappropriate comment determination process performed by the service providing device 1 will be described. FIG. 11 is a flowchart showing the flow of the second determination process, which is the second example of the determination process of the inappropriate comment by the service providing device 1. This second determination process may be performed by a batch process performed at a predetermined timing such as daily or weekly.

First, the service providing device 1 extracts one comment from the comment information 26B stored in the content information storage unit 26, and extracts the report information for the extracted comment from the report information storage unit 28 (S201).

Next, the service providing device 1 determines whether or not the number of reports for the extracted comments is equal to or greater than a predetermined threshold value (S203).

If the service providing device 1 determines that the number of reports for the extracted comment is not equal to or more than a predetermined threshold value, it cannot conclude that the comment is inappropriate, and therefore assigns an "appropriate label" to this comment (S205). ..

On the other hand, when it is determined that the number of reports for the extracted comments is equal to or greater than a predetermined threshold value, the service providing device 1 performs the following processing for determining the reliability of the report contents. It is assumed that inappropriate comments tend to appear in the distribution of attributes of users who report, depending on the type. For example, reports on inappropriate comments regarding "sexism (discrimination against women)" tend to be made by "female" users who are prone to dislike this comment. Therefore, if many of the attributes of the user who made a report on inappropriate comments regarding "sex discrimination (discrimination against women)" are "female", it is assumed that the report is highly reliable. On the other hand, if many of the attributes of the user who made the report are "male", it is assumed that the reliability of the report is low. Therefore, the service providing device 1 defines a reference distribution as a reference for each type of inappropriate comment in advance, and determines whether or not the distribution of the attributes of the user who actually made the report is similar to this reference distribution. (S207), and based on this determination result (similarity), it is determined whether or not the extracted comment is inappropriate.

When the service providing device 1 determines the similarity of the distribution of the user attributes, for example, the service providing device 1 generates a user vector (feature information) based on the user attributes, and the vector between the user vectors. Based on the distance, it may be determined whether or not the acquired distribution of the user's attributes is similar to the reference distribution.

When the service providing device 1 determines that the distribution of the attributes of the user who made the report is similar to the reference distribution, it determines that the content of the report is highly reliable for the comment, and "inappropriate" for this comment. "Label" is given (S209). On the other hand, when the service providing device 1 determines that the distribution of the attributes of the user who made the report does not resemble this reference distribution, the service providing device 1 determines that the reliability of the report content for the comment is low, and the service providing device 1 determines that the reliability of the report content is low. A "quasi-inappropriate label" is given (S211).

The service providing device 1 may set the "quasi-inappropriate label" in more detail based on the similarity of the distribution of the user's attributes as described above. For example, the service providing device 1 may be set so that the "quasi-inappropriate label" is represented by a numerical value from 0 to 1 based on the similarity of the distribution of the user's attributes. The "quasi-inappropriate label" represented by this numerical value may be set to indicate, for example, that the closer it is to 1, the higher the possibility of being inappropriate.

The service providing device 1 repeatedly executes the above processes (S201 to S211) for each of the comments included in the comment information 26B.

Next, the service providing device 1 performs machine learning using a comment with a label (a set of the comment and the label) as training data, and generates a determination model 32A (S213). The service providing device 1 stores the generated determination model 32A in the model storage unit 32. As described above, the service providing device 1 ends the processing of this flowchart.

[News page generation process]
Hereinafter, an example of the news page generation process performed by the service providing device 1 will be described. FIG. 12 is a flowchart showing a flow of a news page generation process by the service providing device 1. This news page generation process is executed every time a news page request from the terminal device T is received.

First, the service providing device 1 receives a request for a news page transmitted by the terminal device T based on the operation of the user (S301).

Next, the service providing device 1 acquires the article information of the requested news page and the comment information posted for this article information from the content information storage unit 26 (S303).

Next, the service providing device 1 tentatively determines the posting order of the acquired comments (S305). The service providing device 1 tentatively determines the posting order of comments based on, for example, the click rate of comments, the evaluation of comments, the expressions included in comments, and the like. For example, the service providing device 1 tentatively determines the posting order of comments in descending order of the click rate of comments.

Next, the service providing device 1 extracts one comment from the comments whose publication order is tentatively determined, and determines whether or not the extracted comment is given an inappropriate label or a quasi-inappropriate label. (S307). When the service providing device 1 determines that the extracted comment is given an inappropriate label or a quasi-inappropriate label, the posting order of the comment is lowered or excluded from the posting target. Is determined (S309). On the other hand, when the service providing device 1 determines that the extracted comment is not given an inappropriate label or a quasi-inappropriate label, that is, when it determines that the extracted comment is an appropriate comment, the service providing device 1 determines the above-mentioned ranking. No processing such as lowering is performed.

Next, the service providing device 1 determines whether or not the processing for all comments is completed (S311). When the service providing device 1 determines that the processing for all the comments has not been completed, the service providing device 1 extracts the unprocessed comments and repeats the above processing. On the other hand, when it is determined that the processing for all the comments is completed, the service providing device 1 generates a news page including the acquired article information and the comment included in the comment information, and transmits the news page to the terminal device T (S313). .. On this news page, comments will be posted in the ranking determined by the above ranking determination process.

FIG. 13 is a diagram showing an example of the news page P1 generated by the service providing device 1. Regarding the comments included in the news page P1, comments with inappropriate labels or quasi-inappropriate labels are either set in the lower rank or excluded from the publication target. This completes the processing of this flowchart.

[Inappropriate judgment processing for newly posted comments]
Hereinafter, an example of the inappropriateness determination process of the newly posted comment performed by the service providing device 1 will be described. The service providing device 1 is a model when, for example, a new comment is posted to the comment input field of the news site page based on the operation of the terminal device T by the user, and a request to post the new comment is received. Using the determination model 32A stored in the storage unit 32, it is determined whether or not this new comment is inappropriate. Based on such a judgment, it is judged in real time whether or not the newly posted comment is inappropriate, and the ranking of the comment judged to be inappropriate is set to the lower rank or excluded from the posting target. Can be controlled to do so.

For example, when the service providing device 1 determines that the new comment is not inappropriate, it assigns an "appropriate label" to this comment. On the other hand, when the service providing device 1 determines that the new comment posted is inappropriate, it assigns an "inappropriate label" to this comment. The service providing device 1 stores the labeled comment in the comment information 26B of the content information storage unit 26.

The service providing device 1 may give a "quasi-inappropriate label" to a comment for which it cannot be determined whether or not it is inappropriate. Further, the service providing device 1 may set the "quasi-inappropriate label" in more detail based on the determination result using the determination model 32A. For example, the service providing device 1 may be set so that the "quasi-inappropriate label" is represented by a numerical value from 0 to 1. The "quasi-inappropriate label" represented by this numerical value may be set to indicate, for example, that the closer it is to 1, the higher the possibility of being inappropriate.

According to the service providing device 1 of the embodiment described above, it is possible to detect inappropriate comments quickly and with high accuracy. Further, according to the service providing device 1 of the embodiment, it is possible to automatically reduce the chance of viewing inappropriate comments by lowering the posting order of inappropriate comments or excluding them from the posting target. Is.

The service providing device 1 may generate a comment vector (feature information) based on the comment. The comment vector may be vector data in a distributed representation. The service providing device 1 may determine whether or not the new comment is an inappropriate comment based on the distance between the vector of the comment vector of the known inappropriate comment and the comment vector of the new comment.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 ... Service providing device, 10 ... Communication unit, 12 ... Content acquisition unit, 14 ... Ranking determination unit, 16 ... Page generation unit, 18 ... Report information processing unit, 20 ... First judgment unit, 22 ... Labeling unit, 24 Learning unit, 25 Second judgment unit, 26 Content information storage unit, 28 Report information storage unit, 30 User information storage unit, 32 Model storage unit, T Terminal device, NW network

Claims (9)

A reception unit that accepts reports indicating that the second content is inappropriate by a user who has viewed the second content, which is a comment posted on the first content.
Based on the report received by the reception unit, the first determination unit that determines whether or not the second content is inappropriate, and the first determination unit.
With
The first determination unit determines whether or not the second content is inappropriate based on the degree of bias in the distribution of the attributes of the user who made the report.
Information processing device. When the first determination unit determines that the second content is inappropriate, the order of determining the ranking of the second content so as to lower the ranking of the second content or exclude it from the posting target. The decision department and
A page generation unit that generates a web page including the first content and the second content based on the placement order determined by the ranking determination unit.
The information processing apparatus according to claim 1. When the number of reports received by the reception unit is equal to or greater than the threshold value, the first determination unit determines that the second content is inappropriate.
The information processing device according to claim 1 or 2. The first determination unit determines that the lower the degree of bias, the higher the reliability of the report by the user.
The information processing device according to any one of claims 1 to 3. A reception unit that accepts reports indicating that the second content is inappropriate by a user who has viewed the second content, which is a comment posted on the first content.
Based on the report received by the reception unit, the first determination unit that determines whether or not the second content is inappropriate, and the first determination unit.
With
The reception unit receives a report from the user indicating the type of inappropriate content included in the second content, and receives a report.
The first determination unit does not have the second content based on the similarity between the distribution bias of the attribute of the user who made the report and the reference distribution defined in advance for each type of inappropriate content. Determine if it is appropriate,
Information processing device. A learning unit that learns the relationship between the second content and a label indicating a determination result indicating whether or not the second content by the first determination unit is inappropriate and generates a determination model is further provided. ,
The information processing device according to any one of claims 1 to 5. A second determination unit for determining whether or not the second content is inappropriate is further provided by using the determination model generated by the learning unit.
The information processing device according to claim 6. The computer
We accept reports from users who have viewed the second content, which is a comment posted for the first content, indicating that the second content is inappropriate.
Based on the received report, it is determined whether or not the second content is inappropriate.
Information processing method
It is determined whether or not the second content is inappropriate based on the degree of bias in the distribution of the attributes of the user who made the report.
Information processing method. On the computer
A user who browses the second content, which is a comment posted on the first content, is allowed to accept a report indicating that the second content is inappropriate.
Based on the received report, it is determined whether or not the second content is inappropriate.
It ’s a program
Based on the degree of bias in the distribution of the attributes of the user who made the report, it is determined whether or not the second content is inappropriate.
program.

Images