WO2023148947A1

WO2023148947A1 - Evaluation device, evaluation method, and program

Info

Publication number: WO2023148947A1
Application number: PCT/JP2022/004576
Authority: WO
Inventors: 啓太鈴木; 篤中平; 盛徳大橋; 穂乃香戸田; 志高土屋
Original assignee: 日本電信電話株式会社
Priority date: 2022-02-07
Filing date: 2022-02-07
Publication date: 2023-08-10

Abstract

The present invention provides an evaluation device and the like for evaluating the trustworthiness of a user from information evaluated for trustworthiness. The evaluation device evaluates the trustworthiness of a user of social media. Provided that a user subject to evaluation is a to-be-evaluated user, information identifying the to-be-evaluated user is a to-be-evaluated user identifier, and users followed by the to-be-evaluated user are a primary followed users, the evaluation device includes a followed user collection unit that obtains the to-be-evaluated user identifier and a list of primary followed users, a post information collection unit that obtains information on posts made by the to-be-evaluated user and information on posts made by the primary followed users, and a trustworthiness value calculation unit that evaluates posts made by the primary followed users and uses this evaluation as a basis for evaluating the to-be-evaluated user.

Description

Evaluation device, evaluation method, and program

The present invention relates to a device, method, and program for evaluating the reliability of any person.

Non-Patent Document 1 is known as a conventional technique for evaluating the reliability of information on social media. In Non-Patent Document 1, a classifier that classifies whether or not a post is reliable is created in advance, posts to be classified are analyzed, and from there, user posts, repost behavior, posted sentences, and external sources are analyzed. Citations are extracted as features, and the extracted features are given to a classifier to classify posts into reliable and unreliable posts.

In addition, in Non-Patent Document 2, users who have posted certain information are classified into users who are oriented to disseminate information to the general public and users who mainly use tools for communication within the community, Based on the classification results, evaluate whether the information is widely useful information.

However, in the prior art, even if humans evaluate the reliability and usefulness of information (see FIG. 1), there is a method for evaluating the user's reliability from information whose reliability has been evaluated (see FIG. 2). is not proposed. For example, in Non-Patent Document 1, training posts used when creating a classifier are manually evaluated, and human evaluation is performed indirectly when classifying posts to be classified. ing.

An object of the present invention is to provide an evaluation device, an evaluation method, and a program for evaluating a user's reliability from information whose reliability has been evaluated.

In order to solve the above problems, according to one aspect of the present invention, an evaluation device evaluates the credibility of users of social media. The evaluation device defines a user to be evaluated as an evaluated user, information identifying the evaluated user as an evaluated user identifier, a user followed by the evaluated user as a primary follow user, and an evaluated user identifier and 1 A follow user collection unit that acquires a list of next follow users, a post information collection unit that acquires posted information of rated users and posted information of primary follow users, evaluates posts of primary follow users, a confidence value calculation unit that evaluates the evaluated user based on the evaluation.

According to the present invention, it is possible to evaluate the user's reliability from information whose reliability has been evaluated. The evaluation result can be used for personnel matching and the like.

A diagram for explaining how humans evaluate the reliability and usefulness of information. FIG. 4 is a diagram for explaining a method of evaluating user reliability from information; FIG. 2 is a functional block diagram of the evaluation device according to the first embodiment; The figure which shows the example of the processing flow of the evaluation apparatus which concerns on 1st embodiment. FIG. 5 is a diagram showing an example of data stored in a follow user storage unit; FIG. 4 is a diagram showing an example of data stored in a posted information storage unit; FIG. 4 is a diagram for explaining a processing flow of a reliability value calculation unit; FIG. 4 is a diagram showing an example of data stored in a reliability value storage unit; The figure which shows the functional structural example of a computer.

The embodiment of the present invention will be described below. In the drawings used for the following description, the same reference numerals are given to components having the same functions and steps performing the same processing, and redundant description will be omitted. In the following description, processing performed for each element of a vector or matrix applies to all elements of the vector or matrix unless otherwise specified.

<Definition of terms>
Social media: A general term for various information exchange services formed by individuals transmitting information using Internet technology. Social media users can post and view posts. For example, twitter (registered trademark) is known as a service that provides social media.

Posting: Posting means publishing information such as sentences, images, videos, etc., or information that has been made public, at a designated place on the Internet. For example, posts include tweets on twitter.

Follow: To follow is to register a specific user so that their posts will be displayed continuously.

Followed user: A user followed by user A is called a "followed user of user A".

Follower user: A user who follows user A is called a "follower user of user A".

Profile: Information indicating characteristics of the user. In social media, for example, users register their own profiles.

Post evaluation value: The post evaluation value is a value representing the user's evaluation of a certain post. For example, if there is a function in social media for each user to indicate positive intentions (like, fun, instructable, interested, etc.) for a certain post, the number of users who indicated positive intentions is Post rating value. For example, the number of "likes" for a certain tweet on twitter corresponds to the evaluation value.

Number of reactions: The number of reactions represents the total number of posts (replies) to a certain post, posts quoting a certain post, and the like. For example, the total number of replies and retweets to a certain tweet on twitter corresponds to the number of reactions.

<Points of the first embodiment>
In the present embodiment, social media users (users to be evaluated, hereinafter also referred to as "evaluated users") post by follower B of user A are calculated as follows: The number of follower users of follow user B (number of followers), the number of interactions between evaluated user A and follow user B, the content of posts, the time of posting, and the like are used for evaluation. Hereinafter, for the sake of convenience, the following users of the evaluated user are also referred to as primary following users, and the following users of the primary following user are also referred to as secondary following users. Note that the interaction means a series of exchanges between the rated user A and the follow user B in response to a post by the rated user A or the follow user B. The number of interactions means the number of posts included in a certain interaction. For example, when follower user B makes a post and rated user A posts (replies) to the post, the number of interactions is two.
Furthermore, when evaluating user A, in addition to information viewed by evaluated user A (posts of primary follow users), the degree of involvement and matching of preferences between evaluated user A and primary follow users take degrees into account.

<Evaluation Apparatus According to First Embodiment>
FIG. 3 is a functional block diagram of the evaluation device according to the first embodiment, and FIG. 4 shows its processing flow.

The evaluation device includes a follow user collection unit 110, a follow user storage unit 115, a post information collection unit 120, a post information storage unit 125, a confidence value calculation unit 130, a confidence value storage unit 135, and an output unit 140. including.

The evaluation device receives information identifying the evaluated user (hereinafter also referred to as "evaluated user identifier") and evaluates the evaluated user.

In this embodiment, the evaluation device receives a plurality of evaluated user identifiers, evaluates the plurality of evaluated users, and stores the evaluated user identifiers and the evaluation results. A plurality of user identifiers to be evaluated may be given by the administrator of the evaluation device or the like. A plurality of rated user identifiers may be obtained by using the user as the rated user.

Furthermore, when the user of the evaluation device inputs an evaluated user identifier into the evaluation device, the evaluation device outputs evaluation results corresponding to the input evaluated user identifier.

Each part will be explained below.

<Follow User Collection Unit 110 and Follow User Storage Unit 115>
The follow user collection unit 110 receives a plurality of evaluated user identifiers, acquires the evaluated user identifiers and a list of primary follow users from the server 90 that provides social media (S110), and stores them in the follow user storage unit 115. . For example, if a user can be identified by an account name or user name, the account name or user name may be used as the evaluated user identifier. The list of primary follow users includes information for identifying primary follow users (hereinafter also referred to as "primary follow user identifier"). FIG. 5 shows an example of data stored in the follow user storage unit 115. As shown in FIG. In FIG. 5, the identifier used in the evaluation apparatus (in-apparatus identifier) is given to the evaluated user identifier used in the server 90, but this does not necessarily have to be given. A UID (User Identifier) or a URI (Uniform Resource Identifier) can be used as the intra-device identifier.

<Posted Information Collection Unit 120 and Posted Information Storage Unit 125>
The posted information collection unit 120 extracts the rated user identifier and the list of primary follow users from the follow user storage unit 115, and acquires the posted information of the rated user and the posted information of the primary follow user from the server 90 ( S120). Posted information is information related to a post, and is information that can be obtained from the server 90 . The posted information collection unit 120 extracts or generates the following information from the posted information of the rated user and the posted information of the primary follow user, and stores the information in the posted information storage unit 125 .

・Identifier that indicates the combination of the rated user and the primary follow user (hereinafter also referred to as "combination identifier")
・Evaluated user identifier ・Primary follow user identifier ・Contents posted by the primary follow user ・Profile of the primary follow user ・Contents posted by the rated user ・Profile of the rated user Number of interactions (denoted as “number of interactions” in the figure)
Posting Time of Primary Follow User Further, the posted information collection unit 120 may extract or generate the following information and store it in the posted information storage unit 125 .

・Information indicating whether or not a URL is included in the post of the primary follower ・Quoted URL included in the post ” and other URLs are stored)
・Information indicating whether or not an image is included in the post of the 1st follower ・Evaluation value of the post of the 1st follower ・Reaction value of the post of the 1st follower ・Number of secondary followers of the 1st follower ・Number of Follower Users of Primary Follow User/Number of Posts of Primary Follow User FIG. 6 shows an example of data stored in the posted information storage unit 125 . The post information collection unit 120 generates a record for each post by the primary follow user, and stores the record in the post information storage unit 125 . In addition, "Posted content of the rated user" is a post related to the "Post of the 1st follow user" of the record. Posts that are replied by "posts", posts that quote "posts of the primary follow user", posts that are quoted by "posts of the primary follow user", and the like.
Furthermore, the posted information collection unit 120 extracts “posts in which the rated user has indicated positive intentions” from the posted information of the rated user, and stores them in the posted information storage unit 125 in association with the rated user identifier. do. However, "posts in which the evaluated user has indicated a positive intention" are not illustrated.

<Confidence Value Calculation Unit 130 and Confidence Value Storage Unit 135>
The confidence value calculation unit 130 evaluates the post of the rated user's primary follow user, evaluates the rated user based on this evaluation (S130), and stores the evaluation value of the rated user in the confidence value storage unit 135. Remember. With such a configuration, the rated user can be evaluated based on the information surrounding the rated user.

At this time, the confidence value calculation unit 130 updates the evaluation of the post by the primary follow user of the rated user, taking into consideration the categories that are assumed to be of interest to the rated user and the primary follow user.

In addition, the reliability value calculation unit 130 updates the evaluation of the post by the primary follow user of the rated user, taking into consideration the posting time of the primary follow user.

FIG. 7 is a diagram for explaining the processing flow of the reliability value calculation unit 130. FIG.

(S130-1) Evaluation of Reliability of Posting For example, the reliability value calculation unit 130 evaluates the reliability of a post by a primary follow user of an evaluated user through the following process.

Let P be the total number of 1st-order followers of a given user, p=1,2,...,P, and _Qp be the total number of posts of the p-th 1st-order follower, _qp =1,2,... , Q _p . Let S _p (q _p ) be the q _p -th post of the p -th primary follower user.

The confidence value calculation unit 130 retrieves a record corresponding to the post S _p (q _p ) of the primary follow user from the post information storage unit 125 . Included in it ・The evaluation value B(S _p (q _p )) of the post S _p (q _p ) of the first follower user
・Reaction value C(S _p (q _p )) of post S _p (q _p ) of primary follower user
・The number of followers D(p) of the primary follower user
is used, the reliability evaluation value A(S _p (q _p )) of the post S _p (q _p ) of the primary follower user is obtained by the following equation.

A(S _p (q _p ))=(b×B(S _p (q _p ))+c×C(S _p (q _p )))/D(p)
Note that b and c are weights for the evaluation value B (S _p (q _p )) and the reaction value C (S _p (q _p )), respectively, and are calculated by simulation or the like prior to evaluation. Here, the higher the evaluation value B (S _p (q _p )) and the reaction value C (S _p (q _p )), the higher the reliability of the post S _p (q _p ). The smaller the evaluation value B (S _p (q _p )) and the reaction value C (S _p (q _p )) with respect to the number of followers D(p), the lower the reaction of other users and the more posts S _p (q _p ) is evaluated as unreliable.
Further, the confidence value calculation unit 130 calculates information indicating whether or not the URL is included in the post S _p (q _p ), which is included in the record corresponding to the post S _p (q _p ), and the post S _p (q _p ). If the post S p (q p ) contains a URL or an image, using information indicating whether the post S _p (q _p ) contains an image
A(S _p (q _p ))←αA(S _p (q _p ))
, the evaluation value A(S _p (q _p )) is updated. Note that α is a constant greater than 1, and is calculated by simulation or the like prior to evaluation. Here, posts S _p (q _p ) that contain URLs or images are evaluated as more reliable than those that do not. The above processing is performed for p=1,2,...,P and _qp =1,2,..., _Qp .

(S130-2) Category Classification Confidence Value Calculation Unit 130 retrieves from post information storage unit 125 posts that the evaluated user has indicated positive intentions (like, fun, can instruct, is interested in, etc.), and posts them. Included in the record corresponding to S _p (q _p ) ・Posted content of the 1st follower user ・Profile of the 1st follower user ・Posted content of the rated user ・Profile of the rated user The category of the post indicating the intention is classified, and the classified result is linked to the post and the profile and stored in the posted information storage unit 125 . Prior to the evaluation, a classifier for category classification may be prepared in a server or the like and obtained from the server or the like. Categories may include various categories such as careers, games, sports, business, computer technology, and so on. A single post or profile may be classified into multiple categories.
(S130-3) Update of evaluation considering category The confidence value calculation unit 130 calculates the contents of the post S p (q p ) included in the record corresponding to the post S _p (q _p ), the post content of the primary follow user and its category classification result, and the evaluated user and the number of interactions with first-order follow users, the coefficient β is set so that the greater the number of interactions with posts in a certain category, the larger the coefficient β. For example, if there are categories A, B, and C, and the number of interactions of posts S _p (q _p ) for category A is large, the coefficient β for category A is set to be large. Set the coefficient β for each primary follow user p. Furthermore, the coefficient γ may be set for each primary follow user p and for each category, or different R coefficients γ may be set for the top R categories for each primary follow user p. . For example, R=1 and β may be set only for the category with the largest number of interactions. β is a constant greater than 1, and is calculated by simulation or the like prior to evaluation.
Further, the confidence value calculation unit 130 uses the contents of posts by the rated user and their categorization results and the profile of the rated user and their categorization results, which are included in the record corresponding to the post S _p (q _p ), to A category in which the evaluated user is highly interested is obtained from the classification result. If the category in which the rated user is highly interested and the category of the post S _p (q _p ) are the same, the number of interactions related to the post S _p (q _p ) is multiplied by a constant, and the constant multiplied number of interactions is used to calculate the coefficient β set.

Here, when the number of interactions between the evaluated user and the primary follow user is large, the category related to that interaction is of interest, and the post of the primary follow user regarding that category is evaluated as highly reliable.
Confidence value calculation unit 130 extracts from post information storage unit 125 posts that the evaluated user has indicated positive intentions for, and their classification results, and sets coefficient γ to increase as the number of posts in each category increases. may Note that the coefficient γ may be set for each category, or R different coefficients γ may be set for the top R categories. For example, R=1 and γ may be set only for the category with the largest number of posts. In short, any setting may be made as long as the coefficient γ is set to increase as the number of posts in each category increases. Here, the category to which many "posts in which the rated user indicated positive intentions" belongs is of high interest to the rated user, and the post of the first-follow user in that category is evaluated as highly reliable. .
The confidence value calculation unit 130 updates the evaluation value A(S _p (q _p )) of the post S _p (q _p ) to one that considers the category, using the following equation.

A(S _p (q _p ))←β×γ×A(S _p (q _p ))
(S130-4) Update of evaluation considering posting time Further, the reliability value calculation unit 130 uses the posting time of the primary follow user included in the record corresponding to the post S _p (q _p ) to calculate the posting time The coefficient δ is set to increase as the elapsed time from . For example, if the elapsed time from the posting time is less than 1 day, δ=3, if the elapsed time from the posting time is less than 1 week, δ=2, and the elapsed time from the posting time is greater than 1 week. set δ=1. Here, it is assumed that the shorter the elapsed time from the posting time, the higher the reliability of the posting.

The confidence value calculation unit 130 updates the evaluation value A(S _p (q _p )) of the post S _p (q _p ) to one that considers the category and the posting time using the following equation.

A(S _p (q _p ))←δ×A(S _p (q _p ))
(S130-5) Evaluated User's Evaluation Confidence Value Calculation Unit 130 calculates the sum of evaluation values A(S _p (q _p )) of all posts by all first-order follow users for an evaluated user.

and the total number of posts by all first-order follow users for a given rated user.

to obtain the evaluation value of the reliability of a certain evaluated user, and the combination of the evaluated user identifier and the evaluation value is recorded in the reliability value storage unit 135 . Furthermore, categories that are presumed to be of interest to the rated user may be stored together. For example, even if "categories in which the rated user is highly interested" or "categories with many interactions" obtained from the content posted by the rated user and the profile of the rated user are assumed to be the categories in which the rated user is interested. Alternatively, the category in which the rated user is interested may be estimated based on the number of posts corresponding to the 'category in which the rated user is highly interested' and the number of posts in the 'category with a large number of interactions'. FIG. 8 shows an example of data stored in the reliability value storage unit 135. As shown in FIG. By storing together the categories that the evaluating user is presumed to be interested in, it becomes clear in which category the reliability value of the evaluated user indicates the reliability value.
<Output unit 140>
The output unit 140 receives the evaluated user identifier, acquires the evaluation value corresponding to the evaluated user identifier from the reliability value storage unit 135, and outputs it (S140).

For example, when the user of the evaluation device wants to obtain an evaluation of the reliability of an evaluated user, the user inputs the identifier of the user to be evaluated into the evaluation device, and the evaluation device outputs the evaluation result.

<effect>
With the above configuration, it is possible to evaluate the reliability of the user from the information whose reliability has been evaluated, and the evaluation result can be used for personnel matching or the like.

<Modification>
The evaluation device may be configured to evaluate the user to be evaluated, that is, to perform S110, S120, and S130 each time the user of the evaluation device inputs the identifier of the user to be evaluated, and output the evaluation result. In this case, the evaluation device does not include the confidence value storage unit 135 and the output unit 140, the follow user collection unit 110 performs the processing S110 based on the input evaluated user identifier, and the output of the confidence value calculation unit 130 is output to the evaluation device. is the output of

The confidence value calculation unit 130 may not consider whether or not the post S _p (q _p ) includes a URL or an image.
A(S _p (q _p ))←αA(S _p (q _p ))
omits the process of updating the evaluation value A(S _p (q _p )).

The confidence value calculation unit 130 may not consider the category, in which case
A(S _p (q _p ))←β×γ×A(S _p (q _p ))
omits the process of updating the evaluation value A(S _p (q _p )). Therefore, processing S130-2 and S130-3 can be omitted. Furthermore, the processing related to only one of β and γ may be omitted.

The confidence value calculation unit 130 may not consider the posting time, in which case
A(S _p (q _p ))←δ×A(S _p (q _p ))
omits the process of updating the evaluation value A(S _p (q _p )). Therefore, processing S130-4 can be omitted.

In this embodiment, the evaluation values are designed so that the higher the evaluation value, the higher the evaluation of the post and the user being evaluated. An evaluation value may be designed. In short, any index may be used as long as it is possible to determine whether the evaluation is high or low based on the magnitude of the evaluation value.

When the user of the evaluation device inputs a category into the evaluation device, the evaluation device may output an evaluated user identifier with a high reliability value for the input category. In this case, the output unit 140 receives information specifying a category, acquires from the reliability value storage unit 135, and outputs the evaluated user identifiers for the top R evaluation values in the category.

<Other Modifications>
The present invention is not limited to the above embodiments and modifications. For example, the various types of processing described above may not only be executed in chronological order according to the description, but may also be executed in parallel or individually according to the processing capacity of the device that executes the processing or as necessary. In addition, appropriate modifications are possible without departing from the gist of the present invention.

<Program and recording medium>
In the above-described various processes, the recording unit 2020 of the computer 2000 shown in FIG. It can be implemented by

A program that describes this process can be recorded on a computer-readable recording medium. Any computer-readable recording medium may be used, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or the like.

In addition, the distribution of this program is carried out, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Further, the program may be distributed by storing the program in the storage device of the server computer and transferring the program from the server computer to other computers via the network.

A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or the program transferred from the server computer once in its own storage device. Then, when executing the process, this computer reads the program stored in its own recording medium and executes the process according to the read program. Also, as another execution form of this program, the computer may read the program directly from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to this computer. Each time, the processing according to the received program may be executed sequentially. In addition, the above-mentioned processing is executed by a so-called ASP (Application Service Provider) type service, which does not transfer the program from the server computer to this computer, and realizes the processing function only by its execution instruction and result acquisition. may be It should be noted that the program in this embodiment includes information used for processing by a computer and conforming to the program (data that is not a direct command to the computer but has the property of prescribing the processing of the computer, etc.).

In addition, in this embodiment, the device is configured by executing a predetermined program on a computer, but at least part of these processing contents may be implemented by hardware.

110 follow user collection unit 115 follow user storage unit 120 post information collection unit 125 post information storage unit 130 confidence value calculation unit 135 confidence value storage unit 140 output unit

Claims

An evaluation device for evaluating the credibility of a user of social media,
The user to be evaluated is defined as the evaluated user, the information identifying the evaluated user is defined as the evaluated user identifier, and the user followed by the evaluated user is defined as the primary follow user,
a follow user collection unit that acquires a list of rated user identifiers and primary follow users;
a posted information collecting unit that acquires the posted information of the rated user and the posted information of the primary follow user;
a confidence value calculation unit that evaluates the post of the primary follow user and evaluates the evaluated user based on this evaluation;
Evaluation device.
The evaluation device of claim 1,
The confidence value calculation unit
updating the rating of the post of the primary follow user, taking into account the categories presumed to be of interest to the rated user and the primary follow user;
Evaluation device.
The evaluation device according to claim 2,
The number of posts included in a series of interactions between the rated user and the primary follow user for the posts of the rated user or the primary follow user is defined as the number of interactions,
The reliability value calculation unit updates the evaluation value so that the evaluation of the post by the primary follow user increases as the number of interactions of the post regarding a certain category increases.
Evaluation device.
The evaluation device according to claim 3,
The reliability value calculation unit multiplies the number of interactions related to the post by a constant when the category in which the evaluated user is highly interested and the category of the post by the primary follow user are the same, and evaluates based on the number of interactions multiplied by the constant. update the value,
Evaluation device.
The evaluation device according to any one of claims 2 to 4,
The reliability value calculation unit uses the results of categorizing posts in which the rated user has indicated positive intentions, and determines that the greater the number of posts in each category, the more posts the primary follow user belonging to the category. Update the rating value so that the rating is higher,
Evaluation device.
The evaluation device according to any one of claims 1 to 5,
The confidence value calculation unit
updating the evaluation of the post of the primary follow user in consideration of the posting time of the post of the primary follow user;
Evaluation device.
An evaluation method for evaluating the reliability of a social media user using an evaluation device,
The user to be evaluated is defined as the evaluated user, the information identifying the evaluated user is defined as the evaluated user identifier, and the user followed by the evaluated user is defined as the primary follow user,
a follow user collection step of obtaining a list of rated user identifiers and primary follow users;
a posted information collecting step of acquiring the posted information of the rated user and the posted information of the primary follow user;
a confidence value calculation step of evaluating the post of the primary follow user and evaluating the evaluated user based on this evaluation;
Evaluation method.
A program for causing a computer to function as the evaluation device according to any one of claims 1 to 6.