JP7453191B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

Conventionally, methods for predicting user behavior on the Internet have been known. For example, in Patent Document 1, with the aim of predicting user behavior with higher accuracy than before, the tendency of search queries of users on the Internet is learned, and the learned tendency of search queries and subsequent user behavior are disclosed. A technique has been proposed in which a model is generated by calculating a relevance score with a user, and this model is used to predict the behavior of other users.

Japanese Patent Application Publication No. 2016-177377

However, with the above-mentioned techniques, it is difficult to obtain certainty as to whether or not the generated information can be trusted. For example, there is a possibility that a model that produces effective results may be included, and even if an AB test is conducted, an inappropriate model may be adopted depending on the content of the test.

The present application has been made in view of the above, and aims to provide an information processing device, an information processing method, and an information processing program that can confirm the reliability of generated information.

The information processing device according to the present application includes a selection section, a specification section, and an evaluation section. The selection unit selects target users whose value indicating the relationship with the predetermined event exceeds a predetermined threshold using an estimation model that estimates the relationship between the user and the predetermined event. The identification unit identifies results for each action performed by the target user in response to presentation of information regarding a predetermined event to the target user. The evaluation unit evaluates the reliability of the estimation model based on the content of the outcome for each action specified by the identification unit.

According to one aspect of the embodiment, the reliability of generated information can be confirmed.

FIG. 1 is a diagram illustrating an example of an information processing system according to an embodiment. FIG. 2 is a diagram showing an overview of information processing according to the embodiment. FIG. 3 is a diagram illustrating a configuration example of an information processing device according to an embodiment. FIG. 4 is a diagram showing an overview of user information according to the embodiment. FIG. 5 is a diagram showing an overview of advertisement history according to the embodiment. FIG. 6 is a diagram illustrating an overview of estimated model information according to the embodiment. FIG. 7 is a flowchart illustrating an example of a processing procedure by the information processing apparatus according to the embodiment. FIG. 8 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device according to the embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing apparatus, an information processing method, and an embodiment of an information processing program (hereinafter referred to as "embodiments") according to the present application will be described in detail below with reference to the drawings. Note that the information processing apparatus, information processing method, and information processing program according to the present application are not limited to the embodiments described below. Furthermore, the embodiments described below can be combined as appropriate within the range that does not conflict with the processing contents. Further, in the embodiments described below, the same parts are given the same reference numerals, and redundant explanations will be omitted.

Furthermore, in the following description, the terminal device 10 may be referred to as user U. That is, the user U can be read as the terminal device 10.

[1. Information processing system]
FIG. 1 is a diagram illustrating an example of an information processing system according to an embodiment. As shown in FIG. 1, the information processing system SYS according to the embodiment includes a terminal device 10 and an information processing device 100. Note that the information processing system SYS is not limited to the example shown in FIG. 1, and may include multiple terminal devices 10 and multiple information processing devices 100.

The terminal device 10 and the information processing device 100 are each connected to a network N (see FIG. 3, for example) by wire or wirelessly. The network N is a communication network such as a LAN (Local Area Network), a WAN (Wide Area Network), a telephone network (mobile phone network, fixed telephone network, etc.), a local IP (Internet Protocol) network, or the Internet. . The network N may include a wired network or a wireless network. The terminal device 10 and the information processing device 100 can communicate with each other through the network N.

The terminal device 10 shown in FIG. 1 is an information processing device used by a user U who uses various services provided by the information processing device 100 and the like. For example, the terminal device 10 is realized by a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a PDA (Personal Digital Assistant), or the like.

The terminal device 10 also supports wireless communication networks such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation), Bluetooth (registered trademark), and wireless LAN (Local Area It is possible to connect to the network N via short-range wireless communication such as Network).

Further, the terminal device 10 can display information for using various services using a web browser or an application. Note that when the terminal device 10 receives control information for realizing information display processing from the information processing device 100 or the like, it realizes the display processing according to the control information.

Further, the user U of the terminal device 10 provides user information to the information processing device 100 through the use of various services. User information includes search history including search queries entered by each user U, purchase history of each user U, history of transaction objects viewed on the online shopping mall, purchase history of transaction objects purchased, etc. , viewing history, which is the history of content such as news that has been viewed, posting history, which is the history of text, images, etc. posted to SNS (Social Networking Service), and location history. In addition, user information includes, for example, the usage history of payment services realized by reading QR codes (registered trademarks) installed in stores using smartphones owned by each user U, and the usage history of payment services realized using the reading results. It may also include a history of usage of payment services (for example, purchase date and time, purchased products, purchase amount, etc.) that is performed by displaying a QR code on an owned smartphone and reading the QR code with a store terminal. Further, the user information may include the usage history of a credit card, the usage history of a bank account, etc. held by each user U.

Further, the user information may include information estimated based on collected information. For example, user information may be based on purchase history if the average monthly grocery purchase total is 100,000 yen or more, household monthly income or personal monthly income is 300,000 yen or more, or a family of three people. It may also include information estimating the family composition, the composition of cohabitants, etc. Further, the user information may include, for example, information that is estimated by estimating the items owned by each user U, the place of residence or workplace of each user U, etc. from the location history and purchase history.

Further, the user information may include, for example, information obtained by acquiring a service usage history from a server device other than the information processing device 100 that provides various services. Further, the user information may include various information estimated from a service usage history by a server device or a single server device that provides such a service. Further, the user information may include information indicating psychographic attributes such as various geographical attributes and hobbies and preferences registered by each user U or estimated for each user U.

Further, the user information may include sensor information acquired by various sensors installed in the terminal device 10. For example, sensor information includes position information acquired by a position sensor, acceleration information detected by an acceleration sensor, angular velocity information detected by a gyro sensor, operation information acquired by a touch panel, and information acquired by an illuminance sensor. The information may include illuminance information obtained by a barometric pressure sensor, atmospheric pressure information obtained by a barometric pressure sensor, sound information obtained by a microphone, and the like.

The information processing apparatus 100 shown in FIG. 1 is an information processing apparatus that provides various services to each user U who is a service user. The various services provided by the information processing device 100 may include API (Application Programming Interface) services corresponding to various applications and various online services. Online services include Internet connectivity, search services, SNS (Social Networking Services), electronic commerce services, electronic payment services, online games, online banking services, online trading services, accommodation reservation services, etc. Ticket reservation services, video distribution services, music distribution services, news distribution services, map information services, route search services, route guidance services, route information services, operation information services, weather information services, etc. Services may be applicable.

Further, the information processing apparatus 100 is an information processing apparatus that executes information processing according to the embodiment using user information collected through provision of various services. The information processing device 100 is typically a server device, but may also be realized by a mainframe, a workstation, or the like.

When the information processing device 100 is implemented as a server device, it may be implemented as a single server, or may be implemented as a cloud system in which multiple server devices and multiple storage devices operate cooperatively. Note that the information processing device 100 may function as a distribution device that distributes control information to the terminal device 10 used by each user U. Here, the control information is written in, for example, a script language such as JavaScript (registered trademark) or a style sheet language such as CSS (Cascading Style Sheets). Note that the application itself distributed from the information processing device 100 may be regarded as control information.

Further, the information processing device 100 may function as a distribution device that distributes advertisements to each user U by providing various services. In this case, the information processing device 100 manages advertisement history regarding advertisements distributed to each user U for each user U. Note that the information processing device 100 may cooperate with another distribution device that distributes advertisements. In this case, the information processing device 100 acquires the advertisement history of each user U from another distribution device communicably connected through the network, and manages the acquired advertisement history for each user U.

[2. Information processing]
An example of information processing in the information processing system SYS will be described below. In the example shown in FIG. 1, the information processing device 100 stores user information collected from each user U. The information processing apparatus 100 then generates a plurality of estimation models using the user information and manages the generated estimation models. The estimation model generated by the information processing apparatus 100 is a model for estimating the relationship between each user U and a predetermined event such as the user's U's actions, interests, and preferences.

For example, the information processing device 100 collects seed users who satisfy the condition (title) of "bought ____" or "interested in ____" from among each user U, and Generates a score list with scores based on commonality of search queries. The information processing device 100 uses this score list as an estimation model. Specifically, we collected seed users who say ``wear glasses,'' search query: ``glasses,'' ``70 points,'' search query: ``eyeglass cleaning,'' ``50 points,'' and search query: ``apple.'' A score such as "-50 points" is assigned to the search query, and a score list that is a combination of the search query and the score is created.

The seed user described above can be identified using any method. For example, the information processing device 100 may identify a seed user by estimating a user who has performed an assumed action based on various information such as purchase history, usage history, and location history. Specifically, the seed user may be identified by inputting the name of a certain predetermined store as a search query and then estimating the number of visitors who actually visited the predetermined store. As another example, seed users may be identified by inputting the name of an event as a search query and then estimating the participants who actually participated in the event. As yet another example, a seed user may be identified by inputting the name of a certain product as a search query and then estimating the purchaser who actually purchased the product with that name. Further, the information processing device 100 may identify the seed user by estimating the number of users who actually used a predetermined store (for example, a store corresponding to a distributed advertisement). Furthermore, the information processing apparatus 100 may identify seed users by estimating the number of viewers who have viewed content such as a predetermined advertisement, web page, movie, or music.

Further, the information processing device 100 can obtain the score of the search query in the above-mentioned score list from a behavior model generated by machine learning as described below. For example, when a search query input by a seed user is taken as a positive example, and a search query input by a randomly selected user other than the seed user is taken as a negative example, when the information processing apparatus 100 inputs a positive example, The learning model is trained to output a higher value and output a lower value when a negative example is input. At this time, any method can be adopted for learning. For example, if a neural network is used as the model, it can be realized by backpropagation or the like.

Furthermore, the information processing apparatus 100 may learn the model so that higher numerical values are output for search queries that are commonly input by seed users. For example, the model may be trained to output a higher value as the number of seed users inputting positive example search queries increases. Furthermore, the model may be trained to output a smaller value (negative value) as the number of other users U who input negative example search queries increases. Then, the information processing device 100 generates a list of pairs of each search query and a score output when the search query is input to such a learning model as a score list. That is, the information processing device 100 calculates each search query input by a seed user who has performed a predetermined action, and the above-mentioned score (score indicating the relationship between the search query and the predetermined action) corresponding to each search query. ) is generated as a score list. The information processing device 100 manages the generated score list in association with a predetermined event estimated using the score list. The information processing device 100 may directly adopt the condition at the time of seed user collection (for example, "wearing glasses") as a predetermined event, or may use the information based on this condition (for example, "wearing glasses" or " "Owning glasses," etc.) may be newly labeled as a predetermined event associated with the score list.

Note that the information processing device 100 can clarify the relationship with a predetermined event by using the score list (estimated model) described above. For example, the information processing device 100 obtains the scores associated with each of the plurality of search queries input by the arbitrarily selected user U from the score list, and calculates the total value of the obtained scores. Then, when the total value of the calculated scores exceeds a predefined threshold, the information processing apparatus 100 determines that the event has a high relevance to the event corresponding to the score list.

The information processing apparatus 100 that generates and manages a plurality of estimation models (score lists) as described above executes information processing according to the embodiment using the generated estimation models.

For example, as shown in FIG. 1, the information processing device 100 uses the above-mentioned estimation model to select target users whose value (score) indicating relevance to a predetermined event exceeds a predetermined threshold (step S1 ). In other words, the information processing device 100 identifies users who are highly relevant to the predetermined event from among the users U.

Next, the information processing device 100 specifies the results of each action performed by the target user in response to the presentation of information regarding a predetermined event to the target user (step S2). For example, the information processing device 100 identifies multiple conversion rates for advertisements related to a predetermined event. Note that the conversion rate may be set using any information related to a predetermined event as a starting point. For example, the information processing device 100 may calculate any information such as the conversion rate of viewing a predetermined web page among web pages serving as advertisements or search results, or the rate of reposting of predetermined posted information among SNS posted information, etc. You may set up multiple conversions for information.

Conversion is a user who is expected to respond to the content of an ad, such as opening an email, clicking an ad, staying on a landing page, requesting information, responding to a survey, requesting a quote, requesting a sale, or purchasing a product (concluding a contract). It may include multiple actions of U. Furthermore, the conversion rate is given by the percentage of users who have achieved the above-mentioned various conversions among all target users.

The information processing device 100 then evaluates the reliability of the estimation model based on the content of the outcome for each identified action (step S3). For example, the information processing device 100 evaluates the reliability of the estimation model based on the contents of a plurality of conversion rates.

Information processing according to the embodiment will be specifically described below. FIG. 2 is a diagram showing an overview of information processing according to the embodiment. FIG. 2 shows an overview of information processing when the predetermined event is "supplement use." Further, in FIG. 2, as examples of a plurality of conversion rates, an email opening rate, a material request rate, and a product purchase rate are shown.

As shown in FIG. 2, the information processing device 100 has an estimation model A that estimates users who use supplement A, and an estimation model that estimates users who use supplement B, as estimation models that estimate the relationship with "supplement use." Model B is obtained from among the plurality of estimated models. Then, the information processing device 100 uses the estimated model A and the estimated model B to identify target users who are highly relevant to the predetermined event: "supplement use" (step S11).

For example, the information processing device 100 generates a user list LT_A using estimation model A that estimates users who use supplement A, and uses the generated user list LT_A to determine the relationship between “use of supplement A” and Identify target users with high Specifically, the information processing device 100 determines the relationship between “use of supplement A” and each user U based on the search query input by each arbitrary user U and the estimated model A (score list). A user list LT_A is generated in which each user U is ranked based on the numerical value indicating the numerical relationship. The information processing apparatus 100 then selects each user U positioned at the top of the user list LT_A (for example, the top 100 users) as target users.

Similarly, the information processing device 100 quantifies the relationship between “use of supplement B” and each user U based on the search query input by each arbitrary user U and the estimated model B (score list). , generates a user list LT_B in which each user U is ranked based on a value indicating a numerical relationship. Then, the information processing device 100 selects each user U positioned at the top of the user list LT_B (for example, the top 100 users) as target users.

Subsequently, the information processing apparatus 100 refers to the user information of the target user identified in step S11 and obtains conversion information of the target user (step S12).

For example, the information processing device 100 acquires, for each target user specified from the user list LT_A, an advertisement history of an advertisement whose advertisement target is “Supplement A” and a purchase history linked to the advertisement history. The information processing device 100 then calculates the email opening rate, material request rate, and product purchase rate using the number of all target users as a parameter based on the advertising history and purchase history acquired for each target user. Then, conversion information CV_A regarding all target users is acquired.

For example, the information processing device 100 acquires, for each target user specified from the user list LT_B, an advertisement history of an advertisement whose advertisement target is “Supplement B” and a purchase history linked to the advertisement history. The information processing device 100 then calculates the email opening rate, material request rate, and product purchase rate using the number of all target users as a parameter based on the advertising history and purchase history acquired for each target user. Then, conversion information CV_B regarding all target users is acquired.

Then, the information processing device 100 determines which of the estimated model A and the estimated model B based on the contents of the plurality of conversion rates included in the conversion information CV_A and the contents of the plurality of conversion rates included in the conversion information CV_B. It is evaluated whether the reliability of the model is high (step S13).

For example, the information processing device 100 estimates the conversion rate based on a comparison between a first type of conversion rate and a second type of conversion rate corresponding to an action performed after the action corresponding to the first type of conversion rate. Evaluate model reliability. Information according to the advertisement type is used for the first type conversion rate and the second type conversion rate. To explain specifically using the example shown in FIG. 2, the information processing device 100 calculates the email opening rate to be adopted as the first type of conversion rate, and the material request rate and product to be adopted as the second type of conversion rate. Evaluate the reliability of the estimation model by comparing it with the purchase rate. Note that the number of conversion rates of the second type to be compared with the conversion rate of the first type does not need to be plural, and may be singular.

In the example shown in Fig. 2, if conversion information CV_A based on estimated model A has a higher email opening rate than the materials request rate and product purchase rate, and conversion information CV_B based on estimated model B has an email opening rate If the rate is smaller than the material request rate and the product purchase rate, the information processing device 100 evaluates that the estimated model B is more reliable than the estimated model A. In other words, the information processing device 100 determines that the higher the execution rate of the actions performed by the user U after opening the email, the more closely related the user U is to the predetermined event.

As described above, the information processing device 100 tracks the behavior of target users who are determined to be highly relevant to a predetermined event based on the estimation model toward advertisements. The final behavior can be analyzed, and as a result, the reliability of the estimated model can be evaluated. In this way, the information processing device 100 can confirm the reliability of the estimated model that is the generated information.

Although FIG. 2 describes an example in which the information processing device 100 performs a relative evaluation as the reliability of the estimated model A and the estimated model B, individual evaluations for each of the estimated model A and the estimated model B are performed. You may do so. For example, the information processing apparatus 100 evaluates that the reliability of the estimation model is high when the conversion rate of the second type is higher than the conversion rate of the first type.

Furthermore, when there are three or more estimation models for estimating the relationship with a predetermined event, the information processing device 100 may perform relative evaluation of the plurality of estimation models or may perform individual evaluation. good.

Furthermore, when there are multiple types of advertisements distributed to target users for a predetermined event, the information processing device 100 obtains conversion information (multiple conversion rates) for each advertisement type and estimates the conversion rate for each advertisement type. The reliability of the model may be evaluated.

Furthermore, in the example of the information processing described above, the information processing device 100 determines whether the first type of conversion rate and the second type of conversion rate each satisfy a predetermined standard. It's okay.

Furthermore, when acquiring conversion information as a result of each action performed by a target user, the information processing device 100 calculates a conversion rate for each target user instead of a conversion rate using the number of all target users as a parameter. You may obtain it. Then, the information processing device 100 may use the number of target users whose conversion rate satisfies a predetermined standard among the conversion rates for each target user as information indicating the results of each action performed by the target user. .

In addition, in Figure 2, when the advertisement type is an email advertisement, the conversion rate of the first type is the email opening rate, and the conversion rate of the second type is the material request rate and product purchase rate. However, the present invention is not limited to this example, and information according to the advertisement type can be used as the first type conversion rate and the second type conversion rate. For example, if the ad type is a banner ad (pure ad) or native ad, the ad click rate may be used as the first type of conversion rate, and the landing page stay rate may be used as the second type of conversion rate. good.

In addition, when there is a single estimation model for estimating the relationship with a predetermined event, the information processing device 100 corresponds to the first type of conversion rate and the first type of conversion rate for the corresponding estimation model. The reliability of the estimation model can be evaluated from comparison with the conversion rate of the second type corresponding to the action performed after the action performed.

Furthermore, the information processing apparatus 100 is not limited to the example in which the reliability of the estimation model is evaluated from the comparison of the contents of a plurality of conversion rates obtained through distribution of an email advertisement as one of the Internet advertisements. For example, the information processing device 100 may acquire information on the target user through information presentation such as mass advertisement that does not use the network N. Specifically, the information processing device 100, based on the response rate of target users to the questionnaire postcard published in a free paper introducing products of store X and the actual visit rate of target users to store X, An estimation model for estimating users who frequently use store X may be evaluated.

[3. Configuration of information processing device]
The configuration of the information processing device 100 according to the embodiment will be described using FIG. 3. FIG. 3 is a diagram illustrating a configuration example of an information processing device according to an embodiment.

As shown in FIG. 3, the information processing device 100 includes a communication section 110, a storage section 120, and a control section 130. Note that FIG. 3 shows an example of the configuration of the information processing apparatus 100, and the information processing apparatus 100 is not limited to the configuration shown in FIG. 3, and may have a configuration including other functional units other than those shown in FIG.

(Communication Department 110)
The communication unit 110 is connected to the network N by wire or wirelessly, for example, and transmits and receives information to and from other devices such as the terminal device 10 via the network N. The communication unit 110 is realized by, for example, a NIC (Network Interface Card), an antenna, or the like. The network N is a communication network such as a LAN (Local Area Network), a WAN (Wide Area Network), a telephone network (such as a mobile phone network or a fixed telephone network), a local IP (Internet Protocol) network, or the Internet. . The network N may include a wired network or a wireless network.

The communication unit 110 receives information such as captured images from the terminal device 10. Furthermore, the communication unit 110 transmits proposal information to the terminal device 10.

(Storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 stores programs and data for implementing information processing executed by each unit of the control unit 130.

As shown in FIG. 3, the storage unit 120 includes a user information storage unit 121 and an estimated model storage unit 122.

(User information storage unit 121)
The user information storage unit 121 stores user information provided by each user in an arbitrary format. FIG. 4 is a diagram showing an overview of user information according to the embodiment.

As shown in Figure 4, the user information includes items such as "user ID", "advertisement history", "search history", "purchase history", and "access history". . These items included in the user information are associated with each other.

The "User ID" item stores identification information that is individually assigned to each user by the information processing apparatus 100 in order to identify users who use various services when providing various services. The "advertisement history" item stores an ad history that is a history of advertisements distributed to each user. The "search history" item stores a search history that is a history of search queries and the like input by each user. In addition, the "Purchase History" item stores a purchase history that is a history of purchases provided by each user as user information. The "access history" item stores an access history that is a history of contents accessed by each user.

For example, the information processing device 100 acquires the above-mentioned information from each user through providing various services to each user, associates the acquired information with identification information given to each user, and stores it in the storage unit 120. By storing it, it can be saved and managed as user information. Note that when the information processing device 100 cooperates with another device that distributes advertisements, the information processing device 100 acquires each user's advertisement history, associates the acquired advertisement history with the identification information given to each user, and stores it in the storage unit 120. Store in.

Hereinafter, an overview of the advertisement history included in user information will be explained using FIG. 5. FIG. 5 is a diagram showing an overview of advertisement history according to the embodiment. As shown in FIG. 5, the advertisement history included in the user information includes items such as "advertisement ID," "advertisement type," "advertisement target," and "delivery date and time." These items included in the advertisement history are associated with each other.

Identification information for identifying advertisements is stored in the "advertisement ID" item. Identification information for identifying the type of advertisement is stored in the "advertisement type" item. The advertisements according to the embodiments are typically Internet advertisements, such as pure advertisements (banner advertisements), listing advertisements, email advertisements, SNS advertisements, affiliate advertisements, video advertisements, and native advertisements (article advertisements). ) etc. In addition, advertisements according to the embodiments include mass advertisements such as television advertisements, radio advertisements, and newspaper advertisements that are linked to Internet advertisements through two-dimensional codes and URLs (Uniformed Resource Locators), as well as flyers and free advertisements that are linked to Internet advertisements. Sales promotion advertisements such as paper, event promotion, and in-store promotion may also be used.

The "advertisement target" item stores information regarding advertising targets such as products and services. The item "Distribution date and time" stores information indicating the date and time when the advertisement was distributed.

For example, the information processing device 100 identifies users for whom products or services related to a predetermined event set by the operator are stored in the "advertisement target" item, and selects target users from among the identified users. can.

(Estimated model storage unit 122)
The estimated model storage unit 122 stores estimated model information regarding estimated models in an arbitrary format. FIG. 6 is a diagram illustrating an overview of estimated model information according to the embodiment.

As shown in FIG. 6, the estimated model information includes a "model ID" item, an "event" item, and a "data" item. These items included in the estimated model information are associated with each other.

The "Model ID" item stores identification information that is individually assigned to each estimated model in order to identify the estimated model.

"Event" stores information indicating the contents of the user's behavior, interests, and preferences estimated by the associated estimation model.

The "data" item stores data of the associated estimation model. Although Fig. 6 shows an example in which conceptual information such as "Data A" and "Data B" is stored in the "Data" item, in reality, the configuration of the estimation model (network This includes various information constituting the corresponding estimation model, such as information on the configuration (configuration) and information on parameters. For example, the individual information that makes up the estimation model includes nodes in each layer of the network, functions adopted by each node, connection relationships between nodes, and connection coefficients set for connections between nodes. may be included.

(Control unit 130)
The control unit 130 shown in FIG. 3 is a controller that controls the information processing device 100. The control unit 130 uses a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), etc. to execute various programs (for example, information processing programs) stored in a storage device inside the information processing device 100. This is realized by executing using RAM as a work area. Further, the control unit 130 may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 3, the control unit 130 includes a selection unit 131, a specification unit 132, and an evaluation unit 133. The control unit 130 uses these units to realize or execute information processing functions and operations described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 3, and may be any other configuration as long as it performs information processing to be described later. Further, the connection relationship between the respective units included in the control unit 130 is not limited to the connection relationship shown in FIG. 3, and may be other connection relationships.

(Selection unit 131)
The selection unit 131 selects target users whose value indicating the relationship with the predetermined event exceeds a predetermined threshold using an estimation model that estimates the relationship between the user and the predetermined event.

(Specific unit 132)
The identifying unit 132 identifies the results of each action performed by the target user in response to the presentation of information regarding a predetermined event to the target user. For example, the identifying unit 132 identifies multiple conversion rates for advertisements related to a predetermined event. The specifying unit 132 passes the specified conversion rates to the evaluating unit 133.

(Evaluation unit 133)
The evaluation unit 133 evaluates the reliability of the estimation model based on the content of the outcome for each action specified by the identification unit 132. For example, the evaluation unit 133 evaluates the reliability of the estimation model based on the contents of a plurality of conversion rates. Specifically, the evaluation unit 133 compares the conversion rate of the first type with the conversion rate of the second type corresponding to an action performed after the action corresponding to the first type of conversion rate. , evaluate the reliability of the estimated model.

[4. Processing procedure]
Hereinafter, a processing procedure by the information processing apparatus 100 according to the embodiment will be described using FIG. 7. FIG. 7 is a flowchart illustrating an example of a processing procedure by the information processing apparatus according to the embodiment. The processing procedure shown in FIG. 7 is executed by the control unit 130 of the information processing device 100. The processing procedure shown in FIG. 7 is repeatedly executed while the information processing device 100 is in operation.

As shown in FIG. 7, the selection unit 131 selects target users whose value indicating the relationship with the predetermined event exceeds a predetermined threshold using an estimation model that estimates the relationship between the user and the predetermined event. (Step S101).

In addition, the identifying unit 132 identifies the results of each action performed by the target user in response to the presentation of information regarding a predetermined event to the target user (step S102).

Furthermore, the evaluation unit 133 evaluates the reliability of the estimation model based on the content of the results for each specified action (step S103).

[5. Modified example]
The information processing device 100 described above may be implemented in various different forms other than the embodiments described above. Therefore, a modification of the embodiment will be described below.

(5-1. Presentation of evaluation results)
In the embodiment described above, the information processing device 100 may present information regarding the evaluation results of the estimation model to the operator. For example, the information processing device 100 can present to the operator information about the estimated model used to select the target user and information regarding the evaluation of the estimated model. The information regarding the evaluation of the estimated models may be individual information about the estimated models, or may be relative information between the estimated models.

(5-2. Use of estimation model based on conversion rate)
In the embodiment described above, the information processing device 100 may distribute advertisements using a plurality of conversion rates for advertisements related to a predetermined event. For example, if the information processing apparatus 100 wants to increase the email open rate of an advertisement to be distributed, the information processing apparatus 100 may distribute the advertisement using a user list that satisfies the desired email open rate.

[6. Hardware configuration]
The information processing apparatus 100 according to the embodiment is realized, for example, by a computer 1000 having a configuration as shown in FIG. FIG. 8 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device according to the embodiment.

The computer 1000 is connected to an output device 1010 and an input device 1020, and has an arithmetic device 1030, a primary storage device 1040, a secondary storage device 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 connected by a bus 1090. has.

The arithmetic unit 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, programs read from the input device 1020, and performs various processes. The primary storage device 1040 is a memory device such as a RAM that temporarily stores data used by the arithmetic unit 1030 for various calculations. Further, the secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), an HDD, a flash memory, or the like.

The output IF 1060 is an interface for transmitting information to be output to the output device 1010 that outputs various information such as a monitor or printer, and is, for example, a USB (Universal Serial Bus), a DVI (Digital Visual Interface), This is realized using a connector compliant with standards such as HDMI (registered trademark) (High Definition Multimedia Interface). Further, the input IF 1070 is an interface for receiving information from various input devices 1020 such as a mouse, a keyboard, and a scanner, and is realized by, for example, a USB or the like.

Note that the input device 1020 is, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), or a tape. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like. Furthermore, the input device 1020 may be an external storage medium such as a USB memory.

Network IF 1080 receives data from other devices via network N and sends it to computing device 1030, and also sends data generated by computing device 1030 to other devices via network N.

Arithmetic device 1030 controls output device 1010 and input device 1020 via output IF 1060 and input IF 1070. For example, the arithmetic device 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040, and executes the loaded program.

For example, when the computer 1000 functions as the information processing device 100, the arithmetic device 1030 of the computer 1000 executes a program (for example, an information processing program) loaded on the primary storage device 1040, thereby performing the same operation as the control unit 130. Realize the functions of That is, the arithmetic device 1030 realizes processing by the information processing device 100 according to the embodiment in cooperation with a program (for example, an information processing program) loaded onto the primary storage device 1040.

[7. others]
Among the processes described in the embodiments described above, all or part of the processes described as being performed automatically can be performed manually, or all of the processes described as being performed manually can be performed manually. Alternatively, some of the steps can be performed automatically using known methods. In addition, information including the processing procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified.

In the embodiment described above, in order to realize the information processing method by the information processing apparatus 100 (for example, see FIG. The processing function corresponding to 133) may be realized as an add-on to the information processing program installed in advance in the information processing device 100, or it may be flexibly described as a dedicated provided program using a lightweight programming language or the like. This may be realized by

Furthermore, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads and usage conditions. Can be integrated and configured. For example, the selection section 131 and the identification section 132 of the control section 130 may be functionally integrated.

Further, the above-described embodiments and modified examples can be combined as appropriate within the range that does not conflict with the processing contents.

[8. effect]
The information processing device 100 according to the embodiment described above includes a selection section 131, a specification section 132, and an evaluation section 133. The selection unit 131 selects target users whose value indicating the relationship with the predetermined event exceeds a predetermined threshold using an estimation model that estimates the relationship between the user and the predetermined event. The specifying unit 132 specifies the results of each action performed by the target user in response to the presentation of information regarding the predetermined event to the target user. The evaluation unit 133 evaluates the reliability of the estimation model based on the content of the outcome for each action specified by the identification unit 132.

For this reason, the information processing apparatus 100 according to the embodiment tracks the behavior of target users toward advertisements that are determined to be highly relevant to a predetermined event based on the estimation model. The final behavior of the target user can be analyzed, and as a result, the reliability of the estimation model can be evaluated. In this way, the information processing device 100 can confirm the reliability of the estimated model that is the generated information.

Furthermore, in the information processing device 100 according to the embodiment, the identification unit 132 identifies a plurality of conversion rates for advertisements related to a predetermined event, and the evaluation unit 133 uses an estimated model based on the contents of the plurality of conversion rates. Evaluate the reliability of

Furthermore, in the information processing device 100 according to the embodiment, the evaluation unit 133 calculates the first type of conversion rate and the second type corresponding to the action performed after the action corresponding to the first type of conversion rate. Evaluate the reliability of the estimation model by comparing it with the conversion rate of

With these, the information processing apparatus 100 according to the embodiment can objectively evaluate the reliability of the estimation model from the advertising effect on the target user.

As mentioned above, the embodiments of the present application have been described in detail based on several drawings, but these are merely examples, and various modifications and variations can be made based on the knowledge of those skilled in the art, including the embodiments described in the disclosure section of the invention. It is possible to carry out the invention in other forms with modifications.

Further, the above-mentioned "section, module, unit" can be read as "means", "circuit", etc. For example, the control section can be read as a control means or a control circuit.

10 Terminal device 100 Information processing device 110 Communication unit 120 Storage unit 121 User information storage unit 122 Estimated model storage unit 130 Control unit 131 Selection unit 132 Specification unit 133 Evaluation unit

Claims (4)

a selection unit that selects target users whose value indicating the relationship with the predetermined event exceeds a predetermined threshold using an estimation model that estimates the relationship between the user and the predetermined event;
a specifying unit that specifies an outcome for each action performed by the target user in response to information presentation regarding the predetermined event to the target user;
an evaluation unit that evaluates the reliability of the estimation model based on the content of the outcome for each action specified by the identification unit ,
The specific part is
identifying a plurality of conversion rates for information related to the predetermined event;
The evaluation department is
An information processing device characterized in that the reliability of the estimation model is evaluated based on the contents of the plurality of conversion rates . The evaluation department is
Evaluate the reliability of the estimation model from a comparison between the first type of conversion rate and the second type of conversion rate corresponding to an action performed after the action corresponding to the first type of conversion rate. The information processing device according to claim 1 , characterized in that: An information processing method performed by a computer, the method comprising:
a selection step of selecting target users whose value indicating the relationship with the predetermined event exceeds a predetermined threshold using an estimation model that estimates the relationship between the user and the predetermined event;
a specifying step of specifying an outcome for each action performed by the target user in response to information presentation regarding the predetermined event to the target user;
an evaluation step of evaluating the reliability of the estimation model based on the content of the outcome for each action identified in the identification step ,
The specific step is
identifying a plurality of conversion rates for information related to the predetermined event;
The evaluation step includes:
Evaluating the reliability of the estimation model based on the contents of the plurality of conversion rates.
An information processing method characterized by: to the computer,
a selection procedure of selecting target users whose value indicating the relationship with the predetermined event exceeds a predetermined threshold using an estimation model that estimates the relationship between the user and the predetermined event;
a specific procedure for specifying results for each action performed by the target user in response to information presentation regarding the predetermined event to the target user;
an evaluation procedure for evaluating the reliability of the estimation model based on the content of the outcome for each action identified by the identification procedure ;
The identification procedure is
identifying a plurality of conversion rates for information related to the predetermined event;
The evaluation procedure is
An information processing program characterized in that the reliability of the estimation model is evaluated based on the contents of the plurality of conversion rates .

Images