JP2022144282A - Information processing apparatus, information processing method, and information processing program - Google Patents

Abstract

To provide an information processing apparatus configured to determine deterioration state of an attribute estimation model.SOLUTION: An information processing apparatus 10 includes: a history acquisition unit 131 which acquires action histories of users on a network; an attribute estimation unit 133 which inputs the action histories of the users to an attribute estimation model generated by machine learning using action histories of correct-answer users corresponding to a target attribute, and associates target attribute scores output from the attribute estimation model with the users; a list generation unit 134 which generates a top user list by selecting a predetermined number of users in descending order of the target attribute score; and a determination unit 137 which determines a deterioration state of the attribute estimation model on the basis of the change in number of the correct-answer users included in the top user list.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing device, an information processing method, and an information processing program for determining the deterioration state of an attribute estimation model.

Conventionally, in order to optimize services provided to users, there is known a method of estimating user attributes by analyzing a user's action history obtained by using a service.
For example, Patent Literature 1 discloses a method of estimating a user's action attribute from an action history (for example, search history) using an attribute estimation model generated by machine learning. This attribute estimation model is generated by performing machine learning using the action history of the correct user as learning data for each attribute. A corresponding score is output. Then, the attribute for which a high score is output is estimated as the main attribute of the user, and content based on this attribute is provided to the user.

JP 2019-160178 A

Since user behavior patterns generally change over time, it is desirable that the scores given to users for each attribute be updated as appropriate.
However, attribute estimation models may deteriorate over time. For example, for an attribute related to fashion, the words (e.g., search words) included in the behavioral history related to the attribute change along with changes in fashion over time, so attribute estimation generated based on the behavioral history Models deteriorate over time. When using such a deteriorated attribute estimation model, there is a problem that the user's attributes cannot be updated accurately.
In order to solve the above problem, frequent updating of the attribute estimation model is conceivable. However, since the attribute estimation model is generated for each attribute, it is necessary to frequently update the attribute estimation model for all attributes that exist in large numbers. will increase the processing load.

An object of the present invention is to provide an information processing apparatus, an information processing method, and an information processing program capable of determining the deterioration state of an attribute estimation model.

The information processing apparatus of the present invention includes a history acquisition unit that acquires a user's action history on a network; and an attribute estimating unit that inputs the action history of the attribute estimation model and associates the score of the target attribute output from the attribute estimation model with the user; A list generation unit that generates a user list, and a determination unit that determines a deterioration state of the attribute estimation model based on a change in the number of correct users included in the top user list.

According to the present invention, it is possible to determine the deterioration state of the attribute estimation model by monitoring the change in the number of correct users included in the top user list. As a result, it is possible to update the attribute estimation model when necessary while suppressing the processing load. As a result, it is possible to continue to accurately estimate the user's attributes.

1 is a schematic diagram showing a schematic configuration of an information processing system according to an embodiment of the present invention; FIG. 2 is a block diagram showing a schematic configuration of the information processing apparatus of the embodiment; FIG. 4 is a flowchart for explaining an information processing method according to the embodiment; 4 is a graph illustrating changes in accuracy rate calculated in the embodiment;

An embodiment according to the present invention will be described below with reference to the drawings.
[Information processing system 1]
FIG. 1 is a schematic diagram showing a schematic configuration of an information processing system 1 of this embodiment.
As shown in FIG. 1, an information processing system 1 of this embodiment includes an information processing apparatus 10 of the present invention, a plurality of user terminals 20, and a plurality of service providing servers 30, which are connected to a network such as the Internet. communicatively connected via
In the information processing system 1 , the user terminal 20 accesses the service providing server 30 according to the user's operation and uses the service provided by the service providing server 30 . The services provided by the service providing server 30 are not particularly limited, and examples thereof include web search services, shopping services, auction services, news services, and the like.
In the present embodiment, the information processing apparatus 10 acquires the user's action history (specifically, search history) acquired from the user terminal 20 or the service providing server 30 for each user, and acquires an attribute estimation model corresponding to each attribute. to get the score corresponding to each attribute. That is, the information processing device 10 estimates the user's score for various attributes.
Here, the information processing apparatus 10 determines the update timing of the attribute estimation model by performing the process of determining the deterioration state of the attribute estimation model.
Note that the attributes described here are the user's interest, interest, lifestyle, life stage, consumption tendency, or the like.

[Information processing device 10]
FIG. 2 is a block diagram showing a schematic configuration of the information processing device 10. As shown in FIG.
The information processing apparatus 10 is configured by a general computer, and as shown in FIG. 2, includes components such as a communication unit 11, a storage unit 12, a processor 13, etc. that constitute a computer.
Note that the number of computers constituting the information processing apparatus 10 is not particularly limited. In the present embodiment, an example in which the information processing apparatus 10 is configured by one computer is shown for simplification of explanation, but a cloud server constructed by connecting a plurality of computers via a network is used as the information processing apparatus 10. good too.

The communication unit 11 is connected to the Internet and communicates with each device such as the user terminal 20 and the service providing server 30 via the Internet.
The storage unit 12 records various information and information processing programs for controlling the information processing apparatus 10 .
The storage unit 12 also includes an action history information storage unit 121, an estimated attribute information storage unit 122, a model storage unit 123, an accuracy rate storage unit 124, and the like.
In the present embodiment, an example in which the storage unit 12 of the information processing apparatus 10 is provided with the action history information storage unit 121, the estimated attribute information storage unit 122, the model storage unit 123, and the accuracy rate storage unit 124 is shown. Such information may be recorded in another data server or cloud storage that is communicably connected to the processing device 10 via a network.

The action history information storage unit 121 stores user identification information (user ID) and action history information for each user.
The behavior history information is history information about user behavior acquired through the Internet. For example, the user action history information includes search history information in search processing performed by the user, browsing history information of content browsed by the user, product trading history information in which the user bought and sold products, and the user's electronic payment method at a physical store. including payment history information using User behavior history information may also include the results of questionnaires given to users via the Internet.
Note that in the present embodiment, search history information is used for the process of determining the deterioration state of the attribute estimation model. This search history information includes items such as a search query and search date and time.

The estimated attribute information storage unit 122 stores user identification information (user ID) and estimated attribute information for each user.
The estimated attribute information is information related to attributes estimated based on the user's action history. For example, estimated attribute information includes one or more attributes and a score assigned to each attribute. Attributes are managed by, for example, attribute identification information (attribute ID).

The model storage unit 123 stores an attribute estimation model generated for each attribute.
The attribute estimation model is a model that inputs user's search history information and outputs scores of attributes corresponding to the attribute estimation model. This attribute estimation model is generated by machine learning based on the action history of correct users selected as users having corresponding attributes.

The accuracy rate storage unit 124 stores the accuracy rate for each attribute in chronological order. In this embodiment, the accuracy rate corresponding to the target attribute corresponds to the proportion of correct users included in the high-ranking user list corresponding to the target attribute.
Note that the accuracy rate storage unit 124 may store the user list for each target attribute in chronological order.

The processor 13 is composed of an arithmetic circuit such as a CPU and a recording circuit such as a RAM. The processor 13 expands the information processing program recorded in the storage unit 12 into the RAM, and executes various processes in cooperation with the program expanded into the RAM.
By reading and executing the information processing program, the processor 13, as shown in FIG. It functions as part 136 and determination part 137 .

The history acquisition unit 131 acquires the user's action history from the user terminal 20, the service providing server 30, or the like, and stores it in the action history information storage unit 121. FIG.
The learning unit 132 selects a user having a corresponding attribute for each attribute as a correct user, and generates an attribute estimation model using machine learning using the action history of the correct user.
The attribute estimating unit 133 estimates a user's score for each attribute using an attribute estimating model corresponding to each attribute, and stores it in the estimated attribute information storage unit 122 .

The list generating unit 134 selects a predetermined number of users in descending order of score for the attribute of the attribute estimation model to be determined (target attribute), thereby generating a top user list of the target attribute.
The accuracy rate calculation unit 135 calculates the percentage of correct users included in the top user list corresponding to the target attribute as the accuracy rate.
The normalization unit 136 normalizes the accuracy rate calculated at each time point after the first time point (second time point) by the accuracy rate calculated immediately after the attribute estimation model to be determined is generated (first time point). become
The determination unit 137 determines the deterioration state of the attribute estimation model corresponding to the target attribute based on the normalized accuracy rate.

[User terminal 20]
The user terminal 20 is a computer managed by a user, and is composed of, for example, a smart phone, a tablet terminal, a personal computer, or the like.
Although the illustration of the specific configuration of the user terminal 20 is omitted, the user terminal 20 has the basic configuration of a general computer. That is, the user terminal 20 includes an input operation unit that receives operations from an operator, a display that displays image information, a recording device that records various types of information, and an arithmetic circuit (such as a CPU) that performs arithmetic processing on various types of information.

[Service providing server 30]
The service providing server 30 is a server device that provides services when accessed from the user terminal 20 . For example, the service providing server 30 provides, as services, various contents related to web search services, shopping services, auction services, content providing services, news services, weather forecast services, and the like.
In the present embodiment, the information processing device 10 and the service providing server 30 are configured separately, but the information processing device 10 may replace some or all of the functions of any one of the service providing servers 30. good.

[Information processing method]
Next, an information processing method executed in the information processing apparatus 10 will be described with reference to the flowchart shown in FIG.
At an arbitrary timing before starting the flowchart of FIG. 3, the history acquisition unit 131 acquires the user's action history and stores it in the action history information storage unit 121 (history acquisition step).
In the following description, processing when an arbitrary attribute estimation model is used as a determination target and the attribute of this attribute estimation model is used as a target attribute will be described, but the same processing is performed for each attribute estimation model.

First, the learning unit 132 refers to the action history information included in the action history information storage unit 121, and selects the user who performed the definite action for the target attribute as the correct user. Then, an attribute estimation model is generated by machine learning using search histories corresponding to a plurality of correct users as learning data (step S1). The generated attribute estimation model is stored in the model storage unit 123. FIG.

It should be noted that, for example, the purchase history or payment history of a product having a relationship with the target attribute, or the results of a questionnaire can be used as the determined action for the target attribute.
Any known technique can be used as a method of generating an attribute estimation model. For example, as training data, a query vector having weights assigned to search queries as components, which are weighted by the number of searches assigned to search queries included in the search history by a method such as TF-IDF, can be used.

Next, when the present time is the predetermined determination timing (step S2; Yes), the attribute estimation unit 133 generates estimated attribute information for each user regarding the target attribute (step S3; attribute estimation step).
Specifically, the attribute estimation unit 133 refers to the action history information storage unit 121, inputs the user's search history into the attribute estimation model generated in step S1 for each user to be estimated, and performs attribute estimation. The target attribute score output from the model is associated with the user and stored in the estimated attribute information storage unit 122 .
Note that when the search history of a user who has a search tendency similar to that of the correct user is input to the attribute estimation model, the attribute estimation model outputs a high score similar to that of the correct user.

The determination timing of step S2 can be set arbitrarily. For example, it is preferable that the first determination timing after generation or update of the attribute estimation model is immediately after step S1. Further, it is preferable that the second and subsequent determination timings are set for each preset period (for example, every month).

Next, the list generation unit 134 refers to the estimated attribute information storage unit 122 and generates a top user list by selecting a predetermined number of users in descending order of the score of the target attribute (step S4; list generation step).
Here, the predetermined number is, for example, a number that is a predetermined percentage (for example, several percent) of the number of users given a score greater than 0 for the target attribute. Alternatively, the predetermined number may be a predetermined number or the number of users given a score equal to or higher than a predetermined threshold.

Next, the accuracy rate calculation unit 135 calculates, as an accuracy rate, the percentage of correct users included in the top user list created in step S3 (step S5; accuracy rate calculation step). Note that the first accuracy rate after the attribute estimation model is generated or updated is treated as data corresponding to "0 months (0M)" after model generation.

Next, the normalization unit 136 normalizes the accuracy rate generated in step S5 (step S6).
Here, the first accuracy rate after generation or update of the attribute estimation model is treated as a normalization reference value and normalized to 100%. Also, the accuracy rate for the second and subsequent times is normalized by dividing by the normalization reference value.
The accuracy rate normalized in step S6 is stored in the accuracy rate storage unit 124. FIG.

Next, the determination unit 137 determines the deterioration state of the attribute estimation model by comparing the correct answer rate normalized in step S6 (hereinafter sometimes simply referred to as the correct answer rate) with a predetermined threshold value (step S7 ; judgment step).
For example, if the accuracy rate is equal to or less than the threshold (step S7; Yes), the determination unit 137 determines that the attribute estimation model is degraded, returns to step S1, and generates (that is, updates) the attribute estimation model. do.
On the other hand, if the accuracy rate is greater than the threshold (step S7; No), the determination unit 137 returns to step S2 and waits until the next determination timing.
According to the flowchart of FIG. 3 described above, the attribute estimation model is continuously monitored, and when deterioration of the attribute estimation model is detected, the attribute estimation model is updated.

[Effect of this embodiment]
The information processing apparatus 10 of the present embodiment includes a storage unit 12 and a processor 13. The processor 13 reads an information processing program stored in the storage unit 12 to obtain a history acquisition unit 131, an attribute estimation unit 133, It functions as the list generation unit 134, the accuracy rate calculation unit 135, and the determination unit 137, and executes each process as described above.
According to this embodiment, the deterioration state of the attribute estimation model can be determined by monitoring the number of correct users included in the user list. As a result, the attribute estimation model can be updated at a necessary time while suppressing the processing load, and the user's attributes can be continuously and accurately estimated.
Further, in the present embodiment, the deterioration state of the attribute estimation model is determined based on the percentage of correct users included in the user list, so more accurate determination is possible.

In this embodiment, the processor 13 also functions as a normalization unit 136 that normalizes the accuracy rate. As a result, a common threshold can be used for a plurality of attribute estimation models with different corresponding attributes, and the deterioration state can be easily determined for the plurality of attribute estimation models.

Here, FIG. 4 is a graph illustrating the transition of the accuracy rate (after normalization) of a plurality of attributes (attribute A to attribute C). Note that FIG. 4 illustrates a graph when the accuracy rate is continuously acquired without updating the attribute estimation model.
In the graph of FIG. 4, for example, attribute A is "interest in domestic dramas", attribute B is "I have a dog", attribute C is "interest in video games", and attribute D is Suppose it is "interest in foreign dramas".
For the attribute A “interest in domestic dramas”, compared to the other attributes B to C, search words related to the attribute A tend to change over time due to factors such as seasonal program revisions. For this reason, the attribute estimation model corresponding to the attribute A ceases to function as a current model after the passage of time and deteriorates. If the attribute estimation model corresponding to attribute A deteriorates, correct users cannot be added to the user list, and the accuracy rate corresponding to attribute A greatly decreases from 100% to nearly 60%. If such a degraded attribute estimation model is used, the user's attributes cannot be estimated accurately.
Therefore, as described above, the deterioration state of the attribute estimation model is determined based on the accuracy rate, and when it is determined that the attribute estimation model has deteriorated, the attribute estimation model is updated to attributes can be continuously and accurately estimated.

[Modification]
The present invention is not limited to the above-described embodiments, and includes modifications shown below as long as the object of the present invention can be achieved.

(Modification 1)
In the above embodiment, the user's search history is used to estimate the attribute score for the user, but the present invention is not limited to this. For example, the attribute estimating unit 133 may estimate the attribute score for the user by using various action histories such as browsing history and purchase history instead of or in addition to the search history.

(Modification 2)
In the above-described embodiment, the determination unit 137 determines that the attribute estimation model is degraded when the normalized accuracy rate is smaller than the threshold, but the degraded state determination method is not limited to this.
For example, the processor 13 may not function as the normalization unit 136, and the determination unit 137 may determine the deterioration state of the attribute estimation model based on changes in the non-normalized accuracy rate.
For example, the determination unit 137 may determine that the attribute estimation model is degraded when the accuracy rate is below the threshold for several consecutive times. Alternatively, the determination unit 137 calculates the rate of decrease in the accuracy rate with respect to the passage of time after the generation of the attribute estimation model, and when the accuracy rate rapidly decreases, regardless of the threshold, the attribute estimation model deteriorates. It may be determined that
Further, in the above-described embodiment, the determination unit 137 may determine the deterioration state of the attribute estimation model based on the change in the number of correct users included in the top user list.

(Modification 3)
In the above-described embodiment, the processor 13 may function as a graph generation unit that graphs changes in accuracy rate corresponding to each attribute. For example, as shown in FIG. 4, the graph generation unit may generate a graph showing changes in the accuracy rate when the attribute estimation model is continued to be used without being updated, and may provide the graph to the service provider or the like. This graph may be used for marketing or the like as information indicating changes in search words related to attributes.

DESCRIPTION OF SYMBOLS 1... Information processing system 10... Information processing apparatus 11... Communication part 12... Storage part 121... Action history information storage part 122... Estimated attribute information storage part 123... Model storage part 124... Accuracy rate storage part , 13... Processor, 131... History acquisition unit, 132... Learning unit, 133... Attribute estimation unit, 134... List generation unit, 135... Accuracy rate calculation unit, 136... Normalization unit, 137... Judgment unit, 20... User terminal , 30 ... service providing server.

Claims (5)

a history acquisition unit that acquires a user's action history on the network;
The action history of the user is input to an attribute estimation model generated by machine learning using the action history of the correct user corresponding to the target attribute, and the score of the target attribute output from the attribute estimation model is calculated by the user an attribute estimator associated with
a list generation unit that generates a top user list by selecting a predetermined number of users in descending order of the score of the target attribute;
an information processing apparatus comprising: a determination unit that determines a deterioration state of the attribute estimation model based on a change in the number of correct users included in the top user list. The information processing apparatus according to claim 1, wherein the determination unit calculates a ratio of the correct users to the upper user list as the number of correct users included in the upper user list. A normalization unit that normalizes the ratio calculated at a second time point after the first time point by the ratio calculated at an arbitrary first time point,
The information processing apparatus according to claim 2, wherein the determination unit determines the deterioration state of the attribute estimation model by comparing the ratio after normalization with a threshold. An information processing method for processing information by a computer,
a history acquisition step of acquiring a user's action history on the network;
The action history of the user is input to an attribute estimation model generated by machine learning using the action history of the correct user corresponding to the target attribute, and the score of the target attribute output from the attribute estimation model is calculated by the user an attribute estimation step that maps to
A list generation step of generating a top user list by selecting a predetermined number of users in descending order of the score of the target attribute;
and a determination step of determining a deterioration state of the attribute estimation model based on a change in the number of correct users included in the top user list. A computer readable and executable information processing program,
An information processing program causing the computer to function as the information processing apparatus according to any one of claims 1 to 3.

Images