JP5661689B2 - Content distribution device - Google Patents

Description

  The present invention relates to a technique for distributing advertising content via a network such as the Internet.

  In Internet homepages and the like, there are often advertisement columns for display advertisements typified by banner advertisements.

  In many cases, an advertiser prepares a plurality of advertisement contents for one product in consideration of viewer's preference. Since it is not possible to know in advance what kind of advertising content is accepted by the viewer, it can be distributed randomly (for example, 0.25 (25%) each when there are four advertising content) or for a predetermined period Distribution probability proportional to the click through rate (CTR) of (e.g., when the CTR of the four advertising contents is 0.8%, 0.6%, 0.4%, 0.2%, (Delivery probability is 40%, 30%, 20%, 10% in proportion to CTR). The CTR is a value obtained by dividing the number of selected (clicked) advertising content such as a display advertisement by the number of distributions, and is an index for evaluating the advertising effect.

JP 2009-271661 A

  As described above, when there are a plurality of advertisement contents for one commercial item, the distribution probabilities have been made uniform or values proportional to the CTR have been used in the past, which is not sufficient for improving the advertising effect. was there.

  In other words, in order to maximize the total number of clicks, it is necessary to focus on one of the most popular advertising content. Dividing a considerable amount of distribution probability into the second or lower advertising content means that the number of clicks This is against maximization.

  However, the CTR fluctuates greatly, and it is dangerous to say that the advertising content that showed the highest value at a certain point in time is the most popular advertising content. For this reason, there was no other way to follow the conventional method.

  In recent years, a plurality of patterns such as buttons, main visuals, and catch phrases are prepared, and the advertising content is often configured by a combination thereof. In many cases, a large amount of advertising content (mass creative) is prepared for one product. .

  FIG. 1 is a diagram showing an example of mass creative, with 6 logo patterns, 6 main visual patterns, and 18 catch phrases. In this case, the total number of advertisement contents is 6 × 6 × 18 = 648 which is a combination of the respective elements.

  In the case of such a large amount of creatives, there is an advertising content in which CTR is not recorded within the advertisement insertion period (when the insertion period is determined by the date and time, or until the predetermined number of impressions (presentation number) is reached). In many cases, it is difficult to determine the distribution probability of advertisement content based on CTR.

  On the other hand, Patent Literature 1 discloses a technique for accurately evaluating the effectiveness of a creative (advertisement) with a small number of exposures so that the creative is efficiently exposed. However, it is based on a statistical method using a normal distribution, and is not necessarily sufficient for improving the advertising effect.

  The present invention has been proposed in view of the above-described conventional problems. The purpose of the present invention is to introduce the concept of reinforcement learning and to predict the CTR of each advertising content, so It is to improve the advertising effect when there is any advertising content.

  In order to solve the above-described problem, in the present invention, for each advertisement content in an advertisement database that manages a plurality of advertisement contents for one product, a feature amount that represents the feature of the advertisement content is generated. Means for selecting a target group of advertisement contents from the advertisement database, means for setting an initial delivery probability for each of the selected group of advertisement contents, means for initially setting a coefficient vector, Means for calculating the predicted CTR from the coefficient vector and the feature amount for each of the selected group of advertisement contents, means for distributing the advertisement content set with the distribution probability and recording the actual CTR, Using the predicted CTR for a predetermined period for each content as a reward, one advertisement component is selected from a plurality of advertisement contents based on the reward. Means for performing reinforcement learning so as to leave a minimum distribution probability for the remaining advertisement content while calculating and setting a new distribution probability for each advertisement content. And a means for updating the coefficient vector on the basis thereof.

  In the content distribution apparatus of the present invention, the concept of reinforcement learning is introduced, and the CTR of each advertising content is predicted, thereby improving the advertising effect when a large amount of advertising content exists for one product. it can.

It is a figure which shows the example of a mass creative. It is a figure which shows the structural example of the system concerning one Embodiment of this invention. It is a figure which shows the example of a data structure of advertisement DB. It is a flowchart which shows the process example of embodiment. FIG. 5 is a diagram (part 1) illustrating an example of generation of a feature amount of advertisement content. FIG. 10 is a second diagram illustrating an example of generation of a feature amount of advertisement content. It is a figure which shows the example of the setting of an initial delivery probability. It is a figure which shows the example of the setting of a delivery probability.

  Hereinafter, preferred embodiments of the present invention will be described.

<Configuration>
FIG. 2 is a diagram illustrating a configuration example of a system according to an embodiment of the present invention.

  In FIG. 2, a plurality of terminal devices 2 such as a PC (Personal Computer), a mobile phone, and a PDA (Personal Digital Assistants) operated by a user are connected to a network 1 such as the Internet. The terminal device 2 includes a general browser (Web browser) 21. The browser 21 performs request / acquisition / display of page data described in a language such as HTML (Hyper Text Markup Language), transmission of form data, and the like according to HTTP (Hyper Text Transfer Protocol) which is a standard protocol of the Internet. It has a function.

  On the other hand, an advertisement information providing device 3 that provides advertisement information to the browser 21 of the terminal device 2 is connected to the network 1.

  The advertisement information providing apparatus 3 includes, as function units, a feature amount generation unit 301, a target advertisement selection unit 302, an initial distribution probability setting unit 303, a coefficient vector initial setting unit 304, a predicted CTR calculation unit 305, and an advertisement information distribution / CTR recording unit. 306, a distribution probability calculation / setting unit 307, and a coefficient vector update unit 308.

  These functional units are realized by a computer program executed on a hardware resource such as a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory) of the computer constituting the advertisement information providing apparatus 3. It is what is done. These functional units do not need to be arranged on a single computer, and may be distributed as necessary.

  An advertisement DB (Data Base) 309 is provided as a database used by the advertisement information providing apparatus 3. This database systematically holds predetermined data on a storage medium such as an HDD (Hard Disk Drive) in a computer constituting the advertisement information providing apparatus 3. Note that the advertisement DB 309 does not need to be arranged in the advertisement information providing apparatus 3 and may be arranged on another apparatus.

  FIG. 3 is a diagram illustrating an example of the data structure of the advertisement DB 309. The advertisement DB 309 includes items such as “advertisement ID”, “number of advertisement contents”, “advertisement content ID”, “advertisement content data / reference”, “feature amount”, “actual CTR”, “predicted CTR (Q value)”, and “delivery probability”. It is out. “Advertisement ID” is information for identifying a group of advertisement contents related to one product of one advertiser. The “number of advertisement contents” is the number of advertisement contents included in a group of advertisement contents related to one product of one advertiser. The “advertisement content ID” is information for identifying individual advertisement content. “Advertisement content data / reference” is actual data or storage location information of the advertisement content.

  The “feature amount” is information representing the feature of each advertisement content. Details will be described later. “Actual CTR” is a record of a CTR actual value for the advertisement content. The “predicted CTR (Q value)” is a predicted CTR calculated based on the “feature amount” for each advertisement content. Details will be described later. “Distribution probability” is a distribution probability for each advertisement content calculated and determined.

  Returning to FIG. 2, the feature amount generation unit 301 of the advertisement information providing device 3 has a function of generating a feature amount for each advertisement content in the advertisement DB 309 and setting it in the advertisement DB 309.

  The target advertisement selection unit 302 has a function of selecting an advertisement to be processed (a group of advertisement contents related to one product of one advertiser) from the advertisement DB 309.

  The initial distribution probability setting unit 303 has a function of setting an initial distribution probability for a group of advertisement contents selected by the target advertisement selection unit 302.

  The coefficient vector initial setting unit 304 has a function of initializing a coefficient vector used for calculating a predicted CTR.

  The predicted CTR calculation unit 305 has a function of calculating a predicted CTR using a feature amount and a coefficient vector for each advertisement content.

  The advertisement information distribution / CTR recording unit 306 distributes the advertisement content registered in the advertisement DB 309 to the browser 21 of the terminal device 2 based on the set distribution probability (generally, a page request is made from the terminal device 2). The advertisement content is distributed in a form added to the original page content), and an advertisement content selection operation (such as a click operation of a banner advertisement) by the browser 21 of the terminal device 2 is detected and the advertisement is displayed as the actual CTR. It has a function of recording in the DB 309.

  The distribution probability calculation / setting unit 307 calculates a new distribution probability for each advertising content by a reinforcement learning method based on the predicted CTR for each advertising content recorded in the advertising DB 309 and sets the advertising probability in the advertising DB 309. have. Reinforcement learning is based on the empirical fact that behavior following large rewards is more reproducible and behavior following small rewards is more difficult to reproduce, in order to maximize the quantified reward signal , Learn what to do (how to choose action based on the situation) ("Reinforcement learning", translation by Sadayoshi Mikami, Masaaki Minagawa, Morikita Publishing Co., Ltd., issued on December 20, 2000) reference.).

  That is, the distribution probability calculation / setting unit 307 uses the predicted CTR for a predetermined period for each advertisement content as a reward, maximizes the distribution probability of one advertisement content from the plurality of advertisement contents based on the reward, and performs the remaining advertisements. Reinforcement learning is performed so as to leave a minimum distribution probability in the content, and a new distribution probability is calculated and set for each advertising content.

  The coefficient vector update unit 308 has a function of updating the coefficient vector based on the actual CTR.

<Operation>
FIG. 4 is a flowchart showing a processing example of the above embodiment.

  First, in FIG. 4A, the feature value generation unit 301 of the advertisement information providing device 3 generates a feature value for each advertisement content in the advertisement DB 309 and sets it in the advertisement DB 309 (step S100).

  FIG. 5 is a diagram illustrating an example of generation of feature amounts of advertisement content.

  For example, if the advertising content is composed of a combination of different types of components such as a main visual, a catch phrase, and a button, the main visual has 7 patterns, the catch phrase has 7 patterns, and the buttons have 4 patterns. As shown in FIG. 5A, there are a total of 18 elements including 7 elements corresponding to the main visual, 7 elements corresponding to the catch phrase, and 4 elements corresponding to the buttons.

  For example, when the main visual is the third pattern, the catch phrase is the fourth pattern, and the button is the second pattern of the advertising content, as shown in FIG. The elements and 16th element are set to “1”, and the others are set to “0”.

  FIG. 6 is a diagram showing another example of generation of the feature amount of the advertisement content, and corresponds to a case where the advertisement content is various and cannot be expressed by a combination of components.

In this case, based on the image similarity, for each advertising content, the distance (image similarity) from other advertising content (including self) is calculated,
(Distance to first advertising content, distance to second advertising content, ...)
Generates a vector.

  Returning to FIG. 4, processing such as setting the distribution probability will be described.

  In FIG.4 (b), if a process is started (step S101), the object advertisement selection part 302 of the advertisement information provision apparatus 3 will be processed from the advertisement DB309 by the advertisement (one group concerning one product of one advertiser). Of advertisement content) is selected (step S102). In the advertisement DB 309 of FIG. 3, one advertisement ID is selected.

  4B, the initial distribution probability setting unit 303 of the advertisement information providing apparatus 3 sets an initial distribution probability for the group of advertisement contents selected by the target advertisement selecting unit 302 (step S103). The initial distribution probability is calculated by obtaining the number of advertisement contents corresponding to the selected advertisement ID and taking the reciprocal number to obtain the same distribution probability for a group of advertisement contents. For example, if there are 196 advertisement contents, 1/196 = 0.0001. FIG. 7 shows the advertisement DB 309 in a state where the initial distribution probability is set.

Next, returning to FIG. 4B, the coefficient vector initial setting unit 304 of the advertisement information providing apparatus 3 initializes the matrix A and the vector b that define the coefficient vector θ (= A ⁻¹ b) (step S104). . That is, a d-dimensional unit matrix corresponding to the number of elements of the feature quantity is set in the matrix A, and a d-dimensional zero vector is set in the vector b.

Then, the prediction CTR calculation unit 305 of the advertisement information providing device 3, for all advertising content a included in the selected ad (features _{x a),} the _{Q a} is predicted CTR (Q value) Q _a ← x _a ^T θ + α · root (x _a ^T A ⁻¹ x _a )
Is calculated and set (step S105). The superscript T indicates transposition. Α is a constant that defines an allowable range of errors.

  Next, the advertisement information distribution / CTR recording unit 306 of the advertisement information providing device 3 receives the advertisement content registered in the advertisement DB 309 based on the set distribution probability when a page request is made from the terminal device 2. While delivering to the browser 21 of the apparatus 2, the selection operation (click operation etc. of a banner advertisement) by the browser 21 of the terminal device 2 is detected, and the result CTR is recorded in the advertisement DB 309 (step S106).

  Thereafter, after the elapse of a predetermined period for reviewing the distribution probability, the distribution probability calculation / setting unit 307 of the advertisement information providing device 3 uses a reinforcement learning method based on the predicted CTR of the predetermined period for each advertising content recorded in the advertisement DB 309. A new distribution probability is calculated for each advertising content (step S107).

For example, when pursuit methods are used, i is an index of advertising content (for example, if the advertising content is 196, i = 0 to 195), a _i is advertising content, and Π (a _i ) is advertising content. When the distribution probability immediately before a _i and β is a predetermined learning rate (a value between 0 and 1), the distribution probability of the advertising content with the maximum predicted CTR is
(Π (a _i ) + β (1−Π (a _i ))) / ΣΠ (a _i )
The delivery probability of other advertising content is
(Π (a _i ) + β (0−Π (a _i ))) / ΣΠ (a _i )
And The reason for dividing by ΣΠ (a _i ) is for normalization.

When the action selection rule (ε-greedy method) is used, when ε is a predetermined constant (a value between 0 and 1) and the number of advertising content is n, the distribution probability of the advertising content with the maximum predicted CTR is
1-ε
The delivery probability of other advertising content is
ε / (n-1)
And

  Next, the distribution probability calculation / setting unit 307 of the advertisement information providing apparatus 3 sets the calculated new distribution probability in the advertisement DB 309 (step S108). FIG. 8 shows the advertisement DB 309 in a state where the distribution probability is updated when ε = 0.1 by the ε greedy method.

Then, back in FIG. 4 (b), the coefficient vector updating unit 308 of the advertisement information providing device 3, the number of impressions the original calculation of the actual CTR of advertisement content a v _a, the coefficient vector based on the number of clicks r _a theta Is updated (step S109). That is,
A ← A + v _a x _a x _a ^T
b ← b + r _a x _a
And

  Then, it returns to calculation of prediction CTR (step S105), and repeats the same process.

<Summary>
As described above, according to the present embodiment, by introducing the concept of reinforcement learning, it is possible to enhance the advertising effect when there are a plurality of advertising contents for one product.

  Further, even when a large amount of advertising content exists, the accuracy of setting the distribution probability can be improved by assigning the predicted CTR for all the advertising content.

  The present invention has been described above by the preferred embodiments of the present invention. While the invention has been described with reference to specific embodiments, various modifications and changes may be made to the embodiments without departing from the broad spirit and scope of the invention as defined in the claims. Obviously you can. In other words, the present invention should not be construed as being limited by the details of the specific examples and the accompanying drawings.

DESCRIPTION OF SYMBOLS 1 Network 2 Terminal apparatus 21 Browser 3 Advertisement information provision apparatus 301 Feature-value production | generation part 302 Target advertisement selection part 303 Initial delivery probability setting part 304 Coefficient vector initial setting part 305 Predictive CTR calculation part 306 Advertisement information delivery and CTR recording part 307 Delivery probability Calculation / setting unit 308 Coefficient vector update unit 309 Advertising DB

Claims (5)

Means for generating, for each advertisement content in an advertisement database that manages a plurality of advertisement contents for one product, a feature value representing the feature of the advertisement content;
Means for selecting a group of targeted advertising content from the advertising database;
A means of setting an initial delivery probability for each of a selected group of advertising content;
Means for initializing the coefficient vector;
Means for calculating a predicted CTR from the coefficient vector and the feature amount for each of the selected group of advertising content;
Means for distributing the advertising content with the distribution probability set and recording the actual CTR;
Using the predicted CTR for a predetermined period for each advertising content as a reward, maximizing the distribution probability of one advertising content out of a plurality of advertising contents based on the reward, leaving the minimum distribution probability for the remaining advertising content Reinforce learning and calculating and setting a new distribution probability for each advertising content,
And a means for updating the coefficient vector based on the actual CTR. The content delivery apparatus according to claim 1,
The feature amount is a vector having the total number of patterns for each type of component of the advertising content, and the element corresponding to the pattern position for each type of component actually used is set to “1”. A content distribution apparatus characterized in that the element of “0” is set to “0”. The content delivery apparatus according to claim 1,
The content distribution apparatus, wherein the feature amount is a vector having an element of image similarity between each advertisement content and all other advertisement content. In the content delivery apparatus as described in any one of Claims 1 thru | or 3,
Either of the tracking method or the action selection rule is used as the reinforcement learning. A step of generating, for each advertisement content in an advertisement database managing a plurality of advertisement contents for one product, a feature amount representing a feature of the advertisement content;
The content distribution device selecting a target group of advertising content from the advertising database;
The content distribution device setting an initial distribution probability for each of the selected group of advertising content;
The content distribution device initializing a coefficient vector;
The content distribution device calculating a predicted CTR from the coefficient vector and the feature amount for each of the selected group of advertisement content;
The content distribution device distributes the advertising content set with the distribution probability and records the actual CTR;
The content distribution device uses the predicted CTR for a predetermined period for each advertisement content as a reward, and maximizes the distribution probability of one advertisement content out of a plurality of advertisement contents based on the reward, while also minimizing the remaining advertisement content Performing reinforcement learning so as to leave a limited delivery probability, calculating and setting a new delivery probability for each advertising content,
The content distribution method comprising: a step of updating the coefficient vector based on the actual CTR.