JP2012141683A - Advertisement information providing device - Google Patents

Advertisement information providing device Download PDF

Info

Publication number
JP2012141683A
JP2012141683A JP2010292483A JP2010292483A JP2012141683A JP 2012141683 A JP2012141683 A JP 2012141683A JP 2010292483 A JP2010292483 A JP 2010292483A JP 2010292483 A JP2010292483 A JP 2010292483A JP 2012141683 A JP2012141683 A JP 2012141683A
Authority
JP
Japan
Prior art keywords
advertisement
advertising content
distribution probability
ctr
probability
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2010292483A
Other languages
Japanese (ja)
Other versions
JP5265659B2 (en
Inventor
Koji Tsukamoto
浩司 塚本
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yahoo Japan Corp
Original Assignee
Yahoo Japan Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yahoo Japan Corp filed Critical Yahoo Japan Corp
Priority to JP2010292483A priority Critical patent/JP5265659B2/en
Publication of JP2012141683A publication Critical patent/JP2012141683A/en
Application granted granted Critical
Publication of JP5265659B2 publication Critical patent/JP5265659B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Transfer Between Computers (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

PROBLEM TO BE SOLVED: To enhance an advertisement effect in a case where a plurality of advertisement contents are present for one commercial material by introducing an idea of reinforcement learning.SOLUTION: An advertisement information providing device comprises: means for selecting a target group of advertisement contents from an advertisement database that manages a plurality of advertisement contents for one commercial material; means for equally setting an initial distribution probability for each of the selected group of advertisement contents; means for distributing an advertisement content with a set distribution probability to record a CTR; and means for performing reinforcement learning, by using the CTR in a predetermined period for each advertisement content as a commission, in such a manner that the distribution probability of one advertisement content of the plurality of advertisement contents is maximized based the commission while a minimum distribution probability is left for the remaining advertisement contents, and calculating and setting a new distribution probability for each advertisement content.

Description

本発明は広告情報提供装置に関する。   The present invention relates to an advertisement information providing apparatus.

インターネットのホームページ等においては、バナー広告と呼ばれる広告欄が設けられることが多い。   Internet homepages and the like often have an advertisement column called a banner advertisement.

また、広告主は１つの商材につき、視聴者の嗜好を考慮して複数の広告コンテンツを用意する場合が多い。どのような広告コンテンツが視聴者に受け入れられるかについては、予め知ることはできないので、ランダムに配信したり（例えば、４つの広告コンテンツがある場合は０．２５（２５％）ずつ）、所定期間のＣＴＲ（Click Through Rate）に比例した配信確率を用いたりする（例えば、４つの広告コンテンツのＣＴＲが０．８％、０．６％、０．４％、０．２％であった場合、配信確率をＣＴＲに比例させて４０％、３０％、２０％、１０％）。なお、ＣＴＲとは、バナー広告のような広告コンテンツが選択（クリック）された数を、配信数で割った値であり、広告効果を評価する指標の一つである。   In many cases, an advertiser prepares a plurality of advertisement contents for one product in consideration of viewer's preference. Since it is not possible to know in advance what kind of advertising content is accepted by the viewer, it can be distributed randomly (for example, 0.25 (25%) each when there are four advertising content) or for a predetermined period Distribution probability proportional to the click through rate (CTR) of (e.g., when the CTR of the four advertising contents is 0.8%, 0.6%, 0.4%, 0.2%, (Delivery probability is 40%, 30%, 20%, 10% in proportion to CTR). CTR is a value obtained by dividing the number of selected (clicked) advertising content such as a banner advertisement by the number of distributions, and is an index for evaluating the advertising effect.

特開２００９−２７１６６１号公報JP 2009-271661 A

上述したように、１商材について複数の広告コンテンツが存在する場合、従来は配信確率を均等にしたり、ＣＴＲに比例した値を用いたりしていたため、広告効果を高める上で充分ではないという問題があった。   As described above, when there are a plurality of advertisement contents for one commercial item, the distribution probabilities have been made uniform or values proportional to the CTR have been used in the past, which is not sufficient for improving the advertising effect. was there.

すなわち、トータルのクリック数を最大化するためには、最も評判のいい広告コンテンツ１本に絞ることが必要であり、２位以下の広告コンテンツに相当量の配信確率を割くということはクリック数の最大化には反するからである。   In other words, in order to maximize the total number of clicks, it is necessary to focus on one of the most popular advertising content. Dividing a considerable amount of distribution probability into the second or lower advertising content means that the number of clicks This is against maximization.

しかしながら、ＣＴＲは変動が大きく、ある時点で最高の値を示していた広告コンテンツが最も評判がいい広告コンテンツであると断ずることは危険である。そのため、従来の手法を踏襲するほか術がなかった。   However, the CTR fluctuates greatly, and it is dangerous to say that the advertising content that showed the highest value at a certain point in time is the most popular advertising content. For this reason, there was no other way to follow the conventional method.

一方、特許文献１には、少ない露出回数で的確にクリエイティブ（広告）の有効性を評価し、クリエイティブが効率よく露出されるようにする技術が開示されている。しかし、正規分布を用いた統計的手法に基づくものであり、必ずしも広告効果を高める上では充分ではなかった。   On the other hand, Patent Literature 1 discloses a technique for accurately evaluating the effectiveness of a creative (advertisement) with a small number of exposures so that the creative is efficiently exposed. However, it is based on a statistical method using a normal distribution, and is not necessarily sufficient for improving the advertising effect.

本発明は上記の従来の問題点に鑑み提案されたものであり、その目的とするところは、強化学習の考え方を導入することで、１商材について複数の広告コンテンツが存在する場合の広告効果を高めることにある。   The present invention has been proposed in view of the above-described conventional problems, and the purpose of the present invention is to introduce the concept of reinforcement learning, thereby providing an advertising effect when a plurality of advertising contents exist for one product. Is to increase.

上記の課題を解決するため、本発明にあっては、請求項１に記載されるように、１つの商材についての複数の広告コンテンツを管理する広告データベースから対象となる一群の広告コンテンツを選択する手段と、選択された一群の広告コンテンツの個々に対して初回の配信確率を等分に設定する手段と、配信確率が設定された広告コンテンツを配信し、ＣＴＲを記録する手段と、広告コンテンツ毎の所定期間のＣＴＲを報酬として、当該報酬に基づき複数の広告コンテンツの中から一の広告コンテンツの配信確率を最大化しつつ、残りの広告コンテンツにも最小限の配信確率を残すように強化学習を行い、新たな配信確率を広告コンテンツ毎に算出して設定する手段とを備える広告情報提供装置を要旨としている。   In order to solve the above problems, in the present invention, as described in claim 1, a target group of advertisement contents is selected from an advertisement database that manages a plurality of advertisement contents for one product. Means for equally setting the initial distribution probability for each of the selected group of advertising content, means for distributing the advertising content set with the distribution probability and recording CTR, and advertising content Reinforcement learning with CTR for each predetermined period as a reward, while maximizing the distribution probability of one advertisement content among a plurality of advertisement contents based on the reward, and leaving the minimum distribution probability for the remaining advertisement content And a means for calculating and setting a new distribution probability for each advertising content.

また、請求項２に記載されるように、広告情報提供装置の制御部が、１つの商材についての複数の広告コンテンツを管理する広告データベースから対象となる一群の広告コンテンツを選択する工程と、前記制御部が、選択された一群の広告コンテンツの個々に対して初回の配信確率を等分に設定する工程と、前記制御部が、配信確率が設定された広告コンテンツを配信し、ＣＴＲを記録する工程と、前記制御部が、広告コンテンツ毎の所定期間のＣＴＲを報酬として、当該報酬に基づき複数の広告コンテンツの中から一の広告コンテンツの配信確率を最大化しつつ、残りの広告コンテンツにも最小限の配信確率を残すように強化学習を行い、新たな配信確率を広告コンテンツ毎に算出して設定する工程とを備える広告情報提供方法として構成することができる。   According to a second aspect of the present invention, the control unit of the advertisement information providing device selects a target group of advertisement contents from an advertisement database that manages a plurality of advertisement contents for one product; The control unit equally sets the initial distribution probability for each of the selected group of advertising content, and the control unit distributes the advertising content with the distribution probability set and records the CTR. And the control unit uses a CTR for a predetermined period for each advertising content as a reward, and maximizes the distribution probability of one advertising content out of the plurality of advertising contents based on the reward, and also applies to the remaining advertising content. Reinforcement learning to leave a minimum distribution probability, and a method for providing an advertisement information comprising a step of calculating and setting a new distribution probability for each advertising content Rukoto can.

また、請求項３に記載されるように、広告情報提供装置を構成するコンピュータを、１つの商材についての複数の広告コンテンツを管理する広告データベースから対象となる一群の広告コンテンツを選択する手段、選択された一群の広告コンテンツの個々に対して初回の配信確率を等分に設定する手段、配信確率が設定された広告コンテンツを配信し、ＣＴＲを記録する手段、広告コンテンツ毎の所定期間のＣＴＲを報酬として、当該報酬に基づき複数の広告コンテンツの中から一の広告コンテンツの配信確率を最大化しつつ、残りの広告コンテンツにも最小限の配信確率を残すように強化学習を行い、新たな配信確率を広告コンテンツ毎に算出して設定する手段として機能させる広告情報提供プログラムとして構成することができる。   According to a third aspect of the present invention, means for selecting a target group of advertisement contents from an advertisement database that manages a plurality of advertisement contents for one product, the computer constituting the advertisement information providing apparatus, Means for equally setting the initial distribution probability for each of the selected group of advertising content, means for distributing the advertising content with the distribution probability set and recording CTR, CTR for a predetermined period for each advertising content As a reward, based on the reward, we will reinforce learning to maximize the probability of distribution of one of the advertising content, and leave the minimum probability of distribution to the remaining advertising content, and new distribution It can be configured as an advertisement information providing program that functions as a means for calculating and setting the probability for each advertisement content.

本発明の広告情報提供装置にあっては、強化学習の考え方を導入することで、１商材について複数の広告コンテンツが存在する場合の広告効果を高めることができる。   In the advertisement information providing apparatus of the present invention, by introducing the concept of reinforcement learning, it is possible to enhance the advertisement effect when a plurality of advertisement contents exist for one product.

本発明の一実施形態にかかるシステムの構成例を示す図である。It is a figure which shows the structural example of the system concerning one Embodiment of this invention. 広告ＤＢのデータ構造例を示す図である。It is a figure which shows the example of a data structure of advertisement DB. 広告情報提供装置のハードウェア構成例を示す図である。It is a figure which shows the hardware structural example of an advertisement information provision apparatus. 実施形態の処理例を示すフローチャートである。It is a flowchart which shows the process example of embodiment. 初期配信確率の設定の例を示す図である。It is a figure which shows the example of the setting of an initial delivery probability. 配信確率の設定の例を示す図である。It is a figure which shows the example of the setting of a delivery probability.

以下、本発明の好適な実施形態につき説明する。   Hereinafter, preferred embodiments of the present invention will be described.

＜構成＞
図１は本発明の一実施形態にかかるシステムの構成例を示す図である。 <Configuration>
FIG. 1 is a diagram showing a configuration example of a system according to an embodiment of the present invention.

図１において、インターネット等のネットワーク１には、ユーザが操作するＰＣ（Personal Computer）、携帯電話、ＰＤＡ（Personal Digital Assistants）等のユーザ端末２が複数接続されている。ユーザ端末２は、一般的なブラウザ（Ｗｅｂブラウザ）２１を備えている。ブラウザ２１は、インターネットの標準プロトコルであるＨＴＴＰ（Hyper Text Transfer Protocol）等に従い、ＨＴＭＬ（Hyper Text Markup Language）等の言語で記述されたページデータの要求・取得・表示およびフォームデータの送信等を行う機能を有している。   In FIG. 1, a plurality of user terminals 2 such as a PC (Personal Computer), a mobile phone, and a PDA (Personal Digital Assistants) operated by a user are connected to a network 1 such as the Internet. The user terminal 2 includes a general browser (Web browser) 21. The browser 21 performs request / acquisition / display of page data described in a language such as HTML (Hyper Text Markup Language), transmission of form data, and the like according to HTTP (Hyper Text Transfer Protocol) which is a standard protocol of the Internet. It has a function.

一方、ネットワーク１には、ユーザ端末２のブラウザ２１に対して広告情報を提供する広告情報提供装置３が接続されている。   On the other hand, an advertisement information providing device 3 that provides advertisement information to the browser 21 of the user terminal 2 is connected to the network 1.

広告情報提供装置３は、機能部として、対象広告選択部３０１と初期配信確率設定部３０２と広告情報配信・ＣＴＲ記録部３０３と配信確率算出・設定部３０４とを備えている。   The advertisement information providing apparatus 3 includes a target advertisement selection unit 301, an initial distribution probability setting unit 302, an advertisement information distribution / CTR recording unit 303, and a distribution probability calculation / setting unit 304 as functional units.

これらの機能部は、広告情報提供装置３を構成するコンピュータのＣＰＵ（Central Processing Unit）、ＲＯＭ（Read Only Memory）、ＲＡＭ（Random Access Memory）等のハードウェア資源上で実行されるコンピュータプログラムによって実現されるものである。これらの機能部は、単一のコンピュータ上に配置される必要はなく、必要に応じて分散される形態であってもよい。   These functional units are realized by a computer program executed on a hardware resource such as a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory) of the computer constituting the advertisement information providing apparatus 3. It is what is done. These functional units do not need to be arranged on a single computer, and may be distributed as necessary.

また、広告情報提供装置３が利用するデータベースとして、広告ＤＢ（Data Base）３０５が設けられている。このデータベースは、広告情報提供装置３を構成するコンピュータ内のＨＤＤ（Hard Disk Drive）等の記憶媒体上に所定のデータを体系的に保持するものである。なお、広告ＤＢ３０５は広告情報提供装置３内に配置される必要はなく、他の装置上に配置してもよい。   An advertisement DB (Data Base) 305 is provided as a database used by the advertisement information providing apparatus 3. This database systematically holds predetermined data on a storage medium such as an HDD (Hard Disk Drive) in a computer constituting the advertisement information providing apparatus 3. Note that the advertisement DB 305 does not need to be arranged in the advertisement information providing apparatus 3 and may be arranged on another apparatus.

図２は広告ＤＢ３０５のデータ構造例を示す図である。広告ＤＢ３０５は、「広告ＩＤ」「広告コンテンツ数」「広告コンテンツＩＤ」「広告コンテンツデータ／リファレンス」「ＣＴＲ」「配信確率」等の項目を含んでいる。「広告ＩＤ」は、一の広告主の一の商材にかかる一群の広告コンテンツを識別する情報である。「広告コンテンツ数」は、一の広告主の一の商材にかかる一群の広告コンテンツに含まれる広告コンテンツの数である。「広告コンテンツＩＤ」は、個々の広告コンテンツを識別する情報である。「広告コンテンツデータ／リファレンス」は、広告コンテンツの実データもしくは格納場所の情報である。「ＣＴＲ」は、その広告コンテンツについてのＣＴＲの記録である。「配信確率」は、算出されて決定された広告コンテンツ毎の配信確率である。   FIG. 2 is a diagram illustrating an example of the data structure of the advertisement DB 305. The advertisement DB 305 includes items such as “advertisement ID”, “number of advertisement contents”, “advertisement content ID”, “advertisement content data / reference”, “CTR”, and “delivery probability”. “Advertisement ID” is information for identifying a group of advertisement contents related to one product of one advertiser. The “number of advertisement contents” is the number of advertisement contents included in a group of advertisement contents related to one product of one advertiser. The “advertisement content ID” is information for identifying individual advertisement content. “Advertisement content data / reference” is actual data or storage location information of the advertisement content. “CTR” is a record of CTR for the advertising content. “Distribution probability” is a distribution probability for each advertisement content calculated and determined.

図１に戻り、広告情報提供装置３の対象広告選択部３０１は、広告ＤＢ３０５から処理対象となる広告（一の広告主の一の商材にかかる一群の広告コンテンツ）を選択する機能を有している。   Returning to FIG. 1, the target advertisement selection unit 301 of the advertisement information providing apparatus 3 has a function of selecting an advertisement to be processed (a group of advertisement contents related to one product of one advertiser) from the advertisement DB 305. ing.

初期配信確率設定部３０２は、対象広告選択部３０１の選択した一群の広告コンテンツにつき、初期の配信確率を設定する機能を有している。   The initial distribution probability setting unit 302 has a function of setting an initial distribution probability for a group of advertisement contents selected by the target advertisement selection unit 301.

広告情報配信・ＣＴＲ記録部３０３は、広告ＤＢ３０５に登録された広告コンテンツを、設定された配信確率に基づいてユーザ端末２のブラウザ２１に配信するとともに、ユーザ端末２のブラウザ２１による広告の選択操作（バナー広告のクリック操作等）を検出し、ＣＴＲを広告ＤＢ３０５に記録する機能を有している。   The advertisement information distribution / CTR recording unit 303 distributes the advertisement content registered in the advertisement DB 305 to the browser 21 of the user terminal 2 based on the set distribution probability, and the advertisement selection operation by the browser 21 of the user terminal 2 It has a function of detecting a banner advertisement click operation and recording the CTR in the advertisement DB 305.

配信確率算出・設定部３０４は、広告ＤＢ３０５に記録された広告コンテンツ毎の所定期間のＣＴＲに基づき、強化学習の手法により新たな配信確率を広告コンテンツ毎に算出して広告ＤＢ３０５に設定する機能を有している。強化学習は、大きい報酬に続く行動はより再現しやすく、小さい報酬に続く行動はより再現しづらくなるという経験的事実を基本とした方法であり、数値化された報酬信号を最大にするために、何をすべきか（どのようにして状況に基づく動作選択を行うか）を学習する（「強化学習」、三上貞芳・皆川雅章 訳、森北出版株式会社、２０００年１２月２０日発行等を参照。）。すなわち、配信確率算出・設定部３０４は、広告コンテンツ毎の所定期間のＣＴＲを報酬として、当該報酬に基づき複数の広告コンテンツの中から一の広告コンテンツの配信確率を最大化しつつ、残りの広告コンテンツにも最小限の配信確率を残すように強化学習を行い、新たな配信確率を広告コンテンツ毎に算出して設定する。   The distribution probability calculation / setting unit 304 has a function of calculating a new distribution probability for each advertising content by a reinforcement learning method and setting it in the advertising DB 305 based on the CTR of each advertising content recorded in the advertising DB 305 for a predetermined period. Have. Reinforcement learning is based on the empirical fact that behavior following large rewards is more reproducible and behavior following small rewards is more difficult to reproduce, in order to maximize the quantified reward signal , Learn what to do (how to choose action based on the situation) ("Reinforcement learning", translation by Sadayoshi Mikami, Masaaki Minagawa, Morikita Publishing Co., Ltd. reference.). That is, the distribution probability calculation / setting unit 304 uses the CTR for a predetermined period for each advertisement content as a reward, maximizes the distribution probability of one advertisement content from the plurality of advertisement contents based on the reward, and maintains the remaining advertisement content In addition, reinforcement learning is performed so as to leave a minimum distribution probability, and a new distribution probability is calculated and set for each advertisement content.

図３は広告情報提供装置３のハードウェア構成例を示す図である。   FIG. 3 is a diagram illustrating a hardware configuration example of the advertisement information providing apparatus 3.

図３において、広告情報提供装置３は、システムバス３１に接続されたＣＰＵ３２、ＲＯＭ３３、ＲＡＭ３４、ＮＶＲＡＭ（Non-Volatile Random Access Memory）３５、Ｉ／Ｆ（Interface）３６と、Ｉ／Ｆ３６に接続された、キーボード、マウス、モニタ、ＣＤ／ＤＶＤ（Compact Disk/Digital Versatile Disk）ドライブ等のＩ／Ｏ（Input/Output Device）３７、ＨＤＤ３８、ＮＩＣ（Network Interface Card）３９等を備えている。Ｍはプログラムもしくはデータが格納されたＣＤ／ＤＶＤ等のメディア（記録媒体）である。   In FIG. 3, the advertisement information providing device 3 is connected to a CPU 32, ROM 33, RAM 34, NVRAM (Non-Volatile Random Access Memory) 35, I / F (Interface) 36, and I / F 36 connected to the system bus 31. Also provided are an input / output device (I / O) 37 such as a keyboard, mouse, monitor, CD / DVD (Compact Disk / Digital Versatile Disk) drive, HDD 38, NIC (Network Interface Card) 39, and the like. M is a medium (recording medium) such as a CD / DVD in which a program or data is stored.

＜動作＞
図４は上記の実施形態の処理例を示すフローチャートである。 <Operation>
FIG. 4 is a flowchart showing a processing example of the above embodiment.

図４において、処理を開始すると（ステップＳ１０１）、広告情報提供装置３の対象広告選択部３０１は、広告ＤＢ３０５から処理対象となる広告（一の広告主の一の商材にかかる一群の広告コンテンツ）を選択する（ステップＳ１０２）。図２の広告ＤＢ３０５では広告ＩＤを一つ選択する。   In FIG. 4, when processing is started (step S <b> 101), the target advertisement selection unit 301 of the advertisement information providing device 3 performs advertisement processing (a group of advertisement contents related to one product of one advertiser from the advertisement DB 305). ) Is selected (step S102). In the advertisement DB 305 in FIG. 2, one advertisement ID is selected.

次いで、図４に戻り、広告情報提供装置３の初期配信確率設定部３０２は、対象広告選択部３０１の選択した一群の広告コンテンツにつき、初期の配信確率を設定する（ステップＳ１０３）。初期の配信確率は同じ配信確率にすべく、選択した広告ＩＤに対応する広告コンテンツ数を得て、逆数とすることで初期の配信確率を算出する。例えば、４つの広告コンテンツがある場合は０．２５ずつとなる。図５は初期の配信確率が設定された状態における広告ＤＢ３０５を示している。   Next, returning to FIG. 4, the initial distribution probability setting unit 302 of the advertisement information providing apparatus 3 sets an initial distribution probability for the group of advertisement contents selected by the target advertisement selection unit 301 (step S103). In order to set the initial distribution probability to the same distribution probability, the initial distribution probability is calculated by obtaining the number of advertisement contents corresponding to the selected advertisement ID and taking the reciprocal number. For example, when there are four advertisement contents, the number is 0.25. FIG. 5 shows the advertisement DB 305 in a state where the initial distribution probability is set.

次いで、図４に戻り、広告情報提供装置３の広告情報配信・ＣＴＲ記録部３０３は、広告ＤＢ３０５に登録された広告コンテンツを、設定された配信確率に基づいてユーザ端末２のブラウザ２１に配信するとともに、ユーザ端末２のブラウザ２１による広告の選択操作（バナー広告のクリック操作等）を検出し、ＣＴＲを広告ＤＢ３０５に記録する（ステップＳ１０４）。   Next, returning to FIG. 4, the advertisement information distribution / CTR recording unit 303 of the advertisement information providing apparatus 3 distributes the advertisement content registered in the advertisement DB 305 to the browser 21 of the user terminal 2 based on the set distribution probability. At the same time, an advertisement selection operation (such as a banner advertisement click operation) by the browser 21 of the user terminal 2 is detected, and the CTR is recorded in the advertisement DB 305 (step S104).

その後、所定の期間経過後、広告情報提供装置３の配信確率算出・設定部３０４は、広告ＤＢ３０５に記録された広告コンテンツ毎の所定期間のＣＴＲに基づき、強化学習の手法により新たな配信確率を広告コンテンツ毎に算出する（ステップＳ１０５）。   Thereafter, after a predetermined period of time, the distribution probability calculation / setting unit 304 of the advertisement information providing apparatus 3 calculates a new distribution probability by a reinforcement learning method based on the CTR for a predetermined period for each advertising content recorded in the advertisement DB 305. Calculation is made for each advertisement content (step S105).

例えば、追跡法（Pursuit Methods）を用いる場合、ｉを広告コンテンツのインデックス（例えば、広告コンテンツが４個の場合はｉ＝０〜３）、ａｉを広告コンテンツ、Π（ａｉ）を広告コンテンツａｉの直前の配信確率、βを所定の学習率（０〜１の間の値）とすると、ＣＴＲが最大の広告コンテンツの配信確率は、
（Π（ａｉ）＋β（１−Π（ａｉ）））／ΣΠ（ａｉ）
その他の広告コンテンツの配信確率は、
（Π（ａｉ）＋β（０−Π（ａｉ）））／ΣΠ（ａｉ）
とする。ΣΠ（ａｉ）で割っているのは正規化するためである。 For example, when using Pursuit Methods, i is an index of advertising content (for example, i = 0 to 3 when there are four advertising content), a i is advertising content, and Π (a i ) is advertising content. If the distribution probability immediately before a i and β is a predetermined learning rate (a value between 0 and 1), the distribution probability of the advertising content with the largest CTR is
(Π (a i ) + β (1−Π (a i ))) / ΣΠ (a i )
The delivery probability of other advertising content is
(Π (a i ) + β (0−Π (a i ))) / ΣΠ (a i )
And The reason for dividing by ΣΠ (a i ) is for normalization.

行動選択規則（εグリーディ手法）を用いる場合、εを所定の定数（０〜１の間の値）、広告コンテンツの数をｎとすると、ＣＴＲが最大の広告コンテンツの配信確率は、
１−ε
その他の広告コンテンツの配信確率は、
ε／（ｎ−１）
とする。 When the action selection rule (ε-greedy method) is used, when ε is a predetermined constant (a value between 0 and 1) and the number of advertising content is n, the distribution probability of the advertising content with the maximum CTR is
1-ε
The delivery probability of other advertising content is
ε / (n-1)
And

次いで、広告情報提供装置３の配信確率算出・設定部３０４は、算出された新たな配信確率を広告ＤＢ３０５に設定する（ステップＳ１０６）。図６は、εグリーディ手法によりε＝０．１とした場合に、配信確率が更新された状態の広告ＤＢ３０５を示している。   Next, the distribution probability calculation / setting unit 304 of the advertisement information providing apparatus 3 sets the calculated new distribution probability in the advertisement DB 305 (step S106). FIG. 6 shows the advertisement DB 305 in a state where the distribution probability is updated when ε = 0.1 by the ε greedy method.

その後、図４に戻り、広告情報の配信およびＣＴＲの記録（ステップＳ１０４）に戻り、同様の処理を繰り返す。   Thereafter, returning to FIG. 4, the process returns to the distribution of advertisement information and the recording of CTR (step S104), and the same processing is repeated.

＜総括＞
以上説明したように、本実施形態によれば、強化学習の考え方を導入することで、１商材について複数の広告コンテンツが存在する場合の広告効果を高めることができる。 <Summary>
As described above, according to the present embodiment, by introducing the concept of reinforcement learning, it is possible to enhance the advertising effect when there are a plurality of advertising contents for one product.

以上、本発明の好適な実施の形態により本発明を説明した。ここでは特定の具体例を示して本発明を説明したが、特許請求の範囲に定義された本発明の広範な趣旨および範囲から逸脱することなく、これら具体例に様々な修正および変更を加えることができることは明らかである。すなわち、具体例の詳細および添付の図面により本発明が限定されるものと解釈してはならない。   The present invention has been described above by the preferred embodiments of the present invention. While the invention has been described with reference to specific embodiments, various modifications and changes may be made to the embodiments without departing from the broad spirit and scope of the invention as defined in the claims. Obviously you can. In other words, the present invention should not be construed as being limited by the details of the specific examples and the accompanying drawings.

１ ネットワーク
２ ユーザ端末
２１ ブラウザ
３ 広告情報提供装置
３０１ 対象広告選択部
３０２ 初期配信確率設定部
３０３ 広告情報配信・ＣＴＲ記録部
３０４ 配信確率算出・設定部
３０５ 広告ＤＢ DESCRIPTION OF SYMBOLS 1 Network 2 User terminal 21 Browser 3 Advertisement information provision apparatus 301 Target advertisement selection part 302 Initial delivery probability setting part 303 Advertisement information delivery and CTR recording part 304 Delivery probability calculation and setting part 305 Advertisement DB

Claims (3)

１つの商材についての複数の広告コンテンツを管理する広告データベースから対象となる一群の広告コンテンツを選択する手段と、
選択された一群の広告コンテンツの個々に対して初回の配信確率を等分に設定する手段と、
配信確率が設定された広告コンテンツを配信し、ＣＴＲを記録する手段と、
広告コンテンツ毎の所定期間のＣＴＲを報酬として、当該報酬に基づき複数の広告コンテンツの中から一の広告コンテンツの配信確率を最大化しつつ、残りの広告コンテンツにも最小限の配信確率を残すように強化学習を行い、新たな配信確率を広告コンテンツ毎に算出して設定する手段と
を備えたことを特徴とする広告情報提供装置。 Means for selecting a target group of advertisement contents from an advertisement database that manages a plurality of advertisement contents for one product;
Means for equally setting the initial delivery probability for each of the selected group of advertising content;
Means for distributing advertising content with a distribution probability and recording CTR;
Using CTR for a predetermined period for each advertising content as a reward, maximizing the distribution probability of one advertising content among a plurality of advertising content based on the reward, and leaving the minimum distribution probability for the remaining advertising content An advertisement information providing apparatus comprising: means for performing reinforcement learning and calculating and setting a new distribution probability for each advertisement content. 広告情報提供装置の制御部が、１つの商材についての複数の広告コンテンツを管理する広告データベースから対象となる一群の広告コンテンツを選択する工程と、
前記制御部が、選択された一群の広告コンテンツの個々に対して初回の配信確率を等分に設定する工程と、
前記制御部が、配信確率が設定された広告コンテンツを配信し、ＣＴＲを記録する工程と、
前記制御部が、広告コンテンツ毎の所定期間のＣＴＲを報酬として、当該報酬に基づき複数の広告コンテンツの中から一の広告コンテンツの配信確率を最大化しつつ、残りの広告コンテンツにも最小限の配信確率を残すように強化学習を行い、新たな配信確率を広告コンテンツ毎に算出して設定する工程と
を備えたことを特徴とする広告情報提供方法。 A step of selecting a target group of advertisement contents from an advertisement database for managing a plurality of advertisement contents for one commercial product by a control unit of the advertisement information providing apparatus;
The control unit equally setting the initial delivery probability for each of the selected group of advertising content;
The controller distributes the advertising content with the distribution probability set and records the CTR;
The control unit uses CTR for a predetermined period for each advertising content as a reward, maximizes the distribution probability of one advertising content out of a plurality of advertising contents based on the reward, and minimizes the distribution to the remaining advertising content A method for providing advertisement information, comprising: performing reinforcement learning so as to leave a probability, and calculating and setting a new distribution probability for each advertisement content. 広告情報提供装置を構成するコンピュータを、
１つの商材についての複数の広告コンテンツを管理する広告データベースから対象となる一群の広告コンテンツを選択する手段、
選択された一群の広告コンテンツの個々に対して初回の配信確率を等分に設定する手段、
配信確率が設定された広告コンテンツを配信し、ＣＴＲを記録する手段、
広告コンテンツ毎の所定期間のＣＴＲを報酬として、当該報酬に基づき複数の広告コンテンツの中から一の広告コンテンツの配信確率を最大化しつつ、残りの広告コンテンツにも最小限の配信確率を残すように強化学習を行い、新たな配信確率を広告コンテンツ毎に算出して設定する手段
として機能させる広告情報提供プログラム。 A computer constituting the advertisement information providing apparatus;
Means for selecting a target group of advertisement contents from an advertisement database for managing a plurality of advertisement contents for one product;
Means for equally setting the initial delivery probability for each of a selected group of advertising content;
Means for distributing advertising content with a distribution probability and recording CTR;
Using CTR for a predetermined period for each advertising content as a reward, maximizing the distribution probability of one advertising content among a plurality of advertising content based on the reward, and leaving the minimum distribution probability for the remaining advertising content An advertisement information providing program that performs reinforcement learning and functions as a means for calculating and setting a new distribution probability for each advertisement content.
JP2010292483A 2010-12-28 2010-12-28 Advertisement information providing device Active JP5265659B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2010292483A JP5265659B2 (en) 2010-12-28 2010-12-28 Advertisement information providing device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2010292483A JP5265659B2 (en) 2010-12-28 2010-12-28 Advertisement information providing device

Publications (2)

Publication Number Publication Date
JP2012141683A true JP2012141683A (en) 2012-07-26
JP5265659B2 JP5265659B2 (en) 2013-08-14

Family

ID=46677943

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2010292483A Active JP5265659B2 (en) 2010-12-28 2010-12-28 Advertisement information providing device

Country Status (1)

Country Link
JP (1) JP5265659B2 (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014174700A (en) * 2013-03-07 2014-09-22 Konami Digital Entertainment Co Ltd Advertisement page providing apparatus, advertisement page providing method, and advertisement page providing program
JP2014174699A (en) * 2013-03-07 2014-09-22 Konami Digital Entertainment Co Ltd Advertisement page providing apparatus, advertisement page providing method, and advertisement page providing program
JP2015148689A (en) * 2014-02-05 2015-08-20 日本電信電話株式会社 Advertising display control method, advertising display control device, and program
JP2016122241A (en) * 2014-12-24 2016-07-07 株式会社Ｎｔｔドコモ Advertisement selection device, advertisement selection method, and program
JP2019057105A (en) * 2017-09-20 2019-04-11 ヤフー株式会社 Content distribution management device, content distribution management method, and content distribution management program
JP2020047066A (en) * 2018-09-20 2020-03-26 Ｚホールディングス株式会社 Determination device, determination method, and determination program
CN111445289A (en) * 2020-03-31 2020-07-24 深圳前海微众银行股份有限公司 Resource delivery method, device, equipment and storage medium
JPWO2021070229A1 (en) * 2019-10-07 2021-04-15

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10312344A (en) * 1997-04-02 1998-11-24 Lucent Technol Inc System for scheduling and controlling distribution of advertisement over communication network
JP2006120135A (en) * 2004-09-27 2006-05-11 Yafoo Japan Corp Program, method, and system for calculating advertisement content distribution ratio, content distribution control system, and system, method, and program for controlling advertisement content distribution
WO2009099879A1 (en) * 2008-02-01 2009-08-13 Qualcomm Incorporated Multiple actions and icons for mobile advertising
JP2009271661A (en) * 2008-05-02 2009-11-19 Kenji Sudo Creative content optimization server, creative content optimization system, creative content optimization method and program

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10312344A (en) * 1997-04-02 1998-11-24 Lucent Technol Inc System for scheduling and controlling distribution of advertisement over communication network
JP2006120135A (en) * 2004-09-27 2006-05-11 Yafoo Japan Corp Program, method, and system for calculating advertisement content distribution ratio, content distribution control system, and system, method, and program for controlling advertisement content distribution
WO2009099879A1 (en) * 2008-02-01 2009-08-13 Qualcomm Incorporated Multiple actions and icons for mobile advertising
JP2009271661A (en) * 2008-05-02 2009-11-19 Kenji Sudo Creative content optimization server, creative content optimization system, creative content optimization method and program

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
CSNB201000401001; 安間　文彦: 人工知能と教育工学 第１版, 20080220, ｐ．２１１-ｐ．２１７, 株式会社オーム社 *
JPN6012068680; 安間　文彦: 人工知能と教育工学 第１版, 20080220, ｐ．２１１-ｐ．２１７, 株式会社オーム社 *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014174700A (en) * 2013-03-07 2014-09-22 Konami Digital Entertainment Co Ltd Advertisement page providing apparatus, advertisement page providing method, and advertisement page providing program
JP2014174699A (en) * 2013-03-07 2014-09-22 Konami Digital Entertainment Co Ltd Advertisement page providing apparatus, advertisement page providing method, and advertisement page providing program
JP2015148689A (en) * 2014-02-05 2015-08-20 日本電信電話株式会社 Advertising display control method, advertising display control device, and program
JP2016122241A (en) * 2014-12-24 2016-07-07 株式会社Ｎｔｔドコモ Advertisement selection device, advertisement selection method, and program
JP2019057105A (en) * 2017-09-20 2019-04-11 ヤフー株式会社 Content distribution management device, content distribution management method, and content distribution management program
JP2020047066A (en) * 2018-09-20 2020-03-26 Ｚホールディングス株式会社 Determination device, determination method, and determination program
JP7174580B2 (en) 2018-09-20 2022-11-17 ヤフー株式会社 Decision device, decision method and decision program
JP7443432B2 (en) 2018-09-20 2024-03-05 Ｌｉｎｅヤフー株式会社 Determination device, determination method and determination program
JPWO2021070229A1 (en) * 2019-10-07 2021-04-15
JP7290170B2 (en) 2019-10-07 2023-06-13 日本電気株式会社 Optimization device, optimization method and optimization program
US11949809B2 (en) 2019-10-07 2024-04-02 Nec Corporation Optimization apparatus, optimization method, and non-transitory computer-readable medium in which optimization program is stored
CN111445289A (en) * 2020-03-31 2020-07-24 深圳前海微众银行股份有限公司 Resource delivery method, device, equipment and storage medium

Also Published As

Publication number Publication date
JP5265659B2 (en) 2013-08-14

Similar Documents

Publication Publication Date Title
JP5265659B2 (en) Advertisement information providing device
US9396444B2 (en) Predictive analytics with forecasting model selection
CN105653545B (en) Method and device for providing service object information in page
US9083562B2 (en) Predictive analysis of network analytics
CN102150175A (en) System and method for word-of-mouth advertising
US10007645B2 (en) Modifying the presentation of a content item
US20170017986A1 (en) Tracking digital design asset usage and performance
JP2014512613A (en) Aggregation of conversion paths using user interaction grouping
CN103329151A (en) Recommendations based on topic clusters
EP3126940A1 (en) Systems and methods for optimizing content layout using behavior metrics
JP2009157907A (en) Information processing device and method, and program
CN103797474A (en) Multi-channel conversion path position reporting
CN103608834A (en) Priority dimensional data conversion path reporting
EP2339526A1 (en) System and method for monitoring visits to a target site
JP5661689B2 (en) Content distribution device
JP2012141682A (en) Advertisement information providing device
JP5905551B1 (en) Information processing device, terminal device, information processing method, and information processing program
US20140289036A1 (en) Marketing Prediction, Analysis, and Optimization
JP2014002620A (en) Information value evaluation device, information value evaluation method and information value evaluation program
US9460163B1 (en) Configurable extractions in social media
CN110674404A (en) Link information generation method, device, system, storage medium and electronic equipment
JP2005234689A (en) Community evaluation support method and apparatus
US9430779B1 (en) Determining visual attributes of content items
JP2016119123A (en) Information processing device, terminal device, information processing method, and information processing program
US11373210B2 (en) Content interest from interaction information

Legal Events

Date Code Title Description
A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20121219

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20130108

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20130311

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20130402

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20130501

R150 Certificate of patent or registration of utility model

Ref document number: 5265659

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

S531 Written request for registration of change of domicile

Free format text: JAPANESE INTERMEDIATE CODE: R313531

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

S533 Written request for registration of change of name

Free format text: JAPANESE INTERMEDIATE CODE: R313533

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313111

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313111

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250