JP6355582B2 - Evaluation value calculation method, evaluation value calculation program, and evaluation value calculation device - Google Patents

Description

  The present invention relates to a technique for calculating an evaluation value of content posted on the Internet such as an online advertisement.

  Click through rate (CTR) and conversion rate are used as indexes for evaluating contents such as advertisements and hyperlinks (link information to other files) posted on the web. The click rate is represented by a value obtained by dividing the number of clicks by the number of impressions. The number of impressions is the number of times the advertisement or hyperlink is displayed. If the number of times of display is large, there will be more opportunities for Internet users to see content such as advertisements. However, since the number of times users actually click (number of clicks) varies depending on the content, , It is an index indicating whether or not the content attracted the user's interest. The conversion rate is an index indicating whether or not the user has actually purchased a product or content, and is represented by a value obtained by dividing the number of conversions (contracts) by the number of clicks. For post-pay type content where revenue is determined based on click rate and conversion rate, click rate and conversion rate must be calculated correctly.

  CTR is not reliable when the number of impressions is small. The greater the number of impressions, the higher the reliability of the evaluation result. However, if a sufficient number of impressions are collected, the advertisement period may be missed. In view of such a state, a method for evaluating the effectiveness of content such as an online advertisement with a small number of impressions has been proposed (see, for example, Patent Document 2).

JP 2009-271661 A

  However, the prior art method is complicated, and a simpler and more accurate method is desired. Therefore, it is an object of the present invention to provide an evaluation value calculation technique and configuration capable of appropriately calculating an evaluation value even when a reference value such as the number of impressions is insufficient.

  Taking online advertising as an example, there are several hierarchical steps between the classification of “advertising” and “advertiser (account)”. For example, in the stage of advertisement <advertisement group <campaign <advertiser, the number of turns increases. The present invention uses this hierarchical structure for calculating the evaluation value. When the reference value such as the number of impressions is insufficient in the subdivided hierarchy, the evaluation value is calculated with a finer granularity by using the number of impressions in the hierarchy with a coarse granularity (that is, the degree of subdivision).

Specifically, the evaluation value calculation method according to one aspect of the present invention is performed by a processor.
Collect data that includes criteria used to evaluate hierarchically organized content,
Obtaining a first reference value of the content from data of the first layer of the data;
Determining whether the first reference value satisfies a predetermined condition;
When the first reference value does not satisfy the predetermined condition, the second reference value of the content included in the data of the second layer, which is coarser than the first layer, or the same group as the content The evaluation value of the content is calculated using the reference value in the first layer of the other content included.

  By the above method, even when the reference value necessary for calculating the evaluation value of the content is insufficient, the evaluation value can be calculated appropriately.

It is a figure which shows an example of a hierarchical classification. 3 is a flowchart of an evaluation value calculation method according to the first embodiment. 10 is a flowchart of an evaluation value calculation method according to the second embodiment. 10 is a flowchart of an evaluation value calculation method according to the third embodiment. It is a figure which shows the structural example of an evaluation value calculation apparatus. It is a figure which shows the hardware constitutions of the general purpose computer which implement | achieves the apparatus of FIG.

  FIG. 1 shows an example of a hierarchical structure when content such as an online advertisement is posted. The left column of the table shows the hierarchy, the middle column shows the number of clicks, and the right column shows the number of impressions. In this hierarchical structure, the framework increases in the order of advertisement (ad), advertisement group, campaign, and advertiser, and the hierarchy becomes higher. “Advertiser” is represented by an account of “IO (Insertion Order)”. Depending on the company, different accounts may be set for each business division, but here, for convenience of explanation, it is assumed that the advertiser and the account have a one-to-one correspondence.

  “Campaign” is a series of advertising activities over a period of time, targeting age, gender, distribution time, etc., destination device (whether it is a wireless terminal such as a smartphone or a cabled terminal), and distribution method The arrangement on the web page is set. For example, if the cosmetic manufacturer is an advertiser, the campaign can be divided into skin care (basic) cosmetics delivered by interest match, beauty supplements targeting middle-aged and older women, and cosmetics for new products posted on the top page. The same product may be classified into multiple campaigns.

  An “ad group” is a group of advertisements grouped together with the same or common theme or concept within a campaign. For example, a campaign for skin care cosmetics distributed by matching based on the user's interest level may include a group of “moisturizing x face wash”, a group of “no additive x lotion”, a group of “whitening x lotion”, etc. it can. One advertisement group includes one or more advertisements, and each advertisement includes a title and a description.

  Generally, the number of impressions is small immediately after the advertisement is displayed. In the example of FIG. 1, the number of impressions of an advertisement of interest in a certain source is 50, and the number of clicks is 1. Moving to the ad group hierarchy, the number of impressions is 5500 and the number of clicks is 30. In the campaign hierarchy, the number of impressions is 18000 and the number of clicks is 35. At the advertiser (account) level, the number of impressions is 40000 and the number of clicks is 65.

  It is assumed that the advertisement of interest is an advertisement titled “Excellent moisturizing effect after face washing!” Made from the viewpoint of “moisturizing × face washing”. There is not enough impressions to evaluate the value of this ad, and the correct CTR cannot be calculated. This is because a very high click rate occurs when a user clicks immediately after the advertisement is displayed. In the example of FIG. 1, the click rate is a high number of 0.02 (2%).

  Therefore, in the embodiment, if the number of impressions is insufficient in a fine-grained hierarchy for content that is subdivided in stages, the reliability of evaluation is improved by using the number of impressions in the upper hierarchy. For example, when evaluating an advertisement, if the number of impressions of the noticed advertisement itself is insufficient, the number of impressions of the advertisement group which is the upper hierarchy is used. This is because the same advertisement group often includes advertisements for products and services on the same or common theme, and it is considered that the evaluation of the noticed advertisement itself is well reflected. Thereby, even when the number of impressions of the target content does not reach the number necessary for the evaluation, the reliability of the evaluation can be ensured.

  FIG. 2 is a flowchart of the evaluation value calculation method according to the first embodiment. In Example 1, if there is a sufficient track record in the “src.ad” hierarchy, which is the finest granularity, use it, and if there are not enough track records, change to the “src.ad” hierarchy, For example, “sec.adg” above “src.ad”. The calculation method of this evaluation value is represented by the formula (1).

広告レベルのインプレッション数Imp_src.adが不足する場合は、広告グループレベルでのインプレッション数とクリック数を用いて、クリック率（ＣＴＲ）をClk_src.adg／Imp_src.adgとして計算する。広告グループレベルでのインプレッション数が広告グループレベルでの評価に必要な数に足りない場合は、さらに上のキャンペーンレベルでのインプレッション数とクリック数を用いてClk_src.camp／Imp_src.campを計算する。

When the impression level Imp _src.ad is insufficient, the click rate (CTR) is calculated as Clk _src.adg / Imp _src.adg using the number of impressions and the number of clicks at the advertisement group level. If the number of impressions at the ad group level is not enough to evaluate at the ad group level, the number of impressions and clicks at the higher campaign level is used to calculate Clk _src.camp / Imp _src.camp To do.

  Since thresholds suitable for evaluation in the hierarchy differ depending on the target hierarchy, a plurality of thresholds are set in advance. The threshold value may vary depending on the size of the advertiser, the type of product or service handled, and the like.

Referring to the flow of FIG. 2, first, the number of impressions A ₁ and the number of clicks B ₁ of the content with the finest granularity are acquired (S11), and it is determined whether or not A ₁ reaches the threshold value 1 (S12). ). If the number of impressions A ₁ has reached the threshold value 1 (YES in S12), the content evaluation value is calculated as B ₁ / A ₁ (S13). When the number of impressions A ₁ is less than the threshold value 1 (NO in S12), the number of impressions A ₂ and the number of clicks B ₂ of the items in the hierarchy 2 including the content are acquired (S14). It determines whether impressions A ₂ of the hierarchy 2 reaches the threshold 2 (S15), when A ₂ has reached the threshold value 2 (YES in S15), calculates an evaluation value as B ₂ / A ₂ (S16). When the number of impressions A ₂ is less than the threshold value 2 (NO in S15), the number of impressions A ₃ and the number of clicks B ₃ of the items in the hierarchy 3 including the content are acquired (S17). Determines whether impressions A ₃ of the hierarchy 3 reaches the threshold value 3 (S18), when A ₃ has reached the threshold value 3 calculates an evaluation value as B ₃ / A ₃ (YES in S18) (S19). If the impression number A ₃ is less than the threshold value 3 (NO in S18), the evaluation value Clk _src / Imp _src using the impression number and click number of the source is calculated as a default (S20). An additional hierarchy determination may be inserted between the determination of the number of impressions in hierarchy 3 (S18) and the default value calculation.

For example, if the number of impressions at the advertiser level in a certain source is less than the corresponding threshold as shown in Equation (2), the impression number Imp _ad and the click number Clk _ad are not limited to a specific source. The evaluation value may be calculated using.

さらに、特定のソースに限定しない広告グループ、キャンペーン、広告主、と順次階層を上げていってもよい。

Further, the hierarchies may be raised sequentially with an advertisement group, a campaign, and an advertiser that are not limited to a specific source.

  In this method, even when the targeted advertisement itself has a small number of impressions, it is possible to appropriately evaluate the value of the advertisement by using the evaluation value of the nearest hierarchy that satisfies the number of impressions necessary for the evaluation.

  In Example 1, when the number of impressions of an advertisement of interest is insufficient, the click rate of the advertisement group including the advertisement is used instead of the click rate of the advertisement. In this case, since separate threshold values suitable for evaluation are used for each hierarchy, setting of threshold values may be complicated.

  Therefore, in the second embodiment, an evaluation value is calculated by using only one threshold and borrowing only the number of impressions that are insufficient from the upper hierarchy. This method is represented by Formula (3).

式３の意味は、あるソースで着目する広告のインプレッション数が閾値に達する場合は（Imp_src.ad≧Threshold）、その広告のインプレッション数とクリック数を用いてクリック率をClk_src.ad／Imp_src.adで計算する。広告のインプレッション数が閾値に満たない場合でも広告のインプレッション数と広告グループのインプレッション数の合計が閾値に達する場合は、不足分だけを上の階層の広告グループのインプレッション数から補う。このときの評価値は、
［Clk_src.ad＋(Threshold−Imp_src.ad)・CTR_src.adg］／Threshold
となる。

The meaning of Equation 3 is that when the number of impressions of an advertisement of _interest at a certain source reaches a threshold (Imp _src.ad ≧ Threshold), the click rate is calculated using the number of impressions and the number of clicks of that advertisement as Clk _src.ad / Imp Calculate with _src.ad. If the total number of ad impressions and ad group impressions reaches the threshold even if the number of ad impressions is less than the threshold, only the shortage is compensated from the number of impressions in the upper-level ad group. The evaluation value at this time is
[Clk _src.ad + (Threshold-Imp _src.ad ) / CTR _src.adg ] / Threshold
It becomes.

Even if the total number of impressions at the ad level and ad group level is less than the threshold, if the total number of impressions for the ad, ad group, and campaign reaches the threshold, the shortage of impressions It supplements sequentially from the number. The evaluation value at this time is
[Clk _src.ad + Clk _src.adg + (Threshold-Imp _src.ad -CImp _src.adg ) / CTR _src.camp ] / Threshold
It becomes.

  With reference to the example of FIG. 1, the number of impressions necessary for evaluating the advertisement of interest is set to 20000. Since the number of impressions of the advertisement is insufficient, the evaluation value is calculated using the number of impressions 5500 of the advertisement group and the number of impressions 14450 of the campaign. The click rate at this time is 0.00245 (0.245%). When the number of clicks and the number of impressions only for the advertisement are used, the evaluation becomes excessive. However, the value of the advertisement is appropriately evaluated by making up the shortage from the upper hierarchy and setting the number of impressions suitable for evaluation. Further, if all the shortage is compensated by the number of impressions of the advertiser, the evaluation of the advertisement itself is not sufficiently reflected and it tends to be underestimated, but this problem can also be avoided.

FIG. 3 is a flowchart of the evaluation value calculation method according to the second embodiment. First, to get the impressions A ₁ and clicks B ₁ content hierarchy 1 (S21), it is determined whether A ₁ reaches the threshold Th (S22). If the number of impressions A _{1 of the} content has reached the threshold Th (YES in S22), the evaluation value is calculated as B ₁ / A ₁ (S23). When the number of impressions A ₁ is less than the threshold value Th (NO in S22), the number of impressions A ₂ and the number of clicks B ₂ of the item in the hierarchy 2 including the content are acquired (S24), and A ₁ and A ₂ It is determined whether or not the total reaches the threshold value Th (S25). If the sum of A ₁ and A ₂ has reached the threshold value Th (YES in S25), the evaluation value is calculated as [B ₁ + (Th−A ₁ ) B ₂ / A ₂ ] / Th (S26). Since B ₂ / A ₂ in this evaluation value is the click rate of the item of level 2, the evaluation value is expressed as [B ₁ + (Th−A ₁ ) * CTR ₂ ] / Th as shown in Expression (3). be able to.

When the sum of A ₁ and A ₂ is less than the threshold value Th (NO in S25), the impression number A ₃ and the click number B ₃ of the item in the hierarchy 3 including the content of interest are acquired (S27), and A ₁ It is determined whether or not the sum of A ₂ and A ₃ reaches the threshold value Th (S28). If the sum of A ₁ , A ₂ and A ₃ has reached the threshold value Th (YES in S28), the evaluation value is [B ₁ + B ₂ + (Th−A ₁ −A ₂ ) B ₃ / A ₃ ] / Calculate as Th (S29). Since B ₃ / A ₃ in this evaluation value is the click rate of the item in the hierarchy 3, the evaluation value is expressed as [B ₁ + B ₂ + (Th−A ₁ −A ₂ ) * CTR as shown in Expression (3). ₃ ] / Th. When the total number of impressions A ₁ + A ₂ + A ₃ is less than the threshold Th (NO in S28), the evaluation value Clk _src / Imp _src using the number of impressions and the number of clicks at that source is calculated as the default. (S30). Between the threshold determination (S28) including the number of impressions of the hierarchy 3 and the default value calculation, the shortage may be supplemented from the determination of an additional hierarchy, for example, the number of impressions of advertisements not limited to a specific source.

  In the method of the second embodiment, only the deficiency is supplemented in order from the coarser level (for example, the level immediately above) with the noticed advertisement as a main subject. The click rate of the advertisement itself to be evaluated is appropriately reflected, and the reliability of the evaluation can be maintained even when the number of impressions is small. Further, since one threshold value is used for one advertisement evaluation, it is possible to reduce the complexity of setting the threshold value.

  In Example 2, when the number of impressions of an advertisement in a certain source is insufficient, the shortage is compensated from the number of impressions in a coarser level. In the third embodiment, when the number of advertisement impressions in a certain source is insufficient, the number of impressions of other advertisements in the same advertisement group is supplemented. In many cases, advertisements created with the same or common theme or concept are included in the same advertisement group, and it is expected to achieve the same or similar advertisement effect. The reliability of evaluation is maintained even when the number of impressions is small by compensating the shortage with the number of impressions of another advertisement in the same advertisement group with the number of impressions of the advertisement of interest as a subject.

FIG. 4 is a flowchart of the evaluation value calculation method according to the third embodiment. First, the number of impressions A _1a and the number of clicks B _1a of the content with the finest granularity are acquired (S31), and it is determined whether or not the number of impressions A _1a has reached the threshold value Th (S32). When the number of impressions A _{1a of the} content of interest has reached the threshold Th (YES in S32), the evaluation value of the content is calculated as B _1a / A _1a (S33). When the number of impressions A _1a is less than the threshold value Th (NO in S32), it is determined whether there is another content in the same group as the content of interest (S34). If there is other content in the same group, the number of impressions A _1b and the number of clicks B _1b of the other content are acquired (S35), and it is determined whether or not the sum of A _1a and A _1b reaches the threshold value Th ( S36). If the sum of A _1a and A _1b has reached the threshold value Th (YES in S36), the evaluation value is calculated as [B _1a + (Th−A _1a ) B _1b / A _1b ] / Th (S27). Since B _1b / A _1b in this evaluation value is a click rate of another content included in the same group as the content of interest in layer 1, the evaluation value is [B ₁ + (Th−A ₁ ) * CTR _1b ]. / Th.

When the sum of A _1a and A _1b does not reach the threshold value Th (NO in S36), the process returns to step S34, and S34 to S36 are repeated until the threshold value Th is reached. In this case, the number of impressions of other contents in the same group is sequentially added to the left side of the inequality in S36. The (Th-A _1a ) * CTR _1b term in the evaluation value of S37 is also updated as (Th-A _1a -A _1b ) * CTR _1c .

If the threshold value Th is not reached even if S34 to S36 are repeated for the content in the same group (NO in S36 and S34), the number of impressions A ₂ and the number of clicks B ₂ of the items in the hierarchy 2 including the content of interest are calculated. Obtain (S38). It is determined whether or not the total of the total number of impressions A _1total acquired at level 1 and the number of impressions A ₂ at level 2 reaches the threshold value Th (S39). If the sum of the impression numbers A _1total and A ₂ has reached the threshold value Th (YES in S39), the evaluation value is _calculated as [B _1total + (Th−A _1total ) B ₂ / A ₂ ] / Th (S40). . Since B ₂ / A ₂ in this evaluation value is the click rate of layer 2, the evaluation value can be expressed as [B _1total + (Th−A _1total ) * CTR ₂ ] / Th. When the total number of impressions A _1total + A ₂ is less than the threshold value Th (NO in S39), the evaluation value Clk _src / Imp _src using the number of impressions and the number of clicks at the source is calculated as a default (S41). ). Between the threshold determination (S39) including the number of impressions of level 2 and the default value calculation, an additional level may be determined, for example, the shortage may be compensated from the number of impressions of advertisements not limited to a specific source.

  As a modified example of the third embodiment, similarities between advertisements in the same group are determined in advance, and when there are not enough impressions of the advertisement of interest, the shortage is compensated in order from the number of impressions of the advertisement with high similarity. It is possible to continue. According to this method, the value of the noticed advertisement can be evaluated with higher reliability. Further, an advertisement having a level of similarity below a certain level may not be used for replenishment of the number of impressions, but may be shifted to use of the number of impressions in the upper hierarchy.

In the embodiment described above, the click rate is used as the evaluation value, but the methods of the first to third embodiments can also be used when the conversion rate of the advertisement is used as the evaluation value. In this case, a threshold is set for the number of clicks as the denominator, and when the number of clicks is less than the threshold, the number of clicks in the upper hierarchy is used. As in the modification of the third embodiment, the click numbers of some or all advertisements in the same group may be used before the click number of the upper hierarchy is used.
<Evaluation value calculation device>
FIG. 5 is a schematic diagram of the evaluation value calculation device 50 of the embodiment. The calculation of the content evaluation value described above is automatically performed by the evaluation value calculation device 50. The evaluation value calculation device 50 includes a data input unit 57, a processing unit 51, a threshold database (DB) 58, and an evaluation value output unit 59. The processing unit 51 includes a statistical processing unit 52, a hierarchical data acquisition unit 53, a threshold setting unit 54, and an evaluation value calculation unit 55.

  The data input unit 57 inputs impression values and click counter values of contents such as advertisements posted on the Internet. The data input unit 57 itself may have a counter function, or an input value from an external counter may be used. An impression is counted when an advertisement or the like is distributed to an end user. When an end user accesses a certain source, the user's identification information is read, and an advertisement to be placed in the source is determined. When a page containing an advertisement is sent to the user, an impression is counted once. The number of clicks may be counted using a redirect page, or a counter value by the advertiser may be input. Also, the number of conversions when the end user finally purchases a product or service is collected.

  The statistical processing unit 52 classifies the information collected by the data input unit 57 into hierarchies on an advertiser (account) basis and sequentially updates the information. For example, when the count information of the advertisement X is input, the counter value of the advertisement group, the campaign, and the advertiser including the advertisement is updated in addition to the counter value of the advertisement itself.

  The hierarchical data acquisition unit 53 acquires information necessary for calculating the evaluation value from the statistical processing unit 52. For example, when evaluating the value of the advertisement X, the number of impressions, the number of clicks, the number of conversions, etc. in each hierarchy of the advertisement X are acquired.

  The threshold setting unit 54 refers to the threshold database 58 and sets a threshold used for evaluation value calculation. The threshold database 58 stores thresholds used by the processing unit 51. In the case of the first embodiment, a plurality of threshold values used in each hierarchy are stored in association with one advertisement. In the case of Examples 2 and 3, since one threshold value is used for evaluating one advertisement, the advertisement and a single threshold value are stored in association with each other.

  The evaluation value calculation unit 55 calculates the evaluation value of content such as an advertisement using the data value acquired by the hierarchical data acquisition unit 53 and the threshold set by the threshold setting unit 54. Further, it is determined using a threshold value whether or not the number that serves as a reference for evaluation value calculation (the number of impressions in the click rate, the number of clicks in the conversion rate, etc.) is an appropriate number. The evaluation value output unit 59 outputs the calculated evaluation value.

  The evaluation value calculation device 50 of FIG. 5 may be realized by the general-purpose computer of FIG. 6 using an evaluation value calculation program. FIG. 6 shows the hardware configuration of the general-purpose computer 60. In the general-purpose computer 60, a CPU (Central Processing Unit) 61, a ROM (Read Only Memory) 62, a RAM (Random Access Memory) 63, and an input / output interface 65 are interconnected by a bus 64. An input unit 66, an output unit 67, a storage unit 68, and a drive 71 are connected to the input / output interface 65. A removable medium 81 may be connected via the drive 71.

  The CPU 61 executes a series of processing procedures according to a program recorded in the storage unit 68 such as the ROM 62 or a hard disk drive. The RAM 83 appropriately stores data of processing steps and processing results executed by the CPU 61. Programs and data used for processing may be acquired from the outside via the communication unit 69. The drive 71 drives them when a removable medium 81 including various recording media (such as an optical disk) is inserted. The input unit 66 includes a keyboard, a mouse, a touch panel screen, and the like. The output unit 67 includes a display, a speaker, and the like.

  The computer recorded in the ROM 62 or the storage unit 68 causes the CPU 61 to execute, for example, the following procedure.

A procedure to collect data, including criteria (such as impressions) used to evaluate hierarchically structured content (such as online advertising)
A procedure for obtaining a first reference value of the content from data of the first layer (for example, “advertising” level) of the collected data;
Determining whether the first reference value satisfies a threshold;
If the first reference value does not satisfy the threshold, the second reference value of the content included in the data of the second layer (for example, the “ad group” level), which is coarser than the first layer, or the A procedure for calculating an evaluation value of content to be evaluated using a reference value in the first layer of other content included in the same group as the content.

  As a result, even when a sufficient reference value cannot be obtained for the content to be evaluated, the value of the content can be appropriately evaluated using data of a higher level of coarseness.

  In the embodiment, an online advertisement is taken as an example of content to be evaluated, but the present invention can be used for evaluation of arbitrary content classified hierarchically. For example, it can be used to evaluate online news articles and online shopping products. In the case of articles, categories of articles (sports, economy, society, art) are classified for each newspaper company. Each category is further divided into several groups (soccer, baseball, tennis, etc. for sports, and paintings, movies, books, etc. for art), and the click rate for the article titles within each group is evaluated. Can be a value. When the number of impressions (display count) of an article title at a certain source is less than the threshold value, the number is used in order from the number of impressions of the upper layer to which the article belongs. This method is particularly useful for evaluation of post-charging type content whose charging amount is determined by the evaluation value.

  In the embodiment, advertisement evaluation values such as a click rate and a conversion rate are used as evaluation values. However, a user's stay time or bounce rate for a certain content may be used as the evaluation value of the content. The staying time indicates how long the user has stayed for a certain content once per display, and is expressed as an average of staying times of a plurality of users. The bounce rate is the ratio of leaving the site without visiting another page from the first page (landing page). For example, if 10 users access a page first, 8 people leave the site without looking at the other page, and 2 people move from the page to another related page, the bounce rate is 80% It is. The staying time and bounce rate are indicators of how much the content attracted the user. In these examples, the number of users becomes a reference value, and the reliability of evaluation increases as the number of users increases. When the number of users is insufficient, the reliability of evaluation can be improved by supplementing the number of users from the upper layer (the layer with coarser granularity) to which the page or content belongs. In addition to the number of impressions, the number of clicks, and the number of users, the number of arbitrary logs accompanying advertisement distribution may be used as the reference value.

50 Evaluation Value Calculation Device 51 Processing Unit 52 Statistical Processing Unit 53 Hierarchical Data Acquisition Unit 54 Threshold Setting Unit 55 Evaluation Value Calculation Unit 57 Data Input Unit 58 Threshold Database 59 Evaluation Value Output Unit

Claims (9)

In the processor
Collect data that includes criteria used to evaluate hierarchically organized content,
Obtaining a first reference value of the content from data of the first layer of the data;
Determining whether the first reference value satisfies a predetermined condition;
When the first reference value does not satisfy the predetermined condition, the second reference value of the content included in the data of the second layer, which is coarser than the first layer, or the same group as the content Calculating an evaluation value of the content using a reference value in the first layer of the other content included ;
The predetermined condition is to reach a predetermined threshold, and the threshold is referred to a threshold database in which the threshold and the content hierarchy are associated with each other, or the threshold and the content are associated with each other. Set,
When the first reference value is less than the threshold, an insufficient value for the threshold is supplemented from the second reference value or the reference value of the other content .   The evaluation value of the content is calculated using the second reference value instead of the first reference value when the first reference value does not satisfy the predetermined condition. 2. The evaluation value calculation method according to 1.   2. The evaluation value of the content is calculated using the first reference value and the second reference value when the first reference value does not satisfy the predetermined condition. The evaluation value calculation method described in 1. When the first reference value does not satisfy the predetermined condition, the reference value of the other content is acquired,
When the sum of the first reference value and the reference value of the other content satisfies the predetermined condition, the evaluation value of the content is calculated using the first reference value and the reference value of the other content. The evaluation value calculation method according to claim 1, wherein calculation is performed. If the sum of the reference value of the other contents and said first reference value does not reach the threshold value, wherein the shortage with respect to the threshold to claim 4, wherein the replenishing from the second reference value Evaluation value calculation method. The evaluation value is an advertisement evaluation value, the evaluation value calculation method according to any one of claims 1 to 5, featuring that said reference value is a log associated with ad serving. The evaluation value, CTR, conversion rate, residence time, comprising at least one bounce rate, impressions for the reference value calculation of the CTR, Clicks for calculating the conversion rate, The evaluation value calculation method according to claim 6 , comprising at least one of the number of users who have accessed the content. A program for causing a computer to execute the following procedure,
A procedure to collect data including reference values used to evaluate hierarchically organized content;
Obtaining a first reference value of the content from data of the first layer of the data;
Determining whether the first reference value satisfies a predetermined condition;
When the first reference value does not satisfy the predetermined condition, the second reference value of the content included in the data of the second layer, which is coarser than the first layer, or the same group as the content A procedure for calculating an evaluation value of the content using a reference value in the first layer of other content included;
Was executed,
The predetermined condition is to reach a predetermined threshold, and the threshold is referred to a threshold database in which the threshold and the content hierarchy are associated with each other, or the threshold and the content are associated with each other. Set,
When said first reference value is less than the threshold value, the evaluation value calculation program the shortfall you characterized by supplementing the reference value of the second reference value or the other content to the threshold. An input unit for inputting data including a reference value used for evaluation of contents organized hierarchically;
An acquisition unit that acquires a first reference value of the content from data of a first layer of the data;
It is determined whether or not the first reference value satisfies a predetermined condition, and when the first reference value does not satisfy the predetermined condition, the data of the second layer having a coarser grain than the first layer A calculation unit that calculates an evaluation value of the content using a second reference value of the content included in the content, or a reference value in the first layer of other content included in the same group as the content;
A predetermined threshold value have a <br/> a threshold database that stores in association with the hierarchy of the content or the content,
The predetermined condition is to reach the predetermined threshold set with reference to the threshold database;
When the first reference value is less than the threshold value, the calculation unit supplements the shortage with respect to the threshold value from the second reference value or the reference value of the other content. Calculation device.

Images