JP6463400B2 - Advertiser evaluation apparatus, advertiser evaluation method, and advertiser evaluation program - Google Patents

Description

  The present invention relates to an advertiser evaluation device, an advertiser evaluation method, and an advertiser evaluation program.

  In recent years, with the rapid spread of networks such as the Internet, advertisement distribution via networks has been actively performed. In performing such advertisement distribution, for example, an advertisement distributor accepts submission of an advertisement to be posted on a web page or the like from an advertiser, and examines whether or not the advertisement is appropriate for distribution.

  As a screening method, for example, there is a technology that analyzes the content of a web page where an advertisement is posted and determines whether there is a violation in light of violation information in which inappropriate text or images are registered in advance. It is known (see, for example, Patent Document 1).

JP 2002-189925 A

  However, the above-mentioned conventional technology only confirms whether or not the web page on which the advertisement is displayed is appropriate, and it is determined whether or not the advertiser of the advertisement is a malicious and malicious advertiser. It is not something to judge.

  The present application has been made in view of the above, and an object thereof is to provide an advertiser evaluation device, an advertiser evaluation method, and an advertiser evaluation program that can determine an unauthorized advertiser.

Advertisers evaluation apparatus according to the present application includes a receiving unit, and the evaluation unit, and a determination unit. The receiving unit receives a submission of advertisement information from an advertiser. The evaluation unit includes elements of the advertisement information submission pattern by the advertiser and the advertiser's behavior pattern after the advertisement information is included in the information related to the advertiser who submitted the advertisement information. Based on an evaluation model having at least an element indicating the parameter , an evaluation score indicating the degree to which the advertiser is a malicious fraudster is calculated . The determination unit determines that the advertiser is the unauthorized person when the evaluation score is within a predetermined range.

  According to one aspect of the embodiment, an advertiser evaluation device, an advertiser evaluation method, and an advertiser evaluation program that can determine an unauthorized advertiser can be provided.

FIG. 1 is an explanatory diagram of an advertisement examination system and an advertiser evaluation process according to the embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of the advertiser evaluation device according to the embodiment. FIG. 3A is a diagram illustrating an example of a submission information table. FIG. 3B is a diagram illustrating an example of the advertiser attribute information table. FIG. 3C is a diagram illustrating an example of an advertiser behavior information table. FIG. 3D is a diagram illustrating an example of an examination history information table. FIG. 4 is a diagram illustrating a specific example of a feature when an illegal advertiser is a positive example. FIG. 5 is a flowchart illustrating a processing procedure of an advertiser evaluation process executed by the advertiser evaluation apparatus according to the embodiment. FIG. 6 is a hardware configuration diagram illustrating an example of a computer that realizes the function of the advertiser evaluation apparatus.

  Hereinafter, embodiments of an advertiser evaluation apparatus, an advertiser evaluation method, and an advertiser evaluation program according to the present application will be described in detail with reference to the drawings. In the following, “database” is described as “DB”.

  Further, in the following, description will be made by taking as an example a case where the advertiser evaluation device is included in an advertisement examination system that examines advertisements.

[1. Ad review system and advertiser evaluation process)
First, the advertisement examination system and the advertiser evaluation process according to the embodiment will be described. FIG. 1 is an explanatory diagram of an advertisement examination system and an advertiser evaluation process according to the embodiment.

[1.1. Ad review system)
As shown in FIG. 1, the advertisement examination system 1 includes advertiser terminals 2-1 and 2-2 and an advertiser evaluation device 3. Advertiser terminals 2-1 and 2-2 (hereinafter may be collectively referred to as advertiser terminal 2) are managed and operated by advertisers SA and SB (hereinafter may be collectively referred to as advertiser S), respectively. The Further, the advertiser evaluation device 3 is managed and operated by an advertisement distributor.

  The advertiser terminal 2 and the advertiser evaluation device 3 are connected to each other via a communication network 4 so that they can communicate with each other. The communication network 4 is a WAN (Wide Area Network) such as the Internet, for example.

  Note that the communication methods of the advertiser terminal 2 and the advertiser evaluation device 3 may be wired communication or wireless communication, respectively. In the example shown in FIG. 1, two advertisers SA and SB are taken as an example, but the number of advertisers S may be one, or may be three or more.

  The advertiser S is a business operator who requests the advertisement distributor to place an advertisement. The advertiser S produces advertisement information including, for example, TD (Title & Description) and submits it to the advertisement distributor. For example, FIG. 1 shows a case where the advertisement information X is submitted from the advertiser SA and the advertisement information Y is submitted from the advertiser SB.

  Then, the advertisement information X and Y submitted from the advertiser S is notified to the advertisement distributor side of the advertisement examination system 1 including the advertiser evaluation device 3 via the communication network 4, and the advertisement content is notified by the advertisement distributor side. After judging whether or not there is a violation, the advertiser S is notified of whether or not the submission is possible based on the examination result.

  Incidentally, the advertiser S may be a malicious person with malicious intent (hereinafter referred to as “illegal advertiser”), for example, as the advertiser SA in FIG. An unauthorized advertiser SA is a person who intends to act illegally, for example, by distributing malicious software by causing an advertisement viewer to click an advertisement.

  If there is no violation in the advertisement content etc. for the advertisement information X submitted by the unauthorized advertiser SA, the advertisement information X is permitted to be submitted and may be posted on a web page or the like.

  Here, the advertisement information X, Y includes information related to the advertiser S such as “account ID” required for submission. Therefore, the advertiser evaluation device 3 of the advertisement examination system 1 according to the embodiment performs an advertiser evaluation process for evaluating the advertiser S using information related to the advertiser S.

[1.2. Advertiser evaluation process)
An outline of the advertiser evaluation process will be described. The advertiser evaluation device 3 includes a control unit 32 (see FIG. 2 and subsequent figures). The control part 32 performs an advertiser evaluation process based on the information regarding the advertiser S that submitted the advertisement information X and Y.

  As shown in FIG. 1, first, the control unit 32 of the advertiser evaluation device 3 receives a submission of advertisement information X and Y including information related to the advertiser S (step S1).

  The information related to the advertiser S includes, for example, “account ID”, “advertisement content”, “submission route”, “submission time”, and the like. As shown in FIG. 1, the advertiser evaluation apparatus 3 has a DB, in which information related to the advertiser S including the “past examination history” of the advertiser S is stored. .

  Then, the control unit 32 includes various elements included in the information related to the advertiser S such as “account ID”, “advertising content”, “submission route”, “submission time”, and “past examination history”. Advertiser S is evaluated based on this (step S2). Such evaluation is performed, for example, based on an evaluation model using a plurality of elements based on the above elements as parameters.

  Then, the control unit 32 determines whether or not the advertiser S can submit the manuscript based on the evaluation result (for example, the evaluation score) (step S3).

  As described above, in the advertiser evaluation device 3 according to the embodiment, the control unit 32 executes the advertiser evaluation process for evaluating the advertiser S based on the information about the advertiser S including the past examination history. In other words, the advertiser evaluation process extracts an action pattern at the time of submission of an unauthorized advertiser SA based on information related to the advertiser S, that is, the “input pattern”, and extracts the “question pattern” of the advertiser S. The “significance” or “degree of fraud” is evaluated based on the aforementioned evaluation model.

  Therefore, it is possible to evaluate the “suspiciousness” of the advertiser S that is difficult to determine from only the advertisement information X and Y, and it is possible to determine an unauthorized advertiser SA based on the evaluation result.

  Note that the elements of the parameters described above are not limited to those related to the actions of the advertiser S at the time of submission, but for example, the advertiser S after the submission such as the number of accesses to the site of the advertisement distributor after the submission. The element indicating the behavior pattern may be the element of the parameter. This will be described later with reference to FIG. Hereinafter, the advertiser evaluation apparatus 3 according to the embodiment will be described in more detail with reference to FIG.

[2. Configuration of advertiser evaluation device]
Next, the configuration of the advertiser evaluation device 3 will be specifically described. FIG. 2 is a block diagram illustrating an example of the configuration of the advertiser evaluation device 3 according to the embodiment.

  In FIG. 2, only components necessary for the description of the advertiser evaluation device 3 are shown, and descriptions of general components are omitted. In addition, description of components that have already been described may be simplified or omitted.

  As shown in FIG. 2, the advertiser evaluation device 3 includes a communication unit 31, a control unit 32, and a storage unit 33.

(About the storage unit 33)
The storage unit 33 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. In the example of FIG. And an advertiser information DB 332.

(About the communication unit 31)
The communication unit 31 is an interface such as a NIC (Network Interface Card). The control unit 32 can transmit and receive various types of information to and from the advertiser terminal 2 via the communication unit 31 and the communication network 4.

(About the control unit 32)
The control unit 32 performs overall control on the execution of the advertiser evaluation process described with reference to FIG. Specifically, the control unit 32 is configured such that various programs stored in a storage device inside the advertiser evaluation device 3 are stored in a RAM (Random Access Memory) by, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). This is realized by being executed as a work area. The control unit 32 may be realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  As illustrated in FIG. 2, the control unit 32 includes a reception unit 321, an evaluation unit 322, a determination unit 323, and a notification unit 324, and realizes or executes the information processing functions and operations described below. . Note that the internal configuration of the control unit 32 is not limited to the configuration illustrated in FIG. 2, and may be other configurations as long as the information processing described below can be performed. In addition, the connection relationship between the processing units included in the control unit 32 is not limited to the connection relationship illustrated in FIG. 2, and may be another connection relationship.

(About the receiving unit 321)
The receiving unit 321 receives the submission of the advertisement information X and Y from the advertiser S, and sets information related to the submission pattern included in the advertisement information X and Y in the submission information table of the advertisement information DB 331. To do.

(Evaluation unit 322)
The evaluation unit 322 includes information regarding the advertiser S that has submitted the advertisement information X and Y, the submission information table of the advertisement information DB 331, the advertiser attribute information table of the advertiser information DB 332, the advertiser behavior information table, and the examination history information. The advertiser S is obtained from the table and the advertiser S is evaluated based on the obtained information.

(About various tables)
Here, the submission information table included in the advertisement information DB 331, the advertiser attribute information table, the advertiser behavior information table, and the examination history information table included in the advertiser information DB 332 will be described with reference to FIGS. 3A to 3D.

  FIG. 3A is a diagram illustrating an example of a submission information table. As shown in FIG. 3A, the submission information table includes, for example, “submission ID”, “submission date”, “submission time”, “account ID” used by the advertiser S at the time of submission, Information associated with “advertisement type”, “advertising content”, “channel”, “payment amount”, “payment route”, and the like is included.

  “Submission ID” is identification information assigned for each submission request from the advertiser terminal 2. “Submission date” and “submission time” are the submission date and submission time from the advertiser terminal 2. The “advertisement type” is the type of advertisement requested by the advertiser S. For example, “click” indicates a click charge advertisement, and “impression” indicates an impression charge advertisement.

  “Advertisement content” is the content of the advertisement requested by the advertiser S, for example, content such as TD. “Channel” is information corresponding to the submission route (see FIG. 1). “Direct” is, for example, directly submitted from the advertiser S via online without going through an agency. “Indirect” is, for example, Each case is via an agency.

  The “payment amount” is an amount received from the advertiser S at the time of submission. “Deposit path” is a deposit path. For example, “A country → JP” indicates that the deposit was made from A country overseas to Japan.

  For example, in the example of FIG. 3A, submissions with “account ID” = “aaa — 1” are grouped at about “22:00” on “2015/3/18” on the same day, and the same contents, the same amount, and It can be seen that the same route was taken from Country A. In addition, it can be seen that the “advertising content” includes the same error that should be “our company”, such as ““ the company is ... ”. As described above, the information related to the submission pattern of the submission information table includes an element for inferring the suspicious behavior pattern of the advertiser S.

  Next, FIG. 3B is a diagram illustrating an example of the advertiser attribute information table. As shown in FIG. 3B, in the advertiser attribute information table, for each “advertiser ID” assigned to each advertiser S, “account ID”, “contract period”, “domain”, “browser language”, Information associated with “time zone”, “business type”, “settlement method”, and the like is included.

  The “account ID” is an account ID held by the advertiser S identified by the “advertiser ID” and can be held by a single advertiser S. The submission information table and the advertiser attribute information table can be associated with each other by the “account ID”.

  The “contract period” is the contract period of each account ID, and “domain” is a domain name acquired from the email address of the advertiser S, for example. “Browser language” is browser language setting information that can be acquired from the advertiser terminal 2, and “time zone” is time zone setting information of the advertiser terminal 2 that can be acquired from the advertiser terminal 2. “Business type” is business type information of the business of the advertiser S. The “settlement method” is a settlement method of the advertiser S. For example, “Kureka” indicates a credit card.

  For example, in the example of FIG. 3B, the advertiser SA has a plurality of “account IDs” = “aaa_1” to “aaa_n” having a short “contract period”, “domain”, “browser language”, and It can be seen that “browser language” and “time zone” are inappropriate. It can also be seen that “industry” and “settlement method” are “service” and “cleca”, which are likely to be fraudulent, respectively. As described above, the advertiser attribute information table also includes an element for inferring the suspicious behavior pattern of the advertiser S.

  Next, FIG. 3C is a diagram illustrating an example of the advertiser behavior information table. As illustrated in FIG. 3C, the advertiser behavior information table includes information in which “account ID”, “management screen access count”, and the like are associated with each “advertiser ID”.

  The advertiser behavior information table can be associated with the submission information table and the advertiser attribute information table by the “advertiser ID” and the “account ID”. The “number of times of management screen access” is the number of accesses indicating how many times each submission has been made to the management screen for the advertiser S of the advertisement distributor that the advertiser S can access. Such access to the management screen is performed when, for example, when it is desired to confirm sales after being submitted and submitted as an advertisement after submission.

  In the example of FIG. 3C, it can be seen that the advertiser SA is accessing the management screen more frequently than the advertiser SB, for example. In addition, it can be seen that the “account ID” = “aaa_n” of the advertiser SA has an access count of 0 and is unused despite being registered. As described above, the advertiser behavior information table also includes an element for inferring the suspicious behavior pattern of the advertiser S.

  Next, FIG. 3D is a diagram illustrating an example of the examination history information table. As illustrated in FIG. 3D, the examination history information table includes information in which “account ID”, “examination history”, and the like are associated with each “advertiser ID”. The “examination history” is examination history information for past submissions for each “account ID”. By analyzing the examination history information, for example, the submission rejection rate (rejection rate) can be derived for each “account ID” or “advertiser ID”.

(Continued on the evaluation unit 322)
Returning to FIG. 2, the details of the processing executed by the evaluation unit 322 will be described. The evaluation unit 322 evaluates the advertiser S based on an evaluation model using a plurality of elements included in the information related to the advertiser S as parameters.

  Specifically, the evaluation unit 322 has an evaluation model that calculates the evaluation score y of the advertiser S using, for example, a regression equation represented by the following equation (1).

y = a ₁ · x ₁ + a ₂ · x ₂ + ... + a _n · x _n (1)

In the above formula (1), a _n (n is an arbitrary number) is a coefficient, and x _n is a feature (explanatory variable). First, the evaluation unit 322 analyzes each item data of the submission information table of the advertisement information DB 331, the advertiser attribute information table, the advertiser behavior information table, and the examination history information table of the advertiser information DB 332 described above, and each item data The identity for an unauthorized advertiser S based on

For example, when an illegal advertiser S is taken as a positive example, “x _n ” = “1” is obtained when the illegal advertiser S has a feature vector, and “x _n ” = “0” is obtained when there is no illegal advertiser S. It becomes. Further, a _{n which} is a weight value of x _n is adjusted by learning a linear model in which an illegal advertiser S according to the above formula (1) is a positive example.

For example, the evaluation unit 322, the a _n in formula (1) as a parameter, to adjust the weight of a _n by regression analysis techniques, such as support vector regression using pattern recognition machine, such as of SVM (Support Vector Machine) .

  In this example, the illegal advertiser S is taken as a positive example. However, the illegal advertiser S may be extracted by taking the legitimate advertiser S as a positive example. Moreover, the evaluation model which the evaluation part 322 has is not limited to the evaluation model mentioned above.

(Specific examples of _xn )
Next, FIG. 4 shows a specific example of _xn that is obtained by the evaluation unit 322 for evaluation with an illegal advertiser S as a positive example. FIG. 4 is a diagram illustrating a specific example of a feature when an illegal advertiser S is a positive example.

  Here, examples where n = 1 to 16 are given. The “corresponding item” is an entry information table of the advertisement information DB 331, an advertiser attribute information table of the advertiser information DB 332, an advertiser action information table, and an examination history information table that are inputs for obtaining the feature. . N1 to N16 are arbitrary constants, and are used as thresholds corresponding to each “condition”.

As shown in FIG. 4, for example, x ₁ is obtained by setting “condition” as “whether there are N1 or more buzzwords” based on “advertisement content” in the submission information table of the advertisement information DB 331. Can do. In other words, it x ₁ if "advertising content" x ₁ if the buzzword is N1 or more is "1", and the buzzword is less than the N1 is "0".

  This is because the more the buzzword is included in the “advertising content”, the easier it is to estimate that the advertiser is an unauthorized advertiser S.

Further, for example, x _2, like based on the "Ad Content" trafficking information table, the "condition" can be determined by a "whether mistake is N2 or more." In other words, if any mistake is N2 or more to the "advertising content" x ₂ becomes "1", x ₂ if clerical error is less than the N2 is "0".

  This is because the more erroneous entries are included in the “advertising content”, the easier it is to estimate that the advertiser is an unauthorized advertiser S.

Further, for example, x _3, like on the basis of the "Account ID" and "advertising content" trafficking information table, the "conditions" be determined by a "whether keywords in different trafficking identical" Can do. That is, x ₃ is “1” if the keywords of “advertising content” are the same in a plurality of submissions with the same “account ID”, and x ₃ is “0” if the keywords are different.

  This is because it is presumed that the same “account ID” and the same content are repeatedly submitted manually or by a program, etc., and it is easy to estimate that the advertiser is an unauthorized advertiser S.

Further, for example, x ₄ is also based on the "channel" trafficking information table, the "condition" can be determined by a "whether the channel is a direct". That is, if the “channel” is “direct”, x ₄ is “1”, and if it is “indirect”, x ₄ is “0”. This is because it is unlikely that the unauthorized advertiser S will bother to go through the agency.

Further, for example, x _5, like on the basis of the "deposit amount" trafficking information table, the "condition" can be determined by a "whether the payment amount is N5 yen or more". In other words, if the "deposit amount" is N5 yen or more x ₅ becomes "1", the x ₅ if it is less than N5 yen becomes "0". This is because the unauthorized advertiser S tends to concentrate in a short time and make a large amount of money. The possibility of fraudulent use such as money laundering is also inferred.

Further, for example, x _6, like on the basis of the "Deposit pathway" trafficking information table, the "condition" can be determined by a "whether the deposit route is domestic from foreign". In other words, x ₆ if the domestic "payment route" from overseas is "1", x ₆ if the domestic from domestic "0". This is because unauthorized advertisers S tend to have many overseas businesses.

Further, for example, x _7, like "Account ID" trafficking information table, "Trafficking date" and based on the "trafficking time", a plurality of "conditional" within N7 seconds from the same date in the "same account Trafficking It can be obtained by "whether or not it is done". That is, if are more trafficked within N7 seconds from the same date and time in the same account x ₇ becomes "1", if otherwise x ₇ is "0". This is because, as described above, it is assumed that an unauthorized advertiser S repeatedly submits the same content with the same “account ID” at a high speed by a program or the like.

Further, for example, x _8, like based on the "trafficking time" trafficking information table, the "condition", "trafficking time 18: 00 next 6:00 is whether" be determined by the Can do. In other words, "submission time" is 18: 00~ _{x 8} if a well-06:00 becomes "1", if otherwise _{x 8} is "0". This is because if it is submitted in a time zone that is difficult for the examiner to handle, it is easy to estimate that the advertiser is an unauthorized advertiser S.

Further, for example, x _9, like based on the "Ad type" trafficking information table, the "condition" can be determined by a "whether advertisement type is CPC." That is, if "ad type" is "pay per click" x ₉ becomes "1", if otherwise x ₉ is "0". This is because, for example, there is an unauthorized advertiser S who distributes malicious software by causing an advertisement viewer to click. In addition, if it is assumed that the purpose is to allow an unauthorized advertiser S to browse anyway, x _n with “condition” as “whether or not the advertisement type is impression charge” may be obtained.

Further, for example, x ₁₀ is above the calculated continuation rate of the contract on the basis of the "contract period" advertiser attribute information table of the advertiser information DB332, whether the "condition", "retention rate is less than N10% "Can be obtained. That is, if the continuation rate is less than N10% _{x 10} becomes "1", if continuation rate is N10% or more _{x 10} becomes "0". As described above, this is because unauthorized advertisers S tend to concentrate in a short time and make a large amount of money.

Further, for example, x _11, like based on the "domain" and "browser language" advertiser attribute information table, the "conditions" be determined by a "whether the language of the domain and the browser is disproportionate" Can do. That is, if the “domain” and the “browser language” are incompatible, x ₁₁ is “1”, and if it is appropriate, x ₁₁ is “0”.

Similarly, for example x _12, like based on the "domain" and "Time Zone" advertiser attribute information table, it is obtained by making the "condition", "whether the domain and time zone are disproportionate" it can. That is, if the “domain” and the “time zone” are not appropriate, x ₁₂ is “1”, and if they are appropriate, x ₁₂ is “0”. These are due to the fact that it is easy to presume that the advertiser is an unauthorized advertiser S if the relations such as “domain”, “browser language”, and “time zone” are inappropriate.

Further, for example, x _13, like based on the "sector" advertiser attribute information table, can be obtained by making the "condition" for example, "whether industry is a service industry." In other words, if the "industry" is a service industry x ₁₃ becomes "1", if other than the service sector x ₁₃ becomes "0". Note that this example is based on the assumption that fraud using a service industry site is likely to occur, and does not necessarily match the actual example.

Further, for example, x _14, like based on the "settlement" advertiser attribute information table, the "condition" can be determined by a "whether payment is credit card." That is, if the “settlement method” is a credit card, x ₁₄ is “1”, and if it is not a credit card, x ₁₄ is “0”. This is because fraudulent advertisers S tend to cheat credit cards.

Further, for example, x _15, based on the "UI access number" advertiser behavior information table advertiser information DB332, the "condition" by the "whether UI access count is equal to or larger than N15 times" Can be sought. That is, if "interface access count" is equal to or larger than N15 times x ₁₅ becomes "1", x ₁₅ if less than N15 times is "0". This is because it is presumed that an unauthorized advertiser S will come to frequently check information related to sales after submission, for example.

Also, if for example x _16, the top of the calculated rate of rejection advertiser S based on the "audit history" audit history information table of the advertiser information DB332, the "condition", "rejection rate is N16% or more not It can be determined by That is, if rejection rate is N16% or more _{x 16} becomes "1", if less than N16% _{x 16} becomes "0". This is because an advertiser S having a high rejection rate cannot be said to be a legitimate advertiser S in the first place.

(Continued on the evaluation unit 322)
Returning to FIG. 2, the evaluation unit 322 will be described. The evaluation unit 322 calculates the evaluation score of the advertiser S by the above formula (1) after obtaining x ₁ to x _n along each “condition” shown as an example in FIG. Then, the evaluation unit 322 notifies the determination unit 323 of the calculated evaluation score.

(About determination unit 323)
The determination unit 323 determines whether or not the advertisement information X and Y requested to be submitted can be submitted depending on whether or not the evaluation score by the evaluation unit 322 is within a predetermined range. That is, when the evaluation score by the evaluation unit 322 is within a predetermined range, the determination unit 323 rejects submission of the advertisement information X and Y requested for submission. Further, the determination unit 323 also rejects the submission from the advertiser S. That is, the advertiser S is blacklisted as an unauthorized advertiser S, and for example, all previous or subsequent advertisement placements are rejected.

  In order to indicate that the advertiser S has entered the black list, the determination unit 323 enters, for example, the “advertiser ID” or “account” of the advertiser S into a black list table (not shown) provided in the advertiser information DB 332. ID "is registered.

  In addition, when the evaluation score by the evaluation unit 322 is not within the predetermined range, the determination unit 323 permits submission of the advertisement information X and Y for which submission is requested.

  For example, the determination unit 323 can feed back the evaluation score by the evaluation unit 322 to the evaluation unit 322 and change the evaluation level of the advertiser S corresponding to the evaluation score from the next time. Specifically, when the evaluation score indicates a high score, for example, close to an unauthorized advertiser S, the evaluation level for the advertiser S will be severe from the next time. Then, the evaluation unit 322 is caused to change the evaluation level. In this case, the evaluation unit 322 performs evaluation with respect to the corresponding advertiser S from the next time, for example, according to the evaluation score fed back, by weighting the incorrect advertiser S.

  On the other hand, when the evaluation score indicates a high score near the legitimate advertiser S, for example, the evaluation level for the advertiser S will be loose from the next time. The evaluation unit 322 is caused to change the evaluation level. In this case, the evaluation unit 322 performs evaluation with respect to the relevant advertiser S from the next time, for example, according to the feedback evaluation score that is weighted closer to the legitimate advertiser S. When the advertiser S clearly shows a high score as a valid advertiser S, the advertiser S is registered in the white list table provided for the black list table, and then entered into the white list. From the next time, the advertiser S may be allowed to submit a document without evaluation, and the advertisement may be distributed.

(About the notification unit 324)
The notification unit 324 notifies the advertiser S of the determination result by the determination unit 323.

[3. (Advertiser evaluation processing procedure)
Next, a processing procedure of an advertiser evaluation process executed by the advertiser evaluation device 3 according to the embodiment will be described. FIG. 5 is a flowchart illustrating a processing procedure of an advertiser evaluation process executed by the advertiser evaluation device 3 according to the embodiment.

  As shown in FIG. 5, the receiving unit 321 receives the advertisement information X and Y from the advertiser S (step S101). Here, the control unit 32 first determines whether or not the advertiser S of the received advertisement information X and Y is clearly a valid advertiser S (step S102). Such a determination is made based on, for example, whether or not the advertiser S is registered in the above-described white list table. Here, when the advertiser S is clearly an authorized advertiser S (step S102, Yes), submission of the advertiser S without permission is permitted (step S103), and control is transferred to step S109. Further, when the advertiser S is obviously not a valid advertiser S (No at Step S102), the control unit 32 next determines whether or not the advertiser S is a rejection target (Step S104). Such a determination is made based on, for example, whether or not the advertiser S is registered in the black list table described above.

  Here, when the advertiser S is not to be rejected (No at Step S104), the evaluation unit 322 evaluates the advertiser S based on information about the advertiser S (Step S105). If the advertiser S is to be rejected (step S104, Yes), the control unit 32 moves the control to step S107.

  And the determination part 323 determines whether the evaluation score by the evaluation part 322 is a predetermined range (step S106). Here, when the evaluation score is within the predetermined range (step S106, Yes), the determination unit 323 rejects the submission of the advertiser S (step S107). If the evaluation score is not within the predetermined range (No at Step S106), the determination unit 323 permits submission of the advertiser S (Step S108).

  And the notification part 324 notifies the determination result by the determination part 323 to applicable advertiser S (step S109), and the control part 32 complete | finishes an advertiser evaluation process.

[4. Hardware configuration)
Note that the advertiser evaluation device 3 according to the embodiment is realized by a computer 60 having a configuration as illustrated in FIG. 6, for example. FIG. 6 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the advertiser evaluation device 3. The computer 60 includes a central processing unit (CPU) 61, a random access memory (RAM) 62, a read only memory (ROM) 63, a hard disk drive (HDD) 64, a communication interface (I / F) 65, an input / output interface (I). / F) 66 and a media interface (I / F) 67.

  The CPU 61 operates based on a program stored in the ROM 63 or the HDD 64 and controls each unit. The ROM 63 stores a boot program executed by the CPU 61 when the computer 60 is activated, a program depending on the hardware of the computer 60, and the like.

  The HDD 64 stores a program executed by the CPU 61, data used by the program, and the like. The communication interface 65 corresponds to the communication unit 31, receives data from other devices via the communication network 4, sends the data to the CPU 61, and transmits the data generated by the CPU 61 to other devices via the communication network 4. .

  The CPU 61 controls an output device such as a display and a printer and an input device such as a keyboard and a mouse via the input / output interface 66. The CPU 61 acquires data from the input device via the input / output interface 66. Further, the CPU 61 outputs the generated data to the output device via the input / output interface 66.

  The media interface 67 reads a program or data stored in the recording medium 68 and provides it to the CPU 61 via the RAM 62. The CPU 61 loads the program from the recording medium 68 onto the RAM 62 via the media interface 67, and executes the loaded program. The recording medium 68 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. Etc.

  When the computer 60 functions as the advertiser evaluation device 3, the CPU 61 of the computer 60 executes each program of the reception unit 321, the evaluation unit 322, the determination unit 323, and the notification unit 324 by executing a program loaded on the RAM 62. Is realized. The HDD 64 realizes the function of the storage unit 33, and stores an advertisement information DB 331, an advertiser information DB 332, and the like.

  The CPU 61 of the computer 60 reads these programs from the recording medium 68 and executes them, but as another example, these programs may be acquired from other devices via the communication network 4.

[5. effect〕
The advertiser evaluation device 3 of the advertisement examination system 1 according to the embodiment includes a receiving unit 321 and an evaluation unit 322. The receiving unit 321 receives the submitted advertisement information X and Y from the advertiser S. The evaluation unit 322 evaluates the advertiser S based on information related to the advertiser S that has submitted the advertisement information X and Y.

  Thereby, it becomes possible to evaluate the “suspiciousness” of the advertiser S that is difficult to see from the contents of the advertisements of the advertisement information X and Y, and the unauthorized advertiser S can be determined based on the evaluation result. In addition, even if there is no violation in the content of the advertisement information X, Y, the advertisement information X, Y can be rejected by the unauthorized advertiser S, so that the advertisement is posted. Thus, it is possible to prevent the malicious fraud of the unauthorized advertiser S from spreading.

  In addition, the evaluation unit 322 evaluates the advertiser S based on an evaluation model using a plurality of elements included in the information related to the advertiser S as parameters.

  As a result, for example, an unauthorized advertiser S can be determined based on an evaluation model based on machine learning, so that an unknown behavior pattern of the unauthorized advertiser S can be appropriately handled by machine learning and high speed with high accuracy. Advertiser evaluation processing can be performed.

  Further, the information related to the advertiser S includes elements of the submission pattern of the advertisement information X and Y by the advertiser S.

  Thereby, the elements of the submission pattern of the advertisement information X and Y by the advertiser S can be analyzed and the characteristics and tendencies in the submission of the advertiser S can be extracted. It can help to judge.

  The submission pattern includes at least one of the contents of the advertisement information X and Y, the submission route, the submission time, the advertisement type, the deposit amount, and the deposit route.

  Thereby, the feature or tendency of the submission pattern of the advertiser S is extracted from at least one of the contents of the advertisement information X, Y, the submission route, the submission time, the advertisement type, the deposit amount, and the deposit route. Therefore, it is possible to contribute to determining an unauthorized advertiser S with high accuracy.

  The information related to the advertiser S includes the review history of the past submission of the advertisement information X and Y, the account usage status for submitting the advertisement information X and Y, and the advertisement information X and Y. At least one of the action information of the advertiser S after writing is included.

  Thereby, based on the examination history, for example, an advertiser S having a high past rejection rate can be handled and evaluated as an advertiser S having a high possibility of fraud. Further, the advertiser S having a suspicious place in the account usage status can be handled and evaluated as the advertiser S having a high possibility of fraud. Further, the advertiser S who exhibits suspicious behavior based on the behavior information of the advertiser S after submitting the advertisement information X and Y, for example, the number of accesses to the management screen, is also the advertiser who is highly likely to be fraudulent. It can be handled and evaluated as S. Therefore, it is possible to contribute to accurately determining an unauthorized advertiser S.

  Further, the advertiser evaluation device 3 includes a determination unit 323 and a notification unit 324. The determination unit 323 determines whether or not the advertisement information X and Y can be submitted depending on whether or not the evaluation score based on the evaluation model by the evaluation unit 322 is within a predetermined range. The notification unit 324 notifies the advertiser S of the determination result by the determination unit 323.

  As a result, when the advertising information X, Y is submitted from an unauthorized advertiser S, even if there is no violation in the content of the advertising information X, Y, the advertising information X, Y Can be rejected. That is, it is possible to prevent malicious fraud from spreading by an unauthorized advertiser S. In addition, since it is not necessary to examine the contents of advertisements of advertisement information X and Y, it is possible to contribute to a reduction in examination cost for examination work.

  Further, the advertiser evaluation device 3 includes a determination unit 323 and a notification unit 324. The determination unit 323 determines whether the submission from the advertiser S is possible depending on whether the evaluation score based on the evaluation model by the evaluation unit 322 is within a predetermined range. The notification unit 324 notifies the advertiser S of the determination result by the determination unit 323.

  As a result, when the unauthorized advertiser S is blacklisted and the advertisement information X, Y is submitted from the unauthorized advertiser S, the unauthorized advertiser S may be rejected without being evaluated. it can. That is, it can contribute to the reduction of the examination cost concerning examination work.

[6. Others]
Although one aspect of the embodiment of the present application has been described in detail with reference to the drawings, this is an exemplification, and various modifications based on the knowledge of those skilled in the art, including the aspect described in the column of the disclosure of the invention. It is possible to implement the present invention in other forms with improvements.

  For example, as an element included in the above-mentioned submission pattern, when one advertiser S has a plurality of account IDs and serial numbers are assigned to these account IDs, There are cases where submission is made using a plurality of account IDs.

DESCRIPTION OF SYMBOLS 1 Advertisement examination system 2 Advertiser terminal 3 Advertiser evaluation apparatus 4 Communication network 31 Communication part 32 Control part 33 Storage part 321 Reception part 322 Evaluation part 323 Determination part 324 Notification part 331 Advertisement information DB
332 Advertiser Information DB

Claims (8)

A receiver for receiving trafficking of advertising information from advertisers,
At least an element of the advertising information submission pattern by the advertiser and an element of the advertiser behavior pattern after the advertising information has been submitted, included in the information about the advertiser that has submitted the advertising information Based on an evaluation model as a parameter, an evaluation unit that calculates an evaluation score indicating the degree of the advertiser being a malicious fraud,
A determination unit that determines that the advertiser is the unauthorized person when the evaluation score is within a predetermined range;
An advertiser evaluation apparatus comprising: The submission pattern includes
The advertiser evaluation apparatus according to claim 1 , wherein at least one of the content of the advertisement information, a submission route, a submission time, an advertisement type, a deposit amount, and a deposit route is included. Information about the advertiser includes:
Examination history for the past trafficking of the advertisement information, contained before Symbol usage of the account to traffic ads information, and, at least one of the advertiser's access status to the management screen of the Advertiser is The advertiser evaluation apparatus according to claim 1 or 2 , wherein The determination unit
If the evaluation score is given range, advertiser evaluation apparatus according to claim 1, 2 or 3, characterized in that deny traffic of the advertisement information. The determination unit
If the evaluation score is given range, advertiser evaluation apparatus according to claim 1, 2 or 3, characterized in that deny traffic from the advertiser. The determination unit
The advertiser according to any one of claims 1 to 5, wherein the evaluation unit is caused to change a next evaluation level of the advertiser corresponding to the evaluation score according to the evaluation score. Evaluation device. An advertiser evaluation method performed by a computer,
A receiving process for receiving trafficking of advertising information from advertisers,
At least an element of the advertising information submission pattern by the advertiser and an element of the advertiser behavior pattern after the advertising information has been submitted, included in the information about the advertiser that has submitted the advertising information Based on an evaluation model as a parameter, an evaluation step for calculating an evaluation score indicating a degree that the advertiser is a malicious fraud,
A determination step of determining that the advertiser is the unauthorized person when the evaluation score is within a predetermined range;
An advertiser evaluation method characterized by including: On the computer,
A receiving procedure for receiving trafficking of advertising information from advertisers,
At least an element of the advertising information submission pattern by the advertiser and an element of the advertiser behavior pattern after the advertising information has been submitted, included in the information about the advertiser that has submitted the advertising information Based on an evaluation model as a parameter, an evaluation procedure for calculating an evaluation score indicating a degree of the malicious person being a malicious fraud,
A determination procedure for determining that the advertiser is the fraudster when the evaluation score is within a predetermined range;
An advertiser evaluation program characterized in that

Images