KR20090113386A - System for scoring click traffic - Google Patents

Abstract

A system is disclosed for measuring click traffic quality by scoring clicks made on sponsored advertisements. A click score generated by the disclosed system may enable advertisers and publishers to distinguish between legitimate and fraudulent clicks. The disclosed system may filter click data associated with a click made on a sponsored advertisement. The system may generate a click score that may represent the confidence with which the quality of a click may be determined. The system also may generate a confidence interval associated with the click score.

Description

System for scoring click traffic {SYSTEM FOR SCORING CLICK TRAFFIC}

The present invention relates generally to fraud detection, and not exclusively, and specifically to click fraud detection in online advertising.

The availability of powerful tools for developing and distributing Internet content has led to a significant increase in the number and types of consumers using the Internet, as well as improving the information, products, and services provided over the Internet. In addition to this increased consumer traffic, the number of advertisers driving sales of their products and services via the Internet has increased significantly.

Advertisers can pay publishers to host or sponsor their ads on web pages, search engines, browsers or other online media. Publishers can charge advertisers on a "per click basis", which means that publishers can charge advertisers each time one of their ads is clicked. However, the "click unit" payment model can be vulnerable to click fraud. For example, a script or other software agent may be configured to repeatedly click on the advertisement to artificially raise the click unit payment and charge the advertiser for a large number of fraudulent clicks.

To address the likelihood of fraudulent clicks, click-based advertising models may use click fraud detection systems to identify "valid" or legitimate clicks. The publisher can then charge the advertiser only for valid clicks. However, there may not be a standard way to determine whether a click is valid. Also, just specifying a click as a binary category (eg, valid or invalid) may not fully or accurately account for probabilistic decisions that often characterize click quality. Therefore, frequent misclassification may occur. Also, although two clicks may each have been declared valid, the clicks may still include significant differences. Based on the characteristics of the click, one click may be clearly valid, but another click may be a difficult case to determine. Declaring each click as valid may not take into account the relative confidence that classified each click.

Summary of the Invention

A system for measuring click traffic quality by scoring clicks on sponsored advertisements is disclosed. The disclosed system can filter the click data associated with the click on the sponsored ad. The system can generate a click score that indicates a confidence level that can determine the quality of the click. The system may also generate a confidence interval associated with the click score. The click score generated by the disclosed system may enable advertisers and publishers to distinguish legitimate clicks from fraudulent clicks.

The system may include a number of filters for generating filter output data. The filter output data may indicate which of the plurality of filters has operated in response to the click data. The output data may also include synthesis filter scores corresponding to multiple filters. The plurality of filters may be one or more definitive filters. The confinement filter may be configured to work when the click data implies that the click is fraudulent with reasonable confidence. The system can obtain a click classification by comparing the click score with one or more thresholds.

Other systems, methods, features, and benefits will become or become apparent to those skilled in the art upon review of the following figures and detailed description. All such additional systems, methods, features, and benefits are intended to be included within this description, within the scope of the present invention, and protected by the following claims.

Non-limiting and non-exhaustive descriptions are provided with reference to the drawings below. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. Moreover, in the drawings, like reference numerals indicate corresponding parts through different views.

1 is a block diagram of a general architecture of a system for adaptive click traffic scoring.

2 is a flow diagram illustrating a process for scoring user clicks in a system for adaptive click traffic scoring.

3 is a block diagram of a view of a system for adaptive click traffic scoring that includes filtering logic and one or more scoring algorithms.

4 is a block diagram illustrating the relationship between user intent and click score in a click on an advertisement in a system for adaptive click traffic scoring.

5 is a flow diagram illustrating a process for scoring user clicks in the system of FIG. 1 and other systems for adaptive click traffic scoring.

6 is a flow diagram illustrating a process for applying a threshold to a click score in a system for adaptive click traffic scoring.

7 is a flow diagram illustrating a process for applying high and low thresholds to click scores in a system for adaptive click traffic scoring.

8 is a block diagram of a computer system implementing a system for adaptive click traffic scoring.

Systems and methods, generally referred to as systems, generally relate to click traffic scoring based on filtered click data. The principles described herein may be embodied in many different forms. The systems and methods disclosed may enable publishers and / or advertisers to effectively identify unreliable or invalid clicks and / or valid clicks. The disclosed systems and methods can provide a click score that can indicate a relative confidence in the effectiveness of the click. The click score can be used to determine the quality of the click. In this manner, the disclosed systems and methods may enable a publisher to implement versatile click based advertising pricing models. For purposes of explanation, the system is described as being used in a network environment, but the system may operate outside of the network environment.

1 shows a general architecture 100 of a system for adaptive click traffic scoring. Architecture 100 may include user client system 110, publisher 120, advertiser 130, ad network 140, and click traffic scoring system 150. User client system 110 may search, browse, or access content, including advertising content provided by publisher 120 via communication network 160. Publisher 120 may host content such as web pages provided by advertiser 130. Publisher 120 may also display advertisement content provided by an advertiser in response to a user query in a search engine. The components of architecture 100 may be separate, supported on a single server or other network enabled system, or supported by any combination of servers or network enabled systems. The components of the architecture 100 may include one or more databases for storing data, parameters, statistics, programs, webpages, search lists, advertising content, or other information related to advertising, click traffic scoring, or other systems. Or may be accessed via the communication network 160.

Communication network 160 may be any private or public communication network or combination of networks. Communications network 160 may be configured to couple one computing device, such as a server, system, database, or other network enabled device, to another device to enable data communication between the devices. Communications network 160 may generally be enabled to use any form of computer readable media for communicating information from one computing device to another. The communication network 160 may include one or more of a direct connection via a wireless network, a wired network, a local area network (LAN), a wide area network (WAN), a universal serial bus (USB), and the like, and interconnects that make up the Internet. It may include a collection of networks. Communication network 160 may implement any communication method capable of transferring information between computing devices.

The publisher may charge advertiser 130 for hosting the advertising content on a web page, search engine, browser, or other online publishing medium. For example, publisher 120 may charge advertiser 130 on a click-by-click basis, that is, whenever an advertisement hosted by publisher 120 is clicked by a user. User client system 110 may select an advertisement by clicking on the advertisement.

User client system 110 may connect to publisher 120 via the Internet using a standard browser application. The browser-based implementation enables system features to be accessed regardless of the platform of the user client system 110. For example, user client system 110 may be a desktop, laptop, handheld computer, cell phone, mobile management device, network enabled television, digital video recorder such as TIVO, automobile, or various hardware and / or software packages. There may be other network enabled user client systems 110 available. User client system 110 may connect to publisher 120 using a standalone application (eg, a browser over the Internet, a mobile device over a wireless network, or other applications) that may be platform dependent or platform independent. have. Other methods may be used to implement the user client system 110.

Choices and clicks on advertisements from user client system 110 may not always be reliable. Clicks on the same advertisement or a series of multiple clicks can be initiated from an automated script rather than a potential customer.

The click traffic scoring system 150 may generate a click score as well as a confidence interval associated with the click score to measure the quality of the click. Click scores and confidence intervals may provide a scoring mechanism that uses continuous scale, for example, unlike a binary mechanism that only identifies clicks as valid / invalid categories. The continuous scale can range from 1 to N, from 0 to infinity, or can include other numerical ranges. The click traffic scoring system 150 may calculate the click score and confidence interval based in part on the user click data. The publisher 120 or other system that monitors and collects data related to user clicks may obtain user click data and send the user click data through the communication network 160 to the click traffic scoring system 150.

The click traffic scoring system 150 may send the click score and confidence interval through the communication network 160 to the publisher 120, the advertiser 130, and / or the ad network 140. Ad network 140 may act as an intermediary between publisher 120 and advertiser 130. Publisher 120, advertiser 130, and / or ad network 140 may implement a versatile advertising pricing model using click scores and confidence intervals. For example, the fee charged to an advertiser for each click may be a function of the click score, which increases gradually as the click score increases. The pricing model may include a stratified pricing model in which different ranges of click scores correspond to different pricing tiers.

2 shows a process 200 that can be used to score user clicks in a system for adaptive click traffic scoring, such as the click traffic scoring system 150. Process 200 may obtain user click data associated with a user click by monitoring and / or collecting information associated with the user click (act 202). User click data may include reference URL, cookie data, IP address, geographic location, whether the click was made in response to a query, whether the click was made by an automated script, or other click characteristics. Process 200 may compile user click data. Alternatively or in addition, process 200 may receive user click data compiled by another click monitoring process.

Process 200 may filter user click data (act 204). Process 200 may apply the user click data to the filtering logic to generate filter output data. The filtering logic may include one or more filters. The filter may be a function designed to identify some kind of invalid traffic. The filter output data may indicate which filters have been activated in response to the user click data. The filter output data may also include filter scores.

The filter may be a deterministic filter, such as a binary function that is "1" for self-declared robots and "0" otherwise. In this example, the filter can be said to work on a click if the value of the function is not "0".

The filter may also be a probabilistic filter. For example, the filter may determine whether a particular advertisement has been targeted more often than the average clicks on that advertisement by a particular client over a period of time. In this example, if the client generates twice as many clicks as the average for a particular ad, the filter may take into account historical analysis or statistics to determine if the number of clicks above the average indicates random fluctuations rather than fraud attacks. Can be. For example, from historical analysis, it can be known that 60% of the clients generating twice as many clicks as the average are fraudulent at that time and 40% at the time are the result of normal variability. In this case, if the score of the complete click is 1, the filter may score the click as 0.4 with a confidence interval (0.3, 0.5) corresponding to 90% confidence.

The filter score may, for example, comprise a binary output that indicates whether the corresponding filter has been activated. The filter score may include, for example, a fraction, range or other numerical representation that indicates the likelihood that the filtered output corresponds to a valid or invalid click.

The filtering logic may include filters to check specific click characteristics. For example, the filtering logic may include an automated script filter. Such a filter may operate when the click is initiated from a known automated script, for example, rather than from a legitimate user search. The filter may also include black lists that include lists from various agencies or organizations, such as an interactive advertising association.

The filtering logic may also include an IP address filter. The IP address filter can work when the IP address that initiated the click implies that the click is invalid. The IP address filter may include algorithms, search functions, or other processing techniques, such as by comparing the IP address that initiated the click with a list or database of malicious or "black listed" IP addresses. The filter score provided by the IP address filter may be a simple "1" or "0" indicating whether the filter has been activated and thus whether the click is valid or invalid.

The IP address filter may also output a fraction or other numerical filter score indicating the confidence that click traffic from a given IP address may be considered valid or invalid. For example, proxy server X may be known to contain 70% valid traffic and 30% invalid traffic. In this example, if the score of the complete click is 1, the filter may provide a score of 0.7 for the click from proxy server X.

Alternatively or in addition, the filtering logic may include filters corresponding to one or more geographic locations. The geolocation filter can provide a filter score that can indicate a confidence in declaring a click invalid based on the geographic location that initiated the click. The geographic location of a user can be identified by analyzing IP addresses to implement various geocoding techniques, or by other geographic location determination methods. The geolocation filter may include or access data associated with the identified geolocation, such as statistics or extrapolation data indicating the likelihood that the click is valid or invalid for a given location.

The filtering logic may include other filters that operate when the click owns or lacks certain characteristics. The types of click characteristics that the process 200 can look at, namely the types of filters used, can be adapted to the needs of the publisher or advertiser. The types of features filtered by process 200 may also be obtained from other information sources, such as standards published by the Internet Advertising Association or by other associations or organizations.

When the filter or combination of filters is activated, process 200 may determine filter scores using statistical data that includes conversion rates for filters or combinations of filters that have operated in response to user click data. S is called the population of clicks and s is called the element of S. Element s may include one or more click properties, including an IP address, reference URL, cookie data, and other click properties. F is called the subset of S for which the filter or combination of filters operates. F can be expressed as a binary function for S, i.e., F (s) = 1 for a subset of S for which the filter or combination of filters is operating, and F (s) = 0 else. The effect or score of the filter or combination of filters can then be estimated by the ratio Pr (s valid | F (s) = 1) / Pr (s valid), where s belongs to the set of clicks S and the numerator Denotes the probability of a valid click when the click is within F, and the denominator represents the probability of a valid click over the entire space S. A good subset F, i.e., a subset that effectively identifies invalid clicks with minimal misclassification, may have a ratio close to zero. Subset F may correspond to a filter or a combination of filters.

The click may lead to a conversion or “conversion” if the click leads to a desired action defined by the advertiser. Advertisers can define conversions as when a click leads to an actual purchase. Alternatively or in addition, the click may lead to a conversion when the click causes the user to add the item to the "shopping card" regardless of whether the user finally purchases the item. That is, criteria for conversion may be determined by the advertiser and may vary between advertisers.

The ratio Pr (s valid | F (s) = 1) / Pr (s valid) gives the statistical click transform data observed, compiled, or collected by assuming that the transform and F are conditionally independent when validity is given. Can be estimated using. Given a third event C, two events A and B are conditionally independent if the occurrence of event A does not change the probability of occurrence of event B and vice versa. That is, if the click is known to be valid, the occurrence of the transformation does not change the probability that the click falls within subset F, and vice versa. That is, the conversion rate for valid clicks may not change when limited to the set {F (s) = 1}. Based on this conditional independence assumption, the ratio Pr (s conversion | F (s) = 1) / Pr (s conversion) is a measure of Pr (s valid | F (s) = 1) / Pr (s valid). Can be used.

The ratio Pr (s transform | F (s) = 1) / Pr (s transform) can be further estimated using the following assumptions.

1. The support of F probably constitutes a small part of S. That is, Pr (s convergence) ≒ Pr (s convergence | F (s) = 0). Therefore, Pr (s conversion | F (s) = 1) / Pr (s conversion) can be estimated as Pr (s conversion | F (s) = 1) / Pr (s conversion | F (s) = 0). have.

2. The click transformation can be modeled as an independent Bernoulli trial for each click, ie for each click the sample space (converted, unconverted), as well as the associated probabilities p _s and 1-p _s May exist. The probability p _s may be the probability that the click s is converted. For any subset A of S, the amount Pr (s converge | A) may be the average of all p _s of s in A.

For subset F, p _D is called Pr (s transform | F (s) = 1) and p _C is called Pr (s transform | F (s) = 0). The ratio p _D / p _C can then estimate the effect of subset F on identifying invalid clicks. The ratio p _D / p _C may also correspond to the filter score for subset F.

The ratio p _D / p _C may also correspond to the click score described below, when the subset F corresponds to a combination of filters operated in response to user click data. The smaller the ratio p _D / p _C to the subset F (p _C is greater than p _D ), the more the process 200 belongs to the subset F (or causes the filter corresponding to the subset F to work). The confidence that can be determined to be invalid is greater. If subset F corresponds to a combination of filters that operated in response to a click, a smaller ratio of p _D / p _C corresponds to a greater confidence that the click that caused the combination of filters to work is invalid. The values of p _D and p _C can be obtained from the sample data. Sample data may consist of experimentally or statistically compiled values for C (and thus p _C ) and D (and thus p _D ).

Process 200 may analyze the filter output data including the filter scores to generate a click score (act 206). As described above, the filter output data may include a number of filter scores generated by the filters that make up the filtering logic. Process 200 may apply the filter output data to one or more scoring algorithms to calculate the click score. Scoring algorithms can calculate the click score using a variety of techniques.

The scoring algorithm may monitor which filters have been activated in response to user click data. The scoring algorithm may determine the click score based on filter scores corresponding to the combination of filters that operated in response to the user click data. For example, user click data may cause certain combinations of filters to work. The click score can be calculated by comparing the conversion rate for the set of clicks filtered by this combination with the overall conversion rate, for example by calculating the ratio p _D / p _C for the subset F. In this example, subset F may be a set of clicks corresponding to a combination of filters that operated in response to user click data. The scoring algorithm may calculate the ratio p _D / p _C using statistical data including the conversion rates for various combinations of the filters in operation. Statistical data, including conversion rates, may be stored in a database accessible through a communication network, such as communication network 160. Statistical data, including conversion rates, may also be provided by the publisher, advertiser, or advertising network.

Scoring algorithms can also average or aggregate filter scores to obtain a click score. Scoring algorithms may apply weights to filter scores to enable results from different filters to affect the continuous score differently. Scoring algorithms may also set the click score equal or substantially equal to the filter score with the largest magnitude.

The scoring algorithms may be algorithms generated from neural networks or other learning or pattern recognition algorithms for calculating click scores. For example, scoring algorithms can be used to train neural networks on known data related to click traffic, including click conversion rates, conversion counts, and other click conversion statistics, as well as data related to false positives or false negatives of past monitored monitoring. Can be generated from

Process 200 may generate a confidence interval associated with the click score (act 208). Process 200 may apply the click score and / or filter output data to a scoring algorithm to generate a confidence interval. Algorithms for calculating click scores may be the same or different than algorithms for calculating confidence intervals.

Process 200 may generate a confidence interval associated with p _D , p _C and / or ratio p _D / p _C for subset F. Subset F may correspond to a combination of filters that operate in response to user click data. Process 200 may use Fieller's Theorem to generate an appropriate confidence interval for p _D / p _C.

의 p_D/p_C에 대한 신뢰 구간을 생성할 수 있다.

및

의 샘플 데이터 및 1-α의 신뢰도 레벨이 주어지면, p_D 및 p_C에 대한 레벨

에서의 신뢰 구간들

및

이 각각 얻어질 수 있다. p_D 및 p_C는 독립적일 수 있으며, 따라서 비율 p_D/p_C는 1-α의 신뢰도 레벨로 구간

내에 있을 수 있다.For a given confidence level, for example 1-α, the process is also shaped

A confidence interval for p _D / p _C can be generated.

And

Given the sample data of and the confidence level of 1-α, the levels for p _D and p _C

Confidence intervals in

And

Each of these can be obtained. p _D and p _C can be independent, so the ratio p _D / p _C is interval with a confidence level of 1-α

Can be in.

Click scores and / or confidence intervals may be sent to publishers, advertisers, advertising networks, or other systems to calculate advertising fees. The click score may provide an indication of the confidence that the click may be considered valid or invalid. Publishers, advertisers, or other systems may use the reliability information to tailor the advertising fee structure to the relative reliability of each click or set of clicks. The confidence interval may provide additional relevant information to the publisher, advertiser, or other system, including error intensity, margin, or other characteristics of the click score.

3 shows a view of a click traffic scoring system 300 that includes filtering logic 302 and one or more scoring algorithms 304. Click traffic scoring system 300 may receive user click data 306 that includes information related to the click to be scored. The click traffic scoring system 300 can obtain user click data from the publisher. Click traffic scoring system 300 may also include a click monitoring system for monitoring user clicks and extracting user click data 306 associated with the user clicks. User click data 306 may include a reference URL, cookie data, IP address, geographic location, whether the click was made in response to a query, whether the click was made by an automated script, or other click characteristics. .

The filtering logic 302 may include one or more filters 308 to process the user click data 306. The click traffic scoring system 300 can send the user click data 306 to the filtering logic 302. The filtering logic 302 may generate filter output data based on the user click data 306. The filter output data may include information indicating which combinations of filters have been activated in response to the user click data. The filter output data may also include filter scores corresponding to the outputs generated by the individual filters 308 or by combinations of the individual filters 308.

The click traffic scoring system 300 may apply the filter output data to the scoring algorithms 304 to generate the click score 310 and the confidence interval 312. Scoring algorithms 304 may also generate one or more click classifications 314. The click score 310 may be a numerical value that falls within a continuous numerical range and may represent a relative confidence level that may determine the reliability of the click. Confidence interval 312 corresponds to the click score and may provide additional confidence data related to the click.

Click classification 314 may include one or more classifications assigned to clicks based on filter output data, click scores, and / or confidence intervals. Click classification 314 may indicate whether the click is valid or invalid. Scoring algorithms 304 may apply one or more thresholds to the click score or confidence interval to classify the click as valid or invalid. Scoring algorithms 304 may include pattern recognition algorithms for identifying patterns in filter output data and classifying clicks according to the recognized pattern. Alternatively or in addition, scoring algorithms 304 may be algorithms generated from neural networks including trained neural networks.

One or more of the click score 310, the confidence interval 312, and the click classification 314 may be used by an online publisher, ad network, or other system to determine which clicks should be billed to the advertiser. In providing the click score 310, the system 200 may enable the publisher or other system to implement a more robust and versatile pricing model. For example, the fee paid per click by the advertiser may be a function of the click score 310. Thus, the fee per click may vary depending on the relative confidence dictated by the click score.

4 shows a diagram 400 illustrating the relationship between user intent in ad clicks and click scores generated by a scoring system, such as the click traffic scoring system 150. User intentions may include intentional intention 402 (eg, interested customers) and malicious intent 404 (eg, automated scripts). User click data may include information related to user clicks. The disclosed systems and methods may generate click scores based on user click data via process 200 or the like as described above. Click scores can be calculated as numerical values that fall within the numerical range. In diagram 400, a higher click score corresponds to a higher confidence that the click is a good quality click. Lower click scores correspond to the lower confidence that the clicks are of good quality, or in other words, lower click scores correspond to the greater confidence that the clicks are of poor quality.

The good quality distribution curve 406 represents an exemplary distribution of click scores corresponding to clicks made of the user intent 402 of the line. The degraded quality distribution curve 408 represents an exemplary distribution curve of click scores corresponding to clicks made with malicious or fraudulent user intent 404. The substantial difference between the good quality distribution curve 406 and the degraded quality distribution curve 408 indicates that the click score can obtain a relative confidence in determining the click quality, while effectively and accurately reflecting the user's intent. Each of the two clicks corresponding to the click scores falling within the good quality distribution curve 406 may be identified as valid. However, determining the points along the distribution curve 406 to which the click score belongs may indicate the reliability or strength of the validity identification.

In addition, the provision of the click score may enable the publisher or other system to distinguish between "thrilling" clicks and "obviously valid" clicks and thus treat them differently. The "breathtaking" click may correspond to the click belonging to the overlapping portion 410 of the distribution curves 406, 408. A "clearly valid" click may correspond to a click belonging to a large portion of the distribution curve 406.

5 shows a process 500 for scoring user clicks in a system for adaptive click traffic scoring, such as click traffic scoring system 150. Process 500 may obtain user click data (act 502). Process 500 may obtain user click data from the publisher. Process 500 may also include a click monitoring step for monitoring user clicks and extracting user click data associated with the user clicks. User click data may include a reference URL, cookie data, an IP address, a geographic location, whether the click was made in response to a query, whether the click was made by an automated script, or other click characteristics.

Process 500 may filter the user click data to obtain filter output data (act 504). Process 500 may check whether one or more defined filters have been activated (act 506). If one or more qualifying filters have been activated, process 500 may flag the click as valid (act 508). The confinement filter may be a filter that operates when the click includes a certain characteristic or a combination of characteristics that implies a high level of confidence that the click may be invalid.

For example, an automated script filter that can operate when a click is initiated from a known automated script can be set up as a qualifying filter. The validity of clicks initiated from known automated scripts can be questionable. Thus, if the automated script filter works, process 500 can confidently declare the click as invalid even before calculating the click score.

The confinement filter may also include a combination of filters. In this example, process 500 may declare a click as invalid when certain combinations of filters are activated. That is, clicks may include several suspicious click characteristics, each of which may not be critical to their own invalidity, but cumulative effects may be critical to invalidity.

The qualification filters described herein can be characterized as "negative" qualification filters, that is, when they work, clicks are declared as invalid. Process 500 may also use "positive" limit filters. There may be certain click characteristics that suggest that upon detection, a click may be declared valid with a high level of confidence.

If the definite filters do not work, process 500 may proceed to generate a click score (act 510) and a confidence interval (act 512). If the process declares a click as invalid according to act 508, the process can still calculate the click score and confidence interval associated with the click score. The "invalid" click classification and / or click score and confidence interval may be sent to the publisher, advertiser, ad network, or other system. Click classifications can provide additional information that publishers or other systems can use to construct the advertising fee structure.

6 shows a process 600 for applying a threshold to a click score in a system for adaptive click traffic scoring, such as the click traffic scoring system 150. Process 600 may obtain user click data associated with one or more clicks (act 602). Process 600 may obtain user click data from the publisher. Process 600 may also include a click monitoring step for monitoring user clicks and extracting user click data associated with the user clicks. User click data may include a reference URL, cookie data, IP address, geographic location, whether the click was made in response to a query, whether the click was made by an automated script, or other click characteristics.

Process 600 may apply the user click data to the filtering logic to obtain filter output data (act 604). The filter output data may include filter scores. Process 600 may generate a click score and a confidence interval based on the filter output data (acts 606 and 608).

Process 600 may compare the click score to a threshold (act 610). The threshold may be an effectiveness threshold. If the click score exceeds the validity threshold, process 600 may classify the click as "valid" (act 612). Otherwise, process 600 may classify the click as "invalid" (act 614).

Process 600 may compare the higher end point of the click score confidence interval to a threshold. The threshold may be an effectiveness threshold. If the higher endpoint of the click score confidence interval exceeds the validity threshold, process 600 may classify the click as "valid" (act 612). Otherwise, process 600 may classify the click as "invalid" (act 614).

Valid / invalid classifications, as well as click scores and confidence intervals, may be sent to publishers, ad networks, advertisers, or other systems. Thresholds used to distinguish between valid and invalid clicks may be calculated or extrapolated based on statistical data, or may be set manually according to the needs or needs of publishers, advertisers, advertising networks or other systems.

7 shows a process for applying high and low thresholds to click scores in a system for adaptive click traffic scoring, such as click traffic scoring system 150. Like process 600 shown in FIG. 6, process 700 may acquire user click data (act 702) and apply the user click data to the filtering logic to obtain filter output data (act 704). . Process 700 may obtain user click data from the publisher. Process 700 may also include a click monitoring step for monitoring user clicks and extracting user click data associated with the user clicks. User click data may include a reference URL, cookie data, IP address, geographic location, whether the click was made in response to a query, whether the click was made by an automated script, or other click characteristics. Process 700 may generate a click score (act 706) and a confidence interval (act 708) based on the filter output data.

Process 700 may compare the click score with a high score threshold and a low score threshold (act 710). If the click score exceeds the high click threshold, process 700 may classify the click as "valid" (act 712). If the click score is below the low click threshold, process 700 may classify the click as "invalid" (act 714). If the click score is neither greater than the high click threshold or lower than the low click threshold, the click may be in the "gray area". Process 700 may provide click scores and confidence intervals to publishers, ad networks, advertisers, or other systems. Valid / invalid classifications may be provided to publishers, ad networks, advertisers, or other systems in addition to or instead of click scores and confidence intervals.

Process 700 may also compare the score thresholds using end points of confidence intervals for the click score. For example, if the high end point of the click score confidence interval is below the low click threshold, the click may be marked as "invalid".

The high click threshold and the low click threshold may be set manually by the publisher, advertising network, advertiser, or other system. Alternatively or in addition, the high click threshold and the low click threshold may be obtained from statistical data provided by the publisher or other system. Process 700 may use different high and low thresholds for the filters or combinations of filters. For example, process 700 may identify a filter or combination of filters that acted in response to user click data, and tailor the high and low thresholds to that filter or combination of filters. The high and low thresholds can be values extrapolated from experimental or statistical data. High and low thresholds may also be calculated by learning or by trained algorithms such as neural networks.

The disclosed methods, processes, programs, and / or instructions may be encoded in a computer readable medium such as a signal bearing medium, memory, programmed in a device such as one or more integrated circuits, or processed by a controller or computer. . When the methods are performed by software, the software may reside in a memory or any other type of nonvolatile or volatile memory that is located at or interfaced to the communication interface. The memory may include an ordered list of executable instructions for implementing logical functions. Logic functions may be implemented through digital circuits, through source code, through analog circuits, or through analog sources such as those generated through analog electrical, audio or video signals. The software may be implemented in any computer readable or signal bearing medium for use by or in connection with an instruction executable system, machine, or device. Such a system may include a computer based system, a process containing system, or another system capable of selectively withdrawing instructions from a machine, an instruction executable system, a machine or a device that can also execute instructions.

8 illustrates a computer system implementing a click traffic scoring system 800 that includes a processor 802 coupled to a memory 804. Processor 802 may execute the instructions stored in memory 804 to score click traffic. The click traffic scoring system 800 may communicate with the publisher 806, the advertiser 808, and / or the ad network 810 via the communication network 812.

Memory 804 may store user click data 814 associated with the click. User click data 814 may include a reference URL, cookie data, IP address, geolocation, whether the click was made in response to a query, whether the click was made by an automated script, or other click characteristics. have. User click data 814 may be obtained by monitoring and / or collecting information associated with the click. The processor 802 can execute the click filter program 814 stored in the memory 804. The click filter program 814 can generate the filter output data 818 by applying the user click data to one or more filters. The filter output data 818 may include one or more filter scores 820. Filter output data 818 may include an indication 822 of which filters have been activated in response to user click data 814.

The processor 802 can execute the click scoring program 824 stored in the memory 804. The click scoring program 824 can generate the click score 826 and the confidence interval 828 based on the filter output data 818. The click score 826 may be a numerical value that represents a confidence level that may determine the quality of the click. The click scoring program 824 can determine the confidence interval 828 and the click score 826 based in part on the confidence level 830. Click scoring program 824 may include a default confidence level, such as a default of 95%. The click scoring program 824 can tailor the confidence level 830 to the needs or needs of the publisher 806, the advertiser 808, or the ad network 810.

The click scoring system 830 can also apply the thresholds 832-836 stored in the memory 804 to the click score 826 and / or the confidence interval 828 to generate the click classification 838. Click classification 838 may include information related to whether a click is valid or invalid. Thresholds 832-836 can be validity threshold 832, high click threshold 834, and / or low click threshold 836.

It can be seen from the above description that the click traffic scoring system can provide an improved determination of click quality by scoring clicks using the click score. The click score may enable the publisher or other system to determine with improved confidence whether the click is genuine and can be billed for the relevant advertiser. In providing click scores, the click traffic scoring system also enables publishers, advertisers, ad networks, and / or other systems to tailor the advertising pricing model to the needs or needs of advertisers and publishers, such as through tiered pricing models. can do.

Although selected aspects, features or components of implementations are depicted as being stored in memories, all or some of the systems including methods and / or instructions for performing those methods consistently with a click traffic scoring system may be included in another computer. Readable media such as secondary storage devices such as hard disks, floppy disks, and CD-ROMs; A signal received from the network; Or stored in, or distributed to, or read from other forms of ROM or RAM that are now known or later developed.

Certain components of click traffic scoring system 150 may include additional or other components. A processor may be implemented as a microprocessor, microcontroller, application specific integrated circuit (ASIC), discrete logic, or other type of circuitry or combination of logic. Likewise, the memories can be DRAM, SRAM, flash, or any other type of memory. Parameters (eg, popularity ranking), databases, and other data structures can be stored and managed individually, integrated into a single memory or database, or logically and physically organized in many different ways. Programs or instruction sets may be a single program, portions of separate programs, or distributed across multiple memories and processors.

"Computer-readable medium", "machine-readable medium", "propagation signal" and / or "signal bearing medium" includes software for use by or in connection with an instruction executable system, machine or apparatus, or It may include any means for storing, communicating, propagating, or transporting. The computer readable medium may optionally be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, machine, apparatus, or propagation medium. A non-exhaustive list of examples of machine readable media may include electrical connections (“electronics”) with one or more wires, portable magnetic or optical disks, volatile memory such as random access memory “RAM” (electronics), read only memory ” ROM "(electronic), erasable, programmable read only memory (EPROM or flash memory) (electronic) or optical fiber (optical). Computer-readable media can also include tangible media on which software is printed, which is compiled, and / or interpreted after the software has been stored electronically as an image or in another format (eg, via optical scan). Because it can be processed. The processed medium may then be stored in a computer and / or machine memory.

While various embodiments of the invention have been described, it will be apparent to those skilled in the art that more embodiments and implementations are possible within the scope of the invention. Accordingly, the invention should not be limited except as by the appended claims and their equivalents.

Claims (24)

A method for scoring user clicks, the method comprising: Acquiring user click data associated with the user click; Applying the user click data to a plurality of filters; Identifying a filter combination, the filter combination including filters among the plurality of filters acting in response to the user click data; Generating a click score according to the user click data and an identification of which of the plurality of filters has operated in response to the user click data; And Generating a confidence interval associated with the click score How to include. The method of claim 1, Generating the click score, Generating filter output data, the filter output data being generated according to the user click data; And Applying the filter output data to a scoring algorithm to generate the click score. The method of claim 1, Wherein the plurality of filters comprises an automated script filter that operates when the user click is made by an automated script. The method of claim 1, And the plurality of filters comprises a definitive filter. The method of claim 1, Generating the click score, Acquiring first conversion data including click conversion rates associated with the filter combination; Obtaining second transform data comprising click conversion rates associated with the plurality of filters; And And comparing the first converted data with the second converted data. The method of claim 5, Comparing the first transform data with the second transform data includes determining a ratio of the first transform data to the second transform data. The method of claim 1, Comparing the click score with a threshold; And Classifying the click as valid when the click score exceeds the threshold. The method of claim 7, wherein Wherein the click score indicates a confidence in which the user click is classified. The method of claim 1, Implementing an advertising pricing scheme based on the click score. The method of claim 1, The pricing scheme is a tiered pricing scheme. A click traffic scoring system for scoring user clicks, A processor; And Memory coupled to the processor Including, The memory, User click data providing information related to the user click; A click filter program including instructions for causing the processor to apply the user click data to a plurality of filters and to generate filter output data based on the user click data; And And a scoring program including instructions for causing the processor to apply the filter output data to a scoring algorithm to generate a click score based on the filter output data. The method of claim 11, The scoring program further comprises instructions to cause the processor to generate a confidence interval based on the filter output data. The method of claim 11, The scoring program further includes instructions to cause the processor to identify a filter combination, wherein the filter combination includes filters that operate in response to the user click data. The method of claim 13, The scoring program causes the processor to: Obtain first conversion data comprising click conversion rates associated with the filter combination, Obtain second conversion data comprising click conversion rates associated with the plurality of filters, To compare the first converted data with the second converted data The click traffic scoring system further comprising instructions. The method of claim 11, Wherein the plurality of filters include a first filter corresponding to a first click characteristic, wherein the first filter operates when the user click includes the first click characteristic. The method of claim 13, And wherein the plurality of filters comprise confined filters. The method of claim 16, The click scoring program further includes instructions to cause the processor to classify the user click as invalid when the limiting filter is activated. As a product, Computer readable media; And Programmable instructions stored on the computer readable medium Including, The programmable instructions cause the processor in the click traffic scoring system to: Get user click data associated with the user click, Apply the user click data to a plurality of filters that produce filter output data, the filter output data including an identification of which of the plurality of filters has operated in response to the user click data; And apply the filter output data to a scoring algorithm that generates a click score indicative of the quality of the user click, and a confidence interval associated with the click score. The method of claim 18, Programmable instructions stored on the computer readable medium cause the processor to: Compare the click score with a high threshold and a low threshold, Classify the user click as invalid when the click score is below the low threshold, Product to classify the user click as valid when the click score exceeds the high threshold. The method of claim 18, Wherein said plurality of filters comprises a limiting filter. The method of claim 20, Programmable instructions stored on the computer readable medium cause the processor to: Determine if the user click data caused the limit filter to operate, Product for classifying the user click as invalid when the limiting filter is activated. The method of claim 18, The confidence interval is generated according to a confidence level. The method of claim 18, Said scoring algorithm is a neural network. The method of claim 18, Said scoring algorithm generates a click score according to a continuous numerical range.

Images