EP4646684A1

EP4646684A1 - Generating hybrid data from aggregated data for use in training multi-touch attribution models

Info

Publication number: EP4646684A1
Application number: EP24721797.9A
Authority: EP
Inventors: David Xi-Kuan Chan; Kyra Krishan SINGH; Zdravko Krasimirov DANEV; Rebecca Sun; Peter Michael HUETTL; James Donald BELL
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2024-03-19
Filing date: 2024-03-19
Publication date: 2025-11-12
Also published as: WO2025198582A1

Abstract

The systems and methods can include generating hybrid interaction data. Generating the hybrid interaction data can include accessing a datastore including a one or more identifiers and one or more features associated with impression data. The method includes generating a number of cohort data structures by assigning impressions and pseud-impressions to respective identifiers within the cohorts of data. The impressions include confirmed impressions and pseudo-impressions include aggregated impression data assigned to respective identifiers of each respective cohort. The method includes processing by a machine-learned attribution model deployed by the computing system, the hybrid interaction data to determine an attribution value for the channels.

Description

GENERATING HYBRID DATA FROM AGGREGATED DATA FOR USE IN TRAINING

MULTI-TOUCH ATTRIBUTION MODELS

FIELD

[0001] The present disclosure relates generally to systems and methods for training machine-learned multi-touch attribution models.

BACKGROUND

[0002] Computing devices can perform data processing and run machine learning models. Users can engage in various online activities which can result in exposure of information to the user. Subsequent activities by a user can be influenced by prior activity and information exposure.

SUMMARY

[0003] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or can be learned from the descnption, or can be learned through practice of the embodiments.

[0004] In one example aspect, the present disclosure provides for an example computer-implemented method. The example computer-implemented method includes accessing, by a computing system, a datastore comprising a plurality of data comprising one or more identifiers and one or more features associated with conversion data. The example computer-implemented method includes generating, by the computing system, a plurality of cohort data structures by, for each cohort data structure of the plurality of cohort data structures: initializing, by the computing system, a data structure; and populating, by the computing system, the data structure with k identifiers and associated feature data, wherein the k identifiers are grouped based on the associated feature data. The example computer- implemented method includes generating, by the computing system, match tables based on the conversion data. The example computer-implemented method includes transmitting, by the computing system, a request to a query engine for aggregated impression data associated with each respective cohort data structure of the plurality of cohort data structures. The example computer-implemented method includes accessing, by the computing system, aggregated impression data for each respective cohort data structure. The example computer- implemented method includes generating, by the computing system, a hybrid interaction data comprising impressions associated with known interaction events and pseudo-impressions associated with pseudo-randomly assigned impressions. The example computer-implemented method includes training, by the computing system, a machine-learned attribution model using the hybrid interaction data. The example computer-implemented method includes processing, by the machine-learned attribution model deployed by the computing system, the hybrid interaction data to determine an attribution value for one or more channels.

[0005] In some embodiments of the example method, generating the hybrid interaction data includes: hashing each identifier of the cohort data structure; ordering the hashed identifiers; generating an impression data object for each known interaction event of a plurality of known interaction events; and generating a pseudo-impression data object for each remaining impression of the aggregated impressions by distributing the remaining aggregated impressions across the ordered hashed identifiers.

[0006] In some embodiments of the example method, the aggregated impressions are grouped by a plurality of time periods within a 90-day lookback window .

[0007] In some embodiments of the example method, the plurality of time periods within the 90-day lookback window comprises (i) 1-day time periods for day one to day thirty before a conversion event, (ii) 3-day time periods for day thirty-one to day sixty before the conversion event, and (iii) 5 -day time periods for day sixty-one to day ninety⁷ before the conversion event.

[0008] In some embodiments of the example method, the aggregated impressions for a first time period of the plurality of time periods are distributed across hashed user identifiers for the first time period.

[0009] In some embodiments of the example method, the feature data comprises at least one of (i) country code, (ii) region code, (iii) last-click channel, (iv) date range, (v) conversion ty pe.

[0010] In some embodiments of the example method, the feature data further comprises at least one of (i) days to conversion, (ii) conversion path length, (iii) presence of first part}⁷ click or third party click, (iv) number of different channels associated with a conversion path, or (v) non-suppressed users.

[0011] In some embodiments of the example method, the feature data is selected such that the cohort size of each respective cohort is outside a predetermined threshold of the other respective cohorts. [0012] In some embodiments of the example method, one or more feature data types are selected by: dividing a range of values of each feature into a plurality of buckets; creating cohorts using a single feature of the plurality of features; transmitting a query to a data clean room for a number of aggregated impressions within a lookback window; accessing a match table generated in response to the transmitted query; generating an impression count for each cohort based on the match table; calculating a standard deviation and variance of each respective cohort of the plurality of cohorts; and selecting one or more features based on at least one of (i) the standard deviation or (ii) the variance.

[0013] In some embodiments of the example method, the plurality⁷ of buckets comprises three to five buckets.

[0014] In some embodiments of the example method, the features with the highest standard deviation or variance are selected.

[0015] In one example aspect, the present disclosure provides for an example computer-implemented method. The example computer-implemented method includes generating, by a computing system, hybrid interaction data by: accessing, by the computing system, a datastore comprising a plurality of data comprising one or more identifiers and one or more features associated with impression data; generating, by the computing system, a plurality of cohort data structures by, for each cohort data structure of the plurality⁷ of cohort data structures: initializing, by the computing system, a data structure; populating, by the computing system, the data structure with k identifiers and associated feature data, wherein the k identifiers are grouped based on the associated feature data; and populating, by the computing system, the data structure with aggregated impression data associated with the k identifiers of the respective cohort; and assigning, by the computing system, impressions and pseudo-impressions to respective identifiers within the cohorts of data, wherein the impressions comprise confirmed impressions and wherein the pseudo-impressions comprise aggregated impression data assigned to respective identifiers of each respective cohort of the plurality⁷ of cohorts. The example computer-implemented method includes processing, by a machine-learned attribution model deployed by the computing system, the hybrid interaction data to determine an attribution value for one or more channels.

[0016] In some embodiments of the example method, the accessed datastore is associated with a third-party⁷ computing system, and wherein the plurality⁷ of data comprises k hashed identifiers and aggregated impression data. [0017] In some embodiments of the example method, the features comprise at least one of (i) country, (ii) channel of last click prior to conversion event, (iii) number of distinct channels in path, or (iv) intensify of channel clicks within a lookback window.

[0018] In some embodiments of the example method, each impression of the aggregated impression data comprises a timestamp associated with each impression of a number of impressions.

[0019] In some embodiments of the example method, assigning impressions and pseudo-impressions to a plurality of identifiers is performed pseudo-randomly.

[0020] In some embodiments of the example method, pseudo-randomly assigning impressions to the identifiers comprises: hashing, by the computing system, a plurality of identifiers; populating, by the computing system, the initialized data structure with the plurality of hashed identifiers; and assigning, by the computing system, the aggregated impression data to individual hashed identifiers of the hashed identifiers by distributing the impressions across the plurality⁷ of hashed identifiers.

[0021] In some embodiments of the example method, the assigning the impression data is performed based on time to conversion, wherein an aggregate number of impressions in a time period are determined by time periods within a 90-day lookback window.

[0022] In some embodiments of the example method, the method includes automatically adjusting, by the computing system, responsive to determining the attribution value for the one or more channels, one or more resource allocations to the one or more channels.

[0023] In an example aspect, the present disclosure provides for an example system for generating hybrid data from aggregated data for use in training multi-touch attribution models, including one or more processors and one or more memory devices storing instructions that are executable to cause the one or more processors to perform operations. In some implementations, the one or more memory devices can include one or more transitory or non-transitory computer-readable media storing instructions that are executable to cause the one or more processors to perform operations. In the example system, the operations can include accessing, by a computing system, a datastore comprising a plurality of data comprising one or more identifiers and one or more features associated with conversion data. The operations can include generating, by the computing system, a plurality of cohort data structures by, for each cohort data structure of the plurality of cohort data structures: initializing, by the computing system, a data structure; and populating, by⁷ the computing system, the data structure with k identifiers and associated feature data, wherein the k identifiers are grouped based on the associated feature data. The operations can include generating, by the computing system, match tables based on the conversion data. The operations can include transmitting, by the computing system, a request to a query engine for aggregated impression data associated with each respective cohort data structure of the plurality of cohort data structures. The operations can include accessing, by the computing system, aggregated impression data for each respective cohort data structure. The operations can include generating, by the computing system, a hybrid interaction data comprising impressions associated with known interaction events and pseudo-impressions associated with pseudo-randomly assigned impressions. The operations can include training, by the computing system, a machine-learned attribution model using the hybrid interaction data. The operations can include processing, by the machine-learned attribution model deployed by the computing system, the hybrid interaction data to determine an attribution value for one or more channels.

[0024] In an example aspect, the present disclosure provides for an example transitory or non-transitory computer readable medium embodied in a computer-readable storage device and storing instructions that, w hen executed by a processor, cause the processor to perform operations. In the example transitory’ or non-transitory' computer-readable medium, the operations include accessing, by a computing system, a datastore comprising a plurality of data comprising one or more identifiers and one or more features associated with conversion data. The operations can include generating, by the computing system, a plurality' of cohort data structures by, for each cohort data structure of the plurality' of cohort data structures: initializing, by the computing system, a data structure; and populating, by the computing system, the data structure with k identifiers and associated feature data, wherein the k identifiers are grouped based on the associated feature data. The operations can include generating, by the computing system, match tables based on the conversion data. The operations can include transmitting, by the computing system, a request to a query engine for aggregated impression data associated with each respective cohort data structure of the plurality of cohort data structures. The operations can include accessing, by the computing system, aggregated impression data for each respective cohort data structure. The operations can include generating, by the computing system, a hybrid interaction data comprising impressions associated with known interaction events and pseudo-impressions associated with pseudo-randomly assigned impressions. The operations can include training, by the computing system, a machine-learned attribution model using the hybrid interaction data. The operations can include processing, by the machine-learned attribution model deployed by the computing system, the hybrid interaction data to determine an attribution value for one or more channels.

[0025] Other aspects of the present disclosure are directed to various sy stems, apparatuses, non-transitory computer-readable media, user interfaces, and electronic devices. [0026] These and other features, aspects, and advantages of various embodiments of the present disclosure will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate example embodiments of the present disclosure and. together with the description, serve to explain the related principles.

BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Detailed discussion of embodiments directed to one of ordinary skill in the art is set forth in the specification, which makes reference to the appended figures, in which: [0028] FIG. 1 depicts block diagram for multi-touch attribution with aggregated impressions data according to example embodiments of the present disclosure;

[0029] FIG. 2 depicts an example block diagram depicting group of aggregated impressions based on time periods within a 90-day lookback window according to example embodiments of the present disclosure;

[0030] FIG. 3 depicts an example block diagram depicting group of a cohort with unallocated aggregated impressions according to example embodiments of the present disclosure;

[0031] FIG. 4 depicts an example block diagram depicting group of a cohort with allocated pseudo-impressions according to example embodiments of the present disclosure; [0032] FIG. 5 depicts an example block diagram depicting allocation of impressions and pseudo-impressions according to example embodiments of the present disclosure;

[0033] FIG. 6 depicts method flowchart of an example method according to example embodiments of the present disclosure; and

[0034] FIG. 7 depicts a block diagram of an example system for performing multitouch attribution with aggregated impressions data according to example embodiments of the present disclosure. DETAILED DESCRIPTION

[0035] Generally, the present disclosure is directed to generating and utilizing hybrid interaction data with multi-touch attribution models. Multi-touch attribution models obtain user-level interaction data to determine a fractional credit of the conversion to each respective interaction event. This requires user-level data to be accessible. Currently, privacy concerns and limitations provide for little to no access to user-level interaction data (e.g., due to the deprecation of third-party cookies). As such, a new means for approximating user-level interaction data (e.g., impressions, clicks, and the like) and conversion data is needed to continue utilization of the existing multi-touch attribution models. The present disclosure provides for improved systems and methods to allow for continued use of existing models, approximating user-level impression data using aggregate data (e.g., from data clean rooms), and conserving privacy for user-level data by maintaining k-anonymity of impression datasets.

[0036] While click data or other first party data can still be accessible on a user-level, other impression data is less accessible. For instance, impression data may only be available at an aggregated level (e.g., via a data clean room). As third party cookies are deprecated, there is no longer access to individual information relating to impressions of various content. This creates a problem because there is no longer impression level data accessible on a userlevel across parties associated with distinct channels. As such, alternative methods are needed for utilizing aggregate data to provide meaningful insights into the efficiency of resource allocation for various media channels to enhance future resource allocation to the various media channels. Existing approaches simply use aggregated data and derived features from the aggregated data as input into models to determine attribution credit. These existing approaches result in a loss of information as the user-level data is no longer available. The present disclosure improves upon those approaches by selectively determining cohorts of identifiers to access aggregate-level impression data from cleans rooms. The cohorts are generated to optimize the tradeoff between requiring a certain number of users within the cohort to comply with privacy requirements while also providing for meaningful groupings based on feature data as discussed further within.

[0037] The present disclosure provides for solutions to these problems by generating hybrid data. This approach combines user-level click data with aggregated impression data to process it in such a way that existing multi-touch attribution models can be used. This allows for continuing to use user-level conversion data for the existing multi-touch attribution models. More particularly, the present disclosure provides for generation of hybrid datasets by assigning impressions associated with last click data or other known impression data and pseudo-randomly assigning pseudo-impressions to approximate distribution of impressions within a cohort at the user-level. Thus allowing for individual user-level paths which can be used within the multi-touch attribution models.

[0038] Improvements are achieved through both the cohort creation based on features conversion data (e.g., country, channel associated with last click, and the like) and the allocation of pseudo-impressions pseudo-randomly at the user-level within the cohort. This disclosure provides for enhanced hybrid data generation as well as training and tuning the multi-touch attribution models using the hybrid data and ground truth data. The present disclosure provides for a solution to these problems by allowing for meaningful computation while preserving the privacy of individual user data. This can allow- for the use of existing models for generating useful computations instead of requiring the deprecation or modification of such models. Particularly, the proposed approach is compatible with probabilistic models based on averages. This is due to the fact that if impressions, on average, are allocated correctly across user paths, reasonable multi-touch attribution insights can be generated using existing multi-touch attribution models. For instance, the present disclosure can provide for validating the utilization of the hybrid data by utilizing the hybrid data as input in the multi-touch attribution model, utilizing ground truth data in the muti-touch attribution model and optimizing the cohort selection process or hybrid data generation to reduce the difference between output (e.g., attribution credit for different channels) of the hybrid data and the ground truth data.

[0039] The hybrid data described herein can be utilized by multi-touch attribution models which predict credit based on live user session data. The system can generate automated suggestions based on the characteristics of live user session data including proposals for participation in real-time content selection processes (e.g., real-time bidding). The hybrid data provides for maintaining the utility of existing models as traditional input data for the models is no longer available (e.g., due to the deprecation of third party’ cookies and other privacy measures). This is performed by improving the generation of cohorts to group aggregated data in such a way that it produces the most meaningful results when used to generate hybrid data. For instance, multiple potential cohorts can be generated and compared to determine which groups of users provide for the most distinctive characteristics as compared to other cohorts. This can allow for optimizing the tradeoff between k- anonymity of users within a cohort and generating groups of users with shared characteristics such that there is not an over or undersaturation of aggregated impressions to allocate among the users within the cohort. Further, the present disclosure provides for improving both cohort selection and/or creation as well as hybrid data generation by comparing results associated with the cohort selection and hybrid data generation with ground truth data and continually updating the cohort selection and hybrid data generation based on the ground truth data (e.g., a feedback loop using ground-truth data).

[0040] The improvements associated with the systems and methods discussed herein can be further understood with reference to the figures. Reference now is made to the figures, which provide example arrangements of computing systems, model structures, and data flows for illustration purposes only.

[0041] FIG. 1 illustrates an example data flow 100 to perform cohort generation and hybrid interaction data generation for use in a machine-learned attribution model according to example embodiments of the present disclosure. Data flow 100 can include conversion and non-conversion data selection pipeline 102, cohort generation pipeline 105, match table generation pipeline 112. query engine 115, impression allocation pipeline 125. model training pipeline 142, and machine-learned attribution pipeline 145.

[0042] Conversion and non-conversion data selection pipeline 102 can include selecting conversion data and selecting non-conversion data. For instance, conversion data can be associated with data indicative of performance of a particular action. In some instances, an action can include creating an account, signing up for a newsletter, making a purchase, spending a certain amount, reaching a checkpoint of a signup flow, or any other action. A non-conversion data can be associated with the absence of the performance of a particular action within a predetermined number of days of a last touchpoint. For instance, a number of days can be a week, a month, 30-days, 90-days, or any other predetermined set of time. A touchpoint can be a last known interaction (such as data associated with a known click).

[0043] Cohort generation pipeline 105 can include generating cohorts of identifiers based on the obfuscated identifier data 104 and similarity of the obfuscated identifiers’ respective feature data. The identifiers can be grouped based on feature data associated with the identifiers such that the identifiers within the cohort share enough feature data within one another and are distinguishable from other cohorts such that meaningful insights can be derived from the cohort’s creation. The cohort generation pipeline will be discussed further with regard to FIG. 2 and FIG. 7. Cohort generation pipeline 105 can generate cohort data 110 as output. Cohort data 110 can include for each cohort of a plurality of cohorts obfuscated identifiers, associated feature data, and conversion data mapped to the respective cohorts.

[0044] Match table generation pipeline 112 can obtain cohort data 110 as input and generate match tables 113 which can include conversion data mapped to cohort data 110. The conversion data can be generated and stored based on actions performed and associated feature data which can be logged. The match table generation pipeline 112 can include grouping data associated with users together into cohorts. The cohorts can be grouped based on strong signals associated with the members of the cohort. In some instances, the signals can be indicative of an intent to convert or not-convert. For instance, based on historical data associated with users, conversion data, feature data, the system can predict a category of likely to convert or not likely to convert. This can be utilized to group identifiers with similar associated data together into cohorts. These cohorts (and associated cohort data 110) can be utilized by the clean room and match generation pipeline 112 to match tables 113 which can include conversion data mapped to cohort data 110.

[0045] The clean room can utilize the conversion data mapped to cohort data 110 to generate match tables and conversions within the clean room such that no personally identifiable information is accessible and only aggregate data can be obtained.

[0046] Query engine 115 can join conversion data mapped to cohort data with private event-level data 1 14 using match tables 1 13 inside the clean room to generate aggregated impression data 120 for the respective cohorts. Query engine 115 can be associated with a cloud provider system which allows for third party computing systems to join their known interaction data with the cloud provider’s event-level data in a secure and privacy compliant manner. For instance, the third party computing systems can query against joined datasets managed by the cloud provider computing system subject to privacy constraints such as k- anonymity. The k-anonymity provides that any row from a result set of a query must have at least 50 identifiers contributing to the results (otherwise the row is filtered out or otherwise removed). The cloud provider computing system can additionally log queries such that if a query joins against prior queries, it does not violate privacy constraints. The query can result in aggregated impression data 120 which is broken into time windows. Aggregated impression data can include a total count of impressions that satisfy certain criteria of a cohort. For instance, the aggregated impression data 120 can include a count of the number of impressions which occurred, as a whole, for the members of the cohort (e.g., a sum) for each time window. For instance, if there were 5,000 identifiers in a cohort and there were aggregated impressions for the cohort, there would not be data about which identifiers are associated with which impressions

[0047] Impression allocation pipeline 125 can access the aggregated impression data 120 and generate hybrid interaction data 130 as output. The aggregated impression data can include a count of a number of impressions associated with a set of characteristics of a cohort. For instance, aggregated impression data for a first cohort can include the number of total impressions that occurred during various time slices in a lookback window. The aggregated impression data can be allocated amongst identifiers (e.g., obfuscated user identifiers) within the cohort in a pseudo-random manner. For instance, each individual impression of the aggregated impression data can be allocated to individual identifiers. The proposed approach will be discussed further herein with regard to FIG. 2 to FIG. 7. The allocation for each cohort will include impression data 135 which pairs the aggregated impressions for time windows with known interaction events (e.g., clicks) and pseudoimpression data 140 which allocates the remaining aggregated impressions for the time window s to identifiers (e.g., obfuscated user identifiers) in the cohort using a pseudo-random method. The hybrid interaction data 130 includes obfuscated identifiers which have been assigned impressions and pseudo-impressions for the respective time windo s. By w y of example, hybrid interaction data can include known click data which is allocated to identifiers with known click instances and pseudo-impressions can be allocated amongst the identifiers as described herein. Thus, hybrid data can include both known click events which are labeled as ‘‘impressions’' and approximated “pseudo-impressions'’ which have been allocated pseudo-randomly to individual identifiers of a cohort. This approximated impression data associated with the cohorts can be used as input into model training pipeline 142 to train the trained machine-learned model 143. The trained machine-learned model 143 can be used in machine-learned attribution pipeline 145 to glean insights on the attribution credit which should be assigned to various media channels.

[0048] Machine-learned attribution pipeline 145 can include trained machine-learned model 143. For instance, the trained machine-learned model 143 can include an attribution model such as a data driven attribution model which uses probabilistic functions to determine a credit for each respective channel and impression along a user journey within a particular lookback window. The attribution credit can be utilized for adjusting future resource allocation to result in better interactions which in turn result in less wasted resources from transmitting data. For instance, this structure can provide for automated adaptation of the real-time machine-learned models for improved population of user interface input element slots. For example, by presenting input elements (e.g., content items) that a user is more likely to interact with, the limited user interface (e.g., screen space) can be used to optimize user input. Thus, the system can adapt a user interface of a computing device to present items that are actually relevant to a user’s tasks or goals for using the computing device. For example, user activity may provide one or more signals that a particular input element would be relevant to accessing a resource of interest or performing a task at hand. However, systems and methods of the present disclosure can, in some embodiments, determine that rendering that particular input element would not be of sufficient incremental value to effectively improve the user interface (e.g., the user already has access to or otherwise is already- navigating toward the resource of interest). Thus, systems and methods according to the present disclosure can, in some embodiments, update a machine-learned model for generating a proposal that de-prioritizes transmission of that input element to avoid wasting resources on rendering that input element (e.g.. content item, advertisement).

[0049] FIG. 2 to FIG. 5 depict an illustrative example of allocating impressions and pseudo-impressions among individual identifiers within respective cohorts.

[0050] As depicted in FIG. 2, each cohort can be associated w ith aggregated impression data. The aggregated impression data can be divided into lookback windows based on cohorts. For instance, first lookback window 205 can be associated with the day of the conversion event 255 to 30 days before the conversion event 260. First lookback window' 205 can be divided into 1-day increments such that individual impressions can be allocated among the individual days leading up to the respective conversion event 260. Second lookback window 210 can be associated with 30 days before the conversion event 260 to sixty days before the conversion event 265. Second lookback window 210 can be divided into 3-day increments such that individual impressions can be allocated within 3-day spans within the second lookback window 210. Third lookback window 215 can be associated with sixty days before conversion event 265 and ninety days before conversion event 270. Third lookback window 215 can be divided into 5-day increments such that individual impressions can be allocated within 5-day spans within the third lookback window' 215. The lookback window s can be broken into shorter increments the closer to conversion event due to the increase in density of impression events that is observed closer to the time of conversion. As such, breaking the lookback windows into these time periods allows for a more accurate distribution of event density within any given time period. In some implementations, the duration of the lookback windows can be dynamically adjusted based on a density of events with the respective windows. The adjustment of the duration can be determined to improve or otherwise optimize between granularity of data and providing for privacy filters.

[0051] The cohorts can include N number of cohorts. N can be any number of cohorts. FIG. 2 depicts four out of N number of cohorts. While the figure illustrates this number of cohorts, it can be understood that any number of cohorts can be generated (e.g., (10, 100, 1000, and the like). For instance, as depicted in FIG. 2, the cohorts can include cohort A 220, cohort B 225, cohort C 230, and cohort N 235. Each cohort can be made up of identifiers with similar feature data. For instance, the cohorts can be defined in such a way that the cohorts group identifiers based on how similar the user paths within the cohort are being exposed to impressions. Thus, the cohorts are generated based on feature data associated with the user identifiers and user paths.

[0052] Feature data can include (i) country code, (ii) region code, (iii) last-click channel, (iv) date range, (v) conversion type. Additionally, or alternatively, feature data can include (i) days to conversion, (ii) conversion path length, (iii) presence of first party click or third party click, (iv) number of different channels associated with a conversion path, (v) suppressed users, or (vi) other feature data. A country code can be an identifier associated with a country. In some instances, individual countries may have too little data to satisfy privacy requirements, as such, region code can be used for combinations of countries. Lastclick channel can include a channel in which the last-click prior to a conversion (or nonconversion) event occurred. Channels can include organic (e.g.. search), direct, paid media, in product promotion, or other channels. A date range can include a range of dates from the conversion to the interactions (e.g., interactions occurring within one day, one week, one month, and the like from the conversion). A conversion ty pe can include a type of conversion such as a sign up, purchase, account creation, activation, or reactivation of a service, or nonconversion (e.g., no action within 30 days of the last-click or last known interaction event). Days to conversion can include a number of days between the click or other known interaction event and the conversion event. Conversion path length can include a number of clicks or other know n interaction events associated with the user path. Presence of a first party click can include whether a user path included a first party' click (e.g., a click associated with the first party data) or third party click (e.g., click associated with third party data). Number of different channels associated with a conversion path can include a number of distinct ty pes of channels associated with the conversion path. Suppressed users can include users which are not included in the data due the user being suppressed from certain content campaigns (e.g., advertisement campaigns) for reasons such as being an existing customer, frequent visitor, or otherwise denoted as a suppressed user.

[0053] Features used to generate the cohorts can be selected such that the features provide for cohorts that divide users into multiple groups. The size of the groups can be determined such that the exposure to certain content exceeds a predetermined among such that a k > 50 privacy filters (where k is a number of user identifiers associated with the group) are satisfied at a granular level (e.g., one to three day time duration window s) while also not being so large in size that there is no distinction between cohorts. If there is no distinction in the cohorts, there is no signal difference between the cohorts which means the cohorts provide no additional benefit to utilizing aggregated data of the entire set of users compared to dividing the users into the respective cohorts. In some implementations, the number of impressions can be normalized by dividing it by the number of users in the cohort (e.g., group) and measuring the difference in number of impressions (e.g., per 100 users or 1000 users) in each cohort. The feature effectiveness can be determined and utilized to adjust cohort creation. For instance, the feature data ty pes used to group the identifiers and associated impression data into cohorts can be selected by dividing a range of values of each feature into a plurality of buckets (e.g., three to five buckets). Candidate cohorts can be created using a single feature of the plurality' of features. The system can transmit a query to a data clean room (e.g., associated with the cloud provider computing system) for a number of aggregated impressions within a lookback window' for the respective cohort. For instance, the lookback window can include a number of impressions for the past 30 days, 60 days, or 90 days. An impression count for each cohort for the lookback window can be calculated byjoining the obfuscated identifiers in the cohort with the private event-level data in the clean room. For each cohort, the standard deviation or variance of the normalized number of impressions for each cohort can be determined. Based on the standard deviation of the features, one or more features can be selected for cohort creation. This process of determining the standard deviation/variance of impressions per feature can be periodically performed to ensure that the features associated with the largest standard deviation/variance are selected for cohort generation. [0054] The cohorts can be used to query against the data clean room to obtain a total number of impressions across different time periods (e.g.. time slices). It can be observed that generally, the frequency of impressions increases closer to the conversion event time. As such, it is advantageous to divide a lookback window into slices of increasing length, for example, from day zero to day thirty into 1-day time periods, a lookback window of day thirty-one to day sixty into 3-day time periods, and a lookback window of day sixty -one to day ninety into 5-day time slices.

[0055] As depicted in FIG. 2, each time window can include a number of impressions to be distributed. For instance, base icon 250 can be associated with one impression or some other base number of impressions (20, 100, 1000, and the like). Icon 245 can be associated with five times the number of base impressions associated with base icon 250. Icon 240 can be associated with five times the number of impressions associated with icon 245 (e.g., twenty -five times) the number of impressions as the base impressions associated with base icon 250. The icons can be associated with any multiple of the base number of impressions and are used for illustrative purposes only. As such, some time windows associated with respective cohorts can include zero aggregated impressions to allocate within the respective cohort and some of the time windows can include large numbers of impressions to allocate within the respective cohort.

[0056] Each cohort can include a number of identifiers associated with individual users. The identifiers can be anonymized or otherwise obfuscated such that they cannot be tied back to a known identifier or known user. FIG. 3 depicts an example graphical depiction of cohort A 305 which can include user A 310, user B 315, user C 320, and user k 325. Each user can be associated with a conversion event and a known interaction event. For instance, the known interaction event can be a known click.

[0057] By way of example, the graphical depiction shows example breakdowns of times within a 90-day lookback window- from the respective conversion event. For instance, user A 310 can be associated with conversion event 330. Additionally, user A 310 can be associated with known interaction event 350A and known interaction event 350B which occurred during one day before conversion event 330.

[0058] User B 315 can be associated with conversion event 335. Additionally, user B 315 can be associated with known interaction event 350C occurring two days before conversion event 335 and known interaction event 350D occurring three days before conversion event 335. [0059] User C 320 can be associated with conversion event 340. Additionally, user C 320 can be associated with know n interaction event 350E occurring within one day of conversion event 340 and known interaction event 350F occurring between 30 and 60 days before conversion event 340.

[0060] User k 325 can be associated with conversion event 345. Additionally, user k 325 can be associated with know n interaction event 350G occurring within one day of conversion event 345 and known interaction event 350H occurring between 60 and 90 days before conversion event 345.

[0061] FIG. 3 additionally depicts a number of aggregated impressions to distribute during each time window' of the 90 day lookback window. For instance, aggregated impressions 355A are to be distributed amongst the k users during the window being within one day of the conversion event, aggregate impression 355B are to be distributed amongst the k users during the window' from one to tw o days before the conversion event, aggregated impressions 355C are to be distributed amongst k users during the window from a respective 1-day window between two days and 30 days before the conversion event. Impressions 355D are to be distributed amongst the k users during a respective 3-day window from 30 to 60 days before the conversion event. Impressions 355E are to be distributed amongst the k users during a respective 5-day window' from 60 to 90 days before the conversion event.

[0062] The graphical depiction in FIG. 3 can be represented by a number of data structures that are written and read by components of the system. For instance, the system can generate data structures for each respective cohort by initializing a data structure and populating it w ith data associated w ith identifiers that have similar feature data. The data structure can include a row- for each respective user of the plurality of users and can include conversion event data, known interaction event data, and feature data.

[0063] For instance, each cohort can be associated with a plurality of data objects within a data structure (e.g., table). Of the aggregated impression volume, individual impressions can be allocated among users within the given cohort and time slice. Deterministic heuristics can be utilized to assign the impressions. In some instances, the heuristics can include assigning known interaction events, such as known clicks with an impression. In some instances, the aggregated impression data can also include a count for reach or frequency of impressions. By w ay of example, a total number of impressions for a time slice can be five with two users having two impressions and one user having one impression. As such, if the impressions are to be distributed across a cohort of six users, a first user can be assigned two of the impressions, a second user can be assigned two of the impressions, a third user can be assigned one of the impressions, and the fourth, fifth, and six users would be allocated zero impressions. This is compared to an instance where reach or frequency count is not included in the query and would result in users one to five being allocated a single impression while user six is allocated zero impressions.

[0064] In some instances, ground truth panel data can be used to modify a probability associated with a user being allocated an impression. Thus, the ground truth panel data can be used to update the heuristics and allocation of aggregated impressions among individual users. This can include adjusting allocation based on geo-signals, suppression lists, or other high level aggregated statistics.

[0065] As described with regard to icons 250. 245, and 240 in FIG. 2, icons such as base icon 370, icon 365, and icon 360 in FIG. 3 can represent a volume of aggregated impressions to distribute.

[0066] The impression allocation pipeline can distribute the aggregated impressions 355A to 355E to users to create impressions and pseudo-impressions. For instance. FIG. 4 depicts example user paths, each path including distributed impressions and pseudoimpressions across a number of users within cohort A 405. For instance, cohort A 405 can include user A path 410, user B path 415, user. .. path 420, and user k path 425. The known interaction events 450A to 450H are depicted and align with the distributions of known interaction events 350A to 350H as depicted in FIG. 3. FIG. 4 further depicts the distribution of icons (e.g., base icon 470, icon 465, and icon 460) representing a volume of aggregated impressions which have been distributed from aggregated impressions 455A to 455D.

[0067] As depicted in FIG. 4, in some instances the distributed impressions align with known interaction events. For instance, in user A path 410, there are two impressions within one day of conversion event 430. As such, each of the two impressions that are allocated from aggregated impressions 455A are labeled as an actual impression. For instance, the data structure associated with cohort A, user A path can be updated to include an impression event with an associated time that is near the time of the known interaction event. Impression event data can be generated and populated within the data structure indicating an impression for known interaction event 450A and known interaction event 450B. Additionally, user A path 410 can include a number of pseudo-impressions w hich have been pseudo-randomly assigned. For instance, two of aggregated impressions 455B can be allocated to user A path 410, one of aggregated impressions 455C can be allocated to user A path 410, two of aggregated impressions 455D can be allocated to user A path 410, and one of aggregated impressions 455E can be allocated to user A path 410. For each respective pseudoimpression, a pseudo-impression data object can be added to the cohort’s data structure. For instance, if there is one pseudo-impression event, it can have a timestamp generated that places the pseudo-impression in the middle of the time frame (e.g., 12:00 PM noon if the window is a 1-day window, 1.5 days in if the window is a three-day window, and the like). In some implementations, the pseudo-impressions can be assigned times based on original timestamps of the aggregated impressions. If there is more than one impression within a time window , the impressions can be distributed such that they are evenly distributed within the time window. For instance, if the time window' is a 1-day window-, and there are 3 impressions, they can be allocated at 6:00 AM (06:00:00), 12:00PM (00: 12:00) and 6:00 PM (18:00:00). The potential distribution methods are described for exemplary purposes only and are not meant to limit the disclosure. In some implementations, for impressions that are allocated, the timestamp can be the timestamp of the known interaction event offset by a fixed interval of time (e.g.. millisecond, second, minute, and the like). In some implementations, pseudo-impressions can be given a time assigned to the distance in days before the conversion event at 12:00 PM noon.

[0068] As described for user A path 410, each of the other user paths can have associated data objects in the cohort’s hybrid interaction data. Hybrid interaction data can include data such as impressions, conversions, clicks, or other data indicative of an interaction with a content item. For instance, user B path 415 associated with conversion event 435, user. . . path 420 associated with conversion event 440, and user k path 425 associated with conversion event 445.

[0069] FIG. 5 depicts a further example of user k path 425 described in FIG. 4. FIG. 5 depicts a user path k 500 for user k. The user path is depicted over a 90-day lookback window and includes known interaction events such as known click 530 and known click 535. Known click 530 occurred within one day of conversion event 505 and known click 535 occurred between 60 and 90 days before conversion event 505. As described in FIG. 4, aggregated impressions can be distributed among various user paths. An aggregated impression that is allocated in a time window with a known interaction event is labeled as an impression event. For instance, impression 510 is associated with known click 530 and impression 525 is associated with known click 535. Additionally, a user path can include allocated aggregated impressions that are not associated with a known interaction event. These allocated aggregated impressions are labeled pseudo-impressions. User path k 500 includes pseudo-impression 515 and pseudo-impression 520. It is possible that there could be more impressions or pseudo-impressions along user path k 500 which are not depicted but would be present in a hybrid interaction data structure.

[0070] The hybrid interaction data structure includes a number of approximated user paths which can be utilized as input into an attribution pipeline. For instance, the attribution pipeline can be a machine-learned attribution pipeline which can include a machine-learned data driven attribution model. The hybrid interaction data includes approximated user-level known interaction events (e.g., clicks), impression, and conversion data. The approximated user-level data can be used as input into a multi-touch attribution model, such as a data- driven attribution model. The data-driven attribution models can be used to model an intensity of a path to conversion given the characteristics of the path. The data-driven attribution models can determine an incremental credit for each impression or pseudoimpression and allocate credit to the channel associated with the impression or pseudoimpression. In some implementations, the model can include a consideration of time-to-event such that touchpoints that are further back from the conversion event receive less credit than those closer in time to the conversion event. If there is a single impression event, the single impression event receives all credit for the conversion.

[0071] FIG. 6 is a flow diagram of an example method 600 to generate hybrid data from aggregated impression data for use in multi-touch attribution models in accordance with some embodiments of the present disclosure. The method 600 can be performed by processing logic that can include hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e g., instructions run or executed on a processing device), or a combination thereof. In some embodiments, the method 600 is performed by server computing system 702. Although show n in a particular sequence or order, unless otherwise specified, the order of the processes can be modified. Thus, the illustrated embodiments should be understood only as examples, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. Additionally, one or more processors can be omitted in various embodiments. Thus, not all processes are required in every embodiment. Other process flows are possible.

[0072] At operation 602, processing logic can access a datastore including a plurality of data including one or more identifiers and one or more features associated with conversion data. The feature data can include feature data described herein. For instance, the feature data can include at least one of (i) country code, (ii) region code, (iii) last-click channel, (iv) date range, (v) conversion type. Additionally, or alternatively, the feature data can include at least one of (i) days to conversion, (ii) conversion path length, (iii) presence of first party click or third party⁷ click, (iv) number of different channels associated with a conversion path, or (v) suppressed users.

[0073] At operation 604. processing logic can generate a plurality of cohort data structures. For each cohort data structure of the plurality of cohort data structures, processing logic can initialize a data structure and populate the data structure with k identifiers and associated feature data. The k identifiers are grouped based on the associated feature data. In some implementations, each cohort of the plurality of cohorts can include at least 50 identifiers, 100 identifiers, 1,000 identifiers, 10,000 identifiers or any number of identifiers. [0074] The cohort data structures can be generated based on selected feature data (e.g., feature data associated with conversions). The feature data can be selected such that the cohort size of each respective cohort is within a predetermined threshold of the other respective cohorts. The one or more feature data types can be selected. For instance, processing logic can divide a range of values of each feature into a plurality of buckets. For instance, the buckets can include three to five buckets. Processing logic can create cohorts using a single feature of the plurality of features. Processing logic can execute a query to a data clean room for a number of aggregated impressions per cohort within a lookback window . The query' engine can return an impression count for each cohort using the private event-level data and the generated match table. Processing logic can calculate a standard deviation and variance of the number of impressions for each respective cohort of the plurality of cohorts. Processing logic can select one or more features based on at least one of (i) the standard deviation or (ii) the variance. For instance, the features with the highest standard deviation and/or highest variance can be selected.

[0075] In some implementations, processing logic can generate a number of cohort data structures. Generating the number of cohort data structures can include, for each cohort data structure of the number of cohort data structures initializing a data structure. For each cohort data structure, populating the data structure with k identifiers and associated feature data, the k identifiers are grouped based on the associated feature data. For each cohort data structure, populating the data structure with aggregated impression data associated with the k identifiers of the respective cohort. [0076] At operation 606, processing logic can transmit a request to a query engine for aggregated impression data associated with each respective cohort data structure of the plurality of cohort data structures.

[0077] At operation 608, processing logic can access aggregated impression data for each respective cohort data structure. For instance, an impression allocation pipeline can read aggregated impression data. For instance, aggregate impression data can include conversion event time, surrogate identifier, or other relevant data. For instance, aggregated impression data can include a plurality of rows of data including a cohort identifier, inventory type, channel, distance days (e.g., time to conversion event), impression count (number of impressions), users count (number of users the impressions are allocated across).

[0078] At operation 610. processing logic can generate a hybrid interaction data including impressions associated with known interaction events and pseudo-impressions associated with pseudo-randomly distributed impressions. For instance, processing logic can generate user interaction data for each impression and pseudo-impression allocated to the surrogate identifiers. Generating the hybrid interaction data can include reconciling known impression data (such as known click data) and impressions and pseudo-impressions such that proper allocation and deduplication occurs. The hybrid interaction data structure can include a user interaction type label (e.g., impression for impression and pseudo-impression for pseudo-impression) and a cohort id (e.g., an identifier associated with the cohorQ.The data structure can include, for instance, a user interaction identifier (e.g., a generated unknown user identifier for each interaction which is created), a user interaction type (e g., impression or pseudo-impression), interaction time (e.g., time stamp), identifier (e.g., identifier associated with the first party' performing the computation based on known interaction events), campaign identifier, country code, and cohort identifier.

[0079] In some implementations, processing logic can assign impressions and pseudo-impressions to respective identifiers within the cohorts. The impressions include confirmed impressions and pseudo-impressions include aggregated impression data assigned to respective identifiers or each respective cohort of the number of cohorts. Assigning impressions and pseudo-impressions to a plurality of identifiers can be performed pseudo- randomly. Pseudo-randomly assigning impressions to the identifiers can include hashing a plurality of identifiers. Pseudo-randomly assigning impressions to the identifiers can include populating the initialized data structure with the plurality of hashed identifiers. Pseudo- randomly assigning impressions to the identifiers can include assigning the aggregated impression data to individual hashed identifiers of the hashed identifiers by distributing the impressions across the plurality of hashed identifiers. For instance, the hashed identifiers can be ordered and each respective impression of the aggregated impression data can be assigned to respective hashed identifiers. Assigning the impression data can be performed based on time-to-conversion. An aggregate number of impressions in a time period can be determined by time periods within a 90-day lookback window.

[0080] Processing logic can hash each identifier of the cohort data structure. For instance, the identifier (e.g., surrogate identifier) can be concatenated with associated feature data and this concatenated value can be hashed. Processing logic can order the hashed i den ti Tiers. This allows for pseudo-random ordering such that evenly distributing aggregated impressions among the hashed identifiers can be done in a repeatable manner such that the way impressions are allocated does not change over consecutive reruns of the pipelines that created the impressions. Due to cohort creation based on conversion events in the past (e.g., and associated estimated timestamps), they will not change unless there is an erasure of the data. Processing logic can generate an impression data object for each known interaction event of a plurality of known interaction events.

[0081] Processing logic can generate a pseudo-impression data object for each remaining impression of the aggregated impressions by distributing the remaining aggregated impressions across the ordered hashed identifiers. The aggregated impressions can be grouped by a plurality of time periods within a 90-day lookback window. In some implementations, the plurality of time periods within the 90-day lookback window can include (i) 1-day time periods for day one to day thirty before a conversion event, (ii) 3 -day time periods for day thirty-one to day sixty before the conversion event, and (iii) 5-day time periods for day sixty-one to day ninety before the conversion event. The aggregated impressions for a first time period of the plurality of time periods are distributed across hashed user identifiers for the first time period.

[0082] For instance, for each time period, processing logic can assign the remaining number of pseudo-impressions (e.g.. number of aggregated impressions minus number of impressions allocated) as pseudo-impression events to the user interaction evenly. Due to the cohort being ordered based on the hashed identifier (e.g., pseudo-randomly ordered), the first N number of identifiers can be allocated pseudo-impressions.

[0083] Processing logic can generate hybrid interaction data. For instance, processing logic can access a datastore including a number of data including one or more identifiers and one or more features associated with impression data. The accessed datastore can be associated with a third-party computing system. The plurality of data can include k number of hashed identifiers and aggregated impression data. The features can include at least one of (i) country, (ii) channel of last click prior to conversion event, (iii) number of distinct channels in path, or (iv) intensity of channel clicks within a lookback window . Each impression of the aggregated impression data can include a timestamp associated with each impression of a number of impressions.

[0084] At operation 612, processing logic can process the hybrid interaction data to determine an attribution value for one or more channels. In some instances, processing the hybrid interaction data can include generating conversion sequences and conversion events (e.g., as depicted in FIG. 5) by a conversion sequence generator pipeline. The conversion sequences and conversion events can be generated based on the hybrid interaction data.

[0085] In some implementations, processing logic can process, by a machine-learned attribution model, the hy brid interaction data to determine an attribution value for one or more channels. For instance, the machine-learned attribution model can be deployed by a computing system. For instance, processing logic can automatically adjust, responsive to determining the attribution value for the one or more channels, one or more resource allocations to the one or more channels.

[0086] Figure 7 depicts one example system 700 for generating hybrid data from aggregated impression data for use in multi-touch attribution models. The example system 700 can include a server computing system 702, a client computing system 704, a training computing system 706, and/or a cloud provider computing system 760.

[0087] The server computing system 702 can include one or more processor(s) 708. The server computing system 702 can include memory 710. The memory 710 can include data 710A and/or instructions 710B. The server computing system 702 can include a cohort generation pipeline 712, impression allocation pipeline 714, and one or more data store(s) 715. The one or more processors 708 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory 710 can include one or more non -transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 710 can store data 710A and instructions 710B which are executed by the processor 708 to cause the server computing system 702 to perform operations. [0088] The server computing system 702 can include one or more data store(s) 715. Data store(s) 715 can store cohort data 716, hybrid interaction data 717, conversion event data 713, conversion sequence data 719, machine-learned model(s) 725, and other data. For instance, other data can include conversion event data, conversion sequence data, or aggregated impression data. The cohort generation pipeline 712 can obtain conversion event data 713, conversion sequence data 719. or other data to generate cohort data 716. Cohort data 716 can include, for each cohort, a cohort identifier, k user identifiers, feature data associated with each respective user identifier of the k user identifiers. For instance, cohort data 716 can include conversion data associated with cohorts.

[0089] The server computing system can make a call to the cloud provider computer system 760 to initiate query engine 772. Query engine 772 can obtain cohort data 716 as input and provide aggregated impression data as output. Server computing system 702 can utilize the aggregated impression data as input into the impression allocation pipeline 714 to generate hybrid interaction data717 as described herein.

[0090] Hybrid interaction data717 can include a number of user path data structures including a number of impression data and pseudo-impression data. The impression data can be associated with known interaction events. The pseudo-impression data can be distributed among user identifiers within a cohort in a pseudo-random manner such that the allocations can be recreated. The pseudo-impressions provide for a probabilistically accurate distribution of aggregated impressions across individual user identifiers such that the individual user identifier paths are approximately representative of the actual user identifier paths.

[0091] In some instances, server computing system 702 can obtain input elements from an input element database. The input elements can include, for example, content items. Input elements can be data that causes the presentation of a content item at a client device (e.g., user device associated with client computing system 704). Examples of input elements include advertisements webpages, word processing documents, portable document format (PDF) documents, images, videos, search results pages, and feed sources. Native applications (e.g., “apps”), such as applications installed on mobile, tablet, or desktop computing devices are also examples of input elements. Input elements can be provided to user devices (e.g., user device 730) by server computing system 702. For example, server computing system 702 can include servers that host publisher websites. In this example, user 732 can initiate a request for a given publisher webpage, and server computing system 702 can respond to the request by sending machine executable instructions that initiate presentation of the given webpage at user device 730.

[0092] In another example, the server computing system 702 can include content servers including app servers from which user devices (e.g., user device 730) can download apps. In this example, the user device 730 can download files required to install an app at the user device 730, and then execute the downloaded app locally. The app can present organic content, e.g.. content specified by a developer of the app, and in some cases can also present one or more digital components (e.g., input element, content created/distributed by a third party) that are obtained from a digital component server (e.g., input element database), and inserted into the app while the app is being executed at the user device 730.

[0093] Input elements can include a variety of content. For example, an input element can include static content (e.g., text or other specified content) that is within the input element itself and/or does not change over time. Input elements can also include dynamic content that may change over time or on a per-request basis. For example, a publisher of a given input element can maintain a data source that is used to populate portions of the input element. In this example, the given input element can include a tag or script that causes the user device 730 to request content from the data source when the given input element is processed (e.g., rendered or executed) by a user device 730. The user device 730 integrates the content obtained from the data source into the given input element to create a composite input element including the content obtained from the data source.

[0094] In some implementations an input element can include an input field. An input element can be configured to obtain user input via a user interface and generate user input signals. The input element can be configured to obtain user input signals and transmit (e.g., via a computing system) the user input signals to a server for processing. In some embodiments an input element could include a text box, touch screen, or other interactive user interface element. For example, the input element can include a clickable URL, clickable link, an item that when interacted with, causes the server to transmit additional content to the user (e.g., direct the user to an additional website).

[0095] In some embodiments, a given input element can include a digital component tag or digital component script that references the server computing system 702. In these situations, the digital component tag or digital component script is executed by the user device 730 when the given input element is processed by the user device 730. Execution of the digital component tag or digital component script configures the user device 730 to generate a request for digital components (referred to as a “component request'’), which is transmitted over the network 718 to the server computing system 702. For example, the digital component tag or digital component script can enable the user device 730 to generate a packetized data request including a header and payload data. The component request can include event data specifying features such as a name (or network location) of a server from which the digital component is being requested, a name (or network location) of the requesting device (e.g., the user device 730), and/or information that the digital component distribution system can use to select one or more digital components provided in response to the request. A component request is transmitted, by the user device 730, over the network 718 (e.g., a telecommunications netw ork) to a server of the server computing system 702.

[0096] The component request can include event data specifying other event features, such as the input element being requested and characteristics of locations of the input element at which digital component can be presented. For example, event data specifying a reference (e.g., URL) to an input element (e.g., webpage) in which the digital component will be presented, available locations of the input elements that are available to present input elements (e.g.. digital components, content items), sizes of the available locations, and/or media types that are eligible for presentation in the locations can be provided to the server computing system 702. Similarly, event data specifying keywords associated with the input element (“input element keywords”) or entities (e.g., people, places, or things) that are referenced by the input element can also be included in the component request (e.g., as payload data) and provided to the server computing system 702 to facilitate identification of input elements (e.g., digital components, content items) that are eligible for presentation with the input element. The event data can also include a search query that w as submitted from the user device 730 to obtain a search results page, and/or data specifying search results and/or textual, audible, or other visual content that is included in the search results.

[0097] Component requests can also include event data related to other information, such as information that a user of the user device has provided, geographic information indicating a state or region from which the component request was submitted, or other information that provides context for the environment in which the digital component will be displayed (e.g., a time of day of the component request, a day of the week of the component request, a type of device at which the digital component will be displayed, such as a mobile device or tablet device). Component requests can be transmitted, for example, over a packetized network, and the component requests themselves can be formatted as packetized data having a header and pay load data. The header can specify a destination of the packet and the pay load data can include any of the information discussed above.

[0098] The server computing system 702, which can include one or more digital component distribution servers, chooses input elements (e.g., digital components, content items) that will be presented with the given input element in response to receiving the component request and/or using information included in the component request. In some implementations, a digital component is selected in less than a second to avoid errors that could be caused by delayed selection of the digital component. For example, delays in providing input elements (e.g., digital components, content items) in response to a component request can result in page load errors at the user device 730 or cause portions of the input element to remain unpopulated even after other portions of the input element are presented at the user device 730. Also, as the delay in providing the digital component to the user device 730 increases, it is more likely that the input element will no longer be presented at the user device 730 when the digital component is delivered to the user device 730, thereby negatively impacting a user’s experience with the input element. Further, delays in providing the digital component can result in a failed delivery of the digital component, for example, if the input element is no longer presented at the user device 730 when the digital component is provided. [0099] To facilitate searching of input elements, the system 700 can include a search system that identifies the input elements by crawling and indexing the input elements (e.g., indexed based on the crawled content of the input elements). Data about the input elements can be indexed based on the input element with which the data is associated. The indexed and, optionally, cached copies of the input elements are stored in a search index (e.g., hardware memory device(s)). Data that is associated with an input element is data that represents content included in the input element and/or metadata for the input element.

[0100] The search system can include one or more processor(s) and memory. The one or more processors can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory can store data and instructions which are executed by the processor to cause the search system to perform operations. [0101] User devices 730 can submit search queries to the search system over the network 718. In response, the search system accesses the search index to identify input elements that are relevant to the search query. The search system identifies the input elements in the form of search results and returns the search results to the user device 730 in search results page. A search result is data generated by the search system that identifies an input element that is responsive (e g., relevant) to a particular search query, and includes an active link (e.g., hypertext link) that causes a client device to request data from a specified location in response to user interaction with the search result. An example search result can include a web page title, a snippet of text or a portion of an image extracted from the web page, and the URL of the web page. Another example search result can include a title of a downloadable application, a snippet of text describing the downloadable application, an image depicting a user interface of the downloadable application, and/or a URL to a location from which the application can be downloaded to the user device 730. Another example search result can include a title of streaming media, a snippet of text describing the streaming media, an image depicting contents of the streaming media, and/or a URL to a location from which the streaming media can be downloaded to the user device 730. Like other input elements, search results pages can include one or more slots in which digital components (e.g., advertisements, video clips, audio clips, images, or other digital components) can be presented. As described above, a variety of online content can be presented to users as they interact with online resources available through the Internet. That online content can generally be classified as organic content or digital components.

[0102] Organic content is content that is specified by and/or provided by an owner or administrator of the online resource in w hich the content is being presented. Examples of organic content include search results provided by a search engine, and content presented in a w eb page provided by a publisher. In each of these examples, the content presented is specified by the entity providing the online resource and is therefore considered first party content. For example, the search engine identifies online resources relevant to a submitted search query, generates search results identifying those relevant resources, and generates a search results page (in the domain of the search engine) that includes the search results generated by the search engine. Thus, the search results are generated by the search engine, and presented in the search results page generated by the search engine, thereby making the search results first party content, and thus, organic content. Similarly, when a user visits a particular web page, that web page will include content specified by and/or generated by the publisher of that web page, which is also considered first party content, also making it organic content for purposes of the present discussion.

[0103] For purposes of this discussion, digital components are considered third party content because the digital components are created by and/or provided by an entity that differs from the entity providing the online resource on which the digital component is presented. In the context of a search results page, a digital component that includes third party content can be a digital component that is selected for inclusion in the online resource at the time the online resource is presented (e g., weather data, stock data, or advertisements). For example, a digital component (e.g., presenting current weather conditions, stock prices, or advertisements) can be selected by a third party (e.g., a different domain than the search engine domain) at the time a search results page is generated, and provided for presentation within the search results page. As discussed above, digital components presented with a search results page can be selected by an entity other than the entity’ providing the search results page based, at least in part, on the search query submitted by the user. In the context of a web page provided by a publisher (e.g., a blog, news web page, weather web page, stock information web page), a digital component provided by a third party that differs from the publisher of the web page can be selected for presentation in the web page when the web page is requested by a client device. The digital components selected for presentation with a given web page can be selected, for example, based on organic content of the given web page and/or characteristics of the user (e.g.. interests, profile information, etc.) visiting the given w eb page.

[0104] Each impression (e.g., exposure) to an input element (e.g., content) can have an effect on a user's future online (or offline) activity. For example, a user that sees content related to a particular brand of shoe (e.g., reviews, news articles, or advertisements) may be more likely to acquire that particular brand of shoes than the user w ould have been absent from the exposures. In some situations, it can be advantageous to be able to quantify the effects of different content exposures as they relate to a user subsequently performing some specified conversion event 740 (e.g., conversion event).

[0105] In some implementations, the target action can be specified by a digital component provider. For example, a digital component provider can specify that the target action as one or more of the user downloading a white paper, navigating to at least a given depth of a website, viewing at least a certain number of web pages, spending at least a predetermined amount of time on a website or web page, completing a website registration process, subscribing to a digital service, adding items to shopping cart or purchasing a product. When a user performs a specified target action, performance of the specified target action can be referred to as a conversion.

[0106] A user’s performance of the specified target action is often preceded by a series of exposures to online content (e.g., rendering of a content item on a user device associated with a user). For example, assume that the user 732 has interest in a particular camera and wants to know more about the camera. Further assume that a digital component provider that distributes digital components (e.g., content items, input elements) containing information about the particular camera has specified the conversion event 740 as acquisition of the particular camera.

[0107] In this example, the user 732 may search for information about the particular camera on the user device 730 by submitting a search query to the search system over the network 718. The search system identifies search results responsive to the search query and returns the search results to the user device 730 for display, which is considered an exposure of organic content about the particular camera to the user 732 (e.g., assuming that digital components about the particular camera are not presented on the search results page). The user 732 viewing the search results at the user device 730 can visit websites 734, 736, and 738 (e.g., by clicking on several of the search results) which each contain information about the particular camera. Each of these visits to the websites by the user 732 can also be considered exposures of organic content to the user 732. Assuming that the user 732 ultimately acquires the particular camera (i.e., performs the specified conversion event 740), each of these exposures to organic content, referred to as organic events, will have contributed to the user’s performance of the specified conversion event 740, and the relative contribution of these organic exposures can be quantified as described in more detail below. [0108] In the example, above, it is assumed that the user 732 was exposed to organic content about the particular camera prior to performing the specified conversion event 740, but that the user 732 was not exposed to a digital component about the particular camera (e.g., a specified type of digital component). Exposures to digital components can also contribute to the user’s performance of the specified conversion event 740. For example, assume that prior to performing the specified conversion event 740, the user 732 performs another search. Further assume that, in response to this search, the search system returns a search results page 742 including search results (e.g., first search result 742A and second search result 742B), and a digital component server provides a digital component 744 (e.g., an input element) about the particular camera for presentation with the search results.

[0109] In this example, when the user 732 subsequently performs the specified conversion event 740 (e.g., acquiring the particular camera), the user’s exposure to the digital component 744 will also have contributed to the performance of that specified conversion event 740. However, it is not readily apparent, not directly observable, and difficult to determine the level of the effect of the digital component exposure as it relates to the user’s subsequent performance of the specified conversion event 740. Moreover, it is not readily apparent from raw data related to content exposures alone how to differentiate between the contributions of organic exposures and the contributions of exposures to digital components as they relate to influencing the user’s subsequent performance of the conversion event 740. As such, it can be difficult to effectively and efficiently distribute content to users, particularly as it relates to input elements (e.g., digital components).

[0110] To determine the effects of content exposures as they relate to users performing a specified conversion event 740, the system 700 can include a training computing system 706 with a multi-touch attribution model (e.g.. machine-learned model) which is configured to evaluate content exposures, and determine the level of contribution of each of those content exposures to users’ subsequent performance of a specified conversion event 740 and/or the probability that by presenting input elements (e.g., content exposures) that a user is more likely to interact with (e.g.. the system is more likely to obtain data indicative of a user interaction with a content item), the limited user interface (e.g., screen space) can be used to optimize user input. Thus, the system can adapt a user interface of a computing device to present items that are actually relevant to a user's tasks or goals for using the computing device. For example, user activity may provide one or more signals that a particular input element would be relevant to accessing a resource of interest or performing a task at hand. This information can be used to determine the performance of specified type of digital components (e.g., input elements) distributed by the component distribution system, which can be used to improve the relevance of content presented to users, for example, by modifying transmission criteria that control when, where, or how digital components are transmitted for presentation to users. However, systems and methods of the present disclosure can, in some embodiments, determine that rendering that particular input element would not be of sufficient incremental value to effectively improve the user interface (e g., the user already has access to or otherwise is already navigating toward the resource of interest). [0111] As described in more detail below, the training computing system 706 is configured to implement data collection techniques that enable the training computing system 706 to leam relationships between user sessions and baseline performance levels of a specified conversion event 740 (e.g., levels at which users having certain attributes perform the specified target interaction). These relationships can be referred to as a baseline action model that can output a baseline performance level based on attributes input to the system. This baseline performance model can be a stand-alone model, or incorporated into a more complex model structure that also takes into account other data, as described in more detail below.

[0112] The baseline performance levels represent the level of performance of the specified conversion event 740 in the absence of users being exposed to digital components of a specified type. For example, a particular baseline performance level can be created to represent a rate at which users acquire a particular type of shoe when those users have not been exposed to digital components distributed for a seller of that particular type of shoe. [0113] In a specific example, the baseline performance measure can be indicative of the portion of users having a certain set of attributes that will acquire the particular type of shoe without those users being exposed to advertisements for that particular type of shoe. [0114] The data collection techniques implemented by the training computing system 706 also enable the training computing system 706 and the multi-touch attribution model to model the effects of various content exposures over time as it relates to users subsequently performing the specified conversion event 740. For example, the training computing system 706 can create a model that quantifies an initial change in the portion of users that perform the specified conversion event 740 immediately following exposure to a particular type of content (e.g.. organic content or a digital component), and that represents the decay of that initial change over time (e.g., toward the baseline performance measure), as described in more detail below. This ability’ to delineate between the baseline performance level and the remaining effects of various content exposures over time enables the training computing system 706 to determine the incremental effects of each content exposure remaining at the time the specified conversion event 740 is performed, thereby providing an improved attribution model relative to traditional attribution models that are not able to delineate the relative contributions of each content exposure that remain when the specified conversion event 740 is performed. [0115] The server computing system 702 can communicate with other systems via network 718. For example, the server computing system 702 can communicate with the cloud provider computing system 760. The cloud provider computing system 760 can include one or more processors 720. The one or more processors 720 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory 722 can include one or more non-transitory computer- readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 722 can store data 722A and instructions 722B which are executed by the processor 762 to cause the client computing system 704 to perform operations.

[0116] In some implementations, the server computing system 702 can store or include one or more machine-learned models 725. For example, the machine-learned models

725 can be or can otherwise include various machine-learned models such as neural networks (e.g., deep neural networks) or other types of machine-learned models, including non-linear models and/or linear models. Neural networks can include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks or other forms of neural networks. Some example machine- learned models can leverage an attention mechanism such as self-attention. For example, some example machine-learned models can include multi -headed self-attention models (e.g., transformer models).

[0117] In some implementations, the one or more machine-learned models 725 can be received from the server computing system 702 over network 718, stored in the server computing system memory 710, and then used or otherwise implemented by the one or more processors 708. In some implementations, the server computing system 702 can implement multiple parallel instances of a single machine-learned model 725 (e.g., to perform parallel updated notification elements across multiple instances of user input data obtained via a structured user interface).

[0118] The client computing system 704 can also include one or more user input components 726 that receives user input. In some embodiments, the user input components

726 can include the user input elements. For example, the user input component 726 can be a touch-sensitive component (e.g., a touch-sensitive display screen or a touch pad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component can serve to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, or other means by which a user can provide user input. In some embodiments, the client computing system 704 can include user session data 728 indicative of a user session with a user device (e.g., user device 730). For instance, user session data can include website visits, interactions with marketing materials (e.g., advertisements, landing pages, and the like).

[0119] The cloud provider computing system 760 can communicate with other systems via network 718. For example, the cloud provider computing system 760 can communicate with the server computing system 702 and/or the client computing system 704. The cloud provider computing system 760 can include one or more processors 762. The one or more processors 762 can be any suitable processing device (e.g.. a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory 764 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory’ devices, magnetic disks, etc., and combinations thereof. The memory 764 can store data 764A and instructions 764B which are executed by the processor 762 to cause the cloud provider computing system to perform operations.

[0120] The data associated with the user visits to websites 734, 736, and 738 can be stored in a cloud provider computing system 760. For instance, the cloud provider computing system 760 can be associated with a walled garden environment wherein data associated with user visits can be associated with a cloud provider managed user identifier. The cloud provider computing system 760 can maintain one or more cloud data clean room(s) 766 which can be accessed via query engine 772 to provide k-anonymized aggregate insights to server computing system 702 with requisite permissions.

[0121] The cloud data clean room(s) 766 can include event data 768 and match table data 770. For instance, query engine 772 can obtain a request from server computing system 702 for aggregate data associated with respective cohorts in cohort data 716. Query engine 772 can generate match table data 770 which can aggregate event data 768 that is associated with the identifiers associated with cohort data 716. Thus, for each respective cohort, query engine can provide aggregated impression data for the identifiers associated with the cohort while not providing data indicative of a particular association between an individual impression event (e.g., user visit to website 734, 736, and 738) and a particular user (e.g., user device 730). As such, nothing more can be learned about any’ individual user device 730 than what is known by the server computing system 702 which has made the request to the query engine 772 for the aggregate data for each cohort.

[0122] The server computing system 702 can communicate with other systems via network 718. For example, the server computing system 702 can communicate with the training computing system 706. The training computing system 706 can include one or more processors 762. The one or more processors 762 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA. a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory' 764 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 764 can store data 764A and instructions 764B which are executed by the processor 762 to cause the training computing system 706 to perform operations.

[0123] The server computing system 702 can train the model 725 via interaction with the training computing system 706 that is communicatively coupled over the network 718. The training computing system 706 can be separate from the client computing system 704 or can be a portion of the client computing system 704.

[0124] The training computing system 706 includes one or more processors 746 and a memory 748. The one or more processors 746 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory' 748 can include one or more non-transitory' computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 748 can store data 748A and instructions 748B which are executed by the processor 746 to cause the training computing system 706 to perform operations. In some implementations, the training computing system 706 includes or is otherwise implemented by one or more server computing devices.

[0125] The training computing system 706 can include a model trainer 750 that trains the machine-learned models 725 stored at the server computing system 702 using various training or learning techniques, such as, for example, backwards propagation of errors. For example, a loss function can be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on a gradient of the loss function). Various loss functions can be used such as mean squared error, likelihood loss, cross entropy loss, hinge loss, or various other loss functions. Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations.

[0126] In some implementations, performing backwards propagation of errors can include performing truncated backpropagation through time. The model trainer 750 can perform a number of generalization techniques (e.g., weight decays, dropouts, etc.) to improve the generalization capability of the models being trained.

[0127] In particular, the model trainer 750 can train the models 725 based on a set of training data 752. The training data 752 can include, for example, training data can include user input data, conversion data, user device location data, click data, or other data indicative of a content item being displayed via a user device and the user device performing a target action (e.g., visiting a physical location associated with the respective content item).

[0128] The model trainer 750 includes computer logic utilized to provide desired functionality. The model trainer 750 can be implemented in hardware, firmware, or software controlling a general purpose processor. For example, in some implementations, the model trainer 750 includes program files stored on a storage device, loaded into a memory and executed by one or more processors. In other implementations, the model trainer 750 includes one or more sets of computer-executable instructions that are stored in a tangible computer- readable storage medium such as RAM, hard disk, or optical or magnetic media.

[0129] The network 718 can be any type of communications network, such as a local area network (e.g.. intranet), wide area network (e.g., Internet), or some combination thereof and can include any number of wired or wireless links. In general, communication over the network 718 can be carried via any type of wared or wireless connection, using a wide variety of communication protocols (e.g., TCP/IP, HTTP. SMTP, FTP), encodings or formats (e.g., HTML, XML), or protection schemes (e.g., VPN. secure HTTP, SSL).

[0130] The machine-learned models described in this specification may be used in a variety of tasks, applications, or use cases.

[0131] The technology discussed herein makes reference to servers, databases, software applications, and other computer-based systems, as well as actions taken, and information sent to and from such systems. The inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and divisions of tasks and functionality betw een and among components. For instance, processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0132] While the present subject matter has been described in detail with respect to various specific example embodiments thereof, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations and/or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present disclosure covers such alterations, variations, and equivalents.

[0133] The depicted and/or described steps are merely illustrative and can be omitted, combined, and/or performed in an order other than that depicted and/or described; the numbering of depicted steps is merely for ease of reference and does not imply any particular ordering is necessary or preferred.

[0134] The functions and/or steps described herein can be embodied in computer- usable data and/or computer-executable instructions, executed by one or more computers and/or other devices to perform one or more functions described herein. Generally, such data and/or instructions include routines, programs, objects, components, data structures, or the like that perform particular tasks and/or implement particular data types when executed by one or more processors in a computer and/or other data-processing device. The computerexecutable instructions can be stored on a computer-readable medium such as a hard disk, optical disk, removable storage media, solid-state memory, read-only memory (ROM), random-access memory (RAM), or the like. As will be appreciated, the functionality of such instructions can be combined and/or distributed as desired. In addition, the functionality can be embodied in whole or in part in firmware and/or hardware equivalents, such as integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or the like. Particular data structures can be used to implement one or more aspects of the disclosure more effectively, and such data structures are contemplated to be within the scope of computer-executable instructions and/or computer-usable data described herein.

[0135] Although not required, one of ordinary skill in the art will appreciate that various aspects described herein can be embodied as a method, system, apparatus, and/or one or more computer-readable media storing computer-executable instructions. Accordingly, aspects can take the form of an entirely hardware embodiment, an entirely software embodiment, an entirely firmware embodiment, and/or an embodiment combining software, hardware, and/or firmware aspects in any combination.

[0136] As described herein, the various methods and acts can be operative across one or more computing devices and/or networks. The functionality can be distributed in any manner or can be located in a single computing device (e.g., server, client computer, user device, or the like).

[0137] Aspects of the disclosure have been described in terms of illustrative embodiments thereof. Numerous other embodiments, modifications, and/or variations within the scope and spirit of the appended claims can occur to persons of ordinary skill in the art from a review of this disclosure. For example, one or ordinary skill in the art can appreciate that the steps depicted and/or described can be performed in other than the recited order and/or that one or more illustrated steps can be optional and/or combined. Any and all features in the following claims can be combined and/or rearranged in any way possible.

[0138] While the present subject matter has been described in detail with respect to various specific example embodiments thereof, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and/or equivalents to such embodiments. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations, and/or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. For instance, features illustrated and/or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present disclosure covers such alterations, variations, and/or equivalents.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method, comprising: accessing, by a computing system, a datastore comprising a plurality of data comprising one or more identifiers and one or more features associated with conversion data; generating, by the computing system, a plurality of cohort data structures by, for each cohort data structure of the plurality of cohort data structures: initializing, by the computing system, a cohort data structure; and populating, by the computing system, the cohort data structure with k identifiers and associated feature data, wherein the k identifiers are grouped based on the associated feature data; transmitting, by the computing system, a request to a query engine for aggregated impression data associated with each respective cohort data structure of the plurality of cohort data structures; accessing, by the computing system, aggregated impression data for each respective cohort data structure; generating, by the computing system, a hybrid interaction data comprising impressions associated with known interaction events and pseudo-impressions associated with pseudo-randomly assigned impressions; training, by the computing system, a machine-learned attribution model using the hybrid interaction data; and processing, by the machine-learned attribution model deployed by the computing system, the hybrid interaction data to determine an attribution value for one or more channels.

2. The computer-implemented method of claim 1, wherein generating the hybrid interaction data comprises: hashing each identifier of the cohort data structure; ordering the hashed identifiers; generating an impression data object for each known interaction event of a plurality of known interaction events; and generating a pseudo-impression data object for each remaining impression of the aggregated impressions by distributing the remaining aggregated impressions across the ordered hashed identifiers.

3. The computer-implemented method of claim 2, wherein the aggregated impressions are grouped by a plurality of time periods within a 90-day lookback window.

4. The computer-implemented method of claim 3, wherein the plurality of time periods within the 90-day lookback window comprises (i) 1-day time periods for day one to day thirty before a conversion event, (ii) 3-day time periods for day thirty-one to day sixty before the conversion event, and (iii) 5-day time periods for day sixty-one to day ninety before the conversion event.

5. The computer-implemented method of claim 4, wherein the aggregated impressions for a first time period of the plurality of time periods are distributed across hashed user identifiers for the first time period.

6. The computer-implemented method of any preceding claim, wherein the feature data comprises at least one of (i) country code, (ii) region code, (iii) last-click channel, (iv) date range, (v) conversion type.

7. The computer-implemented method of claim 6, wherein the feature data further comprises at least one of (i) days to conversion, (ii) conversion path length, (iii) presence of first party click or third party click, (iv) number of different channels associated with a conversion path, or (v) non-suppressed users.

8. The computer-implemented method of any preceding claim, wherein the feature data is selected such that the cohort size of each respective cohort is outside a predetermined threshold of the other respective cohorts.

9. The computer-implemented method of any preceding claim, wherein one or more feature data types are selected by: dividing a range of values of each feature into a plurality of buckets; creating cohorts using a single feature of the plurality of features; transmitting a query' to a data clean room for a number of aggregated impressions within a lookback window; accessing a match table generated in response to the transmitted query; generating an impression count for each cohort based on the match table; calculating a standard deviation and variance of each respective cohort of the plurality of cohorts; and selecting one or more features based on at least one of (i) the standard deviation or (ii) the variance.

10. The computer-implemented method of claim 9, wherein the plurality of buckets comprises three to five buckets.

11. The computer-implemented method of claim 9 or claim 10, wherein the features with the highest standard deviation or variance are selected.

12. A computer-implemented method, comprising: generating, by a computing system, hybrid interaction data by: accessing, by the computing system, a datastore comprising a plurality of data comprising one or more identifiers and one or more features associated with impression data; generating, by the computing system, a plurality of cohort data structures by, for each cohort data structure of the plurality of cohort data structures: initializing, by the computing system, a cohort data structure; populating, by the computing system, the cohort data structure with k identifiers and associated feature data, wherein the k identifiers are grouped based on the associated feature data; and populating, by the computing system, the cohort data structure with aggregated impression data associated with the k identifiers of the respective cohort; and assigning, by the computing system, impressions and pseudo-impressions to respective identifiers within the cohorts of data, wherein the impressions comprise confirmed impressions and wherein the pseudo-impressions comprise aggregated impression data assigned to respective identifiers of each respective cohort of the plurality of cohorts; and processing, by a machine-learned attribution model deployed by the computing system, the hybrid interaction data to determine an attribution value for one or more channels.

13. The computer-implemented method of claim 12, wherein the accessed datastore is associated with a third-party computing system, and wherein the plurality' of data comprises k hashed identifiers and aggregated impression data.

14. The computer-implemented method of claim 13, wherein the features comprise at least one of (i) country, (ii) channel of last click prior to conversion event, (iii) number of distinct channels in path, or (iv) intensity of channel clicks within a lookback window.

15. The computer-implemented method of claim 12 or claim 13, wherein each impression of the aggregated impression data comprises a timestamp associated with each impression of a number of impressions.

16. The computer-implemented method of any of claim 12 to claim 15, wherein assigning impressions and pseudo-impressions to a plurality of identifiers is performed pseudo-randomly.

17. The computer-implemented method of claim 1 , wherein pseudo-randomly assigning impressions to the identifiers comprises: hashing, by the computing system, a plurality of identifiers; populating, by the computing system, the initialized cohort data structure with the plurality of hashed identifiers; and assigning, by the computing system, the aggregated impression data to individual hashed identifiers of the hashed identifiers by distributing the impressions across the plurality of hashed identifiers.

18. The computer-implemented method of any of claim 13 to claim 17, wherein the assigning the impression data is performed based on time to conversion, wherein an aggregate number of impressions in a time period are determined by time periods within a 90-day lookback window.

19. The computer-implemented method of any of claim 13 to claim 18, comprising: automatically adjusting, by the computing system, responsive to determining the attribution value for the one or more channels, one or more resource allocations to the one or more channels.

20. A computing system, comprising, comprising: one or more processors; and one or more transitory or non-transitory computer-readable media storing instructions that are executable to cause the one or more processors to perform operations, the operations comprising the method described in any of claims 1-19.

21. One or more transitory or non-transitory computer readable media storing instructions that are executable by one or more processors to perform operations comprising the method described in any of claims 1-19.