EP4677531A1

EP4677531A1 - Systems and methods for restructuring account data

Info

Publication number: EP4677531A1
Application number: EP24740270.4A
Authority: EP
Inventors: Krishna Roy YENUGA; Jordan Gergov; Jiansong CHAO; Brooks William ROYSTER
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2024-05-20
Filing date: 2024-06-12
Publication date: 2026-01-14
Also published as: WO2025244656A1; CN121368774A

Abstract

A method efficiently restructures account data indicative a first plurality of keywords each mapped to a respective query space, a first plurality of campaigns, and associations therebetween. The method includes consolidating the first plurality of campaigns into a smaller, second plurality of campaigns, based on a degree of overlap between respective query spaces to which keywords associated with different campaigns are mapped. The method also includes generating a second plurality of keywords consisting of a subset of the first plurality of keywords, which includes, for each campaign in the second plurality of campaigns, determining whether to remove associations to particular keywords based on an incremental value added by the query spaces that are mapped to those keywords. The method also includes storing restructured account data indicative of the second plurality of keywords, the second plurality of campaigns, and new associations therebetween.

Description

SYSTEMS AND METHODS FOR RESTRUCTURING ACCOUNT DATA

FIEED OF TECHNOEOGY

[0001] The present disclosure relates to digital advertising account data, and more specifically, to techniques for restructuring such data.

BACKGROUND

[0002] The background description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventor(s), to the extent it is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.

[0003] Digital advertising has become a highly technical field requiring, inter alia, that advertisers arrange, maintain, and continuously seek to improve complex accounts. Typically, an advertiser arranges such an account by creating a number of different advertising “campaigns” each ostensibly focusing on a different area where there is a marketing/advertising need. Within each campaign, the advertiser must identify which digital assets (digital advertisements) are to be used for the campaign. Further, the advertiser typically must add a number of keywords to the campaign, and arrange the keywords into different keyword groups that are associated with different digital assets or different groups of digital assets. The advertiser may also set other parameters of each campaign, such as budget, location of the desired/expected audience, and so on. The arrangement and settings of the campaigns are typically changed repeatedly over time, with the primary goal generally being to create an account that maximizes exposure of the advertiser’s digital assets to an audience that is relatively likely to purchase the advertiser’s products or services (subject to budgetary and other constraints).

[0004] Such accounts, however, can become incredibly complex. For example, a single large advertiser/company can have an account with thousands of campaigns, which themselves can have varying levels of complexity. The nearly unavoidable result is that a great deal of redundant information is present in the account, with the redundancies being spread across campaigns, keyword groups, and possibly other hierarchical levels of the account. Such redundancies may arise for many reasons, such as redundant coverage resulting from keyword match types (e.g., how broadly or specifically a keyword is linked to queries, such as search engine queries), location setting (e.g., the geographic location of interest for digital assets associated with the keyword), device type settings, and so on. Other types of account redundancy include repetitiveness or overlap in keyword groups (e.g., with one group in a campaign being set up for people searching for “deals” and another group being set up for people searching for “discounts”).

[0005] These and other redundancies manifest themselves as “account bloat” which, especially on a large scale (i.e., across many advertisers/accounts), results in far greater storage/memory requirements. On an individual scale, account bloat can make it far more difficult for the advertiser or other responsible entity to understand the overall state of the advertiser’s account or identify ways to improve the performance of the account, can make it difficult or impossible to effectively automate processes that interface with the account information, and/or can make it difficult to understand the added value that such automated processes provide.

[0006] Accordingly, there is a need to reduce the size and complexity of digital advertising accounts. In particular, there is a need to reduce account size and complexity using efficient processing techniques, and in a manner that does not overly (if at all) degrade the performance of the account. For example, any reduction of account size/complexity would preferably result in an account that maintains the ability of the account holder’s digital advertisements to broadly reach the relevant audience (e.g., as quantitatively measured using any metric(s) that are known in the field, such as average cost per view, cost per thousand impressions, click-through rate, etc.).

SUMMARY

[0007] Generally, in one aspect of the disclosed invention, a system restructures account data representing a number of advertising campaigns. The account data, which may be hierarchically structured, is initially (e.g., prior to application of the disclosed techniques) indicative of a first plurality of keywords each mapped to a respective query space (e.g., to a set of exact-match and/or other queries). The account data is also initially indicative of a first plurality of campaigns, and associations between the first plurality of campaigns and the first plurality of keywords. In some cases, the keywords may be arranged as various keyword “groups” within different campaigns, with each keyword group being linked to a particular digital asset or a particular set of digital assets. Digital assets (e.g., digital advertisements) may be digital content in any suitable format, such as text, image, video, and/or audio. [0008] In general terms, the disclosed system consolidates the first plurality of campaigns into a second, smaller plurality of campaigns by merging together at least some campaigns in the first plurality of campaigns (e.g., creating new campaigns based on two or more original campaigns, or modifying one original campaign at least in part by adding information/associations/etc. from two or more other original campaigns). The consolidation is based at least in part on the degree of overlap between the query spaces mapped to the keywords of the different campaigns. The system may merge two or more campaigns when the query spaces mapped to the keywords of those campaigns overlap by at least some threshold percentage (e.g., 50%), for example.

[0009] After campaign consolidation, the disclosed system reduces the first plurality of keywords to a smaller, second plurality of keywords, at least in part by determining whether to remove (from campaigns of the new, second plurality of campaigns) associations to particular keywords, based on the incremental value added by the query spaces that are mapped to those particular keywords. For example, the system may identify, for each campaign in the second plurality of campaigns, the smallest/minimal set of keywords that collectively map to a new query space satisfying one or more predetermined requirements. As a more specific example, the system may identify the smallest/minimal set of keywords that collectively map to a new query space that encompasses at least as many queries as did all the keywords contained in the constituent campaigns prior to the consolidation/merging.

[0010] The resulting combination of campaign consolidation and keyword reduction can greatly reduce “account bloat” and the attending storage/memory requirements, which can be astronomical when summed across many accounts. Moreover, by greatly simplifying an account, the combination of campaign consolidation and keyword reduction can in turn greatly facilitate understanding and management of the account. As one important example, if the disclosed techniques are repeated periodically over time, the automated and recurring removal of redundancies can guide advertisers to make changes (e.g., keyword additions or removals) that are more incrementally effective in terms of account performance.

[0011] Advantageously, the disclosed techniques can achieve this mitigation of account bloat efficiently and without degrading campaign (or overall account) performance. By consolidating campaigns and removing keywords based on the mapped query spaces (rather than, for example, relying solely on the duplication, semantic similarity, or other overlap of keywords), the disclosed techniques simplify the account in a performance-aware and efficient manner. In particular, the disclosed techniques take advantage of the fact that query spaces are a closer proxy for performance than the keywords themselves, and thus considerations such as redundancy, audience coverage, etc., are more accurately and more efficiently assessed at the query space level. The improvements provided by the disclosed techniques (e.g., the ability to reduce account size/complexity without significantly, if at all, degrading performance) can be quantitatively measured, using any existing performance metrics for the restructured account (e.g., average cost per view, cost per thousand impressions, click-through rate, etc.).

[0012] In some implementations, the system restructures keywords of a campaign (i.e., creates new associations between digital assets and keywords) according to theme. In implementations and scenarios where the system had already consolidated campaigns and/or reduced keywords as described above, the restructuring can take place with respect to some or all campaigns in the second plurality of campaigns. In each campaign, for example, the system may cluster the keywords of the campaign by theme (e.g., as determined for each keyword using a machine learning model), predict the performance of various digital assets with respect to each keyword in each cluster or with respect to all keywords of the cluster (e.g., using another machine learning model), and then use the predicted performance to determine which digital assets to link to which clusters. Such a technique can further reduce account bloat/complexity by removing additional redundancies, and can maximize or otherwise increase asset-to-keyword relevance. Moreover, the resulting thematic consistency can make it far easier to understand, predict the outcome of, and/or measure/assess the outcome of related automated processes, such as automated processes for matching keywords to queries using different match types or algorithms.

[0013] In one aspect, a method efficiently restructures account data indicative of (i) a first plurality of keywords each mapped to a respective query space, (ii) a first plurality of campaigns, and (iii) associations between the first plurality of campaigns and the first plurality of keywords. The method includes: (1) consolidating, by one or more processors, the first plurality of campaigns into a second plurality of campaigns consisting of fewer campaigns than the first plurality of campaigns, wherein the consolidating includes determining whether to combine a given first campaign with a given second campaign based on a degree of overlap between (i) the respective query spaces to which keywords associated with the given first campaign are mapped and (ii) the respective query spaces to which keywords associated with the given second campaign are mapped; (2) generating, by the one or more processors, a second plurality of keywords consisting of a subset of the first plurality of keywords, wherein generating the second plurality of keywords includes, for each campaign in the second plurality of campaigns, determining whether to remove associations to particular keywords of the campaign based on an incremental value added by the query spaces that are mapped to the particular keywords; and (3) storing, by the one or more processors, restructured account data indicative of (i) the second plurality of keywords, (ii) the second plurality of campaigns, and (iii) new associations between the second plurality of campaigns and the second plurality of keywords.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] FIG. 1 is a block diagram of an example system in which techniques for restructuring account data can be implemented.

[0015] FIG. 2A depicts a hierarchical arrangement of an example account.

[0016] FIG. 2B depicts an example mapping between keywords of a campaign and various digital assets.

[0017] FIG. 3 depicts an example process for restructuring account data.

[0018] FIG. 4 depicts an example scenario in which the computing system of FIG. 1 merges multiple campaigns into a single campaign.

[0019] FIG. 5 depicts an example process that the computing system of FIG. 1 can implement to remove keywords of a campaign.

[0020] FIG. 6 depicts an example process that the computing system of FIG. 1 can implement to restructure mappings between keywords of a campaign and digital assets.

[0021] FIG. 7 is a flow diagram of an example method for restructuring account data.

DETAILED DESCRIPTION OF THE DRAWINGS

[0022] FIG. 1 is a block diagram of an example system 100 in which techniques for restructuring account data can be implemented. The example system 100 includes a client device 102 (e.g., a device of a user/consumer), a computing system 104, a content sponsor 106 (e.g., a server of a content sponsor entity), and a network 110. The computing system 104 is remote from the client device 102 and content sponsor 106, and is communicatively coupled to the client device 102 and content sponsor 106 via the network 110. [0023] The network 110 may be a single communication network (e.g., the Internet), and in some implementations also includes one or more additional networks. As just one example, the network 110 may include a cellular network, the Internet, and a server- side local area network (LAN). While FIG. 1 shows only a single client device 102 and content sponsor 106, it is understood that the computing system 104 may also be in communication with a number (e.g., thousands or millions) of other client devices and/or content sponsors that are generally similar to the client device 102 and/or content sponsor 106, respectively.

[0024] Generally, computing system 104 may provide advertising services to content sponsors (e.g., advertisers) such as content sponsor 106, to facilitate the marketing of commercial products and/or services of the content sponsors. To this end, computing system 104 may provide an online interface for content sponsor 106 and others to set up and maintain their own digital advertising accounts. Digital advertising accounts can include any suitable settings and/or parameters that the content sponsors can configure to manage their digital advertising efforts. For example, the online interface may enable content sponsor 106 to set up, within an account of content sponsor 106, a number of digital advertising campaigns (e.g., associated with different areas of the business of the content sponsor 106, or different product lines, etc.). Within a single campaign, the content sponsor 106 may select keywords based on expectations of which types of queries might be entered by users interested in particular products or services of content sponsor 106, and may link those keywords (or particular groups of those keywords, e.g., arranged based on product) to particular digital assets (e.g., digital advertisements in the form of text, images, videos, and/or audio), or to particular sets of digital assets, that content sponsor 106 wishes to use for the particular products or services. The content sponsor 106 may also set other parameters, such as bid amounts for specific keywords. Collectively, the settings of the account, including the hierarchical arrangement of campaigns and keywords (and possibly keyword groups), as well as the associations of keywords or keyword groups to digital assets, may be stored as account data of content sponsor 106, within an account database 112 that stores the account data of many content sponsors.

[0025] Computing system 104, or another system (e.g., a dedicated ad exchange server), may use the account data of the various content sponsors to select and deliver (or arrange for the delivery of) specific digital assets to specific client devices based on a suitable content selection procedure. For example, computing system 104 may select digital assets by using an auction based on keyword bids (and possibly also other factors, such as a relevancy score for a particular digital asset given a particular query entered by a client device user, or a particular context in which a user is using a client device, etc.). As just one example, a user of the client device 102 may access a search engine via a web page hosted by computing system 104 or another system, or via a search engine application (e.g., mobile application) that was previously installed on the client device 102. The search engine may be an all- purpose search engine that identifies/provides web pages and/or other Internet content, such as, for example, a Google Search search engine. Alternatively, the search engine may be associated with the search functionality of any other web page or application, such as a video sharing platform (e.g., YouTube), a service-finding platform, and so on.

[0026] In some implementations, computing system 104 provides only some of the functionality discussed herein (e.g., only account setup, account maintenance, and automated account data restructuring), without providing other functionality such as ad selection/exchange functionality. However, it is understood that computing system 104 may itself, in some implementations, include multiple servers and/or other devices (e.g., a first server that maintains and restructures account data, a second server that selects digital assets to present on a website, mobile application user interface, or other information resource, and so on).

[0027] In some implementations, digital assets/advertisements are associated with links to particular landing pages. For example, if a user clicks on a digital asset presented via client device 102 (e.g., within a web browser or mobile application user interface), the user may be transferred to a URL of a web landing page selling the advertised product, or may be transferred (via a deeplink) to a particular landing page/screen of a mobile application where the product is offered for sale, etc.

[0028] The client device 102 may be or include any stationary, mobile, or portable computing device with wired and/or wireless communication capability (e.g., a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart wearable device such as smart glasses or a smart watch, a vehicle head unit computer, etc.). In the example implementation of FIG. 1, the client device 102 includes a network interface 120, a processor 122, memory 124, and a display 126. The processor 122 may be a single processor (e.g., a central processing unit (CPU)), or may include a set of processors (e.g., multiple CPUs, or one or more CPUs and one or more graphics processing units (GPUs)). [0029] The memory 124 includes one or more computer-readable, non-transitory storage units or devices, which may include persistent (e.g., hard disk) and/or non-persistent memory components. The memory 124 stores instructions that are executable on the processor 122 to perform various operations, including the instructions of various software applications and the data generated and/or used by such applications. In the example implementation of FIG. 1, the memory 124 stores at least an application 130. Generally, the application 130 is executed by the processor 122 to provide one or more user interfaces via display 126, where the user interface(s) may enable a user to enter and submit search queries and view (among other things) digital advertisements in response to the queries, or may otherwise enable a user to view digital advertisements within content slots of information resources. For example, the application 130 may be a web browser application or a dedicated mobile application.

[0030] The display 126 includes hardware, firmware, and/or software configured to enable a user to view visual outputs of the client device 102, and may use any suitable display technology (e.g., LED, OLED, LCD, etc.). In some implementations, the display 126 is incorporated in a touchscreen having both display and manual input capabilities. Moreover, in some implementations where the client device 102 is a wearable device, the display 126 is a transparent viewing component (e.g., lenses of smart glasses) with integrated electronic components. For example, the display 126 may include micro-LED or OLED electronics embedded in lenses of smart glasses.

[0031] The network interface 120 includes hardware, firmware, and/or software configured to enable the client device 102 to exchange electronic data with the computing system 104 via the network 110. For example, the network interface 120 may include a cellular communication transceiver, a WiFi transceiver, and/or transceivers for one or more other wired and/or wireless communication technologies.

[0032] While FIG. 1 shows client device 102 as a single component communicating directly (i.e., via network 110) with the computing system 104, in some implementations the subcomponents of client device 102 shown in FIG. 1 are instead divided among two or more user-side devices. As just one example, a pair of smart glasses may include the processor 122, the memory 124, and the display 126, while a smartphone may include another processing unit, another memory, another display, and the network interface 120. The smart glasses (or smart helmet, etc.) may then communicate as needed with the smartphone (e.g., via Bluetooth) to enable the operations described herein. [0033] The computing system 104 includes a network interface 140, a processor 142, and memory 144. The network interface 140 includes hardware, firmware, and/or software configured to enable the computing system 104 to exchange electronic data with the content sponsor 106 (and other, similar entities), and possibly client devices such as client device 102, via the network 110. For example, the network interface 140 may include a wired or wireless router and a modem. The processor 142 may be a single processor, or may include two or more processors. Computing system 104 may be a single computing device at a single location, or may include multiple, coordinating computing devices that are either co-located or remotely distributed.

[0034] The memory 144 is a computer-readable, non-transitory storage unit or device, or collection of units/devices, that may include persistent and/or non-persistent memory components. The memory 144 stores the instructions of an account restructuring application 150, which can be executed by the processor 142. In the example implementation of FIG. 1, the account restructuring application 150 includes a campaign consolidation module 160, a keyword reduction module 162, and a keyword mapping module 164. While modules 160, 162, and 164 are generally shown and described as being distinct modules of a single application 150, it is understood that the modules may be separate software modules (across one or multiple applications), combined as a single software module, or arranged in any other suitable manner. Moreover, it is understood that, in some implementations, the memory 144 may omit one or more modules/elements shown in FIG. 1, such as keyword mapping module 164.

[0035] Generally, the account restructuring application 150 rearranges, simplifies, and/or reduces the size of account data stored in account database 112, for a number of content sponsors such as content sponsor 106. In particular, account restructuring application 150 restructures the account data in a manner that is both efficient and performance-aware, as discussed above in the Summary section, and as will be clear to one skilled in the relevant art based on the disclosure herein.

[0036] As a part of the restructuring, campaign consolidation module 160 merges multiple campaigns of a single account together such that, in at least some scenarios, the restructured account is associated with fewer campaigns than the original account. As used herein, references to an “account” can encompass an account or a sub-account of any suitable type or form, and can relate to an advertiser or other entity in any suitable way. For example, an account may correspond to a particular advertiser or customer number, may be the only account for an entity or one or many accounts or sub-accounts for the entity, and so on. Moreover, as used herein, references to “merging” multiple campaigns together can encompass adding, to one campaign, certain keywords, groupings, associations, and/or settings (e.g., budget settings, geographic location settings, etc.) of other campaigns, or creating a new campaign that includes keywords, groupings, associations, and/or other settings of each of the multiple campaigns. In the latter case, the original (unmerged) campaigns may be retained, or may be partially or entirely deleted or removed, depending on the implementation. Generally, campaign consolidation module 160 merges campaigns based on query spaces associated with the keywords of those campaigns. An example scenario in which campaign consolidation module 160 merges campaigns, and corresponding example processes, are discussed in more detail below with respect to FIG. 4.

[0037] Keyword reduction module 162 may operate on the campaigns of an account after campaign consolidation module 160 has merged (where appropriate) various campaigns together. Keyword reduction module 162 generally culls/removes keywords from each postmerger campaign (including, in some implementations, from campaigns that were not merged) by assessing the incremental value (performance-wise) of each keyword in the campaign. Similar to campaign consolidation module 160, keyword reduction module 162 may accomplish this by using the query spaces mapped to the keywords. In some implementations, keyword reduction module 162 keeps or removes keywords based on whether maintenance of those keywords would increase the mapped query space for the campaign as a whole. An example process for removing keywords from post-merger campaigns is discussed in more detail below with respect to FIG. 5. In an alternative implementation, keyword reduction module 162 operates independently of modules 160 and/or 164 (e.g., in an alternative aspect in which account restructuring application 150 omits modules 160 and/or 164).

[0038] By consolidating campaigns and removing keywords based on the mapped query spaces (rather than, for example, relying solely on the duplication, semantic similarity, or other overlap of keywords), campaign consolidation module 160 and keyword reduction module 162 can simplify the account in a performance-aware and efficient manner. In particular, the use of query spaces provides a closer proxy for performance than the keywords themselves, and thus considerations such as redundancy, audience coverage, etc., are more accurately and more efficiently assessed at the query space level. The resulting improvements (e.g., the ability to reduce account size/complexity without significantly, if at all, degrading performance) are quantitatively measurable, using any existing performance metrics for the restructured account (e.g., average cost per view or “CPV”, cost per thousand impressions or “CPM”, click-through rate or “CPT”, etc.). Such performance metrics may be generated, collected, and/or stored in performance database 172 by computing system 104, for example, or by another suitable computing system.

[0039] Keyword mapping module 164 may operate on the campaigns of an account after keyword reduction module 162 has removed/culled keywords from the post-merger campaigns. Alternatively, keyword mapping module 164 may operate independently of modules 160 and 162 (e.g., in an alternative aspect in which account restructuring application 150 omits modules 160 and 162). Keyword mapping module 164 generally restructures the associations between keywords of a campaign and digital assets, based on themes that keyword mapping module 164 determines for the keywords. In some implementations, keyword mapping module 164 accomplishes this by clustering the keywords of the campaign into groups based on theme (e.g., as determined using a machine learning classification model), and then associating each cluster/theme/keyword group with one or more digital assets. An example process for restructuring mappings between keywords and digital assets in a campaign is discussed in more detail below with respect to FIG. 6.

[0040] Once an account is restructured by modules 160, 162, and/or 164, the restructured account data indicative of the constituent keywords, campaigns, and associations therebetween, as well as the associations between keywords (e.g., keyword groups) and digital assets, is stored in account database 112 by account restructuring application 150, or by another suitable application of computing system 104.

[0041] FIG. 2A depicts a hierarchical arrangement of an example account 200 that may be stored in account database 112. The account 200 includes some number (A7) of campaigns 210, each of which can include any suitable number of keywords 220 (depicted as x, y, and z for the first, second, and A7-th campaign 210, respectively). References herein to an account “including” or being “associated with” a campaign, or a campaign “including” or being “associated with” a keyword, to indicate that the entities (account, campaign, or keyword) are linked by associations that are a part of the account data and stored in persistent memory (e.g., in account database 112). A similar meaning is indicated when stating that a campaign, keyword, or keyword group is “associated with” a digital asset. Associations may be direct (e.g., a single association of a campaign to a keyword stored in account database 112), or may include two or more links (e.g., a campaign being “associated with” a digital asset by way of a first stored association that associates the campaign with a keyword group, and a second stored association that associates the keyword group with the digital asset).

[0042] Also shown in FIG. 2A is an example mapping 230 of a particular keyword K (of keywords 220) to a number ( of queries in a query space 240. The mapping 230 may be reflected by associations stored in account data 112 for an account of content sponsor 106, for example. As the term is used herein, a “query space” may be a fixed or predetermined set of queries mapped to a particular keyword, or may be a dynamically changing set of queries associated with an algorithm or model that maps keywords to queries (e.g., in a time- and/or context-dependent manner). In either case, however, a given query space such as query space 240 can be equated to a specific set of N queries at any given time and/or in any given context. The mapping of keywords to query spaces may be universal (e.g., not specific to any content sponsor, account, or campaign), and may be performed by computing system 104 or by another suitable computing system. In some implementations, however, the mapping for a given keyword depends on a match “type” for that keyword, which may be selected by the content sponsor adding that keyword to a campaign. For example, a first match type may map a keyword only to queries that include the exact keyword or a semantic equivalent of the keyword, a second match type may more broadly /inclusively match a keyword to queries that are to some degree related to the keyword, and so on. Thus, in some implementations, the query mapping is dependent on the match type selected by the content sponsor associated with the account, but is otherwise independent of the content sponsor and account. Query spaces, or data indicative of the query spaces, may be stored in persistent memory of query database 170 of FIG. 1, or in one or more remote databases, for example.

[0043] FIG. 2B depicts an example mapping 250 between keywords of a campaign and various digital assets 260, in a particular implementation and scenario. The mapping 250 may be reflected by associations stored in account data 112 for an account of content sponsor 106, for example. “Campaign 1” of FIG. 2B may be one of the campaigns 210 of FIG. 2A, and the keywords KI through K10 may be keywords of keyword 220 of FIG. 2A, for example. In FIG. 2B, the mapping 250 maps particular groups 270 of keywords to particular sets of digital assets 260. It is understood that by virtue of being in a particular keyword group 270, each constituent keyword is itself mapped to the same digital asset(s) 260 as the corresponding keyword group 270. Each keyword group 270 may be mapped to a single digital asset or a set of two or more digital assets, and there may or may not be overlap in the digital assets 260 mapped to different keyword groups 270. The digital assets 260 may be digital content, of any suitable format (e.g., text, image, video, 3D/immersive, and/or audio), that is used within, or is itself, an advertisement for a product or service. Depending on the implementation, the digital assets 260 may be generated and/or stored by content sponsors, by computing system 104, and/or by one or more other entities and/or computing systems. In some implementations, digital assets 260 are not themselves stored in account database 112, and instead are represented in account database 112 by digital asset identifiers that the computing system 104 can use to denote, select, retrieve, etc. the corresponding digital assets 260.

[0044] FIG. 3 depicts an example process 300 for restructuring account data, such as account data stored in account database 112. The process 300 may be implemented by modules 160, 162, 164 of account restructuring application 150, for example. Original account data 302 of FIG. 3 may be data representing an account of content sponsor 106 as stored in account database 112, for example. The original account data 302 may generally be arranged as shown in FIGs. 2A and 2B, for example.

[0045] At stage 310 of process 300, campaign consolidation module 160 consolidates (and/or in some cases, determines not to consolidate) campaigns of the account represented by original account data 302 based on a degree of overlap between query spaces of the campaigns. An example scenario 400 of the campaign consolidation at stage 310, according to one implementation, is illustrated in FIG. 4. It is understood that, for ease of explanation and clarity, scenario 400 represents a relatively simple example. In practice, some accounts may include hundreds or thousands of campaigns, with tens or hundreds of keywords per campaign, etc.

[0046] In the example scenario 400, the account includes three campaigns, each of which includes three or four keywords (i.e., K1-K4 in Campaign 1, KI, K5, K6, and K7 in Campaign 2, and K8-K10 for Campaign 3). Each keyword in turn is mapped to a query space with a set of queries, as shown in parentheses (e.g., Q1-Q5 for KI, Q3, Q4, Q6, and Q7 for K2, etc.). In some implementations, for each campaign, campaign consolidation module 160 identifies a complete set of queries representing the union of all queries mapped to all unique keywords of the campaign. In FIG. 4, these complete query sets/spaces are labeled 410-1, 410-2, and 410-3 for Campaign 1, Campaign 2, and Campaign 3, respectively. [0047] Campaign consolidation module 160 then determines the degree of overlap between the various combinations of query sets 410-1, 410-2, and 410-3. Specifically, in the implementation of FIG. 4, campaign consolidation module 160 identifies, for each combination, the set of queries that the complete query sets of the two campaigns have in common. In FIG. 4, these common query sets are labeled 420-1, 420-2, and 420-3 for the comparison of Campaigns 1 and 2, Campaigns 2 and 3, and Campaigns 1 and 3, respectively.

[0048] To determine whether any two campaigns should be consolidated/merged, campaign consolidation module 160 uses the common query sets 420-1, 420-2, and 420-3 to compute/generate metrics representing the degree of overlap, and either merges or does not merge a given pair of campaigns based (at least in part) on those metrics. For example, campaign consolidation module 160 may determine whether to combine two campaigns based on whether their common query set 420 has a number of queries that is at least some threshold percentage or ratio of the complete query set 410 for at least one of the two campaigns. In the scenario 400, for instance, and if campaign consolidation module 160 applies a 50% consolidation threshold, campaign consolidation module 160 may combine Campaigns 1 and 2 because common query set 420-1 (with six queries) is at least 50% as large as the complete query set 410-1 (with eight queries), but not combine Campaigns 2 and 3 or Campaigns 1 and 3 because both pairs fail to reach the 50% consolidation threshold. Alternatively, campaign consolidation module 160 may determine whether to combine two campaigns based on whether their common query set 420 has a number of queries that is at least some threshold percentage or ratio of the complete query set 410 for each of the two campaigns. In either implementation, campaign consolidation module 160 may apply any suitable consolidation threshold (e.g., 50%, 75%, etc.), or may apply multiple thresholds (e.g., requiring that at least one campaign have 60% or greater overlap, and that the other have at least 40% overlap, etc.).

[0049] In some implementations, three or more campaigns can be merged into a single campaign, provided the applicable metric(s) is/are satisfied. In such implementations, campaign consolidation module 160 may iteratively perform the procedure reflected by scenario 400 (e.g., by next determining the overlap between the union of the query sets 410-1 and 410-2 and query set 410-3 of Campaign 3, or between the union of the query sets 410-1 and 410-2 and a different campaign not shown in FIG. 4). [0050] In some implementations, campaign consolidation module 160 additionally, or instead, uses other techniques to determine whether to merge campaigns based on query space overlap. For example, campaign consolidation module 160 may cluster the keywords of the campaigns of the account (here, Campaigns 1-3) based on the degree of overlap in the query spaces, and determine whether to combine campaigns based on how many keywords associated with those campaigns are in the same cluster.

[0051] As another example, campaign consolidation module 160 may also, or instead, determine whether to combine campaigns based on the one or more compatibility factors associated with the campaigns. Examples of such compatibility factors include location information associated with the campaigns (e.g., only permitting consolidation of campaigns that are designed for or associated with different geographic locations), performance information associated with the campaigns (e.g., permitting or restricting consolidation of campaigns based on the absolute or relative performance metrics of the campaigns), and/or audience information associated with the campaigns (e.g., only permitting consolidation of campaigns that are designed for or associated with different types of user devices or operating systems).

[0052] Returning to FIG. 3, in some implementations, stage 310 also includes transferring or otherwise generating associations such that all keywords from the original campaigns maintain their associations with the same digital asset or assets in the new/consolidated campaigns. As discussed below, however, such digital asset associations may be restructured at stage 312 and/or 316.

[0053] At stage 312, campaign consolidation module 160 (or another module of account restructuring application 130) consolidates the digital asset mappings for keywords of each of the (possibly merged) campaigns based on landing pages associated with keywords and/or groups of keywords. For example, campaign consolidation module 160 may assign the same digital asset mapping to all keywords or keyword groups that have the same landing page (i.e., direct a user to the same landing page if the user clicks on a digital asset/ad associated with the keyword or keyword group). In some implementations, campaign consolidation module 160 assigns the same digital asset mapping to all keywords or keyword groups that have a sufficiently similar landing page (e.g., as determined by campaign consolidation module 160 using a suitable metric such as cosine similarity of vectors representing the URLs and/or the semantic content of the respective landing pages). In some implementations, however, the process 300 omits stage 312.

[0054] At stage 314, keyword reduction module 162 reduces/culls the set of keywords in the account based on the incremental value added by the query spaces to which those keywords are mapped. FIG. 5 depicts an example process 500 that keyword reduction module 162 may implement at stage 314. Keyword reduction module 162 may implement process 500 multiple times at stage 314, e.g., once for each campaign. The campaigns upon which keyword reduction module 162 operates at stage 314 can include merged campaigns and/or unmerged campaigns, depending on which (if any) campaigns were merged at stage 310 in a given scenario.

[0055] Within process 500, at stage 510, keyword reduction module 162 scores and ranks keywords 502 of a given campaign according to a suitable metric. For example, keyword reduction module 162 may rank the keywords 502 according to a metric indicative of the value or popularity of each of the keywords 502, such as an average bid value for each keyword, or a performance metric (e.g., CPM, CVR, etc.) for digital assets associated with each keyword, etc. Keyword reduction module 162 then selects the highest-ranked keyword at stage 512 and adds the selected keyword as the first keyword in a subset of keywords that keyword reduction module 162 is building for the campaign. By using a suitable metric to select the first keyword to be added, this technique can prevent an overly broad/ambiguous keyword being initially added. An overly broad/ambiguous keyword is undesirable, as it could severely degrade performance of the remaining stages of process 500 (e.g., by making it such that no other keyword, or only very few other keywords, would appear to reach a greater audience and thereby add incremental value).

[0056] At stage 514, keyword reduction module 162 determines the incremental added by a next-highest ranked keyword, based at least in part on the query space mapped to that keyword. At stage 516, keyword reduction module 162 adds the keyword to the subset or omits the keyword to the subset based on that incremental value. Keyword reduction module 162 may repeat stages 514 and 516 for all remaining keywords of keywords 502 or, alternatively, for some predetermined number or percentage of all remaining keywords (e.g., simply omitting the lower-ranked keywords without further consideration). Collectively, the combination of stage 512 with the iterations of stages 514 and 516 may identify, for each campaign, a smallest subset of keywords that collectively maps to a new query space satisfying one or more predetermined requirements, with the predetermined requirements including a measure of the incremental value determined at stage 514.

[0057] In some implementations, the predetermined requirements include a requirement that the new query space (i.e., the query space to which the subset of keywords being developed is mapped) corresponds to at least a minimum expected performance measure. In some such implementations, stages 514 and 516 collectively include adding a particular keyword to the subset of keywords only if that keyword is mapped to a query not already included in the collective query space of the subset of keywords. As another example, stages 514 and 516 may collectively include adding a particular keyword to the subset of keywords only if that keyword is mapped to a query that causes a predicted performance level to increase. For example, stage 514 may include using a machine learning model to predict performance metric values (e.g., CVR, CPM, CTR, etc.) for the collective query space of the subset of keywords with and without the keyword under consideration, and determining the incremental value of the keyword based on the difference between the two values.

[0058] After all iterations are complete, keyword reduction module 162 outputs restructured account data 504 (e.g., stores the restructured account data 504 in account database 112 or memory 144, at least as an intermediate stage).

[0059] Returning to FIG. 3, at stage 316, the keyword mapping module 164 restructures, for one, some, or all of the campaigns in the account (e.g., the account as represented by restructured account data 504), the mappings between keywords and digital assets to enhance the logical (e.g., thematic) consistency of the mappings, and to maintain or improve relevancy of the digital assets to the campaigns and keywords (or groups of keywords) therein. FIG. 6 depicts an example process 600 that keyword mapping module 164 may implement at stage 316. Keyword mapping module 164 may perform the process 600 once for each campaign. In various implementations, keyword mapping module 164 may perform the process 600 independently (e.g., without stages 310, 312, and 314), keyword mapping module 164 may perform the process 600 on the output of stage 310, 312, or 314, or the process 600 may not be performed at all (e.g., if stage 316 is omitted from the process 300 and keyword mapping module 164 is omitted from computing system 104).

[0060] At stage 610, keyword mapping module 164 clusters the (remaining) keywords 602 of the campaign (e.g., keywords represented by restructured account data 504) according to theme. For example, stage 610 may include using a trained machine learning classification model to determine a theme for each keyword, and clustering keywords based on their respective themes (e.g., clustering keywords with identical themes, or semantically similar themes, etc.). Any suitable clustering algorithm may be used.

[0061] At stage 612, for each keyword cluster from stage 610, keyword mapping module 164 ranks performance of each digital asset associated with the account (i.e., for each digital asset within any campaign of the account). Alternatively, keyword mapping module 164 may rank the performance of only a subset of the digital assets in the account. In some implementations, stage 612 includes using a trained machine learning prediction model to predict the performance of each combination of a keyword (within the cluster) and a digital asset. For example, keyword mapping module 164 may input the keyword and the digital asset into the machine learning model to predict a suitable performance metric (e.g., CVR, CPM, CTR, etc.), and then compute another metric based on the keyword- and asset-specific metrics (e.g., an average performance metric across all keywords in the cluster). In other implementations, keyword mapping module 164 inputs all keywords of the cluster, and the digital asset, into the machine learning model to predict a suitable performance metric. In either implementation, keyword mapping module 164 can then rank the digital assets for each cluster based on the performance metrics of those digital assets with respect to the keywords of the cluster.

[0062] At stage 614, keyword mapping module 164 associates keyword clusters to digital assets based on the cluster- specific rankings of those assets. For example, keyword mapping module 164 may associate the highest-ranked digital assets (e.g., the top asset, the top 10 assets, or the top 15 assets, etc.) for a particular keyword cluster with all keywords within that cluster. The set of keywords within a cluster may be hierarchically arranged within the account structure/data as a keyword “group” associated with the highest-ranked digital asset(s), for example.

[0063] Keyword remapping module 164 may repeat stages 610, 612, and 614 for all campaigns in the account, after which keyword remapping module 164 outputs restructured account data 604 (e.g., stores the restructured account data 604 in account database 112 or memory 144). In implementations where the process 600 is included in stage 316 of process 300, the restructured account data 604 may be the restructured account data 304 of FIG. 3.

[0064] FIG. 7 is a flow diagram of an example method 700 for restructuring account data. The method 700 may be performed by computing system 104 (e.g., by instructions of account restructuring application 150 when executed by processor 142), or by another suitable computing system.

[0065] At (optional) block 702, account data is obtained. The account data is indicative of (1) a first plurality of keywords each mapped to a respective query space, (2) a first plurality of campaigns, and (3) associations between the first plurality of campaigns and the first plurality of keywords. The account data may be arranged in the manner illustrated in FIGs. 2A and 2B, for example. In some implementations, block 702 includes accessing a local database (e.g., account database 112), and/or retrieving the account data from one or more remote databases.

[0066] At block 704, the first plurality of campaigns is consolidated into a second plurality of campaigns consisting of fewer campaigns than the first plurality of campaigns. Block 704 includes determining whether to combine a given first campaign with a given second campaign based on a degree of overlap between (1) the respective query spaces to which keywords associated with the given first campaign are mapped and (2) the respective query spaces to which keywords associated with the given second campaign are mapped. Block 704 may be the same as or similar to stage 310 of process 300, and/or any of the techniques discussed above with respect to FIG. 4, for example.

[0067] At block 706, a second plurality of keywords is generated. The generated second plurality of keywords consists of a subset of the first plurality of keywords, and block 706 includes, for each campaign in the second plurality of campaigns, determining whether to remove associations to particular keywords of the campaign based on an incremental value added by the query spaces that are mapped to the particular keywords. Block 706 may be the same as or similar to stage 314 of process 300, and/or process 500 of FIG. 5, for example.

[0068] At block 708, restructured account data is stored, with the restructured account data being indicative of (1) the second plurality of keywords, (2) the second plurality of campaigns, and (3) new associations between the second plurality of campaigns and the second plurality of keywords. The restructured account data may be stored in account database 112, for example.

[0069] The method 700 may include one or more additional blocks. For example, the method 700 may include two additional blocks (occurring after block 706), with the first additional block including generating one or more campaign-specific keyword clusters by clustering keywords of the campaign (e.g., stage 610 of process 600), and the second additional block including associating each campaign-specific keyword cluster to at least one respective digital asset of a second plurality of digital assets (e.g., stage 614, or the combination of stages 612 and 614, of process 600).

[0070] It is understood that the blocks of FIG. 7 need not be performed strictly in the order shown. In some implementations, for example, block 708 may occur contemporaneously with blocks 704 and 706 (e.g., with each change to the account/account data in blocks 704 and 706 being modified or otherwise stored in account database as the changes are made, rather than waiting for account restructuring to be complete before block 708 is performed, etc.).

[0071] In some implementations, the techniques disclosed herein use artificial intelligence to facilitate the restructuring of account data. Artificial intelligence (Al) is a segment of computer science that focuses on the creation of models that can perform tasks with little to no human intervention. Artificial intelligence systems can utilize, for example, machine learning, natural language processing, and computer vision. Machine learning, and its subsets, such as deep learning, focus on developing models that can infer outputs from data. The outputs can include, for example, predictions and/or classifications. Natural language processing focuses on analyzing and generating human language. Computer vision focuses on analyzing and interpreting images and videos. Artificial intelligence systems can include generative models that generate new content, such as images, videos, text, audio, and/or other content, in response to input prompts and/or based on other information.

[0072] Example machine-learned models include neural networks or other multi-layer nonlinear models. Example neural networks include feed forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some example machine-learned models can leverage an attention mechanism such as self-attention. For example, some machine-learned models can include multi-headed self-attention models (e.g., transformer models).

[0073] The model(s) can be trained using various training or learning techniques. The training can implement supervised learning, unsupervised learning, reinforcement learning, etc. The training can use techniques such as, for example, backwards propagation of errors. For example, a loss function can be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on a gradient of the loss function). Various loss functions can be used such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and/or various other loss functions. Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations. A number of generalization techniques (e.g., weight decays, dropouts) can be used to improve the generalization capability of the models being trained.

[0074] The model(s) can be pre-trained before domain- specific alignment. For instance, a model can be pretrained over a general corpus of training data and fine-tuned on a more targeted corpus of training data. A model can be aligned using prompts that are designed to elicit domain- specific outputs. Prompts can be designed to include learned prompt values (e.g., soft prompts). The trained model(s) may be validated prior to their use using input data other than the training data, and may be further updated or refined during their use based on additional feedback/inputs.

[0075] In some implementations, the computing system 104 (e.g., account restructuring application 150) may use any one or more the machine learning models noted above to perform any one or more of the operations discussed herein in connection with machine learning. For example, the computing system 104 may use one or more such machine learning models to predict the performance of a particular digital asset in connection with a particular keyword or group of keywords, as discussed above, and/or to classify a particular keyword or keyword group according to theme, etc.

[0076] Although the foregoing text sets forth a detailed description of numerous different aspects and implementations of the invention, it should be understood that the scope of the patent is defined by the words of the claims set forth at the end of this patent. The detailed description is to be construed as exemplary only and does not describe every possible implementation because describing every possible implementation would be impractical, if not impossible. Numerous alternative implementations could be implemented, using either current technology or technology developed after the filing date of this patent, which would still fall within the scope of the claims. The disclosure herein contemplates at least the following examples:

[0077] Example 1. A method of efficiently restructuring account data indicative of (i) a first plurality of keywords each mapped to a respective query space, (ii) a first plurality of campaigns, and (iii) associations between the first plurality of campaigns and the first plurality of keywords, wherein the method comprises: consolidating, by one or more processors, the first plurality of campaigns into a second plurality of campaigns consisting of fewer campaigns than the first plurality of campaigns, wherein the consolidating includes determining whether to combine a given first campaign with a given second campaign based on a degree of overlap between (i) the respective query spaces to which keywords associated with the given first campaign are mapped and (ii) the respective query spaces to which keywords associated with the given second campaign are mapped; generating, by the one or more processors, a second plurality of keywords consisting of a subset of the first plurality of keywords, wherein generating the second plurality of keywords includes, for each campaign in the second plurality of campaigns, determining whether to remove associations to particular keywords of the campaign based on an incremental value added by the query spaces that are mapped to the particular keywords; and storing, by the one or more processors, restructured account data indicative of (i) the second plurality of keywords, (ii) the second plurality of campaigns, and (iii) new associations between the second plurality of campaigns and the second plurality of keywords.

[0078] Example 2. The method of example 1, wherein: the account data is further indicative of associations between the first plurality of keywords and a first plurality of digital assets; the method further comprises, after generating the second plurality of keywords and for each campaign in the second plurality of campaigns, generating, by the one or more processors, one or more campaign-specific keyword clusters by clustering keywords of the campaign, and associating, by the one or more processors, each campaign-specific keyword cluster to at least one respective digital asset of a second plurality of digital assets, the second plurality of digital assets including all digital assets associated with at least one campaignspecific keyword cluster; and the restructured account data is further indicative of new associations between the second plurality of keywords and the second plurality of digital assets.

[0079] Example 3. The method of example 2, wherein clustering the keywords of the campaign includes clustering the keywords of the campaign according to theme.

[0080] Example 4. The method of example 2 or 3, wherein associating each campaignspecific keyword cluster to the at least one respective digital asset includes: predicting performance for each of a plurality of combinations using a machine learning model, each of the plurality of combinations being a combination of (i) a particular digital asset of the first plurality of digital assets, and (ii) at least one keyword in the campaign- specific keyword cluster; and associating the campaign- specific keyword cluster to the at least one respective digital asset based on the predicted performance for the plurality of combinations.

[0081] Example 5. The method of any one of examples 1-4, wherein the consolidating includes determining whether to combine the given first campaign with the given second campaign based on whether (i) the respective query spaces to which keywords associated with the given first campaign are mapped, and (ii) the respective query spaces to which keywords associated with the given second campaign are mapped, have at least a threshold percentage of queries in common.

[0082] Example 6. The method of any one of examples 1-4, wherein the consolidating includes: clustering the first plurality of keywords based on the degree of overlap; and determining whether to combine the given first campaign with the given second campaign based on how many keywords associated with the given first campaign and how many keywords associated with the given second campaign are in a same cluster.

[0083] Example 7. The method of any one of examples 1-6, wherein the consolidating includes determining whether to combine the given first campaign with the given second campaign based further on one or more compatibility factors associated with the given first campaign and the given second campaign, the one or more compatibility factors including one or more of: location information associated with the given first campaign and the given second campaign; performance information associated with the given first campaign and the given second campaign; or audience information associated with the given first campaign and the given second campaign.

[0084] Example 8. The method of any one of examples 1-7, wherein the consolidating includes merging multiple campaigns of the first plurality of campaigns into a single new campaign of the second plurality of campaigns such that digital assets associated with any keyword of at least one of the multiple campaigns are associated with the single new campaign.

[0085] Example 9. The method of any one of examples 1-8, wherein generating the second plurality of keywords includes, for each campaign in the second plurality of campaigns, identifying a smallest subset of keywords, from among an initial set of keywords associated with the campaign, that collectively maps to a new query space satisfying one or more predetermined requirements. [0086] Example 10. The method of example 9, wherein the one or more predetermined requirements include a requirement that the new query space corresponds to at least a minimum expected performance measure.

[0087] Example 11. The method of example 9 or 10, wherein identifying the smallest subset of keywords that collectively maps to the new query space includes: selecting a first keyword, from the initial set of keywords associated with the campaign, to add to the smallest subset of keywords based on a metric associated with the first keyword; and for each remaining keyword in the initial set of keywords, determining an incremental value of adding the remaining keyword to the new query space.

[0088] Example 12. A system for efficiently restructuring account data indicative of (i) a first plurality of keywords each mapped to a respective query space, (ii) a first plurality of campaigns, and (iii) associations between the first plurality of campaigns and the first plurality of keywords, the system comprising one or more processors and one or more non- transitory computer-readable media storing instructions. When executed by the one or more processors, the instructions cause the one or more processors to: consolidate the first plurality of campaigns into a second plurality of campaigns consisting of fewer campaigns than the first plurality of campaigns, wherein the consolidating includes determining whether to combine a given first campaign with a given second campaign based on a degree of overlap between (i) the respective query spaces to which keywords associated with the given first campaign are mapped and (ii) the respective query spaces to which keywords associated with the given second campaign are mapped; generate a second plurality of keywords consisting of a subset of the first plurality of keywords, wherein generating the second plurality of keywords includes, for each campaign in the second plurality of campaigns, determining whether to remove associations to particular keywords of the campaign based on an incremental value added by the query spaces that are mapped to the particular keywords; and store restructured account data indicative of (i) the second plurality of keywords, (ii) the second plurality of campaigns, and (iii) new associations between the second plurality of campaigns and the second plurality of keywords.

[0089] Example 13. The system of example 12, wherein: the account data is further indicative of associations between the first plurality of keywords and a first plurality of digital assets; the instructions further cause the one or more processors to, after generating the second plurality of keywords and for each campaign in the second plurality of campaigns, (1) generate one or more campaign-specific keyword clusters by clustering keywords of the campaign and (2) associate each campaign- specific keyword cluster to at least one respective digital asset of a second plurality of digital assets, the second plurality of digital assets including all digital assets associated with at least one campaign-specific keyword cluster; and the restructured account data is further indicative of new associations between the second plurality of keywords and the second plurality of digital assets.

[0090] Example 14. The system of example 13, wherein clustering the keywords of the campaign includes clustering the keywords of the campaign according to theme.

[0091] Example 15. The system of example 13 or 14, wherein associating each campaignspecific keyword cluster to the at least one respective digital asset includes: predicting performance for each of a plurality of combinations using a machine learning model, each of the plurality of combinations being a combination of (i) a particular digital asset of the first plurality of digital assets, and (ii) at least one keyword in the campaign- specific keyword cluster; and associating the campaign- specific keyword cluster to the at least one respective digital asset based on the predicted performance for the plurality of combinations.

[0092] Example 16. The system of any one of examples 12-15, wherein the consolidating includes determining whether to combine the given first campaign with the given second campaign based on whether (i) the respective query spaces to which keywords associated with the given first campaign are mapped, and (ii) the respective query spaces to which keywords associated with the given second campaign are mapped, have at least a threshold percentage of queries in common.

[0093] Example 17. The system of any one of examples 12-15, wherein the consolidating includes: clustering the first plurality of keywords based on the degree of overlap; and determining whether to combine the given first campaign with the given second campaign based on how many keywords associated with the given first campaign and how many keywords associated with the given second campaign are in a same cluster.

[0094] Example 18. The system of any one of examples 12-17, wherein the consolidating includes determining whether to combine the given first campaign with the given second campaign based further on one or more compatibility factors associated with the given first campaign and the given second campaign, the one or more compatibility factors including one or more of: location information associated with the given first campaign and the given second campaign; performance information associated with the given first campaign and the given second campaign; or audience information associated with the given first campaign and the given second campaign.

[0095] Example 19. The system of any one of examples 12-18, wherein the consolidating includes merging multiple campaigns of the first plurality of campaigns into a single new campaign of the second plurality of campaigns such that digital assets associated with any keyword of at least one of the multiple campaigns are associated with the single new campaign.

[0096] Example 20. The system of any one of examples 12-19, wherein generating the second plurality of keywords includes, for each campaign in the second plurality of campaigns, identifying a smallest subset of keywords, from among an initial set of keywords associated with the campaign, that collectively maps to a new query space satisfying one or more predetermined requirements.

[0097] Example 21. The system of example 20, wherein the one or more predetermined requirements include a requirement that the new query space corresponds to at least a minimum expected performance measure.

[0098] Example 22. The system of example 20 or 21, wherein identifying the smallest subset of keywords that collectively maps to the new query space includes: selecting a first keyword, from the initial set of keywords associated with the campaign, to add to the smallest subset of keywords based on a metric associated with the first keyword; and for each remaining keyword in the initial set of keywords, determining an incremental value of adding the remaining keyword to the new query space.

[0099] Example 23. One or more non-transitory computer-readable media for efficiently restructuring account data indicative of (i) a first plurality of keywords each mapped to a respective query space, (ii) a first plurality of campaigns, and (iii) associations between the first plurality of campaigns and the first plurality of keywords, the one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to: consolidate the first plurality of campaigns into a second plurality of campaigns consisting of fewer campaigns than the first plurality of campaigns, wherein the consolidating includes determining whether to combine a given first campaign with a given second campaign based on a degree of overlap between (i) the respective query spaces to which keywords associated with the given first campaign are mapped and (ii) the respective query spaces to which keywords associated with the given second campaign are mapped; generate a second plurality of keywords consisting of a subset of the first plurality of keywords, wherein generating the second plurality of keywords includes, for each campaign in the second plurality of campaigns, determining whether to remove associations to particular keywords of the campaign based on an incremental value added by the query spaces that are mapped to the particular keywords; and store restructured account data indicative of (i) the second plurality of keywords, (ii) the second plurality of campaigns, and (iii) new associations between the second plurality of campaigns and the second plurality of keywords.

[00100] Example 24. The one or more non-transitory computer-readable media of example 23, wherein: the account data is further indicative of associations between the first plurality of keywords and a first plurality of digital assets; the instructions further cause the one or more processors to, after generating the second plurality of keywords and for each campaign in the second plurality of campaigns, (1) generate one or more campaign-specific keyword clusters by clustering keywords of the campaign and (2) associate each campaignspecific keyword cluster to at least one respective digital asset of a second plurality of digital assets, the second plurality of digital assets including all digital assets associated with at least one campaign- specific keyword cluster; and the restructured account data is further indicative of new associations between the second plurality of keywords and the second plurality of digital assets.

[00101] Example 25. The one or more non-transitory computer-readable media of example 24, wherein clustering the keywords of the campaign includes clustering the keywords of the campaign according to theme.

[00102] Example 26. The one or more non-transitory computer-readable media of example 24 or 25, wherein associating each campaign-specific keyword cluster to the at least one respective digital asset includes: predicting performance for each of a plurality of combinations using a machine learning model, each of the plurality of combinations being a combination of (i) a particular digital asset of the first plurality of digital assets, and (ii) at least one keyword in the campaign- specific keyword cluster; and associating the campaignspecific keyword cluster to the at least one respective digital asset based on the predicted performance for the plurality of combinations.

[00103] Example 27. The one or more non-transitory computer-readable media of any one of examples 23-26, wherein the consolidating includes determining whether to combine the 1 given first campaign with the given second campaign based on whether (i) the respective query spaces to which keywords associated with the given first campaign are mapped, and (ii) the respective query spaces to which keywords associated with the given second campaign are mapped, have at least a threshold percentage of queries in common.

[00104] Example 28. The one or more non-transitory computer-readable media of any one of examples 23-26, wherein the consolidating includes: clustering the first plurality of keywords based on the degree of overlap; and determining whether to combine the given first campaign with the given second campaign based on how many keywords associated with the given first campaign and how many keywords associated with the given second campaign are in a same cluster.

[00105] Example 29. The one or more non-transitory computer-readable media of any one of examples 23-28, wherein the consolidating includes determining whether to combine the given first campaign with the given second campaign based further on one or more compatibility factors associated with the given first campaign and the given second campaign, the one or more compatibility factors including one or more of: location information associated with the given first campaign and the given second campaign; performance information associated with the given first campaign and the given second campaign; or audience information associated with the given first campaign and the given second campaign.

[00106] Example 30. The one or more non-transitory computer-readable media of any one of examples 23-29, wherein the consolidating includes merging multiple campaigns of the first plurality of campaigns into a single new campaign of the second plurality of campaigns such that digital assets associated with any keyword of at least one of the multiple campaigns are associated with the single new campaign.

[00107] Example 31. The one or more non-transitory computer-readable media of any one of examples 23-30, wherein generating the second plurality of keywords includes, for each campaign in the second plurality of campaigns, identifying a smallest subset of keywords, from among an initial set of keywords associated with the campaign, that collectively maps to a new query space satisfying one or more predetermined requirements.

[00108] Example 32. The one or more non-transitory computer-readable media of example 31, wherein the one or more predetermined requirements include a requirement that the new query space corresponds to at least a minimum expected performance measure. [00109] Example 33. The one or more non-transitory computer-readable media of example 31 or 32, wherein identifying the smallest subset of keywords that collectively maps to the new query space includes: selecting a first keyword, from the initial set of keywords associated with the campaign, to add to the smallest subset of keywords based on a metric associated with the first keyword; and for each remaining keyword in the initial set of keywords, determining an incremental value of adding the remaining keyword to the new query space.

[00110] The following additional considerations apply to the foregoing discussion. Throughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter of the present disclosure.

[00111] Unless otherwise apparent from the context of use, reference in the present disclosure to a same set of “one or more processors” (or a same “plurality of processors,” etc.) performing multiple operations can encompass implementations in which performance of the operations is divided among the processor(s) in any suitable way. For example, “generating, by one or more processors, X; and generating, by the one or more processors, Y” can encompass: (1) implementations in which a first set of one or more processors (e.g., in a first computing device) generates X and an entirely distinct, second set of one or more processors (e.g., in a different, second computing device) independently generates Y; (2) implementations in which all processors in the set of one or more processors (e.g., all in the same device, or distributed among multiple devices) contribute to the generation of both X and Y; and (3) other variations.

[00112] Unless specifically stated otherwise, discussions in the present disclosure using words such as “processing,” “computing,” “calculating,” “determining,” “presenting,” “displaying,” or the like may refer to actions or processes of a machine (e.g., a computer) that manipulates or transforms data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine components that receive, store, transmit, or display information.

[00113] As used in the present disclosure any reference to “one implementation” or “an implementation” means that a particular element, feature, structure, or characteristic described in connection with the implementation is included in at least one implementation or implementation. The appearances of the phrase “in one implementation” in various places in the specification are not necessarily all referring to the same implementation.

[00114] As used in the present disclosure, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, unless expressly stated to the contrary, “or” refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

[00115] Upon reading this disclosure, those of skill in the art will appreciate still additional alternative structural and functional designs through the principles described herein. Thus, while particular implementations and applications have been illustrated and described, it is to be understood that the disclosed implementations are not limited to the precise construction and components disclosed in the present disclosure. Various modifications, changes and variations, which will be apparent to those skilled in the art, may be made in the arrangement, operation and details of the method and apparatus disclosed in the present disclosure without departing from the spirit and scope defined in the appended claims.

Claims

What is claimed is:

1. A method of efficiently restructuring account data indicative of (i) a first plurality of keywords each mapped to a respective query space, (ii) a first plurality of campaigns, and (iii) associations between the first plurality of campaigns and the first plurality of keywords, wherein the method comprises: consolidating, by one or more processors, the first plurality of campaigns into a second plurality of campaigns consisting of fewer campaigns than the first plurality of campaigns, wherein the consolidating includes determining whether to combine a given first campaign with a given second campaign based on a degree of overlap between (i) the respective query spaces to which keywords associated with the given first campaign are mapped and (ii) the respective query spaces to which keywords associated with the given second campaign are mapped; generating, by the one or more processors, a second plurality of keywords consisting of a subset of the first plurality of keywords, wherein generating the second plurality of keywords includes, for each campaign in the second plurality of campaigns, determining whether to remove associations to particular keywords of the campaign based on an incremental value added by the query spaces that are mapped to the particular keywords; and storing, by the one or more processors, restructured account data indicative of (i) the second plurality of keywords, (ii) the second plurality of campaigns, and (iii) new associations between the second plurality of campaigns and the second plurality of keywords.

2. The method of claim 1, wherein: the account data is further indicative of associations between the first plurality of keywords and a first plurality of digital assets; the method further comprises, after generating the second plurality of keywords and for each campaign in the second plurality of campaigns, generating, by the one or more processors, one or more campaign- specific keyword clusters by clustering keywords of the campaign, and associating, by the one or more processors, each campaign- specific keyword cluster to at least one respective digital asset of a second plurality of digital assets, the second plurality of digital assets including all digital assets associated with at least one campaign- specific keyword cluster; and the restructured account data is further indicative of new associations between the second plurality of keywords and the second plurality of digital assets.

3. The method of claim 2, wherein clustering the keywords of the campaign includes clustering the keywords of the campaign according to theme.

4. The method of claim 2 or 3, wherein associating each campaign- specific keyword cluster to the at least one respective digital asset includes: predicting performance for each of a plurality of combinations using a machine learning model, each of the plurality of combinations being a combination of (i) a particular digital asset of the first plurality of digital assets, and (ii) at least one keyword in the campaign-specific keyword cluster; and associating the campaign- specific keyword cluster to the at least one respective digital asset based on the predicted performance for the plurality of combinations.

5. The method of any one of claims 1-4, wherein the consolidating includes determining whether to combine the given first campaign with the given second campaign based on whether (i) the respective query spaces to which keywords associated with the given first campaign are mapped, and (ii) the respective query spaces to which keywords associated with the given second campaign are mapped, have at least a threshold percentage of queries in common.

6. The method of any one of claims 1-4, wherein the consolidating includes: clustering the first plurality of keywords based on the degree of overlap; and determining whether to combine the given first campaign with the given second campaign based on how many keywords associated with the given first campaign and how many keywords associated with the given second campaign are in a same cluster.

7. The method of any one of claims 1-6, wherein the consolidating includes determining whether to combine the given first campaign with the given second campaign based further on one or more compatibility factors associated with the given first campaign and the given second campaign, the one or more compatibility factors including one or more of: location information associated with the given first campaign and the given second campaign; performance information associated with the given first campaign and the given second campaign; or audience information associated with the given first campaign and the given second campaign.

8. The method of any one of claims 1-7, wherein the consolidating includes merging multiple campaigns of the first plurality of campaigns into a single new campaign of the second plurality of campaigns such that digital assets associated with any keyword of at least one of the multiple campaigns are associated with the single new campaign.

9. The method of any one of claims 1-8, wherein generating the second plurality of keywords includes, for each campaign in the second plurality of campaigns, identifying a smallest subset of keywords, from among an initial set of keywords associated with the campaign, that collectively maps to a new query space satisfying one or more predetermined requirements.

10. The method of claim 9, wherein the one or more predetermined requirements include a requirement that the new query space corresponds to at least a minimum expected performance measure.

11. The method of claim 9 or 10, wherein identifying the smallest subset of keywords that collectively maps to the new query space includes: selecting a first keyword, from the initial set of keywords associated with the campaign, to add to the smallest subset of keywords based on a metric associated with the first keyword; and for each remaining keyword in the initial set of keywords, determining an incremental value of adding the remaining keyword to the new query space.

12. A system comprising: one or more processors; and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to execute the method of any one of claims 1-11.

13. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to execute the method of any one of claims 1-11.