CN113892085A

CN113892085A - Limiting provision and display of redundant digital components on a client device

Info

Publication number: CN113892085A
Application number: CN202080014909.6A
Authority: CN
Inventors: V.卡布恩; M.沙里菲
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2020-04-27
Filing date: 2020-11-23
Publication date: 2022-01-04
Also published as: WO2021221718A1; KR20210134902A; WO2021221718A8; EP3931724A1; KR102590412B1

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for limiting the provision and display of redundant digital components on a client. The method may include storing, by the client device, a digital component list specifying digital components available for provision to the client device. For a first digital component received within a first application, a client device detects a set of signals specifying a first user interaction with the first digital component and a second user interaction with content provided in response to the first user interaction. Based on whether a positive user action is performed, the client device may modify the list of digital components. Upon receiving a request to access a content page within the second application, the client device may receive a second digital component, which may be selected from among the digital components included on the modified list of digital components.

Description

Limiting provision and display of redundant digital components on a client device

Cross Reference to Related Applications

Priority is claimed in this application for RO application No. a/10018/2020 filed on 27/2020 and RO application No. a/00282/2020 filed on 22/5/2020, the disclosures of which are incorporated herein by reference.

Background

The present description is directed to limiting the provision and display of redundant digital components on a client device based at least on prior user interaction and/or user actions with such digital components and/or any additional content related thereto.

A client device may use an application (e.g., a web browser, a native application) to access a content platform (e.g., a search platform, a social media platform, or another platform hosting content). The content platform may display digital components (discrete units of digital content or digital information, such as, for example, video clips, audio clips, multimedia clips, images, text, or another unit of content) that may be provided by one or more content sources/platforms within an application launched on the client device. For example, if a browser application running on a client device is used to perform an internet search for "rental cars," the content source and/or platform may provide a search results page that includes a digital component that provides information about rental cars from a rental car company and a link to the company's website.

Disclosure of Invention

In general, one innovative aspect of the subject matter described in this specification can be embodied in methods that include the following operations: storing, by a client device, a list of digital components, the list of digital components specifying a set of digital components available for provision to an application running on the client device; receiving, within a first application running on a client device, a first digital component provided by a first content provider; detecting, by a client device, a set of signals specifying (i) a first user interaction with a first digital component and (ii) a second user interaction with content provided in response to the first user interaction with the first digital component; determining, by the client device and based on the set of signals, that a positive (affirmative) user action was performed by a user of the client device, wherein the positive user action represents performance of a specified target action by the user after a first user interaction with the first digital component; modifying, by the client device, the list of digital components based on a positive user action by the user after the first user interaction with the first digital component; receiving a request to access a content page within a second application running on the client device, wherein the first application is different from the second application; in response to receiving a request to access a content page, transmitting a content request to a second content provider, wherein the first content provider is different from the second content provider, the content request including a portion of a modified list of digital components that prevents selection of the first digital component in response to the content request; receiving, by the client device and within the second application, a second digital component from the second content provider in response to the content request, wherein the second digital component is selected from among the digital components included on the modified list of digital components; and providing a second digital component for display on a content page within a second application.

Other embodiments of this aspect include corresponding methods, apparatus, and computer programs configured to perform the actions of the methods encoded on computer storage devices. These and other embodiments may each optionally include one or more of the following features.

The method may further include inputting a set of signals associated with the first digital component into a machine learning model that predicts whether the user has a positive user action on the particular digital component based on the particular set of signals associated with the particular digital component, wherein the machine learning model is trained using training data for a plurality of training digital components, wherein the training data for each training digital component includes the set of signals associated with the training digital component and a respective label indicating whether the user has a positive user action on the training digital component; and obtaining an indication from the machine learning model and in response to the set of signals associated with the first digital component being input into the machine learning model that specifies whether the user has a positive user action on the first digital component.

The method may include storing a ranked list of digital components, modifying the list of digital components by lowering the rank corresponding to the first digital component in the ranked list or removing the first digital component from the list of digital components.

The method may include transmitting a content request to a second content provider that includes a portion of the modified list, wherein the portion of the modified list of digital components includes the top N-ranked digital components in the ranked list.

The method may further comprise: receiving, by the client device and within the second application, a third digital component from the second content provider in response to the content request, wherein the third digital component is not among the digital components included on the modified list of digital components; refraining from displaying a third digital component on the content page; and in response to the suppression, modifying a content layout of the content page and providing a message to the second content provider indicating that the third digital component is suppressed.

Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages. For example, the techniques described in this specification can filter out (e.g., prevent retrieval and/or display) certain digital components (or certain types of digital components) for which a user of a client has positive user action (as described further in this specification), thereby saving a significant amount of computing resources required in providing and rendering these digital components. This, in turn, may also facilitate improved user experience and user participation across multiple content platforms by avoiding repeated presentation of the same/similar digital components. Conventional systems do not include the ability to prevent the presentation of the same or similar digital components provided by multiple different content sources/platforms. In contrast, the techniques described in this specification may prevent the digital component(s) for which the user has positive user action from being provided and/or rendered, and thus may be able to provide and display digital components for which the user has not positive user action, rather than presenting content for which the user has experienced and taken action. By preventing the presentation of content that the user has experienced and taken action, the system reduces the amount of wasted computing resources (e.g., processing resources, network bandwidth, limited display space, etc.) that result from providing redundant content to the user. Furthermore, the limited display space of the client device is used more efficiently, since the space occupied by the already prevented content may in turn be beneficially used for other content or other purposes.

The techniques described in this document also facilitate an increase in security and privacy of data associated with processing, analyzing, and/or maintaining interaction with digital components and/or any related device actions on a client device. In some conventional implementations, when data is to be shared with and thus exposed to a third-party system, the security and/or privacy of such data may not be maintained or even feasible. In contrast, the techniques described in this specification may be implemented on a client device such that the detection, processing, and storage of a set of signals for inferring positive user actions and the determination of positive user actions may be performed and stored entirely on the client device. Furthermore, the techniques described in this specification do not require sharing of such data with third-party systems.

Further, the techniques described in this specification enable dynamic modification of an interface displayed on a client device. The techniques described in this specification may enable a client device to suppress, remove, and/or prevent the display of certain digital components for which positive user actions have occurred. In such a case, the techniques described herein may dynamically modify the interface such that the location where the suppressed/removed digital component is to be displayed is replaced by other content (e.g., content already included on the page that may be resized (e.g., reorganized/moved/resized)) or other content available from the content source/platform.

The details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

Drawings

FIG. 1 is a block diagram of an example environment in which digital content is distributed and presented to client devices.

FIG. 2 is a table illustrating an example list of digital components stored on a client device.

FIG. 3 is a flow diagram of an example process for providing one or more digital components to a client device based on user actions on the client device.

FIG. 4 is a block diagram of an example computer system that may be used to perform the described operations.

Detailed Description

The present specification relates generally to restricting the provision and display of redundant digital components and/or any additional content related thereto on a client device based at least on prior user interaction and/or user actions with such digital components.

Client devices (e.g., smartphones, tablets, and personal computers) connected to the internet may be provided with digital content including various digital components. As used throughout this document, the phrase "digital component" refers to a discrete unit of digital content or digital information (e.g., a video clip, an audio clip, a multimedia clip, an image, text, or another unit of content).

User interaction and/or user action with a digital component may ultimately result in a positive user action (affinal user action) that represents performance of a particular target event/action following presentation and/or interaction with the digital component. For example, a user who contacts a digital component for an endangered species may interact with the component (e.g., by selecting or clicking on the component) and be directed to a web page for a particular endangered species in which the user may register newsletters that are intended to help save the endangered species. In this example, the newsletter may be considered a specific target action at registration. Examples of targeted actions may also include, but are not limited to, registering with a website/service, adding items to an online shopping cart, downloading a white paper, obtaining a product, navigating to at least a given depth of a website, viewing at least a certain number of web pages, spending at least a predetermined amount of time on a website or web page, completing a website registration process, and subscribing to a digital service. In other words, the specific target action may be an interaction by the user with content provided to the user after the user interacts with the digital component. The user may perform a first interaction with first content of the digital component and then, after the first interaction, perform a second interaction with second content provided to the user as a result of the first interaction. The first interaction and the second interaction may be the generation of a signal due to one or more inputs from a user. Input from the user may be via a touch screen, keyboard, microphone, video, or any other means of interacting with the client device.

When the client device revisits the same or a different content platform within an application running on the client device, the same or similar digital components as previously provided to the client device may again be provided to the client device. Such redundant provision of such digital components may occur even if the client device previously had a positive user action on the same/similar digital components. The affirmative user action is represented by the execution of a particular target action following user interaction with the digital component. Further, in some systems, even if one content source/platform determines that the user of the client device has had a positive user action on a particular digital component, this determination of a positive user action may not be known to another content source/platform that the client device may subsequently access. As a result, the other content source/platform may provide the same or similar types of digital components that have encountered a positive user action. For example, assume that a particular digital component has been submitted to two disparate content distribution systems that do not share data. In this case, even if one content distribution system knows that a particular user completed the target action after interacting with a particular digital component, another disparate content distribution system will be completely unaware of the performance of the target action so that the particular digital component can continue to be presented to the particular user.

In contrast, the techniques described in this document may be implemented entirely (or mostly) on the client device (i.e., independent of a particular content source/platform). As further described in this specification, the client device may use the plurality of signals to determine a positive user action on the digital component based on past user activity on the client device, user interaction with the digital component on the client device independent of the content source/platform of the digital component, the content platform on which the digital component(s) are provided, or the application(s) within which the digital component is obtained. In some implementations, the plurality of signals are analyzed/processed using a machine learning model (or heuristically based, or another suitable model-based or rule-based technique) that determines whether the client device has a positive user action on the digital component.

Based on whether the client device has a positive user action on a particular digital component, the techniques described herein may modify a stored (i.e., stored on the client device) list of digital components (e.g., a cookie list) that specifies a set of digital components that are available to be provided to an application running on the client device. For example, in response to a user's interaction (e.g., selecting, clicking on, viewing) with a rental car company's digital component, the list of digital components may be updated to include a particular entry that provides an indication of the digital component (e.g., name of the digital component, category/type of the digital component, activity performed by the user while interacting with the digital component). However, if the machine learning model (or another suitable manner/model) determines that the client device has a positive user action on the digital component, the machine learning model (or other suitable manner/model) may update the list of digital components, for example, by removing a particular digital component from the list or reducing its ranking in the list (if it is already in the list) or by not adding the particular digital component to the list (if it is not already in the list).

Subsequently, when the client device uses the application to access the same or a different content platform, the techniques described herein can send a portion of the modified list of digital components (e.g., the entire list or a subset of the list, such as the top N-ranked components included on the list) to the content platform (and/or the content source(s) that provided the content for the content platform). The content platform and/or content source(s) may use the modified list of digital components to provide content for display within an application running on the client device. For example, the content source and/or the content platform can only provide one or more digital components that are included on a portion of the received list of digital components. In this case, an application running on the client device renders the digital components provided by the content source/platform. As another example, the content source(s) and/or content platform(s) may provide any one or more digital components, whether they are on a list or not. In this case, the client device may determine whether the provided digital component is on the modified list of digital components (e.g., whether the provided digital component is one of the top N digital components on the list of digital components). If so, an application running on the client device may render/display the provided digital components. Otherwise, the application may suppress such content and, in turn, modify the interface such that the location at which the suppressed/removed digital component is to be displayed is replaced by other content (e.g., content already included on the page that may be resized (e.g., reorganized/moved/resized)) or other content available from the content source/platform.

These features and additional features are further described below with reference to fig. 1-4.

In addition to the description throughout this document, a user may be provided with controls to allow the user to make selections as to whether and when the systems, programs, or features described herein are capable of collecting user information (e.g., information about the user's social network, social actions or activities, profession, user preferences, or the user's current location) and whether to communicate content or communications from a server to the user. In addition, certain data may be processed in one or more ways before being stored or used, such that personally identifiable information is removed. For example, the identity of the user may be treated so that personally identifiable information of the user cannot be determined, or the geographic location of the user may be tied to a place where location information is obtained (such as a city, ZIP code, or state level) so that a particular location of the user cannot be determined. Thus, the user may have control over what information is collected about the user, how the information is used, and what information is provided to the user.

FIG. 1 is a block diagram of an example environment 100 in which content is distributed and presented to client devices. The example environment 100 includes a network 104, such as a Local Area Network (LAN), a Wide Area Network (WAN), the internet, or a combination thereof. Network 104 connects client device 102, content platform 106, and content source 110. The example environment 100 may include many different content sources 110, content platforms 106, and client devices 102.

Content platform 106 is a computing platform capable of distributing content. Example content platforms 106 include search engines, social media platforms, news platforms, data aggregator platforms, or other content sharing platforms. Each content platform 106 may be operated by a content platform service provider.

Content platform 106 may publish and provide its own content on the platform. For example, the content platform 106 may be a news platform that publishes its own news articles. Content platform 106 may also present content provided by one or more content sources 110. In the above example, the news platform may also present content created by different authors and provided by one or more content sources 110. As another example, the content platform 106 may be a data aggregator platform that does not publish any content of its own, but aggregates and presents news articles provided by different news websites (i.e., content sources 110).

The client device 102 is an electronic device capable of requesting and receiving content over the network 104. Example client devices 102 include personal computers, mobile communication devices, digital assistant devices, and other devices capable of transmitting and receiving data over the network 104.

The client device 102 typically includes an operating system 112 that is primarily responsible for managing device hardware resources and software resources, such as applications. The client device 102 also includes device storage 120 for storing data, either temporarily or permanently, based on the particular implementation, application, and use cases. Client device 102 typically includes user applications 116 and 117 (such as web browsers) to facilitate transmitting and receiving data over network 104, but local applications run by client device 102 may also facilitate transmitting and receiving content over network 104. Examples of content presented at client device 102 include web pages, word processing documents, Portable Document Format (PDF) documents, images, videos, and search result pages and digital advertisements.

In general, the client device 102 interacts with an application running on the client device 102 to access digital content, such as search results, web pages, news articles, and social media posts. The client device may also receive digital components from one or more content providers when accessing digital content.

For example, assume that the user uses application A116 (which is a web browser) to perform an Internet search for "rental car" and to review the search results returned in response to submitting the search query "rental car". In this case, a third party (i.e., an entity other than the client device, such as a content platform or content source) may provide the application with a digital component (e.g., video, text) that may be relevant to the search query and with which the user may interact (e.g., by selecting or clicking on the digital component, or viewing content presented in the digital component for a particular period of time).

Such interaction with these digital components may cause the application to open (or redirect) another content page with additional content/digital components that may be relevant to the subject matter of the selected digital component. For example, assume that a user interacts with a digital component by selecting or clicking on a link while reviewing search results returned in response to submitting a search query "rental cars". In this example, client device 102 may open application B117, which application B117 is an application (e.g., a native application) installed on client device 102 and provided by a particular rental car company. In another example, selecting or clicking on the link may redirect the user within application A116 to another website that includes digital content related to the search query "rental car," such as a review of a different car rental service or a web page of the car rental service.

The client device 102 also includes a content evaluation apparatus 114 (which may be a data processing apparatus, as described in this specification). In some implementations, content evaluation device 114 is implemented as a machine learning model that includes a plurality of trainable parameters and is trained to determine a user positive user action on a digital component presented on client device 102 by analyzing a set of signals (as described further below). The machine learning model may be any model deemed suitable for a particular implementation, such as a decision tree, artificial neural network, genetic programming, logic programming, support vector machine, clustering, reinforcement learning, bayesian inference, and the like. The machine learning model may also include methods, algorithms, and techniques for natural language processing for analyzing signals including text data.

In some embodiments, the training data of a plurality of training digital components is used to train a machine learning model implemented within content evaluation device 114. Each training digital component in the training data is associated with a respective set of signals for the training digital component and a label indicating whether the user has a positive user action on the training digital component. For example, the training digital components include a set of signals generated by user interaction with a particular digital component (and/or actions/interactions with other content/applications/digital components on the client device 102), and a label stating whether the user action on the particular digital component is positive.

In some implementations, training the machine learning model involves adjusting trainable parameters of the machine learning model such that the machine learning model can analyze a plurality of signals generated by user activity and user interaction with the digital component to predict whether a user action on the digital component provided by the content provider is positive. Depending on the particular implementation, the training process of the machine learning model may be supervised, unsupervised, or semi-supervised.

In some implementations, the machine learning model can process a number of additional signals generated from user activity on the client device in addition to user interaction and/or user actions with particular digital components. These signals include, but are not limited to, clicks on different links provided in the digital component, internet search keywords, and the time it takes to view different digital components. For example, if a digital component having a link to a website belonging to a particular product (or service or other content) is provided to a client device during the execution of an application, a user may click on the link and access the linked website. In this case, the OS detects a user interaction (click or selection) and generates a signal that is provided to the content evaluation device. In another example, when providing a digital component including a video to a client device, the OS may detect that the user skipped the video (e.g., by scrolling through the digital component) or may take some time to view the video (e.g., by not scrolling through the page for some threshold amount of time). Thus, the OS generates a signal representative of the amount of time spent on the video digital component and provides the signal for further processing by the machine learning model.

In some implementations, in addition to the user interaction signals described above, the machine learning model implemented within the content evaluation apparatus 114 may predict positive user actions of the user on digital components presented on the client device 102 based on signals generated by the user contacting the digital components. Such implementations may use techniques well known in the art of digital image processing and natural language processing, such as Optical Character Recognition (OCR). For example, assuming that the client device is interacting with a third party application (such as an application provided by a car rental company that subscribes to rental cars), the rendered image of the user interface may be used to identify positive user actions of the user on the digital component. In another example, assume that the user gets a network push notification of a reservation confirmation for a rental car in a browser application running on the client device 102. Depending on the type of implementation, the OS may generate a plurality of signals from the network push notification, such as an image of the network push notification or text of the network push notification presented to the user on the client device 102 in the user interface. These signals may be analyzed by a machine learning model to determine positive user actions of the user on the digital component.

In some implementations, as described above, the machine learning model implemented within the content evaluation apparatus 114 processes the set of signals generated by user interaction with the digital component and other actions on the client device (e.g., a first user interaction with the digital component and one or more additional user interactions and/or user actions with content provided in response to the first user interaction) to determine whether a positive user action was performed. In some implementations, the machine learning model can generate a score or likelihood of positive user action based on the set of input signals. For example, the likelihood or score of a positive user action may be a number ranging from 0 to 10, where a number closer to 0 indicates a lower likelihood or score of a positive user action and a number closer to 10 indicates a higher likelihood/score of a positive user action. In such an embodiment, there may be a preset threshold that may suggest a positive user action if the likelihood of a positive user action is greater than the preset threshold. For example, assume that a preset threshold value of the likelihood is set to 5. If the likelihood of a user action on a digital component generated by the machine learning model is a value greater than 5, the action is determined to be positive. Otherwise, the interaction is determined not to be affirmative.

In some implementations, the content evaluation device 114 implements a heuristic-based approach (rather than a machine learning model) for determining positive user actions on digital components. In such an implementation, the content evaluation apparatus 114 analyzes the same set of signals (as described above) generated by user interaction with the digital components provided by the content source/platform to the client device 102.

In some implementations, the content evaluation apparatus 114 operates at the Operating System (OS) level on the client device 102, rather than at the application level of a particular application. As used in this specification, operations at the OS level are either operations that access higher privileges than operations at the application level and/or operations performed by the operating system.

As another example, OS-level operations may access data processed by a number of different applications and provide the data to a machine learning model (or another suitable model or heuristic-based approach) so that it may determine whether a positive user action was performed. Thus, by operating at the OS level, the machine learning model is agnostic to the application and is able to determine positive user actions by analyzing data accessed by the client device 102.

Although not shown in fig. 1, in some implementations, the content evaluation device 114 may be implemented at an application level (rather than at an OS level). In such an implementation, the application implementing the content evaluation device 114 may be provided with higher rights than other applications running at the application level of the operating system 112. In such an embodiment, the application implementing the content evaluation device 114 may have rights to access the content of another application. When an application implementing the content evaluation device 114 accesses content, the machine learning model (or another suitable model or heuristically based approach) determines whether a positive user action has occurred.

In some implementations, the client device 102 stores the digital component list 130 in the device storage 120. The list of digital components 130 (e.g., a cookie list) includes a list of digital components that are available for provision to an application running on the client device 102. The digital component list 130 will be further explained with reference to fig. 2.

As further described with reference to fig. 3, the list of digital components is modified based on a determination as to whether the client device has a positive user action on a particular digital component. As further described with reference to fig. 3, this modified list of digital components is then used by the content platform and/or content source(s) to provide the digital components for display within the application such that the provided digital components are different from the type of digital component(s) for which the client device has had a positive user action.

Fig. 2 is a table illustrating an example digital component list 130 stored on the client device 102.

In some implementations, each entry in the digital component list 130 includes a particular digital component that can be presented to the client device. Such digital components are identified on the list based on prior user interactions and/or user actions with the digital components. Each entry in the digital component list 130 may also include a number of features based on the type of digital component and the user activity. For example, the column products 202, categories 204, user actions 206, and rankings 208 represent different features or characteristics of user activity or user interaction with the digital components.

In some implementations, the features or characteristics are directly associated with or inferred from user activity or user interaction with the digital component. For example, assume that a user searches for "rental cars" and obtains a content page that includes search results returned in response to submitting a search query "rental cars" and digital components depicting a particular rental car service. In this example, an entry is created in the digital component list 130 on the client device that represents the subject matter of the particular digital component with which the user initially interacted. For example, entry 5 in the table indicates a user action to perform an internet search for rental cars. Further, in this example, the user may interact with one of these digital components (e.g., by selecting or clicking), which may redirect the user to a particular third party rental car website. In this example, an entry is created in the digital component list 130 on the client device 102 that represents the user's interaction with the digital component (web page) of a particular car rental service. For example, entry 6 in the table indicates the user's interaction to access the rental car company Q's web page, while entry 7 in the table indicates the user's interaction to access the rental car company R's web page

In some implementations, a portion of the list of digital components 130 is accessed by a third party, such as a content platform or content source (e.g., in response to being provided with the list when a blind pixel on a client device triggers, or in response to a request for the list). The third party provides the digital component to the user based on the particular entry in the list of digital components. For example, assume that a user is accessing digital content provided by a content provider using a browser application 117 installed on the client device 102. The content source/platform may transmit a request for a list of digital components to the client device 102. In response to the request for the list of digital components, the client device 102 can send a portion of the list of digital components to the content source. Upon receiving the portion of the list of digital components, the content source may provide the digital components to the client device 102 for presentation to the user based on the entries listed in the portion of the list of digital components received by the content source/platform.

In some implementations, each entry in the digital component list 130 is ranked based on a probability that a user is likely to interact with a particular digital component. For example, column 208 includes a ranking of each entry in table 130 that represents a list of digital components. In some implementations, the ranking of each entry in the digital component can be modified as described in the examples below and further described with reference to fig. 1 and 3. For example, if the machine learning model implemented within the content evaluation device 114 determines that the user has subscribed to a rental car by analyzing other digital content accessed by the browser application, the ranking of a particular entry in the digital component list 130 may be lowered. A third party, such as an advertiser or publisher, upon receiving the modified list of digital components 130 from the client device 102, provides the digital components based on the updated rankings of the digital components, thereby providing the user with digital components having a higher ranking (or among the top N ranked digital components).

In some implementations, more than one content source/platform can provide digital content to a user on the client device 102. In such an implementation, the client device 102 may store a list 130 of digital components that may be modified based on positive user actions identified using signals generated by user interaction with the digital components provisioned by the first content source/platform. Subsequently, the second content source/platform can access a portion of the digital component list 130 stored on the client device 102 and provision the digital components based on the modified digital component list 130.

For example, assume that an internet search for "camera" is performed using browser application a 116 running on the client device 102, which results in a search results page being provided within application a 116 and a digital component being provided by a first content source. If the client device 102 detects an interaction with the digital component, the client device 102 adds an entry for the digital component to the list of digital components 130. Later, during another online session, the second content source/platform may access a portion of the list of digital components 130 from the client device 102 and provide digital components related to "camera" based on the entry. If a user performs a target action (e.g., reading a comment on a particular camera on a website to which a digital component is linked) in response to an interaction with the digital component (and additional content provided in response to the interaction and additional content for which additional interactions/actions are detected by the client device), the content evaluation apparatus 114 implemented on the client device 102 may analyze the corresponding signal detected by the OS based on such interaction/action and modify the digital component list 130 on the client device, for example, by removing an entry 212 or lowering the rank of a particular entry 212. In this scenario, the first content source/platform may access the digital content list and not provide advertisements related to "camera".

FIG. 3 is a flow diagram of an example process 300 for providing one or more digital components to a client device based on user actions and/or operations of the client device. The operations of process 300 are described below as being performed by components of the systems described and depicted in fig. 1 and 2. The operations of process 300 are described below for illustrative purposes only. The operations of process 300 may be performed by any suitable device or system (e.g., any suitable data processing apparatus). The operations of process 300 may also be implemented as instructions stored on a non-transitory computer-readable medium. Execution of the instructions causes the one or more data processing apparatus to perform the operations of process 300.

The list of digital components 130 is stored on the client device and specifies a set of digital components available for provision to an application running on the client device (310). In some implementations, and as described above with reference to fig. 2, each entry in the digital component list 130 describes/represents a digital component that is available to be provided to an application running on the client device. The entries in the digital component list 130 are created based at least on user interactions and/or user actions with digital components provided by the content source/platform to the client device 102. For example, the list of digital components 130 represented using the table in FIG. 2 is stored in the device storage 120 of the client device 102. In some implementations, the entries in the digital component list 130 are ranked according to relative importance to the user. For example, column 208 displays a ranking of each item in the list. In some implementations, the entries in the list of digital components can be ranked relative to type/category.

The first digital component is provided by a first content source to an application running on a client device (320). In some implementations, the first content source can provide the first digital component for display within an application running on the client device 102. For example, browser-based application A116 running on client device 102 may be used to perform an Internet search for "rental cars". In response to the search query, the content source/platform provides search results related to the rental car to the client device 102.

A set of signals is detected by the client device, where the set of signals specifies a first user interaction with the first digital component and a second user interaction with content provided in response to the first user interaction (330). In some implementations, the OS of the client device 102 detects a first user interaction with a first digital component. Examples of such first user interactions and/or user actions include a user selecting a first digital component, clicking on the first digital component, and viewing the first digital component (e.g., for a particular period of time). The OS may detect such user interaction and/or user action by using a set of rules that capture different device events (e.g., elapsed time in a particular application, information about scrolling content on a client device, selection/clicking of a digital component, opening of a rendered content/application or page in response to selection or clicking of a digital component), and combine one or more of these rules to determine whether a first interaction was performed. Alternatively, the captured device events may be fed to a model (e.g., a machine learning model or another suitable statistical model) trained to predict whether the first interaction was performed based on a set of device actions rather than a set of rules. For example, such a model may be trained using the actually detected first interaction and the respective set of device events.

In response to the first user interaction, additional content may be provided by the first content source (or by another content source) within an application running on the client device 102. Examples of such additional content include content pages to which the user is redirected after selecting a link in search results provided by the content provider in response to submitting a search query. The OS may use a signal such as the number of clicks or time spent on the web page to detect a second user interaction with the additional content (which may include, for example, purchasing a product/service, leaving a review, or registering with a content delivery service). Likewise, the OS may detect such user interactions and/or user actions by using a set of rules that capture (obtained after performing a first user interaction) different device events of the display content and/or parse/analyze (e.g., using OCR, image processing, etc.), and combine one or more of these rules to determine whether a second interaction was performed. Alternatively, the captured device events and/or parsed/analyzed image content may be fed to a model (e.g., a machine learning model or another suitable statistical model) trained to predict whether a second interaction was performed based on the device action set instead of the rule set. For example, such a model may be trained using the actually detected second user interaction and/or user action and the respective corresponding set of device events.

In addition to the above-described interactions, the OS may also detect additional signals from other applications/services on the device regarding other device actions and/or data (as described above with reference to fig. 1). For example, if the user receives a clickable URL in an email or SMS text after selecting to open a web page showing confirmation of rental car reservation on a browser application running on the client device 102, the OS will detect a signal such as text from the web page and provide a set of signals to the content evaluation device 114.

The set of signals is used to determine whether a user performed a positive user action with respect to the first digital component (340). In some implementations, the content evaluation apparatus 114 implements a machine learning model or heuristically based approach on the client device 102 that analyzes the set of signals (as detected in operation 330 above) to determine whether a positive user action (as further described with reference to fig. 1) has occurred. For example, if a digital component is provided for display and includes a link to a website for a particular product, the user may click on the link and access the linked website. In this case, user activity generates multiple signals, such as the number of clicks and the context of the content presented in the web page. The content evaluation device 114 analyzes the signal to determine a positive user action by the user on the digital components provided by the content source/platform to the client device 102.

In some implementations, and as described with reference to fig. 1, a machine learning model implemented within the content evaluation apparatus 114 processes a set of signals generated by user interactions with the digital component (and other actions on the client device) and performs classification of whether or not to result in a positive user action based on the set of signals associated with the user interactions with the digital component (e.g., a first user interaction with the digital component and one or more additional user interactions and/or user actions with content provided in response to the first user interaction). In some implementations, the machine learning model can generate a score or likelihood of positive user action based on the set of input signals (as described with reference to fig. 1).

The digital component list 130 is modified and the modified list is stored on the client device 102 (350). If the content evaluation apparatus 114 implemented within the client device 102 identifies a positive user action for a particular digital component, it modifies the digital component list 130 stored on the client device 102. In some implementations, if the content evaluation device determines that the first digital component encounters a positive user action, the content evaluation device 114 decreases the rank of the particular entry corresponding to the first digital component in the list of digital components, e.g., so that the particular entry is no longer located in the top-N ranked component. For example, if a machine learning model implemented within the content evaluation device 114 determines that the user has subscribed to a rental car by analyzing other digital content accessed by the same browser application or by other applications, the ranking of a particular entry in the list of digital components may be lowered. Alternatively, the content evaluation device 114 may remove the reference to the digital component from the digital component list 130 instead of lowering the ranking.

A request to access a content page is received from an application running on the client device 102 (360). In order for an application running on the client device 102 to access a content page provided by a content source/platform, the application generates a request to access the content page sent to the content source/platform over the network 104. The content source, upon receiving a request to access a content page, sends the corresponding content page to the client device 102. For example, a user may perform an internet search using browser application a 116 running on client device 102, view search results provided in response to the internet search, and access a number of digital components by clicking on links provided as search results. In this scenario, the client device 102 generates a request to access a content page that includes the search results. Further, when a user clicks on a link provided as a result of a search, for example, the client device 102 generates a corresponding request to access digital content that is sent to the content source over the network 104.

A content request including a portion of the modified list of digital components is transmitted to a second content source (370). In some implementations, in response to a request to access a content page by an application running on the client device 102, a script on the content page runs and causes the content request to be provided by the application to a second content source. In some implementations, the content request includes a portion of the modified list of digital components (e.g., the entire list or a subset of the list, such as the top N-ranked digital components). The content request may also include event data specifying characteristics of the content, such as the requested electronic document, the name or network location of the server from which the digital component was requested, the name or network location of the requesting device (e.g., client device 102).

In some implementations, the portion of the list of digital components 130 provided to the content source/platform includes the top N entries in the list of digital components 130, where the top N entries are selected based on the ranking of each individual digital component based on the relative interest of the user. For example, in FIG. 2, entries 1-7 are ranked relative to the type/category of the numeric component. In this case, the client device 102 may provide a portion of the digital component list 130 that includes an entry ranked 1 to the content source/platform.

In some implementations, the portion of the list of digital components 130 provided to the content source/platform includes a combination of one or more features of the digital components listed in the top N entries of the list of digital components 130, where the top N entries are selected based on a ranking of each individual digital component based on the relative interests of the user. For example, in FIG. 2, the client device 102 may provide a portion of the digital component list 130 that includes one or more characteristics (such as a product 202 or category 204 of an entry named 1) to the content source/platform.

Based on a portion of the modified digital component list 130, a second digital component is received from the content source/platform (380). In some implementations, the content source/platform, upon receiving a portion of the list of digital components 130, provides those digital components to the client device 102 that are among the digital components listed on the received list 130 (e.g., provides digital components from the top N identified digital components on the list 130). In some scenarios, the list of digital components may be a modified list of digital components, where the list of digital components is modified based on a positive user action of the user on other digital components.

For example, assume that the portion of the digital component list 130 provided to the content source/platform includes the top N entries in the digital component list 130, where the top N entries are selected based on the ranking of each individual digital component. In this case, the portion of the digital component list 130 accessed by the content source/platform will include

entries

1, 2, and 5 that indicate the user's interest in cameras, hotels, and rental cars.

In some implementations, the content source(s) and/or content platform can provide any digital components regardless of whether they are on a list. In this case, the client device may determine whether the provided digital component is on the digital component list (e.g., whether the provided digital component is one of the top N digital components on the digital component list). If so, an application running on the client device may render/display the provided digital components. Otherwise, the application may suppress such content by, for example, not displaying it, and optionally, the application may in turn modify the interface such that the location where the suppressed/removed digital component is to be displayed is replaced with other content (e.g., content already included on the page that may be adjusted (e.g., reorganized/moved/resized), or other content) that may be obtained from the content source or content platform.

In some implementations, the client device 102 can notify a content platform or content source about the particular digital components that are suppressed and not presented/displayed on the client device 102. The content platform or content source may use this information to subsequently avoid providing such digital components on the client device 102. The content platform and/or content source may maintain/store information about the suppressed/removed content, which may be used to perform analysis that may inform the type of digital components to be provided to one or more client devices.

The digital components received from the content source/platform are provided for display within an application running on the client device 102 (390). In some implementations, the client device 102, upon receiving the digital components from the second content source, provides the digital components for display within the application. For example, assume that a content source/platform provides digital components, such as content related to cameras, hotels, and rental cars, to client device 102 upon receiving a portion of a list of digital components.

In summary, the above-described operations limit the provision and display on a client device of redundant digital components for which the client device has had a positive user action. Relatedly, the above operations have not provided and displayed digital components (whether previously provided or not) with any limitations when there has been no previous positive user action associated with such digital components.

FIG. 4 is a block diagram of an example computer system 400 that may be used to perform the operations described above. System 400 includes processor 410, memory 420, storage 430, and input/output device 440. Each of the

components

410, 420, 430, and 440 may be interconnected, for example, using a system bus 450. The processor 410 is capable of processing instructions for execution within the system 400. In some implementations, the processor 410 is a single-threaded processor. In another implementation, the processor 410 is a multi-threaded processor. The processor 410 is capable of processing instructions stored in the memory 420 or on the storage device 430.

Memory 420 stores information within system 400. In one implementation, the memory 420 is a computer-readable medium. In some implementations, the memory 420 is a volatile memory unit or units. In another implementation, the memory 420 is a non-volatile memory unit or units.

The storage device 430 is capable of providing mass storage for the system 400. In some implementations, the storage device 430 is a computer-readable medium. In various different implementations, the storage device 430 may include, for example, a hard disk device, an optical disk device, a storage device shared by multiple computing devices (e.g., cloud storage devices) over a network, or some other mass storage device.

The input/output device 440 provides input/output operations for the system 400. In some implementations, the input/output devices 440 can include one or more network interface devices, such as an ethernet card, a serial communication device (e.g., an RS-232 port), and/or a wireless interface device (e.g., an 802.11 card). In another embodiment, the input/output devices may include driver devices configured to receive input data and transmit output data to peripheral devices 460 (e.g., keyboard, printer, and display devices). However, other implementations may also be used, such as mobile computing devices, mobile communication devices, set-top box television client devices, and so forth.

Although an example processing system has been described in fig. 4, implementations of the subject matter and the functional operations described in this specification can be implemented in other types of digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.

Embodiments of the subject matter and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage medium(s) for execution by, or to control the operation of, data processing apparatus. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by the data processing apparatus. The computer storage media may be or be embodied in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Further, although the computer storage medium is not a propagated signal, the computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium may also be or be included in one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

The operations described in this specification may be implemented as operations performed by data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

The term "data processing apparatus" encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or a combination of multiple of the foregoing or the foregoing. The apparatus can comprise special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus can include, in addition to hardware, code that creates a runtime environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and operating environment may implement a variety of different computing model infrastructures, such as web services, distributed computing and grid computing infrastructures.

A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be run on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for performing actions in accordance with the instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such a device. Further, the computer may be embedded in another device, e.g., a mobile telephone, a Personal Digital Assistant (PDA), a mobile audio or video player, a game player, a Global Positioning System (GPS) receiver, or a portable storage device (e.g., a Universal Serial Bus (USB) flash drive), to name a few. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example: semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, speech, or tactile input. Further, the computer may interact with the user by transmitting and receiving documents to and from the device used by the user; for example, by transmitting a web page to a web browser on the user's client device in response to a request received from the web browser.

Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described is this specification, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), intranets (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, the server transmits data (e.g., HTML pages) to the client device (e.g., for displaying data to and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., a result of the user interaction) may be received at the server from the client device.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated within a single software product or packaged into multiple software products.

Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. Moreover, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may be advantageous.

Claims

1. A computer-implemented method, comprising:

storing, by a client device, a list of digital components, the list of digital components specifying a set of digital components available to be provided to an application running on the client device;

receiving, within a first application running on the client device, a first digital component provided by a first content provider;

detecting, by the client device, a set of signals specifying (i) a first user interaction with the first digital component and (ii) a second user interaction with content provided in response to the first user interaction with the first digital component;

determining, by the client device and based on the set of signals, that a positive user action was performed by a user of the client device, wherein the positive user action represents performance of a specified target action by the user after the first user interaction with the first digital component;

modifying, by the client device, the list of digital components based on the positive user action by the user after the first user interaction with the first digital component;

receiving a request to access a content page within a second application running on the client device;

in response to receiving the request to access a content page, transmitting a content request to a second content provider that includes a portion of a modified list of digital components that prevents selection of the first digital component in response to the content request;

receiving, by the client device and within the second application, a second digital component from the second content provider in response to the content request, wherein the second digital component is selected from among the digital components included on the modified list of digital components; and

providing the second digital component for display on a content page within the second application.

2. The computer-implemented method of claim 1, wherein determining that a positive user action was performed by the user of the client device comprises:

inputting the set of signals associated with the first digital component into a machine learning model that predicts whether the user has a positive user action on a particular digital component based on the particular set of signals associated with the particular digital component, wherein:

the machine learning model is trained using training data for a plurality of training digital components, wherein the training data for each training digital component includes a set of signals associated with the training digital component and a respective label indicating whether a user has a positive user action on the training digital component; and

obtaining, from the machine learning model and in response to the set of signals associated with the first digital component input into the machine learning model, an indication specifying whether the user has a positive user action on the first digital component.

3. The computer-implemented method of claim 1 or 2, wherein storing the list of digital components comprises storing a ranked list of digital components.

4. The computer-implemented method of claim 3, wherein modifying the list of digital components comprises lowering a rank corresponding to the first digital component in the ranked list.

5. The computer-implemented method of claim 4, wherein the portion of the modified list of digital components includes a top N digital component in the ranked list.

6. The computer-implemented method of claim 1 or 2, wherein modifying the list of digital components comprises removing the first digital component from the list of digital components.

7. The computer-implemented method of any preceding claim, wherein:

receiving, by the client device and within the second application, a third digital component from the second content provider in response to the content request, wherein the third digital component is not among the digital components included on the modified list of digital components; and

refraining from displaying the third digital component on the content page.

8. The computer-implemented method of claim 7, wherein:

modifying a content layout of the content page in response to the suppressing.

9. The computer-implemented method of claim 7 or 8, further comprising:

providing a message to the second content provider indicating that the third digital component is suppressed.

10. The computer-implemented method of any preceding claim, wherein the first content provider is different from the second content provider and the first application is different from the second application.

11. A system, comprising:

12. The system of claim 11, wherein determining that a positive user action was performed by the user of the client device comprises:

13. The system of claim 11 or 12, wherein storing the list of digital components comprises storing a ranked list of digital components.

14. The system of any preceding claim, wherein:

refraining from displaying the third digital component on the content page.

15. The system of any preceding claim, wherein the first content provider is different from the second content provider and the first application is different from the second application.

16. A non-transitory computer-readable medium storing instructions that, when executed by one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising:

modifying, by a client device, the list of digital components based on the positive user action by the user after the first user interaction with the first digital component;

in response to receiving the request to access a content page, transmitting a content request to a second content provider that includes a portion of the modified list of digital components that prevents selection of the first digital component in response to the content request;

17. The non-transitory computer-readable medium of claim 16, wherein determining that a positive user action was performed by the user of the client device comprises:

18. The non-transitory computer-readable medium of claim 16 or 17, wherein storing the list of digital components comprises storing a ranked list of digital components.

19. The non-transitory computer-readable medium of any preceding claim, wherein:

refraining from displaying the third digital component on the content page.

20. The non-transitory computer-readable medium of any preceding claim, wherein the first content provider is different from the second content provider and the first application is different from the second application.