KR20210134902A - Restrict the provision and display of redundant digital components on client devices - Google Patents

Abstract

A method, system and apparatus comprising a computer program encoded on a computer storage medium for limiting the presentation and display of redundant digital components at a client. The method may include storing, by the client device, a list of digital components specifying digital components capable of providing to the client device. For a first digital component received within the first application, the client device generates a set of signals specifying a first user interaction with the first digital component and a second user interaction with content provided in response to the first user interaction. detect Based on whether a positive user action has been performed, the client device may modify the list of digital components. Upon receiving the request for access to the content page in the second application, the client device may receive a second digital component, which may be selected from among the digital components included in the modified list of digital components.

Description

Restrict the provision and display of redundant digital components on client devices

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Romanian Patent Application No. A/10018/2020, filed on April 27, 2020, and Romanian Patent Application No. A/00282/2020, filed on May 22, 2020, the contents of which are The entirety is incorporated herein by reference.

This disclosure relates to limiting the presentation and display of redundant digital components at a client device based at least on previous user interactions and/or user actions on such digital components and/or additional content related thereto.

A client device may use an application (eg, a web browser, native application) to access a content platform (eg, a search platform, social media platform, or other platform hosting the content). The content platform displays digital components (eg, video clips, audio clips, multimedia clips, images, text or other units of content) that may be provided by one or more content sources/platforms within an application launched on the client device. can do. For example, if a browser application running on the client device is used to perform an internet search for "rental car", the content source and/or platform may obtain information about the car rental from the car rental company and links to their websites. A search result page including the provided digital component may be provided.

Generally, one innovative aspect of the invention described herein comprises the steps of: storing, by a client device, a list of digital components specifying a set of digital components that can be provided to an application running on the client device; receiving, within a first application executing on the client device, a first digital component provided by a first content provider; signals specifying, by the client device, (i) a first user interaction with a first digital component and (ii) a second user interaction with content provided in response to the first user interaction with the first digital component. detecting a set; determining, by the client device, based on the set of signals, that a positive user action has been performed by a user of the client device, the positive user action being performed after a first user interaction with the first digital component represent performance of a specified target action by the user; modifying, by the client device, the list of digital components based on a positive user action by the user after the first user interaction with the first digital component; receiving a request to access a content page in a second application executing on the client device, the first application being different from the second application; in response to receiving the request to access the page of content, sending a request for content including a portion of the modified list of digital components to a second content provider, wherein the portion of the modified list of digital components is prevent selection of a first digital component in response to the request for content, wherein a first content provider is different from the second content provider; receiving, by the client device, within a second application, a second digital component from the second content provider in response to the request for the content, the second digital component being added to the modified list of digital components selected from among the included digital components; and providing the second digital component for display on a content page in the second application.

Another embodiment of these aspects includes computer programs encoded in corresponding methods, apparatuses and computer storage devices configured to perform the actions of the above methods. These or other embodiments may each optionally include one or more of the following configurations.

Methods include inputting a set of signals associated with the first digital component into a machine learning model, wherein the machine learning model is based on a particular set of signals associated with a particular digital component, wherein the user is positive with the particular digital component. Predict whether an action has been taken, wherein the machine learning model is trained using training data for a plurality of training digital components, the training data for each training digital component comprising a set of signals associated with the training digital component and a user a corresponding label indicating whether a positive user action was taken with the training digital component; and in response to a set of signals associated with the first digital component input to the machine learning model, obtaining, from the machine learning model, an indication specifying whether a user performed a positive user action with the first digital component. includes steps.

The methods include storing a ranked list of digital components, modifying a list of digital components by lowering a rank corresponding to the first digital component in the ranked list, or removing the first digital component from the list of digital components. includes steps.

The methods may include sending a request to a second content provider for content that includes a portion of the modified list, wherein the portion of the modified list of digital components includes a top N ranked digital component in the ranked list. do.

The methods include receiving, by the client device, within a second application, a third digital component from the second content provider in response to the request for the content, the third digital component modifying the digital components None of the digital components included on this list; inhibiting the third digital component from being displayed on the content page; and in response to the suppression, modifying the content layout of the content page and providing a message to the second content provider indicating that the third digital component has been suppressed.

Certain embodiments of the invention described herein may be implemented to realize the following advantages. For example, the techniques described herein can be used to filter (eg, search for and/or display) a particular digital component (or type of digital component) for which a user of a client has already taken a positive user action (as further described herein). ), which saves a significant number of computing resources required for the provision and rendering of such digital components. It can also promote improved user experience and user engagement across multiple content platforms by preventing repeated presentation of the same/similar digital components. The existing system did not include a function to prevent the presentation of the same or similar digital components provided by different content sources/platforms. In contrast, the techniques described herein can prevent the provision and/or rendering of digital component(s) in which the user has already taken a positive user action, so that the user can instead of presenting content that the user has already experienced and taken action on. Enables presentation and presentation of digital components that have not yet taken positive user action. By preventing the presentation of content that the user has already experienced and taken action on, the system reduces the amount of wasted computing resources (e.g. processing resources, network bandwidth, limited display space, etc.) incurred by providing duplicate content to the user. . In addition, the limited display space of the client device is used more efficiently because the space that would have been occupied by the prevented content can be advantageously used for other content or other purposes.

The techniques described in this document also enhance the security and privacy protection of data associated with processing, analysis and/or maintenance of digital components and/or interactions of client devices with associated device actions. In some conventional implementations, the security and/or privacy of the data is not maintained when such data is shared with and exposed to third party systems, or even the security and/or privacy of the data may not be feasible. In contrast, the techniques described herein may be implemented in a client device such that detection, processing, and storage of a positive user action and a set of signals used to infer a determination of a positive user action may be performed, and the client Allows it to be stored globally on the device. Moreover, the techniques described herein do not require sharing such data with third-party systems.

In addition, the techniques described herein enable dynamic modification of interfaces displayed on client devices. The techniques described herein allow a client device to suppress, remove, and/or prevent display of certain digital components for which a positive user action has already occurred. In such cases, the techniques described herein can be used for content that is different from where the suppressed/removed digital component will be displayed (e.g., content already contained in a page that can be adjusted (e.g., reconfigured/moved/resized)) or content It should be replaced with other content available from the source/platform.

The details of one or more embodiments of the invention herein are set forth in the accompanying drawings and the description below. Other configurations, aspects and advantages of the invention will become apparent from the description, drawings and claims.

1 is a block diagram of an exemplary environment in which digital content is distributed and provided to client devices.
2 is a table showing an exemplary list of digital components stored in a client device.
3 is a flow diagram of an example process for providing one or more digital components to a client device based on a user action on the client device.
4 is a block diagram of an example computer system that may be used to perform the described operations.

This disclosure generally relates to limiting the presentation and display of redundant digital components at a client device based at least on previous user interactions and/or user actions on such digital components and/or additional content related thereto.

Digital content including various digital components may be provided to a client device (eg, a smartphone, tablet, personal computer) connected to the Internet. As used throughout this document, the phrase “digital component” refers to an individual unit of digital content or digital information (eg, video clip, audio clip, multimedia clip, image, text, or other unit of content).

A user interaction and/or user action with a digital component can ultimately result in a positive user action, which represents the presentation of the digital component and/or performance of a particular target event/action after interaction with it. For example, a user exposed to a digital component about an endangered species can interact with that component (e.g., by selecting or clicking on that component) and can navigate to a web page about a particular endangered species; , where users can sign up for a newsletter that helps save endangered species. In this example, subscribing to the newsletter may be considered a specific targeted action. Examples of targeted actions include registering for a website/service, adding an item to an online shopping cart, downloading a white paper, purchasing a product, browsing to at least a given depth of a website, viewing at least a specific number of webpages, a predetermined time on the website or webpage. This may include, but is not limited to, sending more than one, completing a website registration process, and subscribing to a digital service. In other words, the specific target action may be the user's interaction with the content provided to the user after the user's interaction with the digital component. The user may perform a first interaction with the first content of the digital component, and, following the first interaction, may perform a second interaction with the second content provided to the user due to the first interaction. The first and second interactions may be generation of a signal due to one or more inputs from a user. Input from the user may be via a touchscreen, keypad, microphone, video, or any other means of interacting with the client device.

When the client device revisits the same or a different content platform within an application running on the client device, a digital component that is the same or similar to a digital component previously provided to the client device may be provided back to the client device. Duplicate provision of the same/similar digital component may occur even if the client device has previously taken a positive user action on this digital component. A positive user action is indicated by performing a specific target action after user interaction with the corresponding digital component. Moreover, in some systems, even if one content source/platform determines that a user of a client device has taken a positive user action on a particular digital component, the determination of a positive user action may not affect other content that the client device can then access. It may not be known to the source/platform. As a result, other content sources/platforms may provide digital components of the same or similar type for which positive user action has already occurred. For example, suppose a particular digital component is submitted to two different content distribution systems that do not share data. In this situation, even if one of the content distribution systems recognizes that a particular user has completed the targeted action after interacting with a particular digital component, the other separate content distribution system will not be fully aware of the performance of the targeted action, and thus the particular digital component It may continue to be presented to a particular user.

In contrast, the techniques described herein may be implemented entirely (or in significant portions) on the client device (ie independent of the particular content source/platform). As further described herein, the client device is independent of past user activity on the client device, user interactions with the digital component, the content source/platform of the digital component, the client device, the content platform on which the digital component(s) were provided. Alternatively, the digital component may determine a positive user action with the digital component using multiple signals based on the obtained application(s). In some implementations, the multiple signals use/process a machine learning model (or a heuristic-based approach, or other suitable model-based or rule-based technique) to determine whether the client device has taken a positive user action with the digital component. do.

Based on whether the client device has taken a positive user action with a particular digital component, the techniques described herein may modify a stored list (ie, stored on the client device) of digital components (eg, a cookie list). , which specifies the set of digital components available to be provided to an application running on the client device. For example, in response to a user's interaction (eg, selection, click, inquiry) with a digital component of a rental car company, the list of digital components may be updated to include a specific entry providing a representation of the digital component (eg, For example, the name of the digital component, the category/type of the digital component, the activity the user performed while interacting with the digital component). However, if the machine learning model (or other suitable approach/model) determines that the client device has performed a positive user action on the digital component, then the machine learning model (or other suitable approach/model) may, for example, list You can update the list of digital components by removing a particular digital component from the list, lowering its rank from the list (if it is already on the list), or not adding a particular digital component to the list (if it is not on the list).

Consequently, when a client device uses an application to access the same or a different content platform, the techniques described herein can be combined with a modified list of digital components (eg, the entire list or the top N ranking components included in the list). subset of the same list) to the content platform (and/or the content source(s) that provide content to the content platform). The content platform and/or content source(s) may use the revised list of these digital components when providing content for display within an application running on the client device. For example, the content source and/or content platform may only provide one or more digital components included in the received portion of the digital component list. In this case, the application running on the client device renders the digital component provided by the content source/platform. As another example, the content source(s) and/or content platform(s) may provide any one or more digital components, whether or not on the list. In such a case, the client device may determine whether the provided digital component is in the modified list of digital components (eg, whether the provided digital component is one of the top N digital components in the list of digital components). If so, the application running on the client device may render/display the provided digital component. Alternatively, the application may suppress such content, and instead modify the interface so that the location in which the suppressed/removed digital component is to be displayed may be adjusted (e.g., reconfigured/moved/resized) to other content. content already included on the page) or other content available from the content source/platform.

These and further configurations are described in detail below with reference to FIGS. 1-4 .

In addition to the description of this document, the user may use the systems, programs, or configurations described herein to collect user information (eg, information about the user's social network, social actions or activities, occupation, user's Controls may be provided to the user that allow the user to make choices regarding when to make preferences or when the user's current location) and when the user is sent to the content or communication from the server. Additionally, certain data is treated in one or more various ways before it is stored or used, so that personally identifiable information is removed. For example, the user's identity is treated such that personally identifiable information about the user cannot be determined, or the user's geographic location is generalized (at the city, zip code, or state level) from where the location information was obtained. A specific location cannot be determined. Thus, the user can have control over what information about the user is collected, how the information is used, and what information is provided to the user.

1 is a block diagram of an example environment 100 in which content is distributed and provided to client devices. The example environment 100 includes a network 104 such as a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof. The network 104 connects the client device 102 , the content platform 106 and the content source 110 . Example environment 100 may include many different content sources 110 , content platforms 106 , and client devices 102 .

Content platform 106 is a computing platform that enables distribution of content. Exemplary content platforms 106 include a search engine, social media platform, news platform, data aggregator platform, or other content sharing platform. Each content platform 106 may be operated by a content platform service provider.

The content platform 106 may publish and make available its content on the platform. For example, the content platform 106 may be a news platform that publishes its own news articles. The content platform 106 may also present content provided by one or more content sources 110 . In the example above, the news platform may also present content created by different authors and provided by one or more content sources 110 . As another example, the content platform 106 may be a data aggregator platform that aggregates and presents news articles provided by other news websites (ie, content sources 110 ) without publishing its own content.

The client device 102 is an electronic device capable of requesting and receiving content over the network 104 . Exemplary client device 102 includes personal computers, mobile communication devices, digital assistant devices, and other devices capable of sending and receiving data over network 104 .

The client device 102 generally includes an operating system 112 that primarily manages software resources such as device hardware and applications. Client device 102 also includes device storage 120 for temporarily or permanently storing data based on a particular implementation, application, and use case. The client device 102 generally includes user applications 116 and 117 , such as web browsers, to support sending and receiving data over the network 104 , but a native application executed by the client device 102 . ) may also support transmission and reception of content through the network 104 . Examples of content presented to the client device 102 include web pages, word processing documents, Portable Document Format (PDF) documents, images, video and search results pages, and digital advertisements.

Generally, client device 102 interacts with applications running on client device 102 to access digital content such as search results, web pages, news articles, and social media posts. While accessing digital content, the client device may also receive digital components from one or more content providers.

For example, it is assumed that a user performs an Internet search for "rental car" using application A 116, which is a web browser, and reviews search results returned in response to submission of a "rental car" search query. In such a scenario, a third party (i.e., an entity other than a client device, such as a content platform or content source) may be relevant to a search query and the user may interact with (e.g., selecting or clicking on a digital component, or accessing the digital component for a specified period of time). View the displayed content) Digital components (eg video, text) can be provided to the application.

Such interaction with this digital component may result in the application opening (or redirecting) another content page with additional content/digital components that may be relevant to the subject of the selected digital component. For example, assume that a user interacts with a digital component by selecting or clicking a link while reviewing search results returned in response to a submission of a search query for "car rental". In this example, client device 102 may open Application B 117 , which is an application (eg, a native application) installed on client device 102 and provided by a particular rental car company. In another example, selecting or clicking a link takes the user within application A 116 to another website that contains digital content related to the search query "rental car", such as a review of another car rental service or a webpage of a car rental service. can be redirected.

The client device 102 further includes a content rating apparatus 114 (which may be a data processing apparatus as described herein). In some implementations, the content evaluation apparatus 114 is implemented as a machine learning model comprising a number of trainable parameters and positive user actions of the user with digital components presented to the client device 102 by analyzing a set of signal sets. is trained to determine (described further below). The machine learning model may be any model deemed suitable for a particular implementation, such as decision trees, artificial neural networks, genetic programming, logic programming, support vector machines, clustering, reinforcement learning, Bayesian inference, and the like. Machine learning models may also include methods, algorithms and techniques for natural language processing to analyze signals comprising textual data.

In some implementations, machine learning models implemented within content evaluation device 114 are trained using training data for a number of training digital components. Each training digital component of the training data is associated with a set of corresponding signals for the training digital component and a label indicating whether the user has taken a positive user action on the training digital component. For example, the training digital component may be a set of signals generated by a user interaction with a particular digital component (and/or actions/interactions with other content/applications/digital components on the client device 102) and for a particular digital component. Contains a label indicating whether the user action is positive.

In some implementations, training the machine learning model includes adjusting the trainable parameters of the machine learning model so that the machine learning model analyzes a number of signals generated by user activity and user interaction with digital components to obtain content. Predict whether a user action on a digital component provided by a provider is positive. Depending on the particular implementation, the training process of a machine learning model may be supervised, unsupervised, or semi-supervised.

In some implementations, in addition to user interactions and/or user actions with particular digital components, the machine learning model may process a number of additional signals generated from user activity on the client device. Such signals include, but are not limited to, clicks on other links provided on the digital component, internet search keywords, and time spent viewing other digital components. For example, if a client device is presented with a digital component with a link to a website related to a particular product (or service or other content) during application execution, the user may click on the link and visit the linked website. In this case, the OS detects the user interaction (click or selection), generates a signal, and provides it to the content evaluation device. In another example, when a client device is presented with a digital component that includes a video, the OS may indicate that the user has skipped the video (eg, by scrolling past the digital component) or spend some time watching the video. Presence (eg, page scrolling for a certain threshold amount of time) can be detected. The OS thus generates a signal representing the time spent on the video digital component and provides this signal for further processing by the machine learning model.

In some implementations, the machine learning model implemented within the content rating apparatus 114 is based on a digital component based on a signal generated by a user exposure to a digital component presented at the client device 102 in addition to the user interaction signal described above. It is possible to predict the user's positive user action for Such implementations may use techniques well known in the fields of natural language processing and digital image processing, such as optical character recognition (OCR). For example, suppose a client device interacts with a third-party application, such as an application provided by a rental car company, to book a rental car, and an image in the rendered user interface could be used to identify a user's positive user action with a digital component. can In another example, it is assumed that the user receives a web push notification in a browser application running on the client device 102 in connection with a reservation confirmation of a rental car. Based on the type of implementation, the OS may generate a number of signals from the web push notification, such as the text of the web push notification or an image of a user interface presenting the web push notification to the user at the client device 102 . These signals can be analyzed by a machine learning model to determine a positive user action of the user on the digital component.

In some implementations, the machine learning model implemented within the content rating apparatus 114 processes user interactions with digital components and other actions on the client device as described above (eg, first user interaction and one or more further user interactions and/or user actions with content provided in response to said first user interaction), determine whether a positive user action has been performed. In some implementations, the machine learning model may generate a score or likelihood of a positive user action based on an input set of signals. For example, the likelihood or score of a positive user action may be a number from 0 to 10, where closer to 0 indicates a lower likelihood or score of positive user action and closer to 10 indicates a greater likelihood of positive user action. High probability/score. In such implementations, there may be a preset threshold that may suggest a positive user action if the likelihood of a positive user action is greater than the preset threshold. For example, suppose that the preset threshold of likelihood is set to 5. If the machine learning model generates a probability of a user action on a digital component with a value greater than 5, the action is determined to be positive. Otherwise, it is determined that the interaction is not positive.

In some implementations, the content rating device 114 implements a heuristic-based approach (instead of a machine learning model) to determine positive user actions on digital components. In such an implementation, the content rating apparatus 114 analyzes the same set of signals (as described above) generated by user interaction with digital components provided to the client device 102 by the content source/platform. do.

In some implementations, the content rating apparatus 114 operates at the operating system (OS) level on the client device 102 as opposed to operating at the application level for a particular application. As used herein, an operation at the OS level is an operation performed by the operating system and/or a task that accesses higher privileges than an operation at the application level.

As a further example, OS level actions may access data that is processed by a number of different applications and provide this data to a machine learning model (or other suitable model or heuristic-based approach) so that a positive user action is performed. You can decide whether or not Thus, by operating at the OS level, the machine learning model is application agnostic and can determine positive user actions by analyzing data accessed by the client device 102 .

Although not shown in FIG. 1 , in some implementations, the content evaluation device 114 may be implemented at the application level (instead of being implemented at the OS level). In this implementation, the application implementing the content rating device 114 may be provided with higher privileges than other applications running at the application level of the operating system 112 . In this implementation, an application implementing the content rating device 114 may have permission to access the content of other applications. When content is accessed by an application implementing content rating device 114, a machine learning model (or other suitable model or heuristic-based approach) determines whether a positive user action has occurred.

In some implementations, client device 102 stores a list of digital components 130 in device storage 120 . The list of digital components 130 (eg, a cookie list) includes a list of digital components that can be provided to an application running on the client device 102 . The list of digital components 130 is further described with reference to FIG. 2 .

As further described with reference to FIG. 3 , this list of digital components is modified based on a determination as to whether the client device has had a positive user action on the particular digital component. As further described with reference to FIG. 3 , this modified list of digital components is subsequently used by the content platform and/or content source(s) to provide digital components for display within an application, such that the provided digital Make the components different from the digital component(s) (or types of digital component(s)) for which the client device already had a positive user action.

2 is a table showing an exemplary list of digital components 130 stored on a client device 102 .

In some implementations, each entry in the list of digital components 130 includes a particular digital component that can be presented to a client device. Digital components are identified in this list based on user actions and/or previous user interactions with those digital components. Each entry in the list of digital components 130 may further include a number of features based on the type of digital component and user activity. For example, product 202 , category 204 , user action 206 , and rank 208 columns represent other features or characteristics of user activity or user interaction with a digital component.

In some implementations, the features or characteristics are directly associated with or inferred from user activity or user interaction with the digital components. For example, suppose a user searches for "rental car" and obtains a content page containing search results returned in response to a submission of a search query of "rental car" and a digital component representing a particular rental car service. In this example, an entry is created in the digital component list 130 of the client device representing the subject of a particular digital component with which the user initially interacted. For example, entry 5 in the table represents a user action to perform an Internet search for a rental car. Also in this example, the user may interact (eg, select or click) one of these digital components, which may redirect the user to a specific third-party car rental website. In this example, an entry is created in the digital component 130 list of the client device 102 indicating that the user has interacted with a digital component (webpage) of a particular rental car service. For example, entry 6 of the table represents an interaction of a user visiting a web page of a rental car company Q, and entry 7 of a table represents an interaction of a user visiting a web page of a rental car company R.

In some implementations, a portion of the list of digital components 130 is accessed by a third party, such as a content platform or content source (eg, in response to being provided with this list when a blind pixel of a client device occurs). or in response to a request for this list). Based on a particular entry in the list of digital components, the third party provides the digital component to the user. For example, assume that a user is accessing digital content provided by a content provider using a browser application 117 installed on a client device 102 . The content source/platform may send a request to the client device 102 for a list of digital components. In response to the request for the list of digital components, the client device 102 may send a portion of the list of digital components to the content source. Upon receiving the portion of the list of digital components, the content source may provide the digital components to the client device 102 for presentation to the user based on entries listed in the portion of the list of digital components received by the content source/platform. have.

In some implementations, each entry in the list of digital components 130 is ranked based on a probability that a user can interact with a particular digital component. For example, column 208 contains the rank of each entry in table 130 representing a list of digital components. In some implementations, the rank of each entry of the digital component may be modified as further described with reference to FIGS. 1 and 3 and as described in the examples below. For example, if a machine learning model implemented in the content evaluation device 114 analyzes other digital content accessed by the browser application and determines that the user has booked a rental car, the ranking may be lowered. When a third party, such as an advertiser or publisher, receives the modified list of digital components 130 from the client device 102 , it provides the digital component based on the updated ranking of the digital component to give the user a higher ranking (or It provides digital components (in the top N ranking digital components).

In some implementations, one or more content sources/platforms may provide digital content to users on client device 102 . In such an implementation, the client device 102 provides digital components that may be modified based on positive user action identified using signals generated by user interaction with digital components provided by the first content source/platform. A list of components 130 may be stored. The second content source/platform may then access a portion of the list of digital components 130 stored on the client device 102 and provide the digital component based on the modified list of digital components 130 .

For example, an Internet search for “camera” is performed using a browser application A 116 running on the client device 102 , resulting in a first content source within application A 116 of the search results page. digital components are provided. When the client device 102 detects an interaction with the digital component, the client device 102 adds an entry for the digital component to the list of digital components 130 . Later, during another online session, the second content source/platform may access a portion of the list of digital components 130 from the client device 102 and provide the digital component associated with the “camera” based on the entry. In response to a user's interaction with a digital component (and additional content provided in response to that interaction and for which the additional interaction/action is detected by the client device), a target action (e.g., on a website linked by the digital component reading the review), the content evaluation apparatus 114 implemented in the client device 102 may analyze the corresponding signal detected by the OS based on such interactions/actions, for example, an entry The list of digital components 130 at the client device may be modified by removing 212 or by lowering the rank of a particular entry 212 . In such a scenario, the first content source/platform may access the digital content listing and not provide advertisements related to “camera”.

3 is a flow diagram of an example process 300 for providing one or more digital components to a client device based on a user action and/or action of the client device. The operations of process 300 are described below as being performed by the components of the system described and illustrated in FIGS. 1 and 2 . The operation of process 300 is described for purposes of illustration only. The operations of process 300 may be performed by any suitable device or system, for example any suitable data processing apparatus. The operations of process 300 may also be implemented as instructions stored on a non-transitory computer-readable medium. Execution of the instructions causes one or more data processing devices to perform the operations of process 300 .

The list of digital components 130 specifies ( 310 ) a set of digital components that can be stored on the client device and provided to an application executing on the client device. In some implementations, and as described above with reference to FIG. 2 , each entry in the list of digital components 130 describes or represents a digital component that may be provided to an application executing on a client device. The entry in the list of digital components 130 is generated based at least on user interactions and/or user actions with the digital components provided to the client device 102 by the content source/platform. For example, the list of digital components 130 represented using the table of FIG. 2 is stored in the device storage 120 of the client device 102 . In some implementations, entries in the list of digital components 130 are ranked according to their relative importance to the user. For example, column 208 shows the rank of each entry in the list. In some implementations, entries in the digital component list may be ranked with respect to type/category.

The first digital component is provided ( 320 ) by the first content source to an application executing on the client device. In some implementations, the first content source may provide a first digital component for display within an application executing on the client device 102 . For example, browser-based application A 116 running on client device 102 may be used to perform an Internet search for “rental car”. In response to the search query, the content source/platform provides search results related to the rental car to the client device 102 .

A set of signals is detected by the client device, the set of signals specifying ( 330 ) a first user interaction with a first digital component and a second user interaction with content provided in response to the first user interaction. In some implementations, the OS of the client device 102 detects a first user interaction with the first digital component. Examples of such first user interaction and/or user action include the user selecting the first digital component, clicking the first digital component, viewing the first digital component (eg, for a specified period of time) do. The OS may monitor various device events (e.g., elapsed time in a particular application, information about content scrolling on the client device, digital component selection/click, content presentation, application or page opened in response to digital component selection or click)). The capturing set of rules may be used to detect such user interactions and/or user actions, one or more of which may be combined to determine whether the first interaction was performed. Alternatively, instead of a set of rules, the captured device events may be provided to a model (e.g., a machine learning model or other suitable statistical model) trained to predict whether a first interaction has been performed based on a set of device actions. have. Such a model may be trained using, for example, the first interaction actually detected and each corresponding set of device events.

In response to the first user interaction, additional content may be provided within an application executing on the client device 102 by the first content source (or other content source). Examples of such additional content include content pages to which a user is redirected after selecting a link to a search result provided by a content provider in response to a submission to a search query. The OS may use signals such as clicks or time spent on a webpage to detect a second user interaction with this additional content (which may include, for example, purchasing a product/service, leaving a review, or subscribing to a content delivery service). . Again, the OS combines one or more of these using a set of rules that capture other device events and/or parse/analyze the displayed content (eg, after obtaining whether a first user interaction has been performed). to determine whether the second interaction has been performed. Alternatively, instead of a set of rules, the captured device event and/or the parsed/analyzed image content may be subjected to a model trained to predict whether a second interaction has been performed based on a set of device actions (e.g., a machine learning model). or other suitable statistical models). Such a model may be trained using, for example, the second user interaction and/or user action actually detected and each corresponding set of device events.

In addition to the above interactions, the OS may also detect additional signals regarding other device actions and/or data from other applications/services on the device (as described above with reference to FIG. 1 ). For example, if the user receives a clickable URL in an email or SMS text, selects it to open a webpage in a browser application running on the client device 102 showing a rental car reservation confirmation, the OS will display the same text as the text from the webpage. It will detect the signal and provide the set of signals to the content evaluation device 114 .

The set of signals is used ( 340 ) to determine whether a positive user action has been performed by the user on the first digital component ( 340 ). In some implementations, the content rating apparatus 114 is a machine learning model for the client device 102 that analyzes the set of signals (as detected above in operation 330 ) to determine whether a positive user action has occurred. or implement a heuristic based approach (as further described with reference to FIG. 1 ). For example, if the digital component is provided for display and includes a link to a website relating to a particular product, the user may click on the link and visit the linked website. In this case, user activity generates several signals, such as the number of clicks and the context of the content presented on the webpage. The content rating apparatus 114 analyzes the signal to determine a positive user action of the user to the digital component provided to the client device 102 by the content source/platform.

In some implementations, as described with reference to FIG. 1 , the machine learning model implemented within the content evaluation apparatus 114 is a model of the signals generated by user interaction with the digital component (and other actions on the client device). processing the set, the set of signals associated with a user interaction with the digital component (eg, a first user interaction with the digital component and one or more further user interactions and/or user actions with content provided in response to the first user interaction) Classification is performed based on whether or not a positive user action is a result or not. In some implementations, the machine learning model may generate a score or likelihood of a positive user action based on an input set of signals (as described with reference to FIG. 1 ).

The list of digital components 130 is modified, and the modified list is stored 350 in the client device 102 . When the content evaluation apparatus 114 implemented in the client device 102 identifies a positive user action for a particular digital component, it modifies the list of digital components 130 stored in the client device 102 . In some implementations, if the content rating apparatus determines that a positive user action has been encountered with the first digital component, the content rating apparatus 114 lowers the rank of the specific entry corresponding to the first digital component in the list of digital components to Yes For example, ensure that a particular entry is no longer included in the top N ranking component. For example, if a machine learning model implemented in the content evaluation device 114 analyzes other digital content accessed by the same browser application or by another application and determines that the user has booked a rental car, The rank of the entry may be lowered. Alternatively, instead of lowering the ranking, the content rating device 114 may remove the reference to the digital component from the list of digital components 130 .

A request for access to the page of content is received ( 360 ) from an application running on the client device ( 102 ). In order for an application running on the client device 102 to access a content page provided by the content source/platform, the application generates a request to access the content page transmitted over the network 104 to the content source/platform. . When the content source receives an access request for a content page, it transmits the content page to the client device 102 . For example, a user may perform multiple Internet searches using browser application A 116 running on client device 102 , review search results provided in response to Internet searches, and click links provided as search results. access to the digital component of In this scenario, the client device 102 generates a request to access a page of content that includes the search results. Further, if the user clicks on a link provided, for example, as a search result, the client device 102 generates a corresponding request to access the digital content transmitted over the network 104 to the content source.

A content request including a portion of the modified list of digital components is sent 370 to a second content source. In some implementations, in response to a content page access request by an application executing on the client device 102 , a script on the content page is executed, causing the content request to be provided by the application to the second content source. In some implementations, this request for content includes a portion of the digital component modified list (eg, the entire list or a subset of the list, eg, the top N ranked digital components). The request for content may include event data that specifies the electronic document being requested, the name or network location of the server from which the digital component is requested, and the name or network location of the requesting device (eg, client device 102 ).

In some implementations, the portion of the list of digital components 130 provided to the content source/platform includes the top N entries in the list of digital components 130, the top N entries being based on the user's relative interests. It is selected based on the ranking of each individual digital component. For example, in FIG. 2, entries 1-7 are ranked with respect to the type/category of the digital component. In this case, the client device 102 may provide the content source/platform with a portion of the list of digital components 130 that contains the entry with rank 1 .

In some implementations, the portion of the list of digital components 130 provided to the content source/platform includes a combination of one or more features of the digital component listed in the top N entries in the list of digital components 130 , the top N The entries are selected based on the ranking of each individual digital component based on the user's relative interest. For example, in FIG. 2 , the client device 102 presents a content source/platform to a portion of a list of digital components 130 that includes one or more features, such as product 202 or category 204 of an entry that is ranked 1 can be provided to

A second digital component is received 380 from a content source/platform based on a portion of the modified list of digital components 130 . In some implementations, the content source/platform, upon receiving a portion of the list of digital components 130 , provides the digital component in the digital component listed in the received list 130 to the client device 102 (list 130 ). provides digital components from the top N identified digital components in In a particular scenario, the list of digital components may be a modified list of digital components in which the list of digital components is modified based on a positive user action of the user on other digital components.

For example, suppose that a portion of a list of digital components 130 provided to a content source/platform includes the top N entries in the list of digital components 130 , where the top N entries are of each individual digital component. selected based on ranking. In such a case the portion of the list of digital components 130 accessed by the content source/platform will consist of entries 1, 2 and 5 representing user interest in cameras, hotels and rental cars.

In some implementations, the content source(s) and/or content platform may provide any digital component, whether or not on the list. In such a case, the client device may determine whether the provided digital component is in the list of digital components (eg, whether the provided digital component is one of the top N digital components in the list of digital components). If so, the application running on the client device may render/display the provided digital component. Alternatively, the application may suppress such content, for example by not displaying it, and instead, optionally, the application may modify the interface to display content that is different from where the suppressed/removed digital component would have been displayed (e.g. (e.g. content already contained in a page that can be reconfigured/moved/resized) or replaced with other content available from a content source or content platform.

In some implementations, the client device 102 may notify the content platform or content source about certain digital components that have been suppressed and not presented/displayed on the client device 102 . The content platform or content source may use this information to avoid continuing to provide these digital components to the client device 102 . The content platform and/or content source may maintain/store information about the suppressed/removed content, which may be used to perform analysis that may inform one or more client devices of the type of digital component to provide.

The digital component received from the content source/platform is provided ( 390 ) for display in an application executing on the client device ( 102 ). In some implementations, upon receiving digital components from a second content source, client device 102 provides these digital components for display within an application. For example, suppose a content source/platform receives a portion of a list of digital components and provides the client device 102 with digital components, such as content related to cameras, hotels, and rental cars.

In summary, the actions described above limit the presentation and display of redundant digital components at the client device for which the client device has already taken a positive user action. In this regard, the operations described above also provide and display a digital component (whether or not previously provided) without any limitation, provided no prior positive user action has been associated with these digital components.

4 is a block diagram of an exemplary computer system 400 that may be used to perform the operations described above. The system 400 includes a processor 410 , a memory 420 , a storage device 430 , and an input/output device 440 . Each of components 410 , 420 , 430 and 440 may be interconnected using, for example, a system bus 450 . Processor 410 may process instructions for execution within system 400 . In some implementations, processor 410 is a single-threaded processor. In another implementation, processor 410 is a multi-threaded processor. The processor 410 may process instructions stored in the memory 420 or the storage device 430 .

Memory 420 stores information within system 400 . In one implementation, memory 420 is a computer-readable medium. In some implementations, memory 420 is a volatile memory unit. In another implementation, memory 420 is a non-volatile memory unit.

Storage device 430 may provide large storage for system 400 . In some implementations, storage device 430 is a computer-readable medium. In various different implementations, storage device 430 may be, for example, a hard disk device, an optical disk device, a storage device shared over a network by multiple computing devices (eg, a cloud storage device), or some other It may include a mass storage device.

Input/output device 440 provides input/output operations for system 400 . In some implementations, the input/output device 440 is a network interface device such as an Ethernet card, a serial communication device (eg, an RS-232 port) and/or a wireless interface device (eg, an 802.11 card). may include one or more of In another implementation, the input/output device may include driver devices configured to receive input data and transmit output data to other peripheral devices 460 , eg, keyboard, printer, and display devices. However, other implementations may be used, such as mobile computing devices, mobile communication devices, set top box television client devices, and the like.

Although an exemplary processing system has been described in FIG. 4 , implementations of the invention and the functional operations described herein can be used in other types of digital It may be implemented in electronic circuitry, or in computer software, firmware or hardware.

Embodiments of the present invention and operations described herein may be implemented in digital electronic circuitry or computer software, firmware or hardware comprising the structures disclosed herein and their structural equivalents or combinations of one or more thereof. Embodiments of the invention described herein may be implemented as one or more computer programs. That is, one or more modules of computer program instructions encoded in a computer storage medium (or media) for execution by or control of operation of the data processing apparatus. Alternatively or additionally, the program instructions are an artificially generated propagated signal, ie, a machine-generated electrical, optical or electromagnetic, generated for encoding information for transmission to an appropriate receiver device for execution by the data processing device. may be encoded in the signal. A computer storage medium may be or be included in a computer readable storage device, a computer readable storage substrate, a random or serial access memory array or device, or a combination of one or more thereof. Further, while a computer storage medium is not a propagated signal, a computer storage medium may be a source or destination of computer program instructions encoded in an artificially generated propagated signal. In addition, a computer storage medium may be or be included in one or more separate physical components or media (eg, multiple CDs, disks, or other storage devices).

The operations described herein may be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

The term “data processing apparatus” includes all kinds of apparatus, devices and machines for processing data, including by way of example a programmable processor, computer, system on chip or many or combinations of the foregoing. The device may include a special purpose logic circuit, for example a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The device may also include, in addition to hardware, code that creates an execution environment for the computer program in question, such as processor firmware, protocol stack, database management system, operating system, cross-platform runtime execution environment, virtual machine. or code constituting a combination of one or more of them. The device and execution environment may realize various computing model infrastructures such as web services, distributed computing and grid computing infrastructure.

A computer program (also known as a program, software, software application, script or code) may be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and run standalone It may be distributed as a program type or in any form, including modules, components, subroutines, objects, or other units suitable for use in a computing environment. A computer program may, but does not necessarily, correspond to a file in a file system. A program may be part of a file having one or more scripts stored in other programs or data, eg, a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, eg, one or more modules. files, subprograms, or parts of code. The computer program may be distributed for execution on one computer or multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows may also be performed by a special purpose logic circuit such as an FPGA or ASIC, and the apparatus may also be implemented as a special purpose logic circuit.

Processors suitable for the execution of a computer program include, by way of example, general purpose and special purpose microprocessors. Generally, a processor will receive instructions and data from read-only memory or random access memory or both. Essential elements of a computer are a processor for performing actions according to the instructions and one or more memory devices for storing instructions and data. In general, a computer also includes one or more large storage devices for storing data, such as magnetic, magneto-optical disks or optical disks, or operatively to receive data from or transmit data thereto. can be combined. However, a computer does not necessarily have to have such devices. In addition, the computer may be connected to another device, such as a mobile phone, personal digital assistant (PDA), mobile audio or video player, game console, GPS receiver, or portable storage device, such as a universal serial bus, to name a few. It can be embedded in a (USB) flash drive. Devices suitable for storing computer program instructions and data include, for example, semiconductor memory devices such as EPROM, EEPROM and flash memory devices; magnetic disks such as, for example, internal hard disks or removable disks; and all forms of non-volatile memory, media and memory devices including CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated into special purpose logic circuitry.

In order to provide interaction with a user, embodiments of the invention described herein include a display device such as, for example, a cathode ray tube (CRT) or liquid crystal display (LCD) monitor, and a display device to display information to the user. It may be implemented in a computer with a keyboard and pointing device, such as a mouse or trackball, through which a user can provide input to the computer. Other types of devices may also be used to provide interaction with the user. For example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, including acoustic, voice, or tactile input. can Additionally, the computer may interact with the user by sending and receiving documents to and from the device used by the user, for example, by sending a web page to a web browser on the user's user device in response to a request received from the web browser.

Embodiments of the inventions described herein may be implemented by a user computer or user having a graphical user interface or a backend component such as a data server, a middleware component such as an application server, for example, to interact with an implementation of the invention described herein. It may be implemented in a computing system comprising a front-end component, such as a web browser, or any combination of one or more of the above-mentioned back-end, middleware or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, for example, a communication network. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), inter-networks (eg, the Internet), and peer-to-peer networks (eg, ad hoc peer-to-peer networks). .

A computing system may include users and servers. Users and servers are usually remote from each other, and they usually interact through a communication network. The relationship between the user and the server arises by virtue of computer programs running on the respective computers and having a user-server relationship with each other. In some embodiments, the server sends data (eg, an HTML page) to the user device (eg, to display data and receive user input to the user interacting with the user device). Data generated at the user device (eg, a result of a user interaction) may be received from the user device at a server.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of configurations specific to particular embodiments of particular inventions. . Certain configurations that are described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various configurations that are described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable subcombination. Moreover, although components may operate in certain combinations and even initially described above as claimed as such, one or more components from a claimed combination may in some cases be eliminated from the combination, and the claimed combination may be a sub-combination or It can be guided by a variant of the sub-combination.

Similarly, although acts are shown in the figures in a particular order, this should not be construed as requiring that the acts be performed in the specific order shown or in a chronological order or that all shown acts must be performed. In certain circumstances, multitasking and parallel processing are beneficial. Moreover, the separation of various system components in the above-described embodiments should not be construed as requiring such separation in all embodiments, and generally the described program components and systems may be integrated together into a single software product or multiple software products. It should be understood that they can be packaged in

Accordingly, specific embodiments of the present invention have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve desired results. Additionally, the processes depicted in the accompanying drawings do not necessarily require the specific illustrated order, or time-series order, to achieve desired results. In certain implementations, multitasking and parallel processing are advantageous.

Claims (20)

A computer-implemented method comprising:
storing, by a client device, a list of digital components specifying a set of digital components that can be provided to an application executing on the client device;
receiving, within a first application executing on the client device, a first digital component provided by a first content provider;
signals specifying, by the client device, (i) a first user interaction with a first digital component and (ii) a second user interaction with content provided in response to the first user interaction with the first digital component. detecting a set;
determining, by the client device, based on the set of signals, that a positive user action has been performed by a user of the client device, the positive user action being performed after a first user interaction with the first digital component represent performance of a specified target action by the user;
modifying, by the client device, the list of digital components based on a positive user action by the user after the first user interaction with the first digital component;
receiving a request to access a content page in a second application executing on the client device;
in response to receiving the request to access the content page, sending a request for content comprising a portion of the modified list of digital components to a second content provider, wherein the portion of the modified list of digital components is prevent selection of a first digital component in response to the request for content;
receiving, by the client device, within a second application, a second digital component from the second content provider in response to the request for the content, the second digital component being added to the modified list of digital components selected from among the included digital components; and
and providing the second digital component for display on a page of content within the second application. The method of claim 1 , wherein determining that a positive user action was performed by a user of the client device comprises:
inputting a set of signals associated with the first digital component into a machine learning model, wherein the machine learning model determines, based on the particular set of signals associated with the particular digital component, the user taking a positive user action with the particular digital component. predict whether or not
The machine learning model is trained using training data for a plurality of training digital components, wherein the training data for each training digital component includes a set of signals associated with the training digital component and a user positive for the training digital component. includes a corresponding label indicating whether an action was taken; and
obtaining, from the machine learning model, in response to a set of signals associated with the first digital component input to the machine learning model, an indication specifying whether a user performed a positive user action with the first digital component; A computer-implemented method comprising: 3. The computer-implemented method of claim 1 or 2, wherein storing the list of digital components comprises storing a ranked list of digital components. 4. The computer-implemented method of claim 3, wherein modifying the list of digital components comprises lowering a ranking corresponding to the first digital component in the ranked list. 5. The computer-implemented method of claim 4, wherein the portion of the modified list of digital components includes the top N ranked digital components in the ranked list. 3. The computer-implemented method of claim 1 or 2, wherein modifying the list of digital components comprises removing the first digital component from the list of digital components. In any preceding claim,
receiving, by the client device, within a second application, a third digital component from the second content provider in response to the request for the content, the third digital component being added to the modified list of digital components None of the included digital components; and
and inhibiting the third digital component from being displayed on the content page. 8. The method of claim 7,
in response to the suppression, modifying the content layout of the content page. 9. The method of claim 7 or 8,
and providing a message to the second content provider indicating that the third digital component has been suppressed. The computer-implemented method of any preceding claim, wherein the first content provider is different from the second content provider, and wherein the first application is different from the second application. As a system,
storing, by a client device, a list of digital components specifying a set of digital components that can be provided to an application executing on the client device;
receiving, within a first application executing on the client device, a first digital component provided by a first content provider;
signals specifying, by the client device, (i) a first user interaction with a first digital component and (ii) a second user interaction with content provided in response to the first user interaction with the first digital component. detecting a set;
determining, by the client device, based on the set of signals, that a positive user action has been performed by a user of the client device, the positive user action being performed after a first user interaction with the first digital component represent performance of a specified target action by the user;
modifying, by the client device, the list of digital components based on a positive user action by the user after the first user interaction with the first digital component;
receiving a request to access a content page in a second application executing on the client device;
in response to receiving the request to access the page of content, sending a request for content including a portion of the modified list of digital components to a second content provider, wherein the portion of the modified list of digital components is prevent selection of a first digital component in response to the request for content;
receiving, by the client device, within a second application, a second digital component from the second content provider in response to the request for the content, the second digital component being added to the modified list of digital components selected from among the included digital components; and
and providing the second digital component for display on a page of content within the second application. The method of claim 11 , wherein determining that a positive user action was performed by a user of the client device comprises:
inputting a set of signals associated with the first digital component into a machine learning model, wherein the machine learning model determines, based on the particular set of signals associated with the particular digital component, the user taking a positive user action with the particular digital component. predict whether or not
The machine learning model is trained using training data for a plurality of training digital components, wherein the training data for each training digital component includes a set of signals associated with the training digital component and a user positive for the training digital component. includes a corresponding label indicating whether an action was taken; and
obtaining, from the machine learning model, in response to a set of signals associated with the first digital component input to the machine learning model, an indication specifying whether a user performed a positive user action with the first digital component; comprising, a system. 13. The system of claim 11 or 12, wherein storing the list of digital components comprises storing a ranked list of digital components. In any preceding claim,
receiving, by the client device, within a second application, a third digital component from the second content provider in response to the request for the content, the third digital component being added to the modified list of digital components None of the included digital components; and
and inhibiting the third digital component from being displayed on the content page. The system of any preceding claim, wherein the first content provider is different from the second content provider, and wherein the first application is different from the second application. A non-transitory computer-readable medium having stored thereon instructions, which, when executed by one or more data processing devices, cause the one or more data processing devices to perform operations, the operations comprising:
storing, by a client device, a list of digital components specifying a set of digital components that can be provided to an application executing on the client device;
receiving, within a first application executing on the client device, a first digital component provided by a first content provider;
signals specifying, by the client device, (i) a first user interaction with a first digital component and (ii) a second user interaction with content provided in response to the first user interaction with the first digital component. detecting a set;
determining, by the client device, based on the set of signals, that a positive user action has been performed by a user of the client device, the positive user action being performed after a first user interaction with the first digital component represent performance of a specified target action by the user;
modifying, by the client device, the list of digital components based on a positive user action by the user after the first user interaction with the first digital component;
receiving a request to access a content page in a second application executing on the client device;
in response to receiving the request to access the page of content, sending a request for content including a portion of the modified list of digital components to a second content provider, wherein the portion of the modified list of digital components is prevent selection of a first digital component in response to the request for content;
receiving, by the client device, within a second application, a second digital component from the second content provider in response to the request for the content, the second digital component being added to the modified list of digital components selected from among the included digital components; and
and providing the second digital component for display on a page of content within the second application. 17. The method of claim 16, wherein determining that a positive user action was performed by a user of the client device comprises:
inputting a set of signals associated with the first digital component into a machine learning model, wherein the machine learning model determines, based on the particular set of signals associated with the particular digital component, the user taking a positive user action with the particular digital component. predict whether or not
The machine learning model is trained using training data for a plurality of training digital components, wherein the training data for each training digital component includes a set of signals associated with the training digital component and a user positive for the training digital component. includes a corresponding label indicating whether an action was taken; and
obtaining, from the machine learning model, in response to a set of signals associated with the first digital component input to the machine learning model, an indication specifying whether a user performed a positive user action with the first digital component; A non-transitory computer-readable medium comprising: The non-transitory computer-readable medium of claim 16 or 17 , wherein storing the list of digital components comprises storing a ranked list of digital components. 16. The method of claim 15,
receiving, by the client device, within a second application, a third digital component from the second content provider in response to the request for the content, the third digital component being added to the modified list of digital components None of the included digital components; and
and inhibiting the third digital component from being displayed on the content page. The non-transitory computer-readable medium of any preceding claim, wherein the first content provider is different from the second content provider, and wherein the first application is different from the second application.

Images