WO2013177769A1

WO2013177769A1 - Browser-driven retrieval of content for webpages

Info

Publication number: WO2013177769A1
Application number: PCT/CN2012/076304
Authority: WO
Inventors: Yuzhu Shen
Original assignee: Google Inc.
Priority date: 2012-05-30
Filing date: 2012-05-30
Publication date: 2013-12-05

Abstract

Method, computer-readable media, and systems for browser-driven retrieval and presentation of higher-resolution images for images within a rendered webpage are disclosed. The browser-driven image retrieval and presentation are independent of invocation of any hyperlinks or instructions embedded within the webpage.

Description

BROWSER-DRIVEN RETRIEVAL OF CONTENT FOR WEBPAGES

BACKGROUND

This specification relates generally to retrieval and presentation of web-based content. The Internet provides access to a vast amount of electronic content (e.g., text, images, blog posts, videos, music files, articles, news items, ads, and so on). A content source is a host of content items that provides the content items to user devices over one or more networks. Examples of a content source include servers of a website, a blog, an online discussion board, an online gaming community, an online marketplace, or other forums where content are provided to users over the Internet or another network.

The content provided by the content sources can be accessible in the form of webpages. The author or publisher of a webpage provides specifications and instructions that describe what content items are to be included in the webpage and the layout and styles by which the content items are to be presented in the webpage. Typically, instructions and specifications are stored or dynamically generated at the servers in one or more documents written in a structured language, such as Hypertext Markup Language (HTML) or Extended Hypertext Markup Language (XHTML). A client device can request the data for rendering a webpage by a unique Uniform Resource Locator (URL) associated with the webpage, and the server of the webpage provides the data to the client device in response to the request.

Specifically, a web browser application installed on a user device can submit a request (e.g., an HTTP request) to a content source and receive data including content items as well as instructions and specifications on how the content items are to be presented on the user device. The web browser application utilizes the received instructions and

specifications to generate one or more user interfaces (e.g., browser windows) on the user device and present the content items in the user interfaces.

Examples of the perceivable content rendered on the client device include static content such as text, images, audio and video content, as well as interactive elements such as hyperlinks, embedded media players, and interactive user interface elements. Functionalities of the interactive elements can be provided by scripts (e.g., program code written in

JavaScript) downloaded from the content source, or by the built-in components of the web browser application. One of the basic functionalities provided by the web browser application is the navigation from a currently displayed webpage to another webpage when a hyperlink in the currently displayed webpage is selected (or "clicked") by the user.

Typically, when a user requests a webpage, for example, by entering the URL of the webpage in the destination address input field of a web browser application or by clicking on a hyperlink in a currently displayed webpage, the web browser requests and receives the data needed to render the requested webpage. Then, the browser application renders the webpage in a browser window according to the data received. The rendered webpage may include text, images, media files, as well as interactive elements such as hyperlinks, buttons, or forms. Sometimes, the user may see an image in the rendered webpage for which he or she wishes to obtain a copy that is of better quality (e.g., at a higher resolution, or without digital compression). There are a few ways for the user to fulfill such a wish.

For example, if the author of the webpage already anticipates the user's needs and has available such a higher quality image, the author could provide a hyperlink in the webpage that links to the higher quality image. In such a case, the user can simply click on the hyperlink, and the web browser would navigate to the higher quality image according to the URL of the higher quality image specified in the hyperlink. Unfortunately, most images provided in webpages do not have such hyperlinks associated with them in the webpages.

Alternatively, the author of the webpage can embed a script in the webpage, where the script includes instructions that monitor the user's interaction with the webpage, and dynamically retrieve a higher quality image from a remote server when certain trigger conditions are satisfied. However, in most cases, the authors of webpages do not have the time, resources, intention, or skills to predict and fulfill the user's needs to see a better quality version of a particular image in their webpages.

If the publisher or author of the webpage do not already provide a convenient access to higher quality versions of the images in the webpage, the user has to either manually perform an image search using an image search engine, or settle for the lack of such a high quality image. Many users, however, often forego such a search, and thus many users thus leave their interest to see the higher quality images unfulfilled. Furthermore, to initiate such as search, a user must leave the web page of interest and navigate to a search engine interface. Accordingly, the content of primary interest to the user is then relegated to another tab or sometimes even lost. SUMMARY

This specification describes technologies relating to browser-driven content retrieval and presentation for webpages. In an example implementation, when a webpage containing images is rendered in a browser window, the browser application detects user interaction with an image presented in the rendered webpage. If the user interaction is indicative of the user being interested in the image, then, independent of invocation of any hyperlink or instructions embedded in the webpage, the browser application automatically initiates an image search query to an image search engine for that image. If the image search engine returns an identical image at a higher resolution or better quality than the image currently presented in the webpage, the browser application presents the retrieved image

contemporaneously with the webpage in the browser user interface.

In general, one aspect of the subject matter described in this specification can be embodied in a method that includes the actions of rendering a webpage in a browser user interface. The rendered webpage presents a first copy of an image among other content. User input registered in proximity to the first copy of the image in the browser user interface is detected. In response to detecting the user input and while the rendered webpage is presented in the browser user interface, an image search for a second copy of the image at a higher resolution than the first copy of the image is initiated. The initiation of the image search is independent of invocation of any hyperlink or instruction embedded within the webpage. The second copy of the image at the higher resolution than the first copy of the image is obtained as a result of the image search. The second copy of the image at the higher resolution is rendered contemporaneously with the webpage in the browser user interface.

This, and other aspects, can include one or more of the following features. Detecting the user input registered in proximity to the first copy of the image in the browser user interface can include detecting hovering of a pointer in proximity to the first copy of the image in the browser user interface for at least a threshold amount of time. The initiation of the image search can be independent of any user input registered within the webpage in the browser user interface. Upon successful retrieval of the second copy of the image as a result from the image search, a user interface element can be displayed in proximity to the first copy of the image in the rendered webpage. The user interface element can be operable to cause the rendering of the second copy of the image when invoked by user input. Before initiating the image search, it can be determined that the first copy of the image is located within a predetermined section of the rendered webpage. Initiating the image search can further include submitting the first copy of the image to an image search engine. Initiating the image search can further include submitting a Uniform Resource Locator (URL) of the first copy of the image to an image search engine. That a search result received as a result of the image search is a copy of the image and has a higher resolution than the first copy of the image can be verified. Rendering the second copy of the image at the higher resolution contemporaneously with the webpage in the browser user interface can include rendering the second copy of the image as an overlay placed within the rendered webpage in the browser user interface. Other embodiments of this aspect include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices.

Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages. According to some implementations, the browser application is able to automatically detect a user's interest in an image in a webpage currently presented in the browser window, and automatically initiate an image search at one or more image search engines without requiring the user to take any explicit search actions or navigating away from the current webpage. Therefore, the browser can satisfy the user's information needs and provide a better user experience, rather than letting the user go through the trouble of doing an explicit image search or settle for an unfilled interest in seeing a higher quality version of the image currently available in the webpage. The manner by which the browser presents the retrieved higher quality images also ensures that the images autonomously retrieved by the browser do not interfere with the user's normal viewing of the webpage.

In addition, the browser detects a user's interest in particular images in a webpage and initiates the image searches on the Internet independent of the invocation of any hyperlinks or programming instructions embedded in the webpage. Therefore, the browser alleviates from the author or publisher of the webpage the burden to fulfill the user's interest in seeing higher resolution versions of particular images presented in the webpage. Since the publishers or authors of webpages do not always have the resources, time, and/or skills to obtain the higher resolution images and embed hyperlinks or scripts in the webpage for the automatic retrieval of the higher resolution images, the techniques described in this specification fulfills the needs not met by the publishers and authors of the webpages.

In addition, many webpages are dynamically generated and it is difficult to predict which images may be included in the webpage before the webpage is rendered in the user's browser. At the same time, it is difficult to predict which image each particular user may find interesting before the webpage is rendered in the user's browser. It is more efficient to fulfill a user's interest in seeing a higher resolution version of an image present in a webpage by using the browser to determine user interest on images in a currently displayed webpage, and only initiating image search query or presenting retrieved high resolution/quality images when such interest is detected.

The details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an example graphical user interface of a web browser.

FIG. IB illustrates an example implementation of the browser-drive content retrieval and presentation techniques described in this specification.

FIGS. 1C-1D illustrate another example implementation of the browser-drive content retrieval and presentation techniques described in this specification.

FIGS. IE- IF illustrate an example implementation of the browser-drive content retrieval and presentation techniques described in this specification.

FIG. 2 is a block diagram of an example web browser application implementing the browser-drive content retrieval and presentation techniques described in this specification.

FIG. 3 is a flow diagram of an example process for browser-drive content retrieval and presentation.

Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION

In this specification, implementations of browser-driven content retrieval and presentation techniques are described. According to an example implementation of the browser-driven content retrieval and presentation techniques, a web browser detects a user's interest in a particular image in the rendered webpage, and automatically initiates an image search (e.g., at a remote image search engine). If such an image search returns the same image at a higher quality (e.g., at a higher resolution), the web browser automatically makes the higher quality image available for the user to see alongside the rendered webpage. The browser's actions are independent of the invocation of any hyperlink or programming instructions embedded in a webpage currently rendered in the browser window.

The browser-driven content retrieval and presentation techniques described in this specification alleviates the burden to perform the manual search from the user, and alleviates the burden to anticipate the user's interests and provide mechanisms to fulfill such an interest in the webpage itself from the author of webpages. The browser-driven content retrieval and presentation techniques function generically for all images on webpages rendered in the browser regardless of whether hyperlinks or scripts are associated with the images.

In some implementations, the browser distinguishes various zones or regions in a rendered webpage as have different roles or importance. For example, the browser can be configured such that only images within the primary pane of the rendered webpage will trigger the image search and presentation described in this specification. Alternatively, the browser-driven image search and presentation can be performed for all images currently within view on the user's device. The images currently within view can include images on the webpage under input focus and images within pop-up windows that are not obfuscated by other windows, for example.

FIG. 1A illustrates an example graphical user interface 100 of a web browser application installed on a user device. In this example, the graphical user interface 100 is an application window in which one or more webpages can be rendered, e.g., in separate tabs 102a and 102b. In some implementations, the web browser application can open multiple browser windows for rendering webpages. Unless otherwise stated, in this specification, a webpage rendered in a browser window and/or tab of the browser application refers to the webpage that is rendered in the browser window and/or tab that currently has input focus of the user device.

As shown in FIG. 1A, the web browser application has retrieved and rendered a webpage 104 within the tab 102a of the browser window 100. The rendered webpage 104 includes content such as text 106 and images 108a, 108b, and 108c. The webpage 104 also optionally includes a script (e.g., either literally included in the source code of the webpage 104 or referenced and subsequently downloaded with the source code of webpage 104). When the script is executed by the browser application, the executed script optionally detects user interaction with the text or image on the webpage 104, generates a pop-up window 110 alongside the browser window 100, and cause the browser to dynamically download and present an advertisement image 112 in the popup window 110. The webpage 104 also optionally includes hyperlinks 114a, 114b, and 114c, where the hyperlinks 114a and 114b have their respective anchor text in the textual portions 106 of the webpage 104, and the hyperlink 114c has the image 108b as its anchor image. If the user clicks on one of the hyperlinks 114a-c in the webpage 104, the browser will navigate to the webpage associated with the URL of the clicked hyperlink. The browser window 100 also presents other user interface elements for loading a new webpage and navigating from page to page.

In this example, the browser application that provides the user interface 100 in FIG. 1A implements the browser-driven content retrieval and presentation techniques described in this specification. Independent of any instructions embedded in the downloaded and rendered webpages, the browser application is capable of monitoring the user's interaction with a webpage currently rendered in the browser window and determining whether to initiate and carry out additional content retrieval and presentation actions.

For example, the browser application optionally determines which browser window or webpage has the current input focus, and detects the location of the input focus within that window or webpage. The browser also has information on the layout and positions of the content items on the webpage having the current input focus, e.g., by the DOM structure, the view port dimensions, and the cursor position. Therefore, browser can determine whether the user's interactions with the webpage focus on any particular portion or item on the webpage. When the browser application determines according to predetermined criteria that the user's interactions with the webpage are indicative of the user's interest in a particular portion or item on the webpage, the detection can be used as a trigger for subsequent actions taken by the browser application. These actions of the browser application can be initiated and carried out regardless of the hyperlinks or instructions embedded in or downloaded with the webpage, and therefore, the actions are browser-driven.

In this example implementation, the browser application monitors whether the user' s input focus (e.g., as represented by the pointer 116) hovers over any of the images presented on the webpage for at least a predetermined amount of time. If so, the browser determines that the user is interested in seeing a higher quality version of the image, and proceeds to initiate a search for the image. Alternatively, or in addition, the browser application determines that the user is interesting in seeing a higher quality version of an image if the user clicks on an image which is not an anchor image of a hyperlink. If such an higher quality image is found and successfully retrieved, the browser application presents the retrieved image to the user alongside the webpage, either automatically without further input from the user or upon being prompted by the user.

FIG. IB illustrates an example implementation of the browser-drive content retrieval and presentation techniques described in this specification. As shown in FIG. IB, the browser has detected that the user's pointer 116 has hovered over the image 108a for at least a predetermined threshold amount of time, and initiated an search for a higher quality version of the image 108a. The image 108a is not associated with any hyperlink or script, and the search for the image 108a is browser-driven.

In some implementations, the browser saves the image or the URL of the image, and initiates a search in the background at one or more image search engines using the saved image or URL. In some implementations, the browser also has a component that verifies that the retrieved image(s) are indeed copies of the image shown in the webpage, and that the retrieved image(s) are indeed of higher resolution or quality than the image shown in the webpage. Various techniques exist in the art for verifying whether two images are different versions of the same image and are different only in terms of quality or resolution.

If the browser application is successful in retrieving a copy of the searched image at a higher quality or resolution, the browser application renders the retrieved image alongside the image that was the subject of the image search. In this example, the rendering of the retrieved image is automatic and without further prompt from the user. As shown in FIG. IB, a retrieved image is shown in a pop-up window 118 or overlay placed in proximity to the image 108a in the rendered webpage 104. In some implementations, the retrieved image can also be presented in a pop-up window positioned outside of the browser window 104. The pop-up window or overlay 118 presenting the retrieved image optionally includes one or more user interface elements for manipulating the retrieved image, such as controls for saving the image on the user device, zoom into the image, or closing the pop-up window 118.

In some implementations, the pop-up window or overlay 118 can be automatically closed when the user's input focus (e.g., as indicated by the pointer 116) moves away from the image 108a and/or the window 118. If the retrieved image is presented in a separate pop- window, the user can close the pop-up window by selecting the appropriate user interface element available in the pop-up window. After the pop-up window or overlay 118 showing the higher resolution image is closed, the user interface 100 returns to its original appearance as shown in FIG. 1A.

If the user subsequently moves the pointer 116 to another image (e.g., image 108b) shown in the webpage, and hovers the pointer 116 over the image for an extended period of time, the browser application will perform the image search on the Internet using that image (e.g., image 108b). If the search returns a higher quality version of the image 108b, the browser application presents the retrieved image in the same manner as shown in FIG. IB in proximity to the image 108b. If no higher quality version of the image 108b is found through the automatic search, the user interface is not altered. Alternatively, a text prompt indicating that no higher resolution image is available may be provided in the user interface 100, e.g., at an inconspicuous location at the bottom of the browser window 100.

In some implementations, the browser application may present, in addition to an option for the user to save the image that has been retrieved, an option to navigate to the source webpage of the image that has been retrieved. For example, the pop-up window 118 showing the retrieved image can provide an options menu or menu bar showing these various options. Since the retrieved image may come from a different content source from that providing the webpage 104 currently rendered in the browser window 100, by selecting the option to go to the source of the retrieved image, the browser may replace the webpage 104 with a different webpage originally containing the retrieved image. In some implementations, if the search performed by the browser retrieves multiple versions of the same image at various qualities, the browser chooses to present the copy having the best quality in the user interface 100. Optionally, the initially presented copy may be scaled such that the pop-up window 118 does not obfuscate the entire browser window 100, and the pop-up window 118 can be expanded to its original scale once the user has clicked on it.

FIG. IB shows one example manner in which image search can be triggered and a retrieved image can be presented on the user device. FIGS. 1C-1D illustrate another implementation in which the retrieved image is not presented immediately after it is received by the browser application. Instead, a user interface element indicating the availability of such an image is presented alongside the webpage currently rendered in the browser window, and the retrieved image is only rendered in the browser window when the user interface element is invoked by user input.

For example, as shown in FIG. 1C, as soon as the browser application determines that the user may be interested in seeing a higher quality version of the image 108a, for example, based on the pointer 116 having hovered over the image 108a for more than a threshold amount of time, the browser application proceeds to perform an image search on the Internet for the image 108a. As soon as a better quality image is received by the browser, the browser presents a user interface element 120 next to the image 108a on the webpage 104. The user interface element 120 indicates that the web browser has retrieved a higher quality version of the image 108a from the Internet. If the user wishes to see the higher quality version of the image 108a, the user can simply invoke the user interface element 120 by clicking on it using the pointer 116, for example.

FIG. ID shows that once the user has invoked the user interface element 120, the retrieved image is presented in a pop-up window or overlay 122 displayed in proximity to the image 108a in the rendered webpage 104 in the browser window 100. In some

implementations, the pop-up window 122 can also be presented outside of the browser window 100. The user can close the pop-up window or overlay 122 by either moving the pointer 116 away from the image 108a or manually closing the pop-up window or overlay 122, for example. By providing the user interface element 122 rather than providing the retrieved image immediately, the browser allows the user more control over the browser interface, and avoid unnecessary cluttering of the browser window. In addition, the user can always go back to the user interface element 122 to cause the retrieved image to be displayed again without causing the browser to perform the search again.

In some implementations, the browser does not have to wait for the user to hover the pointer over a particular image in order to initiate the image search on the internet. Instead, the search can begin as long as there is bandwidth and computing resources available for the background process to initiate the image searches. For example, in some implementations, the browser application initiates the image search for each image or a subset of the images (e.g., images within the primary pane of the webpage, or images that have been determined not to be a graphical icon) on the rendered webpage as soon as the webpage is loaded and rendered on the user device. Then, for each image on the webpage for which the browser has received a higher quality version, the browser presents a corresponding user interface element in proximity to the image. If no higher quality image is received for a particular image, no such user interface element is presented in proximity to the particular image.

As shown in FIG. IE, the browser has retrieved higher quality versions of several images presented on the webpage 104 without regard to the user' prior interactions with the webpage 104. For those images (e.g., images 108a and 108b) having a higher quality counterpart, the browser application has rendered corresponding user interface elements (e.g., element 124a and 124b) in proximity to the images. For the images (e.g., image 108c) that no higher quality versions have been found, the images are left as they were originally rendering the webpage 104. The user can select any one of the user interface elements 124a and 124b to see a corresponding higher quality version of the images 108a or 108b.

As shown in FIG. IF, the user has selected the user interface element 124b displayed in proximity to the image 108b rendered in the webpage 104, and the higher quality version of the image is presented in a pop-up window or overlay 126 in proximity to the image 108b as a result.

FIGS. 1A-1F are merely illustrations of example user interfaces of a browser application that implements the browser-driven content retrieval and presentation techniques described in this specification. Other variations of the manner by which browser-driven content retrieval and presentation functions can be implemented are possible. FIG. 2 is a block diagram of an environment 200 in which an example browser application 202 that implements the browser-driven content retrieval and presentation techniques described in this specification operates. In the example environment 200, the browser application 202 is installed on a user device 204 that communicates with servers of content sources 206 and image search engines 208 through one or more networks 210.

Examples of the networks include combinations of one or more local area networks (LAN), wide area networks (WAN), peer-to-peer networks, wireless networks, and/or other equivalent communication networks. The user device 204 can be data processing apparatus such as personal computers, smart phones, tablet computers, and so on.

The browser application 202 is a software application that download data 212 (e.g., source code and content items of webpages) for generating content interfaces (e.g., the user interface 102a shown in FIG. 1A) from content sources 206, and render the content interfaces on the user device 204 according to the data 212. The browser application 202 is further capable of executing scripts downloaded from the content sources 206, for example, as part of the data 212, and performing operations according to the instructions specified in the scripts.

As shown in FIG. 2, the browser application 202 installed on a client device 204 is capable of communicating with one or more content sources 206 (e.g., servers of one or more websites) through a network component 214. For example, the network component 214 is capable of sending a request for a webpage (e.g., an HTTP request specifying the URL of the webpage) to a server of the webpage (or one of the content sources 206), and receiving the data 212 needed for rendering the webpage on the client device 204. The data 212 received from the content source 206 include the source code of the webpage (e.g., in one or more HTML or XHTML documents) as well as content items that are to be presented on the webpage and/or scripts to be embedded in the webpage.

Once the network component 214 receives the data needed to render the webpage on the client device 204, a rendering component 216 of the browser application 202 interprets the data and determines the layout and styles by which the content items are to be rendered in the webpage. The rendering component 216 renders the webpage in a browser window of the browser application 202 according to the layout and styles specified in the received data or the default styles set in the browser's configuration settings. Once the webpage is rendered in the browser user interface (e.g., a browser window), a user interface component 218 of the browser application 202 can monitor the browser user interface, and detect any user interactions with the browser user interface and the webpage rendered inside the browser user interface. For example, the UI component is able to determine the location of the pointer inside the browser window, the particular content item or text located at that location, and any user input (e.g., clicks, hovering, etc.) registered at that location. For example, if the user selects a hyperlink inside the webpage, the user interface component 218 of the browser application 202 will detect the user input at a location on the rendered webpage, determine that there is a hyperlink located at that location, and navigate to the webpage identified by the URL associated with the selected hyperlink. The user interface component 218 can communicate with the network component 214 to carry out the actions initiated by the user input registered within the browser user interface by the user interface component 218.

In some implementations, if the webpage contains embedded scripts, the browser application 202 can execute the script, which may utilize the various components of the browser application 202 through the various APIs provided by the browser application 202. In some implementations, the browser application 202 also provides APIs for browser extensions and plug-in software components to extend the default functions of the browser application 202 by utilizing the components 214, 216, and 218 of the browser application 202, for example.

In this example, as indicated in FIG. 2, the browser application 202 also includes a content-retrieval component 220 that initiates image searches for images on a webpage currently rendered in the browser window. The content-retrieval component 220 is capable of initiating image searches for images presented in a webpage currently rendered in the browser independent of any instructions or hyperlinks embedded in the webpage. The content-retrieval component 220 communicates with the user interface component 218 and instructs the user interface component 218 to monitor the user's interactions with the webpage currently presented in the browser window. The content-retrieval component 220 then determines whether the user's interactions (e.g., as reported by the user interface component 218) indicate the user's interest in any particular image in the webpage currently rendered in the browser window. The content-retrieval component 220 can include one or more criteria for evaluating the user's interest based on parameters (e.g., proximity to image, input type, input duration, and so on) received from the user interface component 218, for example.

In some implementations, if the content-retrieval component 220 determines that the user's interaction with the webpage indicates the user is interested in a particular image in the rendered webpage, the content-retrieval component 220 initiates an image search for the particular image via the network component 214 of the browser application 202. For example, the content-retrieval component 220 can contact one or more image search engines and perform the image search using a copy of the image or the URL of the image.

In some implementations, the content retrieval component 220 specifies a minimum resolution required for search results, and requires that only exact match of the image is to be returned as search results. In some implementations, if the search engine does not provide an option to require exact match or minimum resolution for the search results, the content- retrieval component 220 retrieves the search results from the image search engine, and locally determines whether the retrieved search results satisfy the above-mentioned criteria. In some implementations, the search results may be provided by the search engine in a search result page with a thumbnail and a hyperlink for each result image. In such cases, the content-retrieval component 220 follows each of the hyperlinks to retrieve the result image and determine whether the result image satisfies the above-mentioned criterion.

After the retrieved result images are evaluated by the content-retrieval component 220, for each image that a higher quality/resolution image has been retrieved, the content- retrieval component 220 utilizes the rendering component 216 to render the higher quality /resolution image on the user device 204 alongside the webpage. Examples of how the retrieved images are rendered on the user device are described with respect to FIGS. 1B- 1F.

FIG. 2 is merely an illustration of the functionalities of the browser application 202. The browser application 202 may have more or fewer components than those shown, and the functionalities of the components may be combined into fewer components or divided into more components. In some implementations, the content-retrieval component 220 of the browser application 202 may be a native component of the browser application program. Alternatively, the content-retrieval component may be implemented as a browser extension or a plug-in component that interfaces with the main browser application through the extension or plug-in API of the browser application. The content-retrieval component 220 performs its functions independent of the instructions downloaded with the webpages.

Optionally, the content-retrieval component 220 performs the same functions for all webpages currently displayed in the browser, even if the webpage is a locally stored webpage (e.g., a saved webpage).

FIG. 3 is a flow diagram of an example process 300 implemented in a browser application (e.g., browser application 202 in FIG. 2) executing on a client device.

In the example process 300, the browser application renders a webpage in a browser user interface, where the rendered webpage presenting a first copy of an image among other content (302). The browser application detects user input registered in proximity to the first copy of the image in the browser user interface (304). In response to detecting the user input and while the rendered webpage is presented in the browser user interface, the browser application initiates an image search for a second copy of the image at a higher resolution than the first copy of the image, where the initiation of the image search is independent of invocation of any hyperlink or instruction embedded within the webpage (306). The browser application obtains the second copy of the image at the higher resolution than the first copy of the image as a result of the image search (308). Then, the browser application renders the second copy of the image at the higher resolution contemporaneously with the webpage in the browser user interface (310).

In some implementations, detecting the user input registered in proximity to the first copy of the image in the browser user interface includes detecting hovering of a pointer in proximity to the first copy of the image in the browser user interface for at least a threshold amount of time.

In some implementations, the initiation of the image search is independent any user input registered within the rendered webpage in the browser user interface.

In some implementations, upon successful retrieval of the second copy of the image as a result from the image search, the browser application displays a user interface element in proximity to the first copy of the image in the rendered webpage, where the user interface element is operable to cause the rendering of the second copy of the image when invoked by user input. In some implementations, before initiating the image search, the browser application determines that the first copy of the image is located within a predetermined section of the rendered webpage, such as the primary pane of a multi-pane webpage.

In some implementations, initiating the image search further includes submitting the first copy of the image to an image search engine. Alternatively, initiating the image search further includes submitting a Uniform Resource Locator (URL) of the first copy of the image to an image search engine.

In some implementations, the browser application verifies that a search result received as a result of the image search is a copy of the image and has a higher resolution than the first copy of the image.

In some implementations, rendering the second copy of the image at the higher resolution contemporaneously with the webpage in the browser user interface further includes rendering the second copy of the image as an overlay placed within the rendered webpage in the browser user interface.

The example process shown in FIG. 3 is provided for illustrative purposes. Other processes are described with respect to FIGS. 1A-2, for example. In addition, most of the examples are provided for retrieval and presentation of higher resolution images, however similar techniques can be implemented to retrieve better quality or longer version of other types of content, such as video and audio clips. For example, the browser can initiate video searches and retrieval of longer versions of video clips for a short video clip embedded in a webpage currently rendered in the browser window, independent of invocation of any hyperlink or instructions embedded in the webpage. In addition or alternatively, the browser can initiate searches and retrieval of high definition versions of video clips for a low definition video clip embedded in a webpage currently rendered in the browser window, independent of invocation of any hyperlink or instructions embedded in the webpage. The retrieved content can be presented to the user in the manner similar to that described in this specification. The initiation of the content retrieval can be automatic or upon determination that the user may be interested in seeing such higher quality versions of the content shown in the webpage.

Embodiments of the subject matter and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, data processing apparatus.

Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. A computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

The term "data processing apparatus" encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, or combinations of the foregoing. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and execution environment can realize various different computing model infrastructures, such as web services, distributed computing and grid computing infrastructures. A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for performing actions in accordance with instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few. Devices suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices;

magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser.

Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network ("LAN") and a wide area network ("WAN"), an inter-network (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer- to-peer networks).

The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data (e.g., an HTML page) to a client device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., a result of the user interaction) can be received from the client device at the server.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate

embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

What is claimed is:

Claims

1. A computer-implemented method, the method performed by one or more processors and comprising:

rendering a webpage in a browser user interface, the rendered webpage presenting a first copy of an image among other content;

detecting user input registered in proximity to the first copy of the image in the browser user interface;

in response to detecting the user input and while the rendered webpage is presented in the browser user interface, initiating an image search for a second copy of the image at a higher resolution than the first copy of the image, the initiation of the image search being independent of invocation of any hyperlink or instruction embedded within the webpage; obtaining the second copy of the image at the higher resolution than the first copy of the image as a result of the image search; and

rendering the second copy of the image at the higher resolution contemporaneously with the webpage in the browser user interface.

2. The method of claim 1 , wherein detecting the user input registered in proximity to the first copy of the image in the browser user interface further comprises:

detecting hovering of a pointer in proximity to the first copy of the image in the browser user interface for at least a threshold amount of time.

3. The method of claim 1, wherein the initiation of the image search is independent any user input registered within the rendered webpage in the browser user interface.

4. The method of claim 1, further comprising:

upon successful retrieval of the second copy of the image as a result from the image search, displaying a user interface element in proximity to the first copy of the image in the rendered webpage, wherein the user interface element is operable to cause the rendering of the second copy of the image when invoked by user input.

5. The method of claim 1, further comprising: before initiating the image search, determining that the first copy of the image is located within a predetermined section of the rendered webpage.

6. The method of claim 1, wherein initiating the image search further comprises:

submitting the first copy of the image to an image search engine.

7. The method of claim 1, wherein initiating the image search further comprises:

submitting a Uniform Resource Locator (URL) of the first copy of the image to an image search engine.

8. The method of claim 1, further comprising:

verifying that a search result received as a result of the image search is a copy of the image and has a higher resolution than the first copy of the image.

9. The method of claim 1, wherein rendering the second copy of the image at the higher resolution contemporaneously with the webpage in the browser user interface further comprises:

rendering the second copy of the image as an overlay placed within the rendered webpage in the browser user interface.

10. A computer-readable medium having instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising: rendering a webpage in a browser user interface, the rendered webpage presenting a first copy of an image among other content;

11. The computer-readable medium of claim 10, wherein detecting the user input registered in proximity to the first copy of the image in the browser user interface further comprises: detecting hovering of a pointer in proximity to the first copy of the image in the browser user interface for at least a threshold amount of time.

12. The computer-readable medium of claim 10, wherein the initiation of the image search is independent any user input registered within the rendered webpage in the browser user interface.

13. The computer-readable medium of claim 10, wherein the operations further comprise: upon successful retrieval of the second copy of the image as a result from the image search, displaying a user interface element in proximity to the first copy of the image in the rendered webpage, wherein the user interface element is operable to cause the rendering of the second copy of the image when invoked by user input.

14. The computer-readable medium of claim 10, wherein the operations further comprise: before initiating the image search, determining that the first copy of the image is located within a predetermined section of the rendered webpage.

15. The computer-readable medium of claim 10, wherein initiating the image search further comprises:

submitting the first copy of the image to an image search engine.

16. The computer-readable medium of claim 10, wherein initiating the image search further comprises: submitting a Uniform Resource Locator (URL) of the first copy of the image to an image search engine.

17. The computer-readable medium of claim 10, wherein the operations further comprise: verifying that a search result received as a result of the image search is a copy of the image and has a higher resolution than the first copy of the image.

18. The computer-readable medium of claim 10, wherein rendering the second copy of the image at the higher resolution contemporaneously with the webpage in the browser user interface further comprises:

19. A system, comprising:

one or more processors; and

memory having instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:

in response to detecting the user input and while the rendered webpage is presented in the browser user interface, initiating an image search for a second copy of the image at a higher resolution than the first copy of the image, the initiation of the image search being independent of invocation of any hyperlink or instruction embedded within the webpage;

obtaining the second copy of the image at the higher resolution than the first copy of the image as a result of the image search; and

20. The system of claim 19, wherein detecting the user input registered in proximity to the first copy of the image in the browser user interface further comprises:

21. The system of claim 19, wherein the initiation of the image search is independent any user input registered within the rendered webpage in the browser user interface.

22. The system of claim 19, wherein the operations further comprise:

23. The system of claim 19, wherein the operations further comprise:

before initiating the image search, determining that the first copy of the image is located within a predetermined section of the rendered webpage.

24. The system of claim 19, wherein initiating the image search further comprises:

submitting the first copy of the image to an image search engine.

25. The system of claim 19, wherein initiating the image search further comprises:

26. The system of claim 19, wherein the operations further comprise:

27. The system of claim 19, wherein rendering the second copy of the image at the higher resolution contemporaneously with the webpage in the browser user interface further comprises: