JP2014522534A - Zoom-in and zoom-out techniques using dynamic content - Google Patents

Abstract

  A method and apparatus for providing a zoom experience is provided. In response to detecting the start of the zoom control input, a snapshot of the content is generated. A zoom operation is performed on the snapshot based on a zoom control input to provide a zoomed snapshot. Snapshots are displayed while zooming. The content is rendered to provide the amount of zoom indicated by the zoom control input and to provide post-zoom rendering. The post-zoom is mixed with the snapshot and post-zoom rendering to provide the final image. The final image is displayed.

Description

This US patent application was filed on June 5, 2011 and is entitled US Provisional Patent Application No. 61 / 493,486 (Attorney Docket No. 4860P11597Z / P11597USP1) entitled “Techniques For Zooming In And Out With Dynamic Content”. ).
Embodiments of the present invention relate to the display of images by electronic devices. More particularly, embodiments of the invention relate to techniques that provide the ability to zoom in and out from images displayed on an electronic device.

  With advances in browser technology and network infrastructure, more and more content or applications can be accessed through browser applications. In order to improve the usability of this content, some browser applications provide a function of zooming in and out to enlarge a selected area of the content. Various control mechanisms can be used for zooming control.

The present invention is illustrated by way of example and not limitation in the accompanying drawings in which like reference numerals refer to similar elements.
1 is a block diagram of one embodiment of an electronic system. 1 is a block diagram of a system that displays a page of data. FIG. 3 is a flow diagram of one embodiment of a technique for utilizing a snapshot to provide a zoom-in experience. FIG. 4 is a flow diagram of one embodiment of a technique that utilizes a snapshot to provide a zoom-out experience. FIG. 5 is a flow diagram of one embodiment of zooming in that requires panning and / or scrolling. 2 is a block diagram of one embodiment of a zoom control agent. FIG.

  In the following description, numerous specific details are set forth. However, embodiments of the invention may be practiced without these specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

  When a user of an electronic device (eg, a computer system, tablet device, smartphone, kiosk) uses a browser to access information displayed as a page, the user can view a portion of the page in more detail or more roughly There are things you want to see. That is, the user may wish to zoom in or out. As mentioned above, under certain conditions, the experience provided to the user may never be the best. Described herein are techniques that improve the user experience when zooming in or out from a view of a page.

  Various mechanisms may be used to control zooming in both speed and amount. For example, a knob operation on a control pad may be used in which zooming in is controlled by the user moving their fingers closer together, and zooming out is controlled by moving the user away from each other. Other mechanisms for controlling zooming include, for example, tapping the control pad, rotating the wheel (eg, on a mouse or other input device), using arrows or other keys on the keyboard, graphical users The operation of the slider on the interface can be mentioned.

  The user experience is most pleasant if the interaction is natural and intuitive. Therefore, smooth and natural zooming of the image is preferred when zooming. However, the image manipulation required to provide zooming may not be sufficient in computing power and / or bandwidth to provide an immediate smooth zoom. In one embodiment, a snapshot image may be used during the zooming process while the content to be displayed is processed. When processing is complete, the snapshot is replaced with a zoomed page.

  FIG. 1 is a block diagram of one embodiment of an electronic system. The electronic system shown in FIG. 1 includes various electronic systems including, for example, desktop computer systems, laptop computer systems, tablet devices, mobile phones, personal digital assistants (PDAs) including mobile phone compatible PDAs, and set top boxes. Is intended to represent (whether wired or wireless). Alternative electronic systems may include more, fewer, and / or different components.

  Electronic system 100 includes a bus 105 or other communication device for communicating information, and a processor 110 coupled to bus 105 and capable of processing information. Although electronic system 100 is shown having only one processor, electronic system 100 may include multiple processors and / or coprocessors. The electronic system 100 may further include a random access memory (RAM) or other dynamic storage device 120 (referred to as main memory) coupled to the bus 105 to store information and instructions that can be executed by the processor 110. Can do. Main memory 120 may be used to store temporary variables or other intermediate information during execution of instructions by processor 110.

  The electronic system 100 may include a read only memory (ROM) and / or other static storage device 130 coupled to the bus 105 and capable of storing static information and instructions for the processor 110. Data storage device 140 may be coupled to bus 105 for storing information and instructions. A data storage device 140 such as a magnetic disk or optical disk and a corresponding drive may be coupled to the electronic system 100.

  Magnetic disk, read-only, via either a wired or wireless remote connection (eg, through a network via network interface 130) that provides access to one or more electronically accessible media or the like Instructions are provided to the memory from a memory (ROM) integrated circuit, CD-ROM, DVD, or other storage device. In alternative embodiments, hardwired circuitry may be used instead of or in combination with software instructions. Therefore, execution of a sequence of instructions is not limited to any specific combination of hardware circuitry and software instructions.

  Computer-readable media includes any mechanism that provides content (eg, computer-executable instructions) in a form readable by an electronic device (eg, a computer, a personal digital assistant, a mobile phone). For example, computer readable media include read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, and the like.

  The electronic system 100 may be coupled via a bus 105 to a display device 150, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), for displaying information to a user. An alphanumeric input device 160 that includes alphanumeric characters and other keys may be coupled to the bus 105 for communicating information and command selections to the processor 110. Another type of user input device is a cursor controller 170, such as a mouse, trackball, or cursor direction key, that communicates direction information and command selections to the processor 110 and controls cursor movement on the display 150.

  Electronic system 100 may further include a network interface (s) 180 for providing access to a network, such as a local area network. The network interface (s) 180 may include a wireless network interface having an antenna 185 that may represent one or more antennas, for example. The network interface (s) 180 is, for example, an Ethernet cable, coaxial cable, fiber optic cable, serial cable, or parallel cable for communicating with a remote device via a network cable 187, for example. A wired network interface may be included.

  In one embodiment, the network interface (s) 180 can provide access to a local area network, eg, by complying with the IEEE 802.11b and / or IEEE 802.11g standards, and The wireless network interface may provide access to a personal area network, for example, by complying with the Bluetooth® standard. Other wireless network interfaces and / or protocols may be supported.

  IEEE 802.11b was approved on September 16, 1999, “Local and Metropolitan Area Networks, Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY). the Std., entitled “the 2.4 GHz Band”. It corresponds to 802.11b-1999 and related documents. IEEE 802.11g was approved on June 27, 2003, “Local and Metropolitan Area Networks, Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY). IEEE Std. entitled "in the 2.4 GHz Band". It corresponds to 802.11g-2003 and related documents. The Bluetooth (registered trademark) protocol was issued on February 22, 2001 on the Bluetooth (registered trademark) Special Interest Group, Inc. It is described in “Specification of the Bluetooth (registered trademark) System: Core, Version 1.1” published by Associated, and in addition, previous or subsequent versions of the Bluetooth (registered trademark) standard may be supported. .

  In addition to or instead of communication according to the wireless LAN standard, the network interface (s) 180 may include, for example, a time division multiple access (TDMA) protocol, a global system for mobile communication (GSM®) protocol, Wireless communication using code division multiple access (CDMA) protocol and / or any other type of wireless communication protocol may be provided.

  FIG. 2 is a block diagram illustrating one embodiment of a system 200 for displaying web content in a browser, such as in a data processing system or computer. The system 201 can be a computer operating environment including an operating system and / or an application 207 such as a content display application for displaying web content. The application 207 may be a browser that can move between pages and provides the zooming functionality described above.

  Content acquisition / processing module 209 acquires a document or content from a network (eg, from a remote web server) via network interface 203 or from a file system coupled locally or remotely via file system interface 205 You can do it. In one embodiment, the received content is a hypertext-based document encoded in a language such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or other markup language. It's okay.

  The content acquisition / processing module 209 can process (eg, parse) the document, identify document elements, and generate or build a DOM tree or DOM structure 213 from the document elements. The document element may be a hypertext element such as an HTML tag in an HTML document. In one embodiment, the DOM tree 213 may represent topological or structural relationships between document elements. Each node in the document's DOM tree 213 may correspond to one document element of the document. The document element may include a document specification and / or a layout specification such as a size attribute associated with an HTML tag in the HTML document for displaying the document content. Here, the document content is displayed via the display module 219 for displaying a user interface for rendering a web page on the display screen via the user interface module 225, for example.

  In one embodiment, the content acquisition / processing module 209 generates rendering information such as layout parameters (eg, rectangular area size setting data) for each node in the DOM tree 213 for displaying the corresponding document. An engine may be included. The rendering engine can make rendering decisions according to a configuration including layout specifications in the document, user interface settings for the application 207 and / or other system settings, and the like. The rendered DOM tree may depict web content such as a web page displayed by a browser.

  In one embodiment, the application 207 includes a zoom module 223 that can operate with the DOM interface module 211 to provide the zoom functionality described herein. Good. The zoom module 223 may include a zoom detection module 215 for detecting a zoom event. The image module 217 may generate and / or manage snapshots that can be utilized as described herein. The mode configuration module 221 may allow a user to set parameters related to migration, such as a cache size to be used, migration sensitivity, and the like.

  FIG. 3 is a flow diagram of one embodiment of a technique that utilizes a snapshot to provide a zoom-in experience. In one embodiment, the page is provided by a browser application and the zoom-in responds to user input. User input may be provided using buttons or other graphical components, or by user gestures (e.g., a knob action) provided through an input pad or touch screen, or any combination thereof. Other input mechanisms may be supported.

  A zoom-in event is detected (310). In one embodiment, detecting a zoom event corresponds to monitoring an input device that can be utilized to control the zoom event and reacting to the beginning of the input that causes the zoom. Using the example of knob control, if the user places two fingers on the control pad, the user is ready to start a zoom event. For example, if the user moves his finger open, this can cause a zoom in. In this way, a zoom event can be detected in response to the onset of finger movement.

  In response to detecting the zoom event, the browser (or other system entity) can generate a snapshot of the displayed page (320). Snapshot replaces the page in the browser. The snapshot is then used as the image to be zoomed (330). That is, when a page is zoomed in response to user input, what is zoomed is a snapshot, not the page displayed by the browser.

  When zooming is complete, the browser can determine a magnification between the start of zoom and the end of zoom (340). The magnification indicates the amount of zoom applied to the snapshot. The magnification is applied to the page (350). Once the magnification is applied to the page to be provided, a zoomed page corresponding to the zoom applied to the snapshot can be displayed (360).

  FIG. 4 is a flow diagram of one embodiment of a technique that utilizes a snapshot to provide a zoom-out experience. In one embodiment, the page is provided by a browser application and the zoom-in responds to user input. User input may be provided using buttons or other graphical components, or by user gestures (e.g., a knob action) provided through an input pad or touch screen, or any combination thereof. Other input mechanisms may be supported.

  A zoom-out event is detected (410). In one embodiment, detecting a zoom event corresponds to monitoring an input device that can be utilized to control the zoom event and reacting to the beginning of the input that causes the zoom. Using the example of knob control, if the user places two fingers on the control pad, the user is ready to start a zoom event. For example, if the user moves his finger closer, this can cause a zoom out. In this way, a zoom event can be detected in response to the onset of finger movement.

  In response to detecting the zoom event, the browser (or other system entity) can generate a snapshot of the displayed page (420). Snapshot replaces the page in the browser. The snapshot may be, for example, a bitmap of the content of the page or other graphical representation. In one embodiment, only the page content is drawn in the snapshot, and surrounding elements (eg, browser controls, window controls, etc.) are not drawn.

  A maximum zoom area is determined (430). The maximum zoom area may be determined based on the size of the page being viewed, the position of the user's finger on the control pad, and / or using other information. The maximum zoom area represents the maximum possible zoom out and does not necessarily represent the actual (or expected) zoom. The maximum zoom area allows the browser to handle any zoom amount below that including the maximum zoom.

  A container is created based on the maximum zoom (440). The snapshot is placed in the center of the container so that when the user zooms out, the snapshot stays in the center of the browser window. In one embodiment, the area of the container not covered by the snapshot has a color that matches the background of the page. Therefore, when the user zooms out, the area around the snapshot looks like the page background.

  Peripheral content for the page is collected (450). Peripheral content is content that is part of a larger page that is visible but not visible to the user at the time the zoom event is initiated. In one embodiment, this content is obtained from a window server. However, it may be obtained from any relevant source.

  In one embodiment, an initial snapshot is generated from the window server. This is because this content is provided more quickly. If a higher resolution web page snapshot becomes available, this snapshot may be replaced with a higher resolution web page snapshot. The use of window server content allows a faster transition to the use of snapshots, which provides a more immediate zoom response.

  A container with a snapshot and a background that matches the background of the page is used for zooming (460). That is, when a page is zoomed in response to user input, what is zoomed is a snapshot, not the page displayed by the browser.

  When zooming is complete, the browser can determine a magnification between the beginning of the zoom and the end of the zoom (470). The magnification indicates the amount of zoom applied to the snapshot and the container. The magnification is applied to the page (480). Once the magnification is applied to the page to be provided, a zoomed page corresponding to the zoom applied to the snapshot can be displayed (490).

  FIG. 5 is a flow diagram of one embodiment of zooming in that requires panning and / or scrolling. When using some control mechanisms for zooming, it may be necessary to pan or scroll the page to provide the desired zoom. For example, when zooming in by a preselected amount in response to a tap or other command, zooming may cause an area outside the page content to be displayed. In one embodiment, automatic scrolling or panning is applied to keep the view completely within the page.

  A zoom-out event is detected (510). In one embodiment, detecting a zoom event corresponds to monitoring an input device that can be utilized to control the zoom event and reacting to the beginning of the input that causes the zoom. Using the tap or multi-tap, control example, when the user taps a specified number of fingers on the control pad, the user causes a zoom event.

  The browser may determine whether the zoom is off the page or whether the zoom will include an area outside the current view (520). This may occur during zoom out as described above with respect to FIG. It may also occur, for example, when zooming out near the page boundary or near the top of the window. Therefore, automatic panning and / or scrolling may be utilized to keep the browser window within the area that provides the content.

  If the zoom does not deviate from the page (520), the browser (or other system entity) may generate a snapshot of the displayed page (525). Snapshot replaces the page in the browser. The snapshot is then used as the image to be zoomed (535). That is, when a page is zoomed in response to user input, what is zoomed is a snapshot, not the page displayed by the browser.

  When zooming is complete, the browser can determine the magnification between the beginning of the zoom and the end of the zoom (545). The magnification indicates the amount of zoom applied to the snapshot. The magnification is applied to the page (555). Once the magnification is applied to the provided page, a zoomed page corresponding to the zoom applied to the snapshot can be displayed (565).

  If the zoom is off the page (520), the browser (or other system entity) may generate a snapshot of the displayed page (530). Snapshot replaces the page in the browser. The snapshot may be, for example, a bitmap of the content of the page or other graphical representation. In one embodiment, only page content is rendered in the snapshot and surrounding elements (eg, browser controls, window controls, etc.) are not rendered.

  A zoom area is determined (540). The zoom area may be determined based on the size of the page being viewed, the position of the view within the page, and / or using other information. The zoom area represents a possible zoom-out and does not necessarily represent the actual (or expected) zoom. The zoom area allows the browser to handle any zoom amount below it, including maximum zoom.

  A container is created based on the zoom area (550). Snapshots are positioned within the container based on the panning and / or scrolling required to keep the view within the page content so that the snapshot stays in place in the browser window as the user zooms. . In one embodiment, the area of the container not covered by the snapshot has a color that matches the background of the page. Therefore, when the user zooms out, the area around the snapshot looks like the page background.

  Peripheral content for the page is collected (560). Peripheral content is content that is part of a larger page that is visible but not visible to the user at the time the zoom event is initiated. In one embodiment, this content is obtained from a window server. However, it may be obtained from any relevant source.

  In one embodiment, an initial snapshot is generated from the window server. This is because this content is provided more quickly. If a higher resolution web page snapshot becomes available, this snapshot may be replaced with a higher resolution web page snapshot. The use of window server content allows a faster transition to the use of snapshots, which provides a more immediate zoom response.

  A container with a snapshot and a background that matches the background of the page is used for zooming (570). That is, when a page is zoomed in response to user input, what is zoomed is a snapshot, not the page displayed by the browser.

  When zooming is complete, the browser can determine the magnification between the beginning of the zoom and the end of the zoom (545). The magnification indicates the amount of zoom applied to the snapshot and the container. The magnification is applied to the page (555). Once the magnification is applied to the provided page, a zoomed page corresponding to the zoom applied to the snapshot can be displayed (565).

  FIG. 6 is a block diagram of one embodiment of a zoom control agent that may reside, for example, within a browser application, an electronic system that provides the browser application, or a combination thereof. The zoom control agent 600 includes control logic 610 that implements a logic function controller that directs the operation of the zoom control agent 600 and / or hardware associated with directing the operation of the zoom control agent 600. The logic may be hardware logic and / or software routines. In one embodiment, zoom control agent 600 includes one or more applications 612 that represent code sequences and / or programs that provide instructions to control logic 610.

  The zoom control agent 600 includes a memory 614 that represents access to memory devices and / or memory resources for storing data and / or instructions. The memory 614 may include memory local to the zoom control agent 600, and in addition or alternatively, includes memory of the host system in which the zoom control agent 600 resides. The zoom control agent 600 represents an access interface (eg, input / output interface, application programming interface) to / from the zoom control agent 600 relating to an entity (electronic or human) external to the zoom control agent 600. One or more interfaces 616 are also included.

  The zoom control agent 600 also includes a zoom control engine 620 that represents one or more functions that allow the zoom control agent 600 to provide zoom in and / or out on the pages described herein. . An example of a module that may be included in the zoom detection module 630, the snapshot module 640, the zoom module 650, the magnification change module 660, and the display module 670. Each of these modules may further include other modules to provide other functions. As used herein, a module refers to a routine, subsystem, etc., regardless of whether the implementation is hardware, software, firmware, or some combination thereof.

  The zoom detection module 630 can function to determine when a zoom event has been initiated. As described above, a zoom event may be initiated by a gesture or other control pad interaction, the use of an input device such as a wheel or mouse with a control pad, an interaction using a graphical user interface element, or the like. The zoom detection module 630 can communicate the beginning of a zoom event to one or more other modules.

  The snapshot module 640 can function to generate and manage snapshots that can be utilized for zooming as described herein. The snapshot module 640 may create a snapshot image of the content of the page in response to a zoom event indication from the zoom detection module 630. The snapshot may be displayed for zooming purposes and may be stored for other uses.

  The zoom module 650 may function to provide a zoom operation on the snapshot generated by the snapshot module 640. Zoom module 650 may provide zoom-in and / or zoom-out as described herein utilizing snapshot images.

  The magnification change module 660 may interact with the zoom module to determine the magnification corresponding to the zoom operation. The magnification change module 660 may communicate the magnification to the display module 670 or other components. The display module 670 functions to determine and control whether a snapshot image or a page is displayed. Further, the display module 670 may provide panning and / or scrolling information related to positions within the page.

  References to “one embodiment” or “one embodiment” in the specification include the particular structure or feature described in connection with that embodiment, in at least one embodiment of the invention. Means that. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment.

  In the foregoing specification, the invention has been described with reference to specific embodiments thereof. However, it will be apparent that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.

Claims (28)

In response to detecting the start of the zoom control input, generating a snapshot of the content;
Performing a zoom operation on the snapshot based on the zoom control input to provide a zoomed snapshot;
Displaying the snapshot while performing the zooming;
Rendering the content to provide a zoom amount indicated by the zoom control input and to provide post-zoom rendering;
Mixing the zoomed snapshot and the zoomed rendering to provide a final image;
Displaying the final image;
A method comprising:   The method of claim 1, wherein the zoom control input comprises a double tap with a finger on a control interface of an electronic device.   The method of claim 1, wherein the zoom control includes a finger picking action.   The method of claim 3, wherein the finger picking action includes moving the fingers closer together to cause a zoom in.   The method of claim 3, wherein the finger pinching includes moving the finger further away to cause a zoom out.   The method of claim 2, wherein the control interface includes a control pad of a laptop computer.   The method of claim 2, wherein the control interface includes a control pad that is communicatively coupled to a computing device.   The method of claim 2, wherein the control interface comprises a touch screen of a tablet device.   The method of claim 1, further comprising applying a scroll operation to the snapshot to compensate for zooming beyond the snapshot edge. An article of manufacture comprising a computer-readable medium, wherein when executed by one or more processors, the one or more processors include:
In response to detecting the start of the zoom control input, generating a snapshot of the content;
Performing a zoom operation on the snapshot based on the zoom control input to provide a zoomed snapshot;
Displaying the snapshot while performing the zooming;
Rendering the content to provide a zoom amount indicated by the zoom control input and to provide a post-zoom rendering;
Mixing the zoomed snapshot and the zoomed rendering to provide a final image;
Displaying the final image;
An article of manufacture in which instructions are stored on the computer readable medium.   The article of claim 10, wherein the zoom control input comprises a double tap with a finger on a control interface of an electronic device.   The article of claim 10, wherein the zoom control includes a finger gripping action.   13. The article of claim 12, wherein the finger picking action includes moving the fingers closer together to cause a zoom in.   The article of claim 12, wherein the finger pinching includes moving the finger further away to cause a zoom out.   The article of claim 11, wherein the control interface comprises a control pad of a laptop computer.   The article of claim 11, wherein the control interface includes a control pad communicatively coupled to a computing device.   The article of claim 11, wherein the control interface comprises a touch screen of a tablet device.   The article of claim 10, further comprising instructions that when executed cause the one or more processors to apply a scrolling operation to the snapshot to compensate for zooming beyond a snapshot edge. Means for generating a snapshot of the content in response to detecting the start of the zoom control input;
Means for performing a zoom operation on the snapshot based on the zoom control input to provide a zoomed snapshot;
Means for displaying the snapshot while performing the zooming;
Means for rendering the content to provide a zoom amount indicated by the zoom control input and to provide a post-zoom rendering;
Means for mixing the zoomed snapshot and the zoomed rendering to provide a final image;
Means for displaying the final image;
An apparatus comprising:   The apparatus of claim 19, further comprising means for applying a scrolling operation to the snapshot to compensate for zooming beyond the snapshot edge. A display device;
A zoom control agent coupled to the display device, wherein the zoom control agent is responsive to detecting the start of a zoom control input to generate a snapshot of the content and to provide a zoomed snapshot Performing a zoom operation on the snapshot based on the zoom control input, displaying the snapshot while performing the zooming, providing a zoom amount indicated by the zoom control input, and after zooming Zooming to render the content to provide a rendering of the image, to mix the zoomed snapshot with the zoomed rendering to provide a final image, and to display the final image. A control agent;
A system comprising:   The system of claim 21, wherein the zoom control input comprises a double tap with a finger on a control interface of an electronic device.   The system of claim 21, wherein the zoom control includes a finger picking action.   24. The system of claim 23, wherein the finger pinching includes moving the fingers closer together to cause a zoom in.   24. The system of claim 23, wherein the finger picking action includes moving the finger further away to cause a zoom out.   23. The system of claim 22, wherein the control interface comprises a laptop computer control pad.   23. The system of claim 22, wherein the control interface includes a control pad that is communicatively coupled to a computing device.   23. The system of claim 22, wherein the control interface includes a tablet device touch screen.

Images