TWI465969B - Panning content utilizing a drag-operation - Google Patents

Description

Use drag and drop to pan content

The present invention relates to panning content using a drag operation.

Computers and mobile devices (such as PDAs, cell phones, laptops, wireless handheld devices, etc.) that are used to implement touch screen interfaces are becoming more and more popular. These touch screen interfaces are an important distinguishing feature for mobile devices and other electronic products that tend to interact intuitively with the software integrated into them. Ideally, the touch screen interface should provide a complete software experience to manipulate the software to achieve all the functions that a conventional input device can achieve. However, the large number of software available today has not been designed to incorporate touch concepts, hereinafter referred to as touch-naïve applications. For example, in the case where the non-touch application is configured to scroll the content presented by the display area on the user interface (UI) display, the scrolling function is not fully integrated into the touch screen interface. That is to say, the touch screen interface does not have the ability to scroll content by performing a drag operation on the content area in the display area, whereas conventional input devices have such capabilities (eg, using a mouse, keyboard, etc.).

Until now, users of the touch screen interface have not been able to use the full non-touch application functionality. Even, there is no solution to adapt the gesture to a touch screen interface (the movement is different from the expected movement of the traditional input device), so that the intuitive command manipulation can display the content on In the display area. Therefore, using the gesture interpretation technology to control the scrolling functionality of the non-touch application will enhance the user's dragging when applying a specific gesture (eg, a drag operation at a content area on a display area) to the touch screen interface. The experience in terms of content.

This "Summary of the Invention" is provided to briefly introduce the selection concept of the present invention, which will be further described in the "Embodiment" below. This Summary of the Invention is not intended to identify key or essential features of the subject matter of the present invention, and is not intended to assist in determining the scope of the claimed subject matter.

A specific embodiment of the present invention provides a computer readable medium and computer system including a plurality of computer readable commands for enabling a user of the touch screen interface to utilize the scrolling functionality of the non-touch application. In particular, a technique for controlling scrolling functionality of a non-touch application when the touch screen interface is instructed to receive a particular gesture (eg, a drag operation in a content region of the display area) is provided.

In general, the present invention provides a method of translating content for presentation in a display area on a UI display of a computing device. First, an input initiated by a user applied to the touch screen interface is detected, and the touch screen provides startup information to the operating system. Typically, touch screen information includes a touch location and a gesture. The recognition application supports the scrolling function substantially simultaneously with or before receiving the user-initiated input. The operating system initiates a gesture interpretation process to decide to pan the display area The content, panning the display area within the UI display, or performing other operations. In a specific embodiment, the gesture interpretation program includes, but is not limited to, determining a touch position occurring in a content area of a display area of the touch screen surface, and a distance according to a continuous uninterrupted drag movement from the start position. To clearly distinguish between the drag operation and the click operation, thereby determining that the gesture is a drag operation. Typically, the content within the content area is rendered via a non-touch application integrated into the computing device, wherein the purpose of establishing the non-touch application is not to identify the drag operation within the content area as an indication of panning the content. When the gesture is recognized as a drag operation and the application supports the scroll function, the content can be panned in the content area of the display area according to the drag operation. The indication of the panning content can be presented to the user at the touch screen or any other display component.

The subject matter of the invention is expressly described herein to comply with legal requirements. However, the description itself is not intended to limit the scope of the patent. On the contrary, the inventors have considered that the subject matter of the present invention may be implemented in other ways, and thus includes a combination of steps different from those described herein or steps similar thereto to be combined with other prior art or future technologies. In addition, although the terms "step" and/or "square" are used herein to mean different elements of the method of the method, unless the individual indicates that the individual steps have a specific order, the words should not be It is to be understood that the various steps disclosed herein must be in any particular order.

A specific embodiment of the present invention provides a computer readable medium having computer executable instructions thereon, a computer system, and a computerization step for translating content in a content area of a display area presented by the non-touch application, the non- The touch application supports the functionality of scrolling by performing a gesture (eg, a drag operation) within the content area.

Thus, in one aspect, embodiments of the present invention provide one or more computer readable media having computer executable instructions thereon that, when executed, can perform a configuration for panning display on a mobile device The content of the display area of the touch screen interface. In some embodiments, the method includes detecting a user-initiated input applied to the touch screen interface and receiving activation information from the touch screen interface. Typically, touch screen information includes touch locations and gestures. The method can further include determining a touch location occurring within a content area of the display area of the touch screen interface surface. In some embodiments, the display area can be presented by an application integrated into the operating system of the mobile device. The application is typically non-touch, meaning that gestures that affect the content on the surface cannot be interpreted as scrolling or selecting. In an exemplary embodiment, a continuous uninterrupted movement distance (eg, a single-finger contact within the touch screen interface, or based on the gesture from a user-activated activation position on the touch screen surface, or The use of a stylus or a pen to interact with a touch screen, etc., to clearly distinguish between a drag operation and a click operation, thereby identifying the gesture as a drag operation. For example, continuous uninterrupted movement includes sweeping the touch screen interface over a critical distance in a predetermined Cartesian direction. By using the scrolling functionality of the app, gestures facilitate translation by dragging The content in the content area of the display area. An indication of the translation of the content can be displayed on the touch screen interface.

In general, embodiments of the present invention relate to touch translation in a touch screen interface environment. As explained herein, "touch translation" is not limiting and may include any user action that may initiate scrolling functionality of an application. In one example, touch translation includes scrolling a page or list by touching a content area and performing a drag operation to move the content along a contact position of the drag operation. In the case where the application does not provide scrolling functionality, performing a touch shift can achieve a general panning motion of the display area within the mobile UI display. In this way, interference with applications that do not provide scrolling functionality can be avoided.

In order to facilitate touch translation, a gesture confirmation program can be implemented to identify a user-initiated input gesture on a touch screen display or other touch input device as a drag operation or a click operation. As described above, in one example, the drag operation is significantly different from the tap type operation based on the distance of a continuous uninterrupted movement. As used herein, the term "uninterrupted movement" is not limiting and may include any continuous sweep, semi-continuous input, or recursive input chain across repeated drag contacts across the surface of the touch screen interface. . In one example, continuous uninterrupted movement may include contact between a physical object (eg, a stylus, a pen or a portion of a finger pad, etc.) and a touch screen interface. In an exemplary embodiment, if the activation position occurs within the content area of the display area (eg, a window on the surface of the UI display) and the drag operation is confirmed, the pan mode is initiated. In more detail, in pan mode, The operating system provides a panning message to the application that displays the content, as opposed to a mouse message that typically does not initiate a panning of the content in the content area.

Although a number of different methods for establishing continuous uninterrupted movement have been described above, one of ordinary skill in the art will recognize and appreciate that various other drag inputs can be included herein for application to the touch screen assembly. Continuous uninterrupted movement, and embodiments of the invention are not limited to the methods shown and described. For example, Figures 4 through 7 illustrate other gestures of continuous uninterrupted movement.

In various embodiments, the present invention relates to interpreting gestures received at a touchscreen interface and providing a message to an application based on this interpretation. The term "application" is used without limitation and may include any operable program or software that can be controlled by the operating system, wherein when a user initiates an input, the operating system receives input from the touchscreen interface. News. For example, the application can be a third party application that presents scrollable content at a display component and/or touch screen interface. In this example, this application is considered to support the scrolling feature. In addition, if a reel tool is presented in the display area at the UI display, the scrolling function can be detected.

In a specific embodiment, the application is a gesture-aware application. The gesture recognition application is a type of application that touches the touch design, and does not need to interpret the gesture on a touch interface and communicate the gesture to the application with a special message to work normally. As such, gesture recognition applications can generally be optimized to integrate touch into the programming and receive raw input from the touch screen interface.

In another embodiment, the application is a non-touch application, and most of the applications that exist in the existing market belong to the application. In general, a non-touch application is an application in which no panning features (eg, scrolling through input in the content area of the touch screen interface) are established, implemented, or written. That is to say, such an application cannot know that it is cooperating with the touch screen input, in particular, it cannot know that the gesture performed at the touch screen is intended to translate the content presented thereon. By way of example only, the non-touch application can be an old application (eg, a notebook text compiler) in which the scrolling function is allowed to perform the operation of panning the content only by providing input at the reel tool.

Therefore, the method described in the specific embodiments of the present invention can add touch translation capability to non-touch applications, and is quite suitable for an operating system including an ecosystem of non-touch applications. For example, the operating system can operate as an interface for connecting the touch screen interface and applications to manage communication therebetween. By managing its communication, appropriate commands in the form of messages (eg, a mouse message or a scrolling message) can be passed to the application to properly adjust the presentation of the content. As such, management with the operating system will likely allow the operating system to enable most existing non-touch applications to support touch capabilities.

An overview of various embodiments of the present invention and some of the features of the present invention are briefly described, and an exemplary operating environment suitable for implementing the present invention will be described below.

Referring generally to all of the drawings, and with particular reference to Figure 1, an exemplary operating environment for implementing a particular embodiment of the present invention is generally shown. The computing device 100 is shown and designated. The computing device 100 is only one example of a suitable computing environment and is not intended to impose any limitation on the use or functionality of the present invention. The computing device 100 should not be construed as having to rely on or must have one or a combination of the illustrated components.

The present invention can be described in the context of general computer code or machine usable instructions, including computer executable instructions, such as program components, executable by a computer or other machine, such as a personal data assistant or other handheld device. In general, program components include routines, programs, objects, components, data structures, etc., which are code that can perform specific tasks or implement specific abstract data types. Particular embodiments of the present invention can be implemented in a variety of system configurations, including handheld devices, consumer electronics, general purpose computers, specialized computing devices, and the like. Embodiments of the invention may also be implemented in a distributed computing environment in which tasks are performed by remote processing devices that are coupled through a communications network.

With continued reference to FIG. 1, computing device 100 includes a busbar 110 coupled directly or indirectly to: memory 112, one or more processors 114, one or more display components 116, input/output (I/O) The device 118, the I/O component 120, and an exemplary power supply 122. Busbars 110 may represent one or more busbars (such as a single address bus, a data bus, or a combination thereof). Although the various blocks in Figure 1 are shown in lines for simplicity, the various components are actually ambiguously described, but metaphorically these lines are more accurately blurred. For example, a display component such as a display device can be considered an I/O component. In addition, the processor has a memory. The inventors of the present invention confirmed this as the essence of the art and reiterated FIG. The illustrations are only one example of an exemplary computing device that can be used in conjunction with one or more embodiments of the present invention. The terms "workstation", "server", "laptop", "handheld device", etc. are not clearly defined. These are included in the scope of Figure 1 and are in the form of "computer" or "computing device". "To represent.

The computing device 100 generally includes a variety of computer readable media. In one example (without limitation), computer readable media may include random access memory (RAM); read only memory (ROM); electronically erasable programmable read only memory (EEPROM); cache memory Body or other memory technology; CDROM, digital versatile disc (DVD) or other optical or holographic media; magnetic cards, magnetic tape, magnetic storage or other magnetic storage devices, carrier waves, or any other code that can be used to encode Information and media accessible by computing device 100.

Memory 112 includes computer storage media in the form of volatile and/or non-volatile memory. The memory can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, and the like. The arithmetic memory 100 includes one or more processors that read material from various entities such as the memory 112 or the I/O component 120. Display component (plural) 116 presents data to the user or other device. Exemplary display assemblies include a display device, a speaker, a printing assembly, a vibrating assembly, and the like. I/O 埠 118 allows computing device 100 to be logically coupled to other devices, including I/O component 120, and some of the devices can be built into it. Exemplary components include a microphone, rocker, game handle, satellite dish, scanner, printer, wireless device, and the like.

Referring now to Figure 2, there is shown an exemplary system architecture suitable for implementing embodiments of the present invention in accordance with an embodiment of the present invention. Those of ordinary skill in the art will understand and appreciate that the exemplary system architecture 200 illustrated in FIG. 2 is only one example of a suitable computing environment and is not intended to limit the scope of use or functionality of the invention. This exemplary system architecture 200 should also not be considered to be dependent on or have any single component or combination of components as exemplified herein.

First, an arithmetic device 215 is provided. The computing device 215 can be any form of computing device. In an example, computing device 215 can be a human computing device (eg, computing device 100 of FIG. 1), a handheld device (eg, a personal digital assistant), a mobile device (eg, laptop, cell phone, media) Players), consumer electronic devices, various servers, and the like. Further, the arithmetic device may include two or more electronic devices that share information with each other.

In various embodiments, a display component 250, an input device 221, and a processor (eg, hardware) can be disposed on the computing device 215, with an operating system 220 mounted thereon. The computing device 215 is configured to present a UI display on the display assembly 250. The display component is operatively coupled to the processing component 255 and can be configured as any display device capable of presenting information to the user, such as a monitor, an electronic display panel, a touch screen, a liquid crystal display (LCD), a plasma screen, One or more light emitting diodes (LEDs), incandescent light bulbs, lasers, cold light sources, chemical light, flexible light cables, and/or fluorescent lights, or any other display type, or may include a reflective plane, visible information may be Projected on this plane. In a fan In an exemplary embodiment, the UI display presented by display component 250 can be configured to present a display area 223 having a content area (not shown), wherein the content presented by application 230 is presented within the content area. On the surface. In another exemplary embodiment, the UI display can provide a visual indication that translates content within the content area or pans the display area 223 within the UI display.

In a particular embodiment, computing device 215 is operatively coupled to input device 221 and display assembly 250. As such, communication between input device 221, display component 250, and operating system 220 can be supported by a distributed computing environment, such as a remote server (not shown) or any combination thereof. In a specific embodiment of the present invention implemented in a distributed computing environment, the communication system is linked by appropriate connections. These links can be wired or wireless. Specific wired embodiments within the scope of the present invention include USB connections and cable connections. Specific wireless embodiments within the scope of the present invention include short-range wireless networks and radio frequency technologies. It should be understood and appreciated that the designation of "close-range wireless network" is not intended to be limited only to such technology and should be interpreted as including at least the following techniques: negotiated wireless peripheral (NWP) devices; short-range wireless air intermediaries and between any support means; network (e.g., wireless personal area network (wPAN), wireless local area network (wLAN), wireless wide area network (wWAN), Bluetooth ^TM, etc.); a wireless peer to peer communication (e.g., ultra wideband) Protocol for wireless communication of data. Moreover, those skilled in the art will appreciate that short-range wireless networks can be implemented in a variety of data transfer methods (e.g., satellite transmission, telecommunications networks, etc.). It is therefore emphasized that the embodiment of the connection between the computing device 215 or other remote components that are operatively coupled to the device 215 is not limited to the examples described above, and various communication methods are also included.

An input device 221 is provided for accepting user initiated input 295 to effect content such as panning the content area. In an exemplary embodiment, input device 221 receives user initiation input 295 at display area 223 presented on the UI display. Exemplary devices include the touch screen interface shown in FIG. 1 (eg, can be used as display component 250 at the same time), I/O component 120, or any other user-initiated input 295 (including a gesture) and An indication of the input is communicated to the components of the operating system 220. For example only, the input device 221 may determine a coordinate position (e.g., an activation position) at which the user initiates input 295 to initiate contact with, and a coordinate position at which the user initiates input 295 to terminate contact therewith (e.g., release position) ), refer to the project presented on the GUI. These coordinate positions can be compared to determine the distance and direction of a continuous uninterrupted movement, as described more fully below. Therefore, a drag operation can be extracted from a gesture on the input device 221. In a particular embodiment, prior to transmitting the input to the operating system 220, a screen driver pre-processes the input to extract input information from the original input.

While a number of differently configured input devices 221 have been described above, those skilled in the art will recognize and appreciate that various input devices for receiving input via physical contact 290 or other means may be employed as input device 221. And the specific embodiments of the present invention are not limited to the illustrated input and described input devices 221 integrated in display component 250. For example, display component 250 can be a monitor operatively coupled to a computer (eg, computing device 215), and input device 221 can be an independent capacitive touch that wirelessly communicates with the computing device (as described above) Control the screen. However, for simplicity and clarity, input device 221 and display component 250 will be combined hereinafter as a touch interface as a single unit that provides both display and input elements.

Operating System (OS) 220 generally refers to software that manages resource sharing of processing components and provides a program editor with an interface to access these resources. In operation, operating system 220 interprets system data and detects user initiation input 295 (eg, via input device 210), and then executes the following program in a non-specific order: receiving a physical object 290 at a touch screen. Input 295 (eg, single-finger gesture, pen click, etc.); determine what gesture is performed by input 295; determine that application 230 supports the scrolling function; and if the gesture is mapped to a drag operation in the scrolling direction, filter to application 230 The input 295 is transmitted. The term "scrolling direction" as used herein is not intended to be limiting, and is generally related to the direction in which content can be allowed to move within the content area as specified by application 230. In one example, the scrolling direction is generally parallel to the line of the reel tool (eg, horizontal or vertical).

In an exemplary embodiment, operating system 220 executes on a processor and supports execution of one or more applications, including application 230. In other embodiments, operating system 220 and/or application 230 may reside partially or wholly on a remote location computing device, such as on another operating system, on a server located via a web address. In multiple specific In an embodiment, the operating system 220 determines whether to provide a panning message to the application 230 based on the input information 245 received from the input device 221. As shown in Figure 2, although the user initiated input 295 is received from the touch screen interface, it should be understood and appreciated that other inputs from various input devices (eg, a remote tactile input panel) can be received. It is interpreted by the operating system 220; therefore, the scope of the invention is not limited to the input and input devices described herein. If the distance and/or direction of a gesture included in input 295 satisfies the metric used by the system to evaluate whether to initiate the pan mode, then operating system 220 can provide a scrolling message to application 230 to cause the content to translate in accordance with the gesture. In this example, the scrolling function of the application 230 is mapped to the drag operation of the gesture, which may be initiated automatically.

In general, application 230 is any application that can be launched and manipulated by operating system 220. As previously mentioned, the application 230 manages the content presented on the UI display. Moreover, as previously mentioned, the application 230 is a non-touch application or otherwise capable of accepting raw input from the touch screen interface.

This operating system architecture of operating system 220 is merely a suitable architectural example that can be implemented on a processor and is not intended to impose any limitation on the use or functionality of the present invention. The illustrated operating system 220 should also not be considered as having to rely on or have any of the functions of the user initiated input 295 in the form of receiving input information 245 or the function of providing the message 235, or a combination thereof. Moreover, application 230 may be integrated into operating system 220 and is merely exemplary in nature and quantity and should not be considered as limiting.

In an exemplary embodiment, operating system 220 includes an input The component 205, a gesture engine 280 (including a correction component 255 and a selection component 226), a reel seeker 210, and an application interface component 235.

Additionally, operating system 220 can include a data store 275. In another embodiment, the data store 275 can reside on a remote location computing device (e.g., on another operating system, in a server that is located via a web address). The data store 275 can be configured to store information including suitable messages mapped to particular gestures, including information such as a trigger boundary for performing system metrics from panning content to content selection, and including other information related to panning Information. Moreover, although illustrated as a single, stand-alone component, the data store 275 can in fact be a plurality of databases, such as a server cluster.

This operational system configuration of operating system 220 is merely an example of a suitable configuration that can be implemented on computing device 215, and is not intended to impose any limitation on the use or functionality of the present invention. The illustrated operating system 220 should also not be considered to be dependent on or have one or a combination of the illustrated components 205, 210, 225, 226, 280, or 250. In one such embodiment, one or more of the components 205, 210, 225, 226, 280, or 250 can be implemented as a standalone application. In other embodiments, one or more of the components 205, 210, 225, 226, 280, or 250 can be directly integrated into the display device 221, the application 230, or a combination thereof of the computing device 215. For example only, the application interface component 250 can be integrated with the display device 221. One of ordinary skill in the art will appreciate that the components 205, 210, 225, 226, 280, or 250 shown in FIG. 2 are merely exemplary in nature and quantity and should not be considered as limiting.

Any number of components can be utilized to achieve the desired functionality within the scope of the specific embodiments of the invention. Although the various components of Figure 2 are represented by lines for the sake of brevity, and in fact the individual components/components are less clear, the metaphorically these lines are precisely blurred. In addition, although some of the components and devices in FIG. 2 are illustrated as a single block, these examples are exemplary in nature and number and should not be considered as limiting (eg, although only one display device 215 is shown, A plurality of display devices can be operatively coupled to the computing device 210 to synergistically present the UI display 225).

First, the input component 205 is configured to receive input information 245 from the touchscreen interface. In an exemplary embodiment, input information 245 may be derived from an input 290 when the user touches (eg, a single-finger gesture) the touch screen interface. In other embodiments, the input information 245 includes an activation position (eg, a Cartesian coordinate position) on the touch screen interface and data associated with a gesture (eg, continuous uninterrupted movement distance, movement direction, activation). Period, etc.). This input information 245 can be pre-processed (not shown) by a screen driver to transfer the data to the input component 250. Input component 205 can then process input information 205 and then pass it to gesture engine 280.

The reel lookup component 210 determines if the application 230 supports the scrolling function. In one example, the determination is made by looking up the touch screen interface to determine if a reel tool is present in display area 223. If the application 230 supports the scrolling function, the scrolling message can be sent to the application to perform a panning action on the content based on a drag operation of the received gesture. Otherwise, mention The standard "mouse message" is given to the application 230. The mouse message is an unscreened input 295 that is provided to a non-touch application that controls the application in a manner similar to that expected using a mouse tool.

Although a method for determining whether the application 230 displays the scrolling function at the UI display has been discussed above, there are several different determination methods that can be implemented and applied to a variety of different reel implementations.

The gesture engine 280 is configured to distinguish which gestures the user is currently performing on the computing device 215. For example, gesture engine 280 can use a timer to process a continuous uninterrupted gesture and/or a duration gesture. If the duration gesture does not include an uninterrupted movement of a distance longer than a predetermined distance, or does not reach a predetermined critical time period, the gesture is considered a tap type operation (eg, simulating a mouse click) hit). Otherwise, the gesture is treated as a drag operation.

In a particular embodiment, gesture engine 280 can distinguish between translation and standard motion. For example, if the gesture is considered a drag operation, the gesture engine 280 monitors the UI display of the application 230 to determine if the user is dragging within the scrolling content area. In a particular embodiment, a technique and method for identifying whether a boot location (ie, a touch point that touches a touch screen interface) occurs in a scrolling content region is employed. If the activation location is within the content area and a drag operation is confirmed, the scrolling message (rather than transmitting the mouse message) is transmitted to the application 230 so that the content in the content area is dragged with the user's finger. Thus, this method allows the user to drag their fingers over the content area for translation and to drag or select within the content area using different gestures. For example, the user can Dragging a finger on a surface file list to pan through the items listed in a profile searcher menu and performing a different gesture to select and move a file (eg, executing a contact on the title bar or border of display area 230) .

If the user is dragging a non-rolling content area (eg, title bar, border, scrolling tool, etc.) presented by the application 230, the mouse message is sent to the application to use the application interface component 250, so that the finger is used Dragging is like clicking and dragging with a mouse. This allows the user to drag or pan the display area 223, drag the slider of the scrolling content area, rearrange the desktop icon, etc. within the screen (eg, UI display).

In general, gesture engine 289 filters the input to application 230 by providing appropriate message 235. When the scrolling content is dragged in the scrolling direction, it will move with a touch point on the touchscreen interface and provide an averaging mechanism to smooth the movement of the content. When the user views a scrolling tool presented at display area 223, an indication is provided to the user to indicate that the content can be panned. As such, the user will intuitively drag the finger to pan the content (eg, a page in a file, or a map of the viewer). When the user drags his finger in the scrolling content area in a direction substantially parallel to the scrolling tool, the content will translate in the appropriate direction. The panning will continue until the user lifts his finger or interrupts the dragging operation of continuous movement. In addition, panning the content can drive the slider of the scrolling tool to provide visible feedback to indicate that the panning system is being executed (eg, moving the reel for the user). In an exemplary embodiment, the input 295 is managed to control the translation of the content presented by the application 230, which management can interact with various reel implementations.

In operation, gesture engine 280 identifies the type of gesture being performed, where the gesture originated, and any other attributes of the gesture. When interpreting this information, the selection component 226 is configured to provide appropriate actions (mapped to the interpreted gesture) to the application interface component 250, which sends an appropriate message 235 to the application 230. For example, if the gesture engine determines that the user is dragging the non-scrolling portion of the display area, then a conventional left-click mouse drag action is initiated via the mapping. If the direction in which the user is dragging cannot be scrolled (for example, perpendicular to the reel tool), a selection or left mouse drag action is performed. Alternatively, the user can select a content by performing a first hold and drag gesture to initiate a left mouse drag action.

In a particular embodiment, the user drags a finger or physical object 290 in the general direction of the scrolling tool (eg, above and below a vertical scrolling tool) in the content area of display area 223 (eg, IE browsing) Window, media player, notebook file, Word, Outlook, start menu, control panel, or other flat application UI), content (for example, folder list, email list, group email list, scrolling font) You should pan under your finger. In addition, only the content of the selection display area 223 at the beginning of the drag should be affected, and the other contents of the other panes should not be scrolled or displayed in a hovering state. Furthermore, dragging in the display area 223 that is not allowed to scroll will cause the entire display area 223 to be translated on the page. To perform these actions, the gesture engine provides a communication stream to the application 223 to ensure that the application 223 processes the input 295 in a particular manner.

Selection component 226 is configured to select an appropriate one based on input information 245. The action is passed to the application interface component 250. In addition, selection component 226 is configured to determine when the user pans out of the end of a scrolling zone and provides a translational out-of-bounds motion to provide visual feedback to let the user know that they have reached the range boundary of the available scrolling content. In one embodiment, the translational crossing action moves the entire window in accordance with the direction in which the user slides their finger on the touch screen interface. Advantageously, this visual feedback (which moves the entire display area 223) can be implemented or scaled to manipulate an old application. In particular, since the operating system 220 has the function of sliding the on-screen window, the translational out-of-bounds action for moving the display area on the UI display can be implemented without the need for cooperative operation or interaction with the application 230. Thus, the translational out-of-bounds action provides a clear indication that the scrolling function is available but has reached the boundaries of the page.

Since the new user will often make a mistake of dragging in the wrong direction, the panning outbound action provides an auxiliary action to warn the user that there is no content in the direction of the drag operation. In addition, the translational out-of-bounds action also provides a teaching behavior that can teach the user how to control content movement while panning. The error of panning in the opposite direction of the intended direction occurs because the direction of the generally moved content when dragging the finger in the content area is opposite to the direction of the content being moved when the slider of the scroll tool is pulled in the same direction. Specifically, when a new user is above a file, the new user will often drag his finger down because he is accustomed to the corresponding action when the scrolling tool is operated. When the user drags his finger down in this state, the entire window or display area 223 will slide down. The pan across the boundary action visually conveys that the translation is available but the user Drag in the wrong direction. However, when the touch shift is a "direct manipulation system" in which the content moves substantially in the drag direction, the page dragging up the content or a file upward moves the basis of the page. If no translational transgression or other feedback is provided to the user, the user will often consider the scrolling function to be unavailable or not working properly.

Although one embodiment of a translational out-of-bounds motion is described, those skilled in the art will appreciate and appreciate that other types of appropriate translational out-of-bounds actions can be used to provide visual or audio feedback to the user to convey the scrolling function. It is to be understood that the specific embodiments of the invention are not limited to those that are described herein. In addition, a user of a translational out-of-bounds action is uniquely suited to a multi-tasking system (ie, having a multi-view capable screen), wherein each executing application is present in one or more display areas on the surface of the UI display. In addition, if the user "throws" a page to reach the boundary of the scrolling region, the panning is triggered. In a particular embodiment, the throwing action provides the user with the ability to first move the content under the control of the drag operation, then automatically continue to pan the content and/or slide the display area 223 when the drag operation is interrupted, as if the content had quality Inertial movement.

The correction component 225 is configured to remove an offset tolerance that provides for alignment with the reel tool when dragged in one direction. For example, when the user begins panning (eg, by dragging up in the scrolling content area), the selection component 226 translates the page up and down, even when the user wants to scroll vertically or slightly. right. In other words, when the drag operation is performed, even if the user's finger drag action is not exactly straight Moving up/down or in the right/right direction, the user is still able to pan or pan horizontally. These features can be turned off if the user prefers to think that the oblique translation is more efficient. However, in general, the correction component 225 is enabled because translating the content in a straight or straight down or horizontal direction in the content area is less distracting and is a more traditional/intuitive movement for viewing.

Due to the friction during dragging and the movement of a whole arm to establish an input ergonomics, it is difficult for the user to achieve precise drag on the touch computer. Advantageously, a correction component 226 can be used. Therefore, there is an inherent offset and/or wobble in the drag interaction. Correction component 225 perceives the wobble/offset and suppresses these inputs such that the content is translated in a straight line (i.e., as on a track). In a specific embodiment, the suppressing action is performed by determining a major axis in the direction of the drag operation. The auxiliary movement on the non-spindle is then filtered out to assist in scrolling the specific direction of the drag operation. Therefore, the user is allowed to be more casual in the accuracy of the towing when still seeing a gentle translational motion.

In a particular embodiment, with reference to Figures 4 and 5, the critical range may be related to the main axis and it is defined such that an important auxiliary movement (e.g., a vertical drag when the main axis is horizontal) will trigger a primary action, such as Select the content of the proximity touch point. That is to say, if the user drags away from the spindle far enough, it can cause a first-level action. Thus, by controlling the drag direction on a non-spindle, the user can select from a particular behavior, such as selecting text.

Or, in the case where a large amount of drag occurs in an unexpected direction (ie, delay) With the spindle, the suppression action applied to the drag operation can be interrupted so that the user is free to translate in the direction of his choice. As such, the correction component 225 switches the panning motion into a moving motion when a critical offset is detected. When the user interrupts the panning behavior, it can perform a "general drag operation", such as a mouse drag type operation. Alternatively, the user can still perform a primary action without interrupting the correction component. In a specific embodiment, the user can perform operations on the touch screen interface to perform secondary actions, including: pressing and holding for a predetermined period of time, then dragging; performing a double-click operation (within a specific time window) , then drag; or provide a multi-touch input, such as two-finger drag. Other operations that the user can perform in the scrolling content area to cause a secondary action without scrolling may include dragging left to right in a display area that scrolls only up and down. When dealing with secondary actions, the app will work as if the user is tapping and dragging with the mouse. In general, this secondary action can be used to select text within an application. Upon receiving an indication of this secondary action, the application interface component can transmit a mouse message instead of a scrolling message to the application 230.

As mentioned earlier, the user can hold down and then drag to temporarily disable the panning behavior. When disabled, the user is free to use a drag operation to perform a secondary action instead of a translation (which would be a preset action associated with the drag operation). For example, when an input is interpreted as a hold + drag, a one-level action can be caused to facilitate a selection gesture while temporarily stopping the pan. In another embodiment, a received input gesture is interpreted as being held down + released, the scrolling function is disabled and the user is allowed to perform a right click. A skilled person can perform a selection task by touching, such as copying a web address Or move the file to a folder.

Turning now to FIG. 3, a diagram 300 of a display area 328 on a touch screen interface 320 in accordance with an embodiment of the present invention is shown, illustrating a method for translating content therein. First, a touch screen interface 320, a mobile device 310, or other computing device is provided. The touch screen interface 320 is configured to provide display areas 328, 337, and 340 on the UI display 305. The drag operation 335 is shown as being applied by a physical object 315 (e.g., a single-finger gesture) into a content area 325, and the content 327 is translated in the direction 333 of the drag operation 335. The drag operation includes a starting point 334 or a starting position as described above, and a touch point. Additionally, the reel tool 329 can provide visual feedback of the panning operation.

In the illustrated example, the drag operation 335 is performed in a generally downward direction. Thus, the content 327 will translate in the downward direction 333, thereby moving with the physical object 315. The reel tool 329 can provide visual feedback indicating that a downward translation occurs by automatically moving a slider 332 from a previous position 330 in an upward direction. In the case where the downward drag operation reaches the boundary of the content 327, a panning over action can be caused. In an example, the panning over action can include panning the display area 328 of the UI display 305. Moreover, in an exemplary embodiment, if the correction component 225 of FIG. 2 is enabled, the content 327 can be scrolled generally in the vertical direction 333 even when the drag operation 335 includes a slight offset.

When an input is provided to display area 328, the application underneath is enabled because input under display area 328, display areas 337, and 340 cannot be accepted during translation. For example, if drag operation 335 is by touch point 336 continues to the content area 338 of the display area 337, and the content within the display area 338 will not translate. Rather, when the physical object exceeds the lower boundary of the content area 325, the translation of the vertical direction 333 can be stopped. Alternatively, the entire display area 328 can be slid down in the UI display 305 to indicate that an out-of-bounds phenomenon has occurred.

Display area 337 includes a horizontal reel tool 339. As such, in contrast to the vertical reel tool 328 of the display area 328, a horizontal drag operation in the content area 338 will cause a panning action, while a vertical drag operation of the content area 338 causes a level one action, such as a text in the emphasized content area 338. The choice function.

Display area 340 does not include a reel tool. Therefore, the user is visually notified that one of the applications under the display area 340 does not support the scrolling function. Thus, a drag operation within range 341 of display area 340 will cause the display area to translate over UI display 305 as the physical object 315 moves. To initiate translation of display area 328, a launch position should be outside of content area 325, such as within boundary/title bar 326 of display area 328.

Turning now to Figures 4 and 5, there is illustrated a drag operation development diagram in accordance with a particular embodiment of the present invention, which illustrates performing a drag operation step that is switched from a panned content to a secondary action. First, referring to Fig. 4, a set of predefined critical boundaries 420 is created to clearly distinguish the blurring zone between a click operation and a drag operation. That is, if the gesture extends beyond any of the boundaries of the set of critical boundaries 420 (eg, initial drag 405), the gesture is considered a drag operation. In another specific embodiment, if the gesture is applied to the touch screen interface for a predetermined time window, the gesture is viewed. For a drag operation.

As shown in Fig. 4, the initial drag 405 is moving to an upper quadrant; therefore the gesture is considered a vertical drag operation in an upward direction. Therefore, if the application supports the scrolling function in the vertical direction, the content can be moved with the initial drag 405. In the event that the correction component 225 of FIG. 2 is enabled, if any offset of the initial drag 405 does not cross any of the critical range lines 410 and 415, the offset is suppressed. Turning to Figure 5, a subsequent drag 505 crosses the critical range 515. In a specific embodiment, the panning function is suspended. In another embodiment, subsequent drag 505 crossing the critical range may thus cause a primary action, or disable correction component 225 of Figure 2, such that the inhibit function is deactivated. Therefore, the content will follow the drag operation instead of moving in the direction of the main axis.

The present invention has been described in connection with the specific embodiments thereof, which are intended to be illustrative and not restrictive. Other alternative embodiments will be apparent to those skilled in the art without departing from the scope of the invention.

Based on the foregoing description, it will be appreciated that the invention is to be construed as a It will be appreciated that certain features and subcombinations may be utilized effectively without reference to other features and subcombinations. These are included in the scope of the patent application.

100‧‧‧ arithmetic device

110‧‧‧ busbar

112‧‧‧ memory

114‧‧‧ (plural) processor

116‧‧‧ (plural) display components

118‧‧‧Input/Output (I/O)埠

120‧‧‧I/O components

122‧‧‧Power supply

200‧‧‧ system structure

205‧‧‧ Input components

210‧‧‧Reel checker

215‧‧‧ arithmetic device

220‧‧‧Operating system

221‧‧‧ Input device

223‧‧‧ display area

225‧‧‧calibration components

226‧‧‧Select components

230‧‧‧Application

235‧‧‧Information

240‧‧‧ Application Information

245‧‧‧Enter information

250‧‧‧Display component/application interface component

275‧‧‧Data storage

280‧‧‧ gesture engine

290‧‧‧Physical contacts

295‧‧‧Users start typing

300‧‧‧ display area

305‧‧‧UI display

310‧‧‧Mobile devices

315‧‧‧ physical objects

320‧‧‧Touch screen interface

325‧‧‧Content area

326‧‧‧Boundary/Title Bar

327‧‧‧Content

328‧‧‧ display area

329‧‧‧Reel tool

330‧‧‧Previous location

331‧‧‧ upward direction

332‧‧‧ Slider

333‧‧‧About vertical direction

334‧‧‧ starting point

335‧‧‧ Drag operation

336‧‧‧ Touch point

337‧‧‧ display area

338‧‧‧Content area

339‧‧‧ horizontal scroll tool

340‧‧‧ display area

341‧‧‧Scope

342‧‧‧Content

405‧‧‧ initial drag action

410‧‧‧critical range line

415‧‧‧critical range line

420‧‧‧ critical boundary

505‧‧‧Subsequent towing action

515‧‧‧critical range

The invention has been described above with reference to the accompanying drawings in which: FIG. 1 is a block diagram of an exemplary operational environment suitable for implementing a specific embodiment of the invention; FIG. 2 is a specific implementation in accordance with the present invention. FIG. 3 is a schematic diagram of an exemplary system architecture for implementing a specific embodiment of the present invention; FIG. 3 is a schematic diagram of a display area on a touch screen interface according to an embodiment of the present invention, illustrating an example for translating A method of content thereon; and Figures 4 and 5 are diagrams of successive drag operations in accordance with an embodiment of the present invention, illustrating a phase of switching from a panned content to a initiated secondary action when a drag operation is performed.

200‧‧‧ system structure

205‧‧‧ Input components

210‧‧‧Reel checker

215‧‧‧ arithmetic device

220‧‧‧Operating system

221‧‧‧ Input device

223‧‧‧ display area

225‧‧‧calibration components

226‧‧‧Select components

230‧‧‧Application

235‧‧‧Information

240‧‧‧ Application Information

245‧‧‧Enter information

250‧‧‧Display component/application interface component

275‧‧‧Data storage

280‧‧‧ gesture engine

290‧‧‧Physical contacts

295‧‧‧Users start typing

Claims (18)

A computer readable medium having a plurality of computer executable instructions thereon for performing a method for translating content within a display area when the instructions are executed, the display area being presented on an computing device, the method comprising the Step: detecting a user input input to a touch screen interface of the computing device; receiving touch screen information from the touch screen interface at an operating system of the computing device, wherein the touch screen information Included with the touch mark associated with the time stamp; determining that the touch position occurs in a content area of the display area on the surface of the touch screen interface, wherein the display area is presented by a touch-naïve application, the non-touch The touch application is in communication with the operating system of the computing device, wherein the non-touch application is not equipped to recognize a drag operation in the content area as an indication of panning content in the content area; identifying the non-touch application Whether the content in the content area supports the scrolling function; obtaining a gesture from the touch screen information, wherein the obtaining step includes clear A drag operation is different from a one-click operation or other operations to confirm that the gesture is the drag operation; when the non-touch application is identified as supporting the scroll function, a panning message is generated, and the panning messages act according to the dragging Operating pan display area The content in the upper content area; and when the non-touch application is identified as not supporting the scrolling function, a mouse message is generated, and the mouse messages are transmitted from the operating system to the non-touch application. The computer readable medium of claim 1, wherein the method further comprises the step of causing the touch screen interface to present an indication that the mouse message is consumed by the non-touch application (consume And wherein the indication comprises translating the content area on the display area without translating content within the content area on the display area. The computer readable medium of claim 1, wherein the step of identifying the non-touch application to support scrolling functionality further comprises the step of: determining whether the display area depicted at the touch screen interface includes A reel control item. The computer readable medium of claim 3, wherein the method further comprises the step of: presenting, at the touch screen interface, an indication that the translational message is consumed by the non-touch application, wherein The indication includes a direction of driving a slider of the reel control item according to a translational direction at the touch screen interface. The computer readable medium as described in claim 1, wherein the distance of an uninterrupted drag movement is derived by the following steps: The Cartesian coordinates of the starting position of the user's initial input at the touch screen interface are compared with the Cartesian coordinates of the release position of the dragging movement. The computer readable medium of claim 1, wherein the step of clearly distinguishing between a drag operation and a click operation or the like is performed according to an uninterrupted drag movement from the touch screen interface. The distance at which the start position of the input is initiated, wherein the uninterrupted drag movement includes sweeping the touch screen interface by more than a critical distance in a predetermined direction. The computer readable medium of claim 1, wherein the step of confirming that the gesture is the drag operation comprises the following steps: detecting the difference between the drag operation and the one click operation or the other operation One of the dragging operations continuously drags the direction of the movement; identifying a scrollable direction of the content area, wherein the scrollable direction means a direction in the content area where the content is allowed to translate, the scrollable direction being by the non-touch Controlling the application; and determining that the direction of the uninterrupted drag movement is substantially parallel to the scrollable direction of the content region. The computer readable medium of claim 1, wherein the method comprises the following steps after the mouse message is generated: transmitting the mouse message from the operating system to the non-touch application, and wherein A drag action of the content area is triggered immediately after the mouse message is consumed by the non-touch application. A computer system capable of providing touch translation capability to a touch-naïve application, the system comprising: an input device, the input device receiving a user initiated input, the input comprising being applied to a display area a touch screen, wherein the touch screen interface is capable of receiving a physical touch associated with the user's initial input; the non-touch application, the non-touch application The touch application is configured to present content in a content area of the display area, wherein the non-touch application is not equipped to recognize the user initiation input applied to the touch screen interface as causing translation of the rendered content. An operating system for interacting with the touch screen interface and the non-touch application, wherein the interaction includes transmitting one or more in response to the user initiated input applied to the touch screen interface Multiple messages to control the scrolling function of the non-touch application, wherein the operating system includes: (a) an input component received from the input device Input information of the screen driver, wherein the input information includes a touch location associated with the time stamp; (b) a reel seeker component for detecting the non-touch application to detect the non-touch application Whether to provide a scrolling function for the content of the content area; (c) a gesture engine for acquiring a gesture and activating position from the touch locations combined by the time stamps, the gesture engine determining whether the activation location occurs within the content region, And the gesture engine is configured to determine whether the gesture is a drag operation according to a direction and a distance from the substantially uninterrupted drag movement originating from the activation position; and (d) an application interface component, the application interface component is configured to provide the One or more messages to the non-touch application, wherein when an indication is received indicating that the non-touch application supports the scrolling function, the application interface component generates at least one translation message for communication to the non-touch application, otherwise the application The interface component generates at least one mouse message to the non-touch application immediately after detecting the user input input to the touch screen interface, wherein the at least one translation message is consumed by the non-touch application (consume Afterwards, causing the content to translate within the content area, wherein the at least one translation message is adapted to initiate the input by the user to cause a scrolling action And the at least one mouse message is generated to cause a drag action without adapting the user to initiate input through the operating system; and a display component, the display component presenting an indication of the scrolling function to User, wherein the display component is integrated with the input device. The computer system of claim 9, wherein the step of adapting the user to initiate input to cause a scrolling action comprises: interpreting the input information from the screen driver; Translating the interpreted input information to the scrolling function of the non-touch application; and communicating the at least one translation message, the at least one translation message reflecting the scrolling function to which the non-touch application is mapped. A computerized method for implementing a scrolling function of a touch-naïve application after receiving a gesture at a touch screen interface, the computerized method comprising the steps of: detecting application to the touch a user initiates a gesture of the screen interface, wherein the gesture includes an activation position associated with the touch screen interface and a drag operation associated with the gesture, and wherein the non-touch application is capable of being on a surface of the touch screen interface Displaying a content area in a display area, and wherein the non-touch application is not equipped to recognize a drag operation in the content area as an indication of content translated in the content area; determining whether the drag operation satisfies a predetermined system a metric value; when the non-touch application is identified as supporting a scrolling function and the drag operation is determined to satisfy the predetermined system metric value, causing the non-touch application to enter a pan mode that allows for the display The content in the content area of the area performs a scrolling action; and otherwise the non-touch application is maintained in a preset mode, wherein the preset mode An operating system from the touchscreen interface about the original input gesture transferred to the non-touch application. The computerized method of claim 11, wherein The step of forwarding the original input from the touch screen interface to the non-touch application includes: transmitting the user start gesture as a mouse message recognizable by the non-touch application to the non-touch application. The computerized method of claim 11, wherein the step of determining whether the drag operation satisfies the predetermined system metric comprises the step of: confirming that the drag operation comprises starting substantially uninterrupted from the start position Dragging movement, the substantially uninterrupted drag movement continues for a distance greater than a predetermined critical distance; and determining that the direction of the drag operation is substantially aligned with a predetermined scrollable direction of the non-touch application. The computerized method of claim 13, wherein the non-stop dragging movement is generated by a physical object that is substantially in contact with the touch screen interface, the computerized method further comprising the steps of: receiving a Instructing to indicate that the substantially uninterrupted drag movement generated by the physical object exceeds one or more boundaries surrounding the content region when the drag operation is performed; and initiating a translational outbound motion that provides an operator A visual feedback to a limit of one of the scrollable content is indicated, wherein the visual feedback includes translating the display area on the touch screen interface in a rough direction of the substantially uninterrupted drag movement. The computerized method of claim 13, wherein the step of determining that the direction of the drag operation is substantially aligned with a predetermined scrollable direction of the non-touch application comprises the step of: averaging according to the substantially uninterrupted drag movement Direction identifying a spindle of the dragging action; comparing the spindle with one or more scrolling directions of the scrolling function of the content region of the display area; and determining that the spindle is within a predetermined tolerance range One or more of the direction in which these enablers are scrolling. The computerized method of claim 13, wherein the step of determining whether the drag operation satisfies the predetermined system metrics comprises the step of using a timer to determine when the substantially uninterrupted drag movement is completed. The amount of time consumed by the drag operation; and the difference between the drag operation and the one-click operation is clearly distinguished based on the amount of time consumed and the distance of the drag operation. The computerized method of claim 11, wherein the scrolling action for the content in the content area of the display area comprises: translating corresponding to a physical object and the touch screen interface The content of a contact location. The computerized method according to claim 14, wherein the computerized method further comprises the step of: receiving an indication that the substantially uninterrupted drag movement is performed by the physical object while performing the drag operation Sweeping the amplitude to the touch screen interface; automatically releasing the content in the direction of the substantially uninterrupted drag movement immediately after releasing the drag operation, as the content moves following inertia; and This translational out-of-bound action is triggered when the end of the content is reached in the direction of the intermittent drag movement.