JP2014506369A - Gesture-based control method and apparatus - Google Patents

Abstract

In one implementation, the number of parallel gestures within a particular area of the device's touch screen interface is detected. At least based on the parallel gesture, a command is identified and an action associated with the command is performed.
[Selection] Figure 1

Description

  The present invention relates to the use of gestures. Specifically, the present invention relates to control based on multimedia content gestures.

  The approaches described in this section may have been pursued, but are not necessarily those that have actually been considered or pursued before. Accordingly, unless otherwise specified, any approach described in this section should not be considered prior art simply because they are included in this section.

  Multimedia content such as web pages, images, videos, slides, text, graphics, sound files, audio / video files, etc. can be displayed on the device or played on the device. The user can give commands to play the content on the device or display it on the device itself or another device that functions as a remote control.

  For example, a user can select a remote control button to play, pause, stop, rewind, or fast forward the video displayed on the television.

FIG. 2 is a block diagram illustrating an example system in accordance with one or more implementations. FIG. 6 illustrates a flowchart for detecting a gesture in accordance with one or more embodiments. FIG. Fig. 4 illustrates an example interface in accordance with one or more implementations. 1 shows a block diagram illustrating a system in which embodiments of the present invention may be implemented.

  The present invention is illustrated by way of example, and not by way of limitation, in the accompanying drawings. In these drawings, like reference numerals refer to like elements.

  In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent that the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.

  In the following, some features are described so that each can be used independently or in combination with other features. However, any individual feature may not address any of the issues described above, or may only address one of the issues described above. Some of the issues discussed above are fully addressed by any of the features described in this specification (hereinafter "the specification" including the drawings, the claims, and the abstract). Maybe not. A heading is provided, but information about a particular heading but not found in this heading section can be found elsewhere in the specification.

Specific features are described according to the following outline.
1.0 Overview 2.0 System Architecture 3.0 Gesture 4.0 Gesture Area 5.0 Command 6.0 Gesture Detection within Gesture Area 7.0 Examples of Gestures and Commands 8.0 Remote Control Usage Example 9.0 Implementation Example 10.0 Hardware Overview 11.0 Extensions and Variations

1.0 Overview In one embodiment, a gesture is detected at a specific area of the touch screen interface on the device. This gesture does not necessarily allow any visual object within a particular area to be selected or moved. For example, a gesture can be detected in a blank box, at the top of the video, at the top of the instruction information for performing the gesture, or the like. The video playback command associated with the gesture can be identified and an action can be determined in response to the video playback command. This action can then be performed on the same device that detected the gesture. This action may also be performed on a different device that is communicatively coupled to the device that detected the gesture.

  In one embodiment, multiple input means (eg, multiple fingers) can be used simultaneously to perform parallel or identical gestures on the touch screen interface. The action can be selected based on the number of detected gestures. For example, the number of gestures can be used to select a specific item from a menu or identify a command.

    Although a particular component is described in detail herein as performing a method step, in other embodiments, an agent or mechanism that acts on a particular component performs this method step. be able to. Furthermore, although some aspects of the invention have been discussed with respect to components of a system, the invention can also be implemented by components distributed across multiple systems. Embodiments of the present invention include any system that includes means for performing the method steps described herein. Embodiments of the present invention also include computer-readable media having instructions that, when implemented, cause the method steps described herein to be performed.

2.0 System Architecture Although specific computer architectures are described herein, other embodiments of the present invention can be used to perform the functions described herein. It can be applied to any architecture.

  FIG. 1 is a block diagram illustrating an example system (100) in accordance with one or more embodiments. The exemplary system (100) includes one or more components that function as content sources, touch screen interface devices, multimedia devices (eg, devices that play audio and / or video content) and / or Or a content management device. Although specific components are presented to clarify the functionality described herein, they are not necessarily required to implement one or more embodiments. Each of these components is presented to clarify the functionality described herein, but is not necessarily required to implement one or more embodiments.

  Components not shown in FIG. 1 can also be used to perform the functionality described herein. Functionality described as being performed by one component may alternatively be performed by another component.

  The example system (100) can include one or more of an input device (110), a multimedia device (140), and a data repository (150). One or more devices shown herein can be combined into a single device or divided into multiple devices. For example, the input device (110) and the multimedia device (140) can be executed on a single device. The multimedia device (140) can be configured to play audio and / or video content. The multimedia device (140) can be configured to display one or more still images. In another example, the input device (110) can be used as a remote control to detect commands based on gestures associated with content displayed on another multimedia device (140). The input device (110) can communicate directly with the multimedia device (140) or with an intermediate device (not shown). This intermediate device can serve as a content source for, for example, a multimedia device (140) or media management device. The network bus (102) connecting all components in the system (100) is shown for clarity. The network bus (102) can represent any local network, intranet, Internet, etc. The network bus (102) may include wired and / or wireless segments. All components (shown to be coupled in communication) do not necessarily have to be coupled in communication with all other components in the system (100).

  In one implementation, the input device (110) may include a touch screen interface (115) configured to detect one or more gestures (described herein). The input device (110) may be configured to detect gestures, gesture paths, gesture speeds, gesture accelerations, gesture directions, and the like.

  In one implementation, the input device (110) may include a resistive system in which current flows in two layers that touch at a point / area to be touched on the touch screen interface (115). The coordinates of the contact point or contact spot can be compared to gesture information stored in the data repository (150) to identify gestures performed by the user on the touch screen interface (115). In another example, the input device (110) is a capacitive system having a layer that stores charge (a portion of this charge is transferred to the user where the user touches the touch screen interface (115)). Can be included. In other implementations, the input device (110) may include a surface acoustic wave system having two transducers where electrical signals are sent from one transducer to another. Any interruption of electrical signals (eg, due to a user touch) can be used to detect touch points on the touch screen interface (115). For example, the input device (110) may be configured to first detect an initial user touch on the visual representation of the data displayed on the touch screen interface.

  In one implementation, the input device (110) may include hardware configured to receive data, transmit data, or communicate with other devices in the system (100). For example, the input device (110) can be configured to detect a gesture performed by a user and perform a video playback action associated with the gesture. In another example, the input device (110) is capable of transmitting information related to a gesture (information also referred to herein as “metadata” and can be used in place of “metadata”). Can include sex. For example, the input device (110) can be configured to transmit information comprising a temporal sequence of touch points detected on the touch screen interface (115).

  In one embodiment, the input device (110) may include one or more of the following: read only memory (ROM) (206), central processing unit (CPU), random access memory (RAM), Infrared control unit (ICU), keypad scanning, keypad, non-volatile memory (NVM), one or more microphones, general purpose input / output (GPIO) interface, speaker / tweeter, key transmitter / indicator, microphone, Wireless, infrared (IR) blaster, network card, display screen, radio frequency (RF) antenna, QWERTY keyboard, network card, network adapter, network interface controller (NIC), network interface card, local Area network adapter, Ethernet network card and / or network Other optional components that information can be received over the workpiece. In one embodiment, the input device (110) can be configured to communicate with one or more devices over a wired and / or wireless segment. For example, the input device (110) may be a radio wave (eg, Wi-Fi signal, Bluetooth signal), infrared wave, electromagnetic spectrum of any other suitable frequency, network connection (eg, intranet, internet, world wide web, etc.) ), Or any other suitable method, can be communicated wirelessly.

  In one implementation, input device (110) generally represents any device that can be configured to detect a gesture as a user input. A user (including any operator of the input device (110)) can perform a gesture by touching the touch screen interface (115) on the input device (110). For example, the user can perform a gesture by tapping the touch screen interface (115) with a finger or sliding the finger on the touch screen interface (115).

  For clarity, the examples described herein refer to specific input means (eg, a user's finger) that perform a gesture. However, any input means can be used to perform a gesture in accordance with one or more embodiments, including but not limited to a stylus, a user's finger, a pen, a thimble, and the like.

  Gestures related to touching or touching the touch screen interface (115) referred to herein may be performed by the touch screen interface (115) at a location on the touch screen interface (115), for example. To hover over the touch screen interface (115) with the finger (or other input means) without necessarily touching the touch screen interface (115), such as detecting the finger (by charge transfer in the Including.

3.0 Gesture In one embodiment, a tap gesture is performed by touching a specific location on the touch screen interface (115) and then releasing contact with the touch screen interface (115). can do. The tap gesture can be detected by detecting a contact at a specific position of the touch screen interface (115) and then detecting that the contact with the touch screen interface (115) has been released. it can.

  A tap gesture refers to a gesture that is performed using one or more fingers. For example, a two-finger tap uses two fingers to touch two locations on the touch screen interface (115) simultaneously, and then releases contact with the touch screen interface (115) By doing so, it can be executed. A two finger tap can be detected by detecting simultaneous contact at two locations on the touch screen interface (115) and subsequent release of this contact.

  In one implementation, the slide gesture can include any movement that allows the user to slide one or more fingers over the surface of the touch screen interface (115). Examples of slide gestures include flick gestures, swipe gestures, or gestures related to moving a finger along any path on the touch screen interface (115). The path can be a closed shape such as a circle or a square with the same start and end points, or an open shape such as a right angle with different start and end points. Specific examples of the path include, but are not limited to, a straight line, a curve, a circle, a square, a triangle, an angle, and the like.

  In one embodiment, the flick gesture is performed as follows. That is, one of the sliding actions performed by the user touching a specific position on the touch screen interface (115) of the input device (110) with a finger (or any other item (eg, stylus)) and performed by the user. Keep the touch screen interface (115) in contact with the part, slide your finger from a specific position, and continue the slide action after contact with the touch screen interface (115) ends Is executed. In one implementation, the touch screen interface (115) may be configured to detect finger proximity after physical contact with the touch screen interface (115) is over.

  For example, even though the surface area added in the direction of the sliding action of the touchscreen interface (115) can be used to continue the sliding action while maintaining contact, the user can The touch with the touch screen interface (115) can be released while moving the finger in the direction of the action to be performed.

  In another example flick gesture, the user touches a particular location on the touch screen interface (115) of the input device (110) and then maintains contact with the touch screen interface (115). While sliding the finger across the edge of the touch screen interface (115). Thus, the user can maintain contact (eg, with a finger) with the touch screen interface (115) until the finger reaches the end of the touch screen interface (115), and the touch screen interface ( 115) can continue to move in the same direction across the edge.

  The user performing the flick gesture can continue its slide action after releasing contact with the touch screen interface (115). Based on the duration that the input device (110) was in contact with the touch screen interface at the last contact point, the finger and the touch screen interface (115) It can be detected that the contact between has been released. The release detected while the finger is moving can be determined to be a flick gesture.

  In one embodiment, the swipe gesture touches a specific position on the touch screen interface (115) of the input device (110) with a finger and is in contact with the touch screen interface (115) during the sliding action. This can be done by sliding the finger from a specific position while maintaining contact.

  In another embodiment, the user slides a finger along the touch screen interface (115) from a first position to a second position, and then a second time for a threshold time (eg, 1 second). It can be stopped by maintaining contact with the position. With continued contact detection with the second position, the user can determine that the swipe gesture has already been completed.

  In one implementation, a sliding action (eg, swipe or flick) can be detected before the sliding action is complete. For example, a gesture that slides to the right may be a contact at a first position and a subsequent contact at a second position (or within a specific angle to the right of the first position). The user can also continue the sliding gesture to the third position to the right of the second position, but the direction of the sliding gesture is the first position. And using the second position has already been detected.

  In one implementation, flick gestures (eg, in the same direction) and sliding gestures can be mapped to the same video playback command. Thus, the device can be configured to detect either a slide gesture or a flick gesture and identify the same video playback command in response to the detected flick gesture or slide gesture.

  In one implementation, flick gestures and slide gestures (possibly in the same direction) can be mapped to different commands. For example, a left flick gesture can correspond to a 20 second rewind command and a left swipe gesture can correspond to a command to select a previous bookmarked scene in the video. For example, scenes can be bookmarked by the user, or a code can be inserted in the media recording so that the scene can be selected from a movie recorded on a digital video disc (DVD). You can also.

  In one embodiment, the slide gesture can be performed by using multiple input means simultaneously. For example, the user can simultaneously slide two fingers anywhere on the touch screen interface. Further, the user can simultaneously slide two fingers in parallel (eg, slide two fingers in the same direction from left to right).

  The term “simultaneously” as used herein also includes “substantially simultaneously”. For example, two fingers performing a parallel gesture at the same time also mean two fingers having different lengths performing the same gesture at slightly different times. For example, one finger may move late after the other finger starts and / or finishes a gesture. Thus, the two fingers can start and end gestures at different start and / or end times.

  As used herein, the term “parallel” includes paths that are in approximately the same direction. As used herein, two fingers performing a parallel motion involves the user dragging two fingers in the same direction anywhere on the touch screen interface. The directions of two or more fingers performing parallel motion in the same direction may differ by several degrees because the finger lengths are different or hand angles are different. In one embodiment, the paths that perform two parallel gestures may overlap. The term “parallel” as used herein means any set of two or more gestures performed in the same direction.

4.0 Gesture Area In one embodiment, the touch screen interface (115) includes a gesture area. The gesture region is at least a part of the touch screen interface (115) that is configured to detect gestures performed by the user. The gesture region can include the entire touch screen interface (115) or a portion of the touch screen interface (115). The gesture area can display a blank box or one or more items. For example, the gesture area can display a video. In another implementation, the gesture region can display information on how to perform the gesture.

  In one implementation, a gesture can be detected in the gesture area without the user interacting with any visual object that can be displayed in the gesture area. For example, anywhere in the mobile phone touchscreen interface (115), a swipe gesture can be detected in a gesture area, which is a blank box on the touchscreen interface. In another implementation, the detected swipe gesture associated with the rewind command does not touch the progress indicator displayed in the gesture area.

  In one implementation, no visual object displayed within the gesture area is necessary to detect a gesture or to determine a command associated with the gesture. In one embodiment, any visual object displayed within the gesture area is not selected or dragged by the finger performing the gesture.

  In one implementation, the touch screen interface (115) may include multiple gesture areas. Gestures detected in one gesture area can be mapped to different commands than the same gesture performed in different gesture areas. The device can be configured to identify an area where the gesture is performed and determine an action based on the gesture and the gesture area where the action was performed.

  In one implementation, if a gesture is detected in any of a number of gesture regions, the device can select a gesture region from among the number of gesture regions. The initiated gesture region can be identified as the selected gesture region. For example, the user can initiate a swipe gesture in a first gesture area and finish a swipe gesture in a second gesture area. In response to detecting that a swipe gesture has been initiated in the first gesture region, a command mapped to the gesture and the first gesture region may be selected. In another implementation, a gesture region in which the end of a sliding action is detected can be identified as the intended gesture region. The selected or intended gesture region can then be used to identify the command.

5.0 Commands In one implementation, gestures can be mapped (or associated) to commands. For example, the command mapped to the gesture may be a video playback command related to video playback. The command can relate to the playback of the video at the device that received the command or at another device.

  In one embodiment, the command can specify the playback speed and direction of the video. For example, the command may select to fast forward at a specific fast forward speed or rewind at a specific rewind speed. Examples of other video playback commands include, but are not limited to: interrupting video playback, resuming video playback, replaying a portion of the video that has been played Stop video playback, stop video playback and resume video playback at a specific playback position, play video in slow motion, play video frame by frame, video from scratch Play one or more videos from the next playlist, play a video forward from a specific scene, bookmark the video playback position, stop playback and bookmarked position To resume playback or rate the video.

  In one implementation, the command can select a particular option from a list of options. For example, a list of available media content can be displayed on the screen, and the command can select specific media content among the available media content. In another implementation, a list of configuration settings can be displayed, and the command can select specific settings to modify.

6.0 Detection of Gestures within Gesture Area FIG. 2 shows a flow diagram for detecting gestures within a gesture area. One or more steps described below may be omitted, repeated, and / or performed in a different order. Accordingly, the specific arrangement of steps shown in FIG. 2 should not be construed as limiting the scope of the invention.

  For one or more embodiments, detecting a gesture includes detecting an interface touch at an initial position on the touch screen interface (step 202). The initial contact on the touch screen interface can be made by the user's finger, stylus or any other item that can be used to perform gestures on the touch screen interface. The initial contact with the touch screen interface is maintained at the original position for a quick touch (eg, tap gesture) at the original location, or any time (eg, 1 millisecond, 1 second, 2 seconds, etc.) Touch. The initial contact with the touch screen interface is a finger that is already moving in a certain direction (eg, moving through the air without touching and then at some point during the movement of the touch screen interface). It can be shortened because it is made by a finger that makes initial contact with a part).

  In one embodiment, the “initial contact” described herein can include a finger (or other item) that is close enough to the touch screen interface that the touch screen interface can detect the finger. . For example, when using a device that includes a capacitive system having a charge storage layer, a portion of the charge can be transferred to the user. In this case, the user may touch the touch screen interface or hover without touching near the touch screen interface. Thus, “initial contact” or “maintained contact” as described herein also includes having the user hover a finger or other item on the touch screen interface.

  In one embodiment, the initial touch on the touch screen interface does not select any visual object displayed on the touch screen interface. If no visible object is displayed, the initial contact can be made. The initial contact can be made to the top of the visual object display without selecting the visual object. For example, an initial contact can be made to a touchscreen interface displaying a background image of a user-selected mobile phone. In another embodiment, an initial contact can be made to a blank screen. Initial contact can be detected at the top of the television program being played on the tablet.

  In one or more implementations, detecting the gesture can further include detecting interface contact at an additional location of the touch screen interface (step 204). For example, by detecting a flick gesture or swipe gesture, a sequential sequence of interface contacts at additional locations along the path from the original contact location can be detected. For example, interface contact can be detected continuously in a path leftward from the original contact location on the touch screen interface.

  In this specification, a contact along a path away from the position of the original contact point will be referred to as a sliding gesture. In one or more implementations, the speed of the sliding gesture or the direction of the sliding gesture can be determined. For example, contact at two or more locations on the interface (such as an initial contact point and a second point along the slide gesture path) determines the direction and / or speed of the slide gesture. Can be used for. Contact at multiple points can be used to calculate the acceleration of a sliding gesture.

  In one or more implementations, gestures can be identified based on touch detected at one or more locations on the touch screen interface (step 206). For example, identifying a three-finger tap gesture by detecting simultaneous contact at three locations on the remote control interface and then detecting that contact has been released at all three locations. it can. In one embodiment, detecting the gesture includes identifying a path where contact was detected on the touch screen interface. For example, in response to detecting contact along a circular path on the touch screen interface, a circle gesture is identified. A flick or swipe gesture can be identified based on a sequence of contact points over time on the touch screen interface.

  In one or more implementations, identifying the gesture can include determining the number of simultaneous parallel gestures (step 208). For example, initial contact can be detected simultaneously at multiple locations on the touch screen interface. Following the initial contact at each initial position, the contact can be detected along a path starting from these initial positions. If these paths are determined to be parallel, the number of paths can be identified to determine the number of simultaneous parallel gestures.

  In one implementation, the number of simultaneous parallel gestures can be determined based on the number of paths that match a known configuration. For example, if the path has at least a first contact point and a second contact point that is within a range of about 10 degrees with respect to a horizontal line that goes to the right from the first contact point, It can be determined that it corresponds to a gesture that slides on the screen. By counting the number of detected gestures corresponding to paths that match the same criteria within a particular time period, the number of simultaneous parallel gestures can be determined. In one implementation, other methods not described herein can be used to determine the number of simultaneous parallel gestures.

  In one implementation, the command is determined based on the identified gesture (step 210). The command can be determined while the gesture is still being performed, or after the gesture is completed.

  In one implementation, determining the command can include determining that the particular gesture detected is mapped to a command in the database. For example, to identify a command associated with a two finger swipe, the command database can be queried for a two finger right swipe. In another implementation, a two-finger flick to the bottom of the gesture area can be associated with a command that selects a second menu item from the items currently displayed in the menu.

  In one implementation, the number of parallel fingers in the command can be used to determine the playback speed at which to play the multimedia content. For example, detection of two parallel gestures can be mapped to a command that results in a playback speed that is twice the normal playback speed.

  In one embodiment, the direction of the gesture command can be combined with the number of parallel fingers of the gesture command to determine the playback command. For example, two fingers swiping simultaneously from the right side of the screen to the left side of the screen can be mapped to a rewind twice the normal speed. In another embodiment, two fingers swiping simultaneously from the left side of the screen to the right side of the screen can be mapped to fast forward twice the normal playback speed (not fast forward).

  In one implementation, the command may include resuming playback of the video at a particular bookmark (eg, a user specified bookmark or a manufacturer specified bookmark). The number of fingers used to perform simultaneous parallel gestures can be used to select a bookmark. For example, in response to detecting a downward flick with two fingers, video playback may resume at the second bookmark from the current playback position.

  In one implementation, determining the command can include identifying a device corresponding to the command. For example, the device associated with the command can be identified based on the gesture and / or the gesture region in which the gesture was detected.

  In one implementation, the action corresponding to the command is performed (step 212). The device that detected the command can perform this action. For example, if a fast-forward command gesture is detected on a handheld touchscreen phone (telephone) that is playing video, the handheld touchscreen phone (telephone) will play the video in fast-forward mode. .

  In one implementation, the action corresponding to the command may include sending information related to the command to another device. For example, gestures can be detected with a touch screen remote control. Information associated with the gesture (eg, information identifying the gesture or information identifying the command associated with the gesture) may then be transmitted to the digital video disc player. The digital video disc player can then perform the corresponding action. If the command is to interrupt video playback, the digital video disc player can interrupt video playback on the display screen.

7.0 Examples of Gestures and Commands FIG. 3 shows an example screen shot for an input device that is configured to detect gestures. Gestures, commands, mapping between gestures, commands, gesture regions, visual objects, and any other items discussed with respect to FIG. 3 are examples of items, but should be interpreted as limiting the scope of rights is not. One or more items described with respect to FIG. 3 are not necessarily all executed, but other items described can also be executed in accordance with one or more embodiments.

  FIG. 3 shows an exemplary interface (300) having a circular gesture region (305) and a rectangular gesture region (310). Any gesture detected within the circular gesture area (305) is mapped to a navigation command. For example, a two-finger tap detected in a circular gesture area (305) can be associated with a command that selects a second item in any currently displayed menu. If the second item is a folder, the items in the folder can be displayed in response to detecting a tap with two fingers.

  In one implementation, the rectangular gesture region (310) may identify commands associated with one or more gestures detected within the rectangular gesture region (310). For example, a rectangular gesture region (310) indicates that a left swipe gesture of one finger corresponds to a rewind command, and a one finger tap gesture corresponds to a pause command. A finger swipe gesture to the right corresponds to a fast forward command, a two finger left swipe gesture to rewind 10 seconds, and a two finger tap gesture to play slow motion. A swipe right with two fingers corresponding to a command can include graphics corresponding to a skip to the next bookmark command.

  In one implementation, the example interface (300) may include a progress indicator (315) that is separate from a circular gesture region (305) and a rectangular gesture region (310). The progress indicator (315) may include the current playback position of the video, a bookmark, the current playback speed, and the like. For example, the progress indicator (315) may include a symbol representing the current playback speed (eg, playback, 1x forward forward fast, pause, 2x rewind, etc.).

  In one implementation, symbols can be displayed in response to commands. For example, in response to a 3x rewind command while rewinding multimedia content at 3x speed is being performed by displaying the frame in the reverse direction at 3x normal playback speed, A symbol indicating triple speed rewind can be displayed. However, any gesture associated with the video playback command cannot necessarily select the progress indicator (315). In one implementation, the visual object in the example interface (300) is not necessarily selected when the user is performing a gesture in the example interface (300).

  In one implementation, the example interface (300) may also include a tool (eg, a drop-down box) for selecting a particular media device to be controlled by the detected gesture. In one implementation, the example interface (300) may include an option to switch between input mechanisms (eg, gesture-based input, buttons, text boxes, wireless boxes, etc.).

8.0 Remote Control Use Case In one embodiment, the remote control device is a media device (eg, a digital video recorder, a digital video disc player, a media management device, a video recorder, a Blu-ray player, etc.). Communicate with. The remote control device can communicate with the media device via a wired and / or wireless communication segment. For example, the remote control device can communicate with a network (for example, the Internet, an intranet, etc.) by wireless communication, by Bluetooth, by infrared rays or the like.

  In one embodiment, the remote control displays a progress indicator (315), as shown in the screen shot (300) of FIG. The progress indicator (315) can indicate the playback position of the multimedia content currently displayed on another multimedia device. The progress indicator (315) can display an accurate playback position or an approximate playback position. For example, the progress indicator (315) can include a slider (320) that indicates the playback position displayed along the trick playback bar (330). In one embodiment, a particular playback position can be indicated by a time (eg, 8:09). This time can indicate, for example, the actual streaming time of the currently played content, or an offset from the start point of the content.

  In one embodiment, the information regarding the playback position of the multimedia content is a media device (eg, digital video recorder, cable box, computer, media management device, digital video disc player, multimedia player, Audio player, etc.). For example, a remote control device that is communicatively coupled to a media device can be configured to receive frame information associated with a particular frame that is being displayed (played) by the media device. In one implementation, the media device can periodically send frame information to the remote control device. Alternatively, the remote control device can periodically request frame information from the media device. The remote device uses this information to locate the slider (320) along the trick play bar (330). The remote control device can also receive information from the media device that indicates the coverage of the cache bar (325) that indicates the amount of multimedia content that is stored or recorded by the media device It is. If the media device is in the process of recording or caching multimedia content, the cache bar (325) will increase as the media device records or caches more content Will. If the media device is playing recorded multimedia content, the cache bar (325) will extend the length of the trick play bar (330).

  Another example relates to a remote control device that is configured to receive a time stamp that is closest to the frame being displayed. If the frame does not exactly match the time stamp, the remote control device can also be configured to use a step function (eg, the next frame or the previous frame of the time stamp). Another embodiment may include a remote control device that continuously receives images (eg, bitmaps, display instructions, etc.) from the media device of the progress indicator to a display on the remote control device. In one implementation, the remote control device can include a specific starting position and display rate that the remote control device uses to determine the playback position of the multimedia content. For example, the digital video recorder transmits the initial playback position in the playback of multimedia content to the remote control device along with the progress rate (eg, slider (320) change per unit time, frame rate, etc.). be able to. The remote control device can use this information to first display the progress indicator based on the original playback position, and then calculate the subsequent position as a function of time.

  In one embodiment, when a trick play function is performed (eg, when a 10 second rewind is performed), the slider (320) becomes out of sync with the displayed video. In response to the trick play function, updated information regarding the new play position can be provided to the remote control device.

  In one embodiment, the remote control device may further receive updated information indicating a particular playback position or indicating a change in progress rate. For example, the user may issue one or more commands to interrupt the playback of multimedia content at the current playback position, then skip to 10 seconds before the current playback position and resume playback be able to. In this case, the media device displays the new playback position corresponding to 10 seconds before the current playback position by suspending the slider (320) and moving the slider (320). 320) can be provided to the remote control device to resume periodically updating.

  In one implementation, the slider (320) can be updated when the remote control device is activated. For example, when the user picks up or touches the remote control device, the remote control device can request playback location information from the media device. For example, the remote control device can include an accelerometer configured to detect movement and / or a touch screen interface configured to detect touch. In response, the media device can provide playback position information to the remote control device. The remote control device can then cause the slider (320) to display the current playback position of the multimedia content based on the playback on the position information received from the media device.

  In one embodiment, because the remote control device constantly updates the slider (320), the remote control device can continuously receive information regarding the playback position of the multimedia content. In another embodiment, information regarding the playback position of the multimedia content can be received periodically, and the remote control device can update the slider each time this information is received.

  In one implementation, the remote control device can send multimedia content to the multimedia device for display by the multimedia device. For example, a remote control device can obtain a video stream over the Internet and can send the video stream to the multimedia device for display on the multimedia device. In this specific example, the remote control device can determine the display position of the slider (320) based on the reproduction position information determined by the remote control device itself. For example, the remote control device can calculate playback position information based on frames being transmitted from the remote control device to the multimedia device.

9.0 Embodiment Specifics The method of an embodiment detects a slide gesture from a first position of a specific area to a second position of the specific area in a specific area on the touchscreen interface of the device. The method comprising: identifying a video playback command based on at least a slide gesture; and performing an action associated with the video playback command, the method being performed by at least one device.

  In one embodiment, this sliding gesture is detected without detecting the selection of any video progress indicator displayed within the particular area. A slide gesture can be detected in the specific area while displaying at least a portion of the video in the specific area. While displaying information on how to perform one or more gestures in a specific area, a slide gesture can be detected in the specific area.

  In one embodiment, the step of identifying the video playback command is further based on a specific region in which a slide gesture is detected among a plurality of regions on the touch screen interface.

  In one embodiment, performing the action comprises the first device transmitting information based on the video playback command to the second device. Performing the action associated with the video may comprise performing an action on the same device as the device that detects the slide gesture. The video playback command can select the playback speed and direction.

  In one embodiment, the slide gesture comprises a swipe gesture from a first position to a second position. The slide gesture can comprise a flick gesture starting at the first position.

  In one embodiment, the video playback command can suspend video playback, resume video playback, replay the played portion of video, stop video playback, stop video playback. And restart the video at a specific playback position, play the video in slow motion, step through the video, play the video from the beginning, play one or more videos from the next playlist Playing, playing a video from a specific scene, bookmarking the playback position in the video, stopping playback, restarting playback at the bookmarked position, or rating the video One or more of.

  The method of an embodiment includes the steps of simultaneously detecting a plurality of parallel gestures on the touch screen interface of the device, determining a number of the plurality of parallel gestures, and a plurality of commands based on the number of the plurality of parallel gestures. Selecting a command from and executing an action associated with the command.

  In one embodiment, selecting the command comprises selecting a menu option based on the number of parallel gestures. The plurality of parallel gestures can comprise a plurality of parallel sliding gestures performed in the same direction.

  In one embodiment, determining the number of multiple parallel gestures comprises determining the number of tap gestures performed simultaneously on the touch screen interface.

  Although specific components are described in detail herein as performing these method steps, in other embodiments, agents or mechanisms that operate for specified components may be The method steps can be performed. Furthermore, although some aspects of the invention have been discussed with respect to components on the system, the invention can also be implemented with components distributed across multiple systems. Embodiments of the present invention include any system that includes means for performing these method steps as described herein. Embodiments of the present invention also include computer-readable media having instructions that, when executed, cause the steps of these methods described herein to be performed.

10.0 Hardware Overview According to one embodiment, the techniques described herein are performed by one or more special purpose computing devices. This special purpose computing device may be hardwired to perform these techniques, or it may be continuously programmed to perform one or more application specific integrated circuits (ASICs) or these techniques. A digital electronic device such as a field programmable gate array (FPGA) or programmed to perform these techniques according to program instructions in firmware, memory, other storage devices, or combinations thereof Can include one or more general purpose hardware processors. Such special purpose computing devices can also combine custom hardwired logic, ASICs or FPGAs with custom-made programming to accomplish these techniques. This special purpose computing device can be a desktop computer system, a portable computer system, a handheld device, a networking device, or any other device incorporating hardwired logic and / or program logic that implements these technologies. It can be.

  For example, FIG. 4 is a block diagram illustrating a computer system 400 that can implement embodiments of the present invention. Computer system 400 includes a bus 402 or other communication mechanism for communicating information, and a hardware processor 404 for processing information coupled to bus 402. The hardware processor 404 can be, for example, a general purpose microprocessor.

  Computer system 400 is coupled to a bus 402 for storing information and instructions executed by processor 404 (such as random access memory (RAM) or other dynamic storage device). A memory 406 is included. Main memory 406 can also be used to store temporary variables or other intermediate information during execution of instructions executed by processor 404. When stored on a fixed storage medium accessible to processor 404, such instructions cause computer system 400 to turn into a special purpose machine that is customized to perform the operations specified in these instructions. Change.

  Computer system 400 further includes a read only memory (ROM) 408 or other static storage device coupled to bus 402 for storing static information and instructions for processor 404. A storage device 410, such as a magnetic disk or an optical disk, is provided and coupled to the bus 402 for storing information and instructions.

  Computer system 400 may be coupled via bus 402 to a display 412, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 414 that includes alphanumeric keys and other keys is coupled to the bus 402 for communicating information and command selections to the processor 404. Another type of user input device is a cursor control 416, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selection to the processor 404 and controlling cursor movement on the display 412. The input device typically has two axes (in a first axis (eg, x) and a second axis (eg, y) that allow the device to identify a position in the plane). Has two degrees of freedom.

  The computer system 400 includes customized hardwired logic, one or more ASICs or FPGAs, firmware, and / or computer logic that works with the computer system to turn the computer system 400 into a special purpose machine. Can be used to perform the techniques described herein. According to one embodiment, the techniques herein are performed by computer system 400 in response to processor 404 executing one or more sequences of one or more instructions contained in main memory 406. Is done. Such instructions can be read into main memory 406 from another storage medium, such as storage device 410. Execution of the sequence of instructions contained in main memory 406 causes processor 404 to perform the process steps described herein. In another embodiment, hardwired circuitry can be used instead of or in conjunction with software instructions.

  The term “storage medium” as used herein refers to any fixed medium that stores data and / or instructions that cause a machine to operate in a specific fashion. Such storage media can comprise non-volatile media and / or volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 410. Volatile media includes dynamic memory, such as main memory 406. Common forms of storage media are, for example, floppy disk, flexible disk, hard disk, SSD (solid state drive), magnetic tape, or any other magnetic data storage medium, CD-ROM, etc. Including any optical data storage medium, any physical medium having a hole pattern, RAM, PROM and EPROM, flash EPROM, NVRAM, and any other memory chip or cartridge.

  The storage medium is perceptible but can also be used with a transfer medium. A transfer medium is a medium that transfers information between storage media. For example, the transfer medium includes coaxial cable, including wire with bus 402, copper wire and optical fiber. The transfer medium can also take the form of acoustic waves or light waves, such as those generated during radio wave and infrared data communications.

  Various forms of media pertain to carrying one or more sequences of one or more instructions that cause processor 404 to execute. For example, the instructions can initially be carried on a remote computer magnetic disk or SSD. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 400 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infrared detector can receive data carried by the infrared signal, and appropriate circuitry can place the data on bus 402. Bus 402 carries data to main memory 406, from which processor 404 reads and executes instructions. The instructions received by main memory 406 may optionally be stored on storage device 410 either before or after execution by processor 404.

  Computer system 400 also includes a communication interface 418 coupled to bus 402. Communication interface 418 provides a bidirectional data communication coupling to network link 420 that is connected to local network 422. For example, the communication interface 418 may be an integrated services digital network (ISDN) card, cable modem, satellite modem, or modem that provides a data communication connection to the type that the telephone line supports. As another example, the communication interface 418 may be a local area network (LAN) card that provides a data communication connection to a compatible LAN. A wireless link can also be implemented. In any such implementation, communication interface 418 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

  Network link 420 typically provides data communication to other data devices via one or more networks. For example, the network link 420 may provide a connection over the local network 422 to a host computer 424 or to data equipment operated by an Internet service provider (ISP) 426. On the other hand, ISP 426 provides data communication services through a global packet data communication network now commonly referred to as the “Internet” 428. Local network 422 and Internet 428 both use electrical, electromagnetic or optical signals that carry digital data streams. Signals via various networks and signals via network link 420 and signals via communication interface 418 (which carries digital data to / from computer system 400) are examples of forms of transfer media. .

  The computer system 400 sends messages and receives data including program code via the network, network link 420 and communication interface 418. For the Internet example, the server 430 may send the requested code to the application program over the Internet 428, ISP 426, local network 422, and communication interface 418.

  The processor 404 may execute the received code as it is received and / or store it in the storage device 410, or store it in other persistent storage for later execution.

  In one embodiment, the device is a combination of one or more hardware and / or software components described herein. In one embodiment, the subsystem for performing the steps is a combination of one or more hardware and / or software components that can be configured to perform the steps.

11.0 Extensions and Variations In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that vary from implementation to implementation. Accordingly, the only and exclusive indication of what is an invention and what the applicant intends is the specific form in which such claims arise, including any subsequent amendments. And a set of claims arising from this application. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Hence, no limitation, element, attribute, feature, advantage or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. Accordingly, the specification and drawings are to be regarded as illustrative rather than limiting.

Claims (24)

Detecting a slide gesture from a first position to a second position in a specific area on the touch screen interface of the device;
Recognizing a video playback command for video based at least on the slide gesture;
Performing an action associated with the video playback command;
A method comprising:   The method of claim 1, wherein the sliding gesture is detected without detecting selecting any video progress indicator displayed within the particular region.   The method of claim 1, wherein the slide gesture is detected simultaneously with the display of at least a portion of the video in the specific area.   The method of claim 1, wherein the slide gesture is detected in the specific area while displaying information in the specific area on how to perform one or more gestures. The video playback command is
Identifying the specific area where the slide gesture is detected from a plurality of areas on the touch screen interface;
The method of claim 1, wherein identifying the video playback command comprises selecting the video playback command from a plurality of video playback commands associated with the particular region.   The method of claim 1, wherein performing the action comprises the first device transmitting information based on the video playback command to a second device.   The method of claim 1, wherein performing the action associated with the video comprises performing the action on the same device when the device detects the slide gesture.   The method of claim 1, wherein the video playback command selects a playback speed and a playback direction.   The method of claim 1, wherein the slide gesture comprises a swipe gesture from the first position to a second position.   The method of claim 1, wherein the slide gesture comprises a flick gesture that begins at the first position. The video playback command is
Interrupting the playback of the video;
Resuming playback of the video;
Replaying the played portion of the video;
Stopping playback of the video,
Stopping playback of the video and restarting playback of the video at a specific playback position;
Playing the video in slow motion;
Frame by frame,
Playing the video from the beginning,
Playing one or more of the videos from the next playlist,
Playing the video from a specific scene,
Bookmarking the playback position in the video;
Stop playback and resume playback at the bookmarked position, or rate the video,
The method of claim 1, wherein one or more of the above. Simultaneously detecting a plurality of parallel gestures on the touch screen interface of the device;
Determining the number of the plurality of parallel gestures;
Selecting a command from a plurality of commands based on the number of the plurality of parallel gestures;
Performing an action associated with the command;
A method comprising:   The method of claim 12, wherein selecting the command comprises selecting a menu option based on the number of the plurality of parallel gestures.   The method of claim 12, wherein the plurality of parallel gestures comprises a plurality of sliding gestures performed in the same direction.   The method of claim 12, wherein determining the number of the plurality of parallel gestures comprises determining the number of tap gestures performed simultaneously on the touch screen interface.   The method of claim 12, further comprising determining a playback speed for playing multimedia content based on the number of the plurality of gestures. Simultaneously detecting a plurality of parallel gestures from left to right on a touch screen interface of a remote control device;
Determining the number of the plurality of parallel gestures;
Selecting a playback speed from a plurality of playback speeds based on the number of the plurality of parallel gestures;
Causing the multimedia device to play multimedia content based on the selected playback speed;
A method comprising:   The method of claim 17, wherein the plurality of playback speeds comprises two or more forward fast forward speeds.   The method of claim 17, wherein the multimedia device comprises an audio device.   The method of claim 17, wherein the multimedia device comprises a video device. Simultaneously detecting a plurality of right-to-left parallel gestures on the touch screen interface of the remote control device;
Determining the number of the plurality of parallel gestures;
Selecting a rewind speed from a plurality of rewind speeds based on the number of the plurality of parallel gestures;
Causing the multimedia device to play multimedia content in a rewind mode based on the selected rewind speed;
A method comprising:   The method of claim 21, wherein the plurality of rewind speeds comprises two or more rewind speeds.   A computer readable storage medium comprising a sequence of instructions that, when executed by one or more processors, causes the steps of any one of claims 1 to 22 to be performed.   23. A device comprising one or more processors and configured to perform the steps of any one of claims 1-22.