US7777649B2 - Advanced control device for home entertainment utilizing three dimensional motion technology - Google Patents

Advanced control device for home entertainment utilizing three dimensional motion technology Download PDF

Info

Publication number
US7777649B2
US7777649B2 US10/597,273 US59727305A US7777649B2 US 7777649 B2 US7777649 B2 US 7777649B2 US 59727305 A US59727305 A US 59727305A US 7777649 B2 US7777649 B2 US 7777649B2
Authority
US
United States
Prior art keywords
motion
hand
held device
device
command
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US10/597,273
Other versions
US20080252491A1 (en
Inventor
Boris Emmanuel Rachmund De Ruyter
Detlev Langmann
Jiawen W. Tu
Vincentius Paulus Buil
Tatiana A. Lashina
Evert Jan Van Loenen
Sebastian Egner
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
III Holdings 6 LLC
Original Assignee
NXP BV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US53780004P priority Critical
Application filed by NXP BV filed Critical NXP BV
Priority to PCT/IB2005/050182 priority patent/WO2005071636A1/en
Priority to US10/597,273 priority patent/US7777649B2/en
Assigned to KONINKLIJKE PHILIPS ELECTRONICS, N.V. reassignment KONINKLIJKE PHILIPS ELECTRONICS, N.V. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BUIL, VINCENTIUS PAULUS, DERUYTER, BORIS EMMANUEL RACHMUND, EGNER, SEBASTIAN, LASHINA, TATIANA A., TU, JIAWEN W., VAN LOENEN, EVERT JAN
Assigned to NXP B.V. reassignment NXP B.V. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KONINKLIJKE PHILIPS ELECTRONICS N.V.
Publication of US20080252491A1 publication Critical patent/US20080252491A1/en
Publication of US7777649B2 publication Critical patent/US7777649B2/en
Application granted granted Critical
Assigned to BREAKWATERS INNOVATIONS LLC reassignment BREAKWATERS INNOVATIONS LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NXP B.V.
Assigned to NXP B.V. reassignment NXP B.V. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BREAKWATERS INNOVATIONS LLC
Assigned to III HOLDINGS 6, LLC reassignment III HOLDINGS 6, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NXP B.V.
Assigned to KONINKLIJKE PHILIPS ELECTRONICS N.V. reassignment KONINKLIJKE PHILIPS ELECTRONICS N.V. NUNC PRO TUNC ASSIGNMENT (SEE DOCUMENT FOR DETAILS). Assignors: BUIL, VINCENTIUS PAULUS, LASHINA, TATIANA A., TU, JIAWEN W., DE RUYTER, BORIS EMMANUEL RACHMUND, EGNER, SEBASTIAN, VAN LOENEN, EVERT JAN
Application status is Active legal-status Critical
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G08SIGNALLING
    • G08CTRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C17/00Arrangements for transmitting signals characterised by the use of a wireless electrical link
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/105Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals using inertial sensors, e.g. accelerometers, gyroscopes
    • GPHYSICS
    • G08SIGNALLING
    • G08CTRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C2201/00Transmission systems of control signals via wireless link
    • G08C2201/30User interface
    • G08C2201/32Remote control based on movements, attitude of remote control device

Abstract

A hand held device for generating commands and transferring data between the hand-held device and a base device (including consumer electronic equipment). The hand-held device detects the motion of the device itself, interpreting the motion as a command, and executing or transferring the command. The motion of the device can include gestures made by the user while holding the device, such as the motion of throwing the hand-held device toward a base device. The commands generated by the user range from basic on/off commands to complex processes, such as the transfer of data. In one embodiment, the user can train the device to learn new motions associated with existing or new commands. The hand-held device analyzes the basic components of the motion to create a motion model such that the motion can be uniquely identified in the future.

Description

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. provisional application Ser. No. 60/537,800 filed Jan. 20, 2004, which the entire subject matter is incorporated herein by reference.

The present invention relates to the control of home entertainment devices and applications, and more particularly, to a method and system for controlling and transferring data to home entertainment devices by manipulating a control device.

Hand-held devices, such as remote controls devices, are typically used to control consumer electronic devices, such as televisions and gaming machines. As the hand-held devices and consumer electronic devices have become more sophisticated, new techniques for inputting commands to the hand-held devices have been developed. These techniques include methods that detect the orientation of a hand-held device to generate a command. For example, U.S. Pat. Nos. 4,745,402 and 4,796,019 disclose methods for controlling the position of a cursor on a television. U.S. Pat. No. 6,603,420 discloses a remote control device that detects the direction of movement of the remote control device to control, e.g., the channel and volume selection of a television.

The ability of these hand-held devices to hold data and the development of more sophisticated capabilities in the consumer electronic devices has created new challenges for controlling these consumer electronic devices. For example, it is often necessary to transfer data from the hand-held device to the consumer electronic device or vice versa. The hand-held device should also provide a natural, efficient mechanism for indicating that an action, such as a data transfer, is to be performed. A need therefore exists for an improved hand-held device that is capable of efficiently generating commands and transferring data to or from consumer electronic devices.

An apparatus and method are disclosed for generating commands and transferring data between a hand-held device and a base device (including consumer electronic equipment). The hand-held device is capable of detecting the motion of the hand-held device itself, interpreting the motion as a command, and executing or transferring the command. The motion of the device can include gestures made by the user while holding the device, such as the motion of throwing the hand-held device toward a base device, as a user would do when swinging a tennis racket. The commands generated by the user range from basic on/off commands to complex processes, such as the transfer of data.

In one embodiment, the user can train the device to learn new motions associated with existing or new commands. For example, the user can make the motion of throwing the hand-held device toward the base device. The hand-held device analyzes the basic components of the motion to create a motion model such that the motion can be uniquely identified in the future.

A more complete understanding of the present invention, as well as further features and advantages of the present invention, will be obtained by reference to the following detailed description and drawings.

FIG. 1 shows an exemplary hand-held device of the present invention;

FIGS. 2A-B illustrate gestures that are interpreted as commands by the hand-held device of FIG. 1;

FIG. 3 is a schematic block diagram of the hand-held device of FIG. 1;

FIG. 4 illustrates an exemplary embodiment of a motion detection subsystem;

FIG. 5 is a flowchart describing an exemplary implementation of the system process of the hand-held device of FIG. 1;

FIG. 6 is a flowchart describing an exemplary implementation of a motion training process;

FIG. 7 is a flowchart describing an exemplary implementation of a motion detection process; and

FIG. 8 is a graph illustrating the motion model of a throwing motion based on the expected acceleration in each of three perpendicular planes.

FIG. 1 shows an exemplary hand-held device 300 of the present invention, discussed further below in conjunction with FIG. 3, such as the Philips Super Pronto, modified in accordance with the features of the present invention. The hand-held device 300 is capable of detecting motion of the hand-held device 300, interpreting the detected motion as one or more commands, and executing or transferring the command(s).

FIGS. 2A-B illustrate gestures that a user can make using the hand-held device 300. FIG. 2A, for example, shows a user 201 making the gesture of throwing the device 300 toward a base device, such as television 210. FIG. 2B shows a user making the gesture of pouring from the device 300 into a base device, such as television 210. The gesture and associated motion indicate that the user 201 would like to transfer data from the hand-held device 300 to the television 210. In this case, the user would first locate and identify the data (e.g. a picture or music) and then make the gesture toward the base device. The data could be identified, for instance, by selecting an item from of a list displayed on the hand-held device 300. The data would then be transferred. In addition, if the data is a picture, it could be (optionally) displayed on the television or, if the data is music, it could be (optionally) played through the speakers. Other gestures include making a pulling motion (not shown) directed from a base device towards the user. In this case, the gesture would indicate that the identified data should be transferred to the hand-held device 300. The data would then be retrieved from either the base device itself, or from another device (e.g. a server). Since there are a number of base devices 210 through 214 located in the area of the user 201, the hand-held device 300 has the ability to identify which device 210-214 should receive the data being transferred (as described in more detail below). FIG. 3 is a schematic block diagram of an exemplary hand-held device 300 of the present invention. As is known in the art, the methods and apparatus discussed herein may be distributed as an article of manufacture that itself comprises a computer-readable medium having computer-readable code means embodied thereon. The computer-readable program code means is operable, in conjunction with a computer system such as central processing unit 301, to carry out all or some of the steps to perform the methods or create the apparatuses discussed herein. The computer-readable medium may be a recordable medium (e.g., floppy disks, hard drives, compact disks, or memory cards) or may be a transmission medium (e.g., a network comprising fiber-optics, the world-wide web, cables, or a wireless channel using time-division multiple access, code-division multiple access, or other radio-frequency channel). Any medium known or developed that can store information suitable for use with a computer system may be used. The computer-readable code means is any mechanism for allowing a computer to read instructions and data, such as magnetic variations on a magnetic medium or height variations on the surface of a compact disk.

Memory 302 will configure the processor 301 to implement the methods, steps, and functions disclosed herein. The memory 302 could be distributed or local and the processor 301 could be distributed or singular. The memory 302 could be implemented as an electrical, magnetic or optical memory, or any combination of these or other types of storage devices. The term “memory” should be construed broadly enough to encompass any information able to be read from or written to an address in the addressable space accessed by processor 301.

As shown in FIG. 3, the memory 302 includes motion model database 303, system process 500, discussed further below in conjunction with FIG. 5, motion training process 600, discussed further below in conjunction with FIG. 6, and motion detection process 700, discussed further below in conjunction with FIG. 7. Hand-held device 300 also includes motion detection subsystem 400, discussed further below in conjunction with FIG. 4, radio frequency (RF) communication subsystem 305, and infrared detection subsystem (IDS) 310.

The RF communication subsystem 305 provides communication between the handheld device 300 and one or more base devices 210-214 in a known manner. For example, the RF communication subsystem 305 may utilize the IEEE 802.11 standard for wireless communications or any extensions thereof. The IDS 310 emits infrared light in a directional manner in order to signal a base device 210-214 that it should execute the command being transmitted by the device 300. Only the base device 210-214 that detects the infrared signal should execute the transmitted command. The command is transferred to the base device 210-214 via the RF communication subsystem 305 in a known manner. In an alternative embodiment, the command may be transferred by modulating the infrared signal (utilizing, for example, the IR Blaster standard) in a known manner.

FIG. 4 illustrates an exemplary embodiment of motion detection subsystem 400. Motion detection subsystem 400 contains x-axis accelerometer sensor 410, y-axis accelerometer sensor 411, z-axis accelerometer sensor 412, and corresponding analog to digital converters 415, 416, 417. Accelerometer sensors 410, 411, 412 detect the acceleration of the device 300 along the x-axis, y-axis, and z-axis, respectively. The accelerometer sensors 410, 411, 412 may be embodied, for example, using the 3D Motion Sensors commercially available from NECTokin of Union City, Calif. Analog to digital converters 415, 416, 417 convert the acceleration(s) detected by accelerometer sensors 410, 411, 412, respectively, to a digital form that can be read by processor 301. In alternative embodiments, other components, including stress-sensitive resistive elements, tilt sensors, and magnetic direction sensors, may be utilized to determine the position, orientation and/or speed of movement of the device 300.

FIG. 5 illustrates an exemplary embodiment of system process 500. System process 500 initially waits for a command to be entered during step 505. If, during step 505, a user enters a training command, the system process 500 executes step 510 where motion training process 600 is called. If, during step 505, a user makes a gesture or motion indicative of a command, the system process 500 executes step 515 where motion detection process 700 is called. Upon completion of the called processes 600, 700, system process 500 returns to step 505 to wait for the entry of a new command.

FIG. 6 illustrates an exemplary embodiment of motion training process 600. Motion training process 600 learns new gestures and motions demonstrated by a user to be used for identifying existing or new commands. For instance, a user 201 may want to train the device 300 to detect a throwing motion, such as the motion of throwing the device 300 toward a television 210. The user first presses a switch on the hand-held device 300 to indicate that a new gesture is to be created. (Alternatively, the user can train the hand-held device 300 to interpret a motion as an indication that the training process should be executed.) Motion training process 600 initially waits for motion to be detected by one or more of the accelerometer sensors 410, 411, 412 (step 601) and then records the motion detected by the sensors 410, 411, 412 by periodically sampling and storing data read from analog to digital converters 415, 416, 417 (step 605). After each set of samples have been read during sampling step 605, a test is made to determine if no motion has been detected for a specified period of time indicating that the gesture or motion has been completed (step 608). If motion is detected during step 608, then step 605 is repeated to read the next set of samples; otherwise, motion training process 600 creates and stores a model of the motion captured during step 610. The motion model is created in a known manner For example, the following publications describe methods for analyzing, comparing and modeling motions and gestures: Ho-Sub Yoon, Jung Soh, Younglae J. Bae and Hyun Seung Yang, Hand Gesture Recognition Using Combined Features of Location, Angle and Velocity, Pattern Recognition, Volume 34, Issue 7, 2001, Pages 1491-1501; Cristopher Lee and Yangsheng Xu, Online, Interactive Learning of Gestures for Human/Robot Interfaces, The Robotics Institute, Carnegie Mellon University, Pittsburgh, IEEE International Conference on Robotics and Automation, Minneapolis, 1996; Mu-Chun Su, Yi-Yuan Chen, Kuo-Hua Wang, Chee-Yuen Tew and Hai Huang, 3D Arm Movement Recognition Using Syntactic Pattern Recognition, Artificial Intelligence in Engineering, Volume 14, Issue 2, April 2000, Pages 113-118; and Ari Y. Benbasat and Joseph A. Paradiso, An Inertial Measurement Framework for Gesture Recognition and Applications, MIT Media Laboratory, Cambridge, 2001 each incorporated by reference herein.

The created model will be used to interpret future gestures and motions made by the user 201. During step 615, the model created during step 610 is assigned a command or process that is to be executed when the motion associated with the model is detected. The command to be executed is identified utilizing well known methods, for instance, pressing a switch on the hand-held device 300 associated with the command or entering a code associated with the command on a keypad. In an alternative embodiment, the user could enter (record) a series of commands by performing the actions on the system (e.g., on the touch screen), similar to recording a macro in MS Word. The series of commands can then be associated to a single gesture. The assigned command or process is stored with the associated motion model in the motion model database 303.

FIG. 7 illustrates an exemplary embodiment of motion detection process 700. Motion detection process 700 interprets gestures and motions made by a user 201 to determine the command(s) that are to be executed. For instance, if the user 201 makes the motion of throwing the hand-held device 300 towards the television 210, the hand-held device 300 will interpret the gesture as a command to transfer data from the device 300 to the television 210. Motion detection process 700 initially records the motion detected by the accelerometer sensors 410, 411, 412 by periodically sampling and storing the data read from analog to digital converters 415, 416, 417 (step 705). After each set of samples have been read during sampling step 705, a test is made to determine if no motion has been detected for a specified period of time indicating that the gesture or motion has been completed (step 708). If motion is detected during step 708, then step 705 is repeated to read the next set of samples; otherwise, motion detection process 700 compares the data collected during step 705 to the motion models stored in the device 300 (step 710). During step 710, a score is generated for each model comparison. The command or process associated with the model that attained the highest score during step 710 is then executed during step 715. For example, if the model with the highest score was the “throwing motion” model, then a data transfer process (not shown) would be executed in a known manner. The data transfer process can be accomplished, for example, utilizing the 802.11 standard in a well known manner. During step 720, the IDS 310 is also activated, thereby causing an infrared signal to be emitted in the direction of the throwing motion. Only the base device 210-214 that detects the infrared signal will receive the data transferred via the RF communication subsystem 305.

FIG. 8 shows an exemplary motion model representing the throwing motion of FIG. 2A. As is illustrated, the z-axis accelerometer indicates that the motion is in the x-y plane (no motion along the z-axis). As indicated by the x-axis accelerometer, the motion shows a quick acceleration along the x-axis, a peak speed at the halfway point of the motion, and an increasing deceleration as the motion is completed. A similar, but smaller, action is occurring along the y-axis.

It is to be understood that the embodiments and variations shown and described herein are merely illustrative of the principles of this invention and that various modifications may be implemented by those skilled in the art without departing from the scope and spirit of the invention.

Claims (21)

1. A hand-held device that wirelessly communicates with a base device, the hand-held device comprising:
a memory for storing at least one of picture data and music data;
a motion detection subsystem configured to detect a motion of the hand-held device, the motion of the hand-held device being made by a user holding the device;
a radio frequency (RF) communications subsystem for wirelessly communicating with the base device; and
at least one processor operative to:
interpret the motion of the hand-held device as a command that involves wirelessly transmitting at least one of picture data and music data to the base device; and
execute the command to wirelessly transmit at least one of picture data and music data from the hand-held device to the base device in response to interpreting the motion of the hand-held device as the command that involves wirelessly transmitting at least one of picture data and music data to the base device.
2. The hand-held device of claim 1, wherein said execute said command operation includes transferring a second command to said base device.
3. The hand-held device of claim 1, wherein said detected motion is a throwing motion.
4. The hand-held device of claim 1, wherein said detected motion is a pouring motion.
5. The hand-held device of claim 1, wherein said detected motion is a pulling motion directed from said base device.
6. The hand-held device of claim 1, further operative to add one or more new commands by detecting and recording a demonstration motion.
7. The hand-held device of claim 6, further operative to create a motion model from said recorded demonstration motion.
8. The hand-held device of claim 7, further operative to assign said one or more new commands to said motion model.
9. The hand-held device of claim 1, wherein the motion detection subsystem comprises three dimensional motion sensors for performing said motion detection operation.
10. The hand-held device of claim 1, further comprising one or more motion models, wherein each of said one or more motion models is assigned a command.
11. The hand-held device of claim 10, wherein said interpret said motion operation is performed by comparing said detected motion to one or more of said one or more motion models.
12. A method for transferring at least one of picture data and music data from a hand-held device to a base device, the method comprising:
identifying at least one of picture data and music data that is stored in a memory of the hand-held device;
detecting a motion of the hand-held device, wherein the motion of the hand-held device is made by a user that is holding the hand-held device;
interpreting the motion of the hand-held device as a command that involves wirelessly transmitting at least one of picture data and music data to the base device; and
wirelessly transmitting at least one of the identified picture data and music data that is stored in memory of the hand-held device to the base device in response to interpreting the motion of the hand-held device as a command that involves wirelessly transmitting at least one of picture data and music data to the base device.
13. The method of claim 12, wherein said detecting motion step is a throwing motion.
14. The method of claim 12, wherein said detecting motion step is a pouring motion.
15. The method of claim 12, wherein said detecting motion step is a pulling motion directed from said base device.
16. The method of claim 12, further comprising the step of adding one or more new commands by detecting and recording a demonstration motion.
17. The method of claim 16, further comprising the step of creating a motion model from said recorded demonstration motion.
18. The method of claim 17, further comprising the step of assigning said one or more new commands to said motion model.
19. The method of claim 12, wherein said interpreting said motion step is performed by comparing said detected motion to one or more motion models.
20. The method of claim 12 wherein:
the motion is detected by a motion detection subsystem of the hand-held device;
the motion is interpreted by a processor of the hand-held device; and
the at least one of the identified picture data and music data that is stored in the memory of the hand-held device is wirelessly transmitted to the base device by an RF communications subsystem of the hand-held device.
21. The method of claim 12 further comprising:
interpreting a motion of the hand-held device as a command to display the picture data or to play the music data on the base device;
transmitting the command to display the picture data or to play the music data to the base device.
US10/597,273 2004-01-20 2005-01-17 Advanced control device for home entertainment utilizing three dimensional motion technology Active 2027-06-28 US7777649B2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US53780004P true 2004-01-20 2004-01-20
PCT/IB2005/050182 WO2005071636A1 (en) 2004-01-20 2005-01-17 Advanced control device for home entertainment utilizing three dimensional motion technology
US10/597,273 US7777649B2 (en) 2004-01-20 2005-01-17 Advanced control device for home entertainment utilizing three dimensional motion technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US10/597,273 US7777649B2 (en) 2004-01-20 2005-01-17 Advanced control device for home entertainment utilizing three dimensional motion technology

Publications (2)

Publication Number Publication Date
US20080252491A1 US20080252491A1 (en) 2008-10-16
US7777649B2 true US7777649B2 (en) 2010-08-17

Family

ID=34807125

Family Applications (1)

Application Number Title Priority Date Filing Date
US10/597,273 Active 2027-06-28 US7777649B2 (en) 2004-01-20 2005-01-17 Advanced control device for home entertainment utilizing three dimensional motion technology

Country Status (6)

Country Link
US (1) US7777649B2 (en)
EP (1) EP1709609A1 (en)
JP (1) JP2007518511A (en)
KR (1) KR20060126727A (en)
CN (2) CN104732752A (en)
WO (1) WO2005071636A1 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070200658A1 (en) * 2006-01-06 2007-08-30 Samsung Electronics Co., Ltd. Apparatus and method for transmitting control commands in home network system
US20080059178A1 (en) * 2006-08-30 2008-03-06 Kabushiki Kaisha Toshiba Interface apparatus, interface processing method, and interface processing program
US20080273756A1 (en) * 2007-04-26 2008-11-06 Atlab Inc. Pointing device and motion value calculating method thereof
US20090113354A1 (en) * 2007-10-30 2009-04-30 Samsung Electronics Co., Ltd. Broadcast receiving apparatus and control method thereof
US20090128714A1 (en) * 2007-11-16 2009-05-21 Oki Semiconductor Co., Ltd. Electronic device, remote control device, and remote control system
US20100060569A1 (en) * 2008-09-09 2010-03-11 Lucent Technologies Inc. Wireless remote control having motion-based control functions and method of manufacture thereof
US20120007713A1 (en) * 2009-11-09 2012-01-12 Invensense, Inc. Handheld computer systems and techniques for character and command recognition related to human movements
US20120081216A1 (en) * 2007-03-23 2012-04-05 Lee Yu-Tuan Remote-controlled motion apparatus with acceleration self-sense and remote control apparatus therefor
WO2013124792A1 (en) 2012-02-23 2013-08-29 Koninklijke Philips N.V. Remote control device
US8613666B2 (en) 2010-08-31 2013-12-24 Microsoft Corporation User selection and navigation based on looped motions

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007034787A1 (en) 2005-09-26 2007-03-29 Nec Corporation Mobile telephone terminal, data process starting method and data transmitting method
JP5406188B2 (en) * 2007-08-20 2014-02-05 クアルコム,インコーポレイテッド Advanced vocabulary rejection
US20090136016A1 (en) * 2007-11-08 2009-05-28 Meelik Gornoi Transferring a communication event
US8780278B2 (en) 2007-11-30 2014-07-15 Microsoft Corporation Motion-sensing remote control
CN101499216B (en) 2008-01-28 2012-10-03 曾秋昇 Limb interaction type learning method and apparatus
US8941466B2 (en) * 2009-01-05 2015-01-27 Polytechnic Institute Of New York University User authentication for devices with touch sensitive elements, such as touch sensitive display screens
US10019081B2 (en) * 2009-01-15 2018-07-10 International Business Machines Corporation Functionality switching in pointer input devices
KR101545490B1 (en) * 2009-05-29 2015-08-21 엘지전자 주식회사 Image Display Device and Operating Method for the Same
KR101598336B1 (en) * 2009-05-29 2016-02-29 엘지전자 주식회사 Operating a Remote Controller
KR101572843B1 (en) * 2009-06-03 2015-11-30 엘지전자 주식회사 Image Display Device and Operating Method for the Same
US8380225B2 (en) 2009-09-14 2013-02-19 Microsoft Corporation Content transfer involving a gesture
US20120124481A1 (en) * 2010-03-18 2012-05-17 Robert Campbell Interacting with a device
CN102346431A (en) * 2010-07-30 2012-02-08 鸿富锦精密工业(深圳)有限公司 Portable electronic device with remote control function
CN102008822A (en) * 2010-12-17 2011-04-13 黄振武 Desktop entertainment system based on gesture interaction
US20140195925A1 (en) * 2011-08-24 2014-07-10 Sony Ericsson Mobile Communications Ab Short-range radio frequency wireless communication data transfer methods and related devices
CN103051589A (en) * 2011-10-11 2013-04-17 技嘉科技股份有限公司 Multimedia communication composition system and multimedia communication method thereof
CN102819751A (en) * 2012-08-21 2012-12-12 长沙纳特微视网络科技有限公司 Man-machine interaction method and device based on action recognition
RU2642026C1 (en) * 2017-01-09 2018-01-23 Общество с ограниченной ответственностью "Научно-производственное предприятие "Резонанс" Remote control system of machine with mast attachment

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4745402A (en) 1987-02-19 1988-05-17 Rca Licensing Corporation Input device for a display system using phase-encoded signals
US4796019A (en) 1987-02-19 1989-01-03 Rca Licensing Corporation Input device for a display system
US5598187A (en) * 1993-05-13 1997-01-28 Kabushiki Kaisha Toshiba Spatial motion pattern input system and input method
WO1999022338A1 (en) 1997-10-28 1999-05-06 British Telecommunications Public Limited Company Portable computers
US6249606B1 (en) * 1998-02-19 2001-06-19 Mindmaker, Inc. Method and system for gesture category recognition and training using a feature vector
US6347290B1 (en) * 1998-06-24 2002-02-12 Compaq Information Technologies Group, L.P. Apparatus and method for detecting and executing positional and gesture commands corresponding to movement of handheld computing device
US20020190947A1 (en) 2000-04-05 2002-12-19 Feinstein David Y. View navigation and magnification of a hand-held device with a display
US6603420B1 (en) 1999-12-02 2003-08-05 Koninklijke Philips Electronics N.V. Remote control device with motion-based control of receiver volume, channel selection or other parameters
US6750801B2 (en) * 2000-12-29 2004-06-15 Bellsouth Intellectual Property Corporation Remote control device with directional mode indicator
US7123180B1 (en) * 2003-07-29 2006-10-17 Nvidia Corporation System and method for controlling an electronic device using a single-axis gyroscopic remote control
US7233316B2 (en) * 2003-05-01 2007-06-19 Thomson Licensing Multimedia user interface

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7148879B2 (en) 2000-07-06 2006-12-12 At&T Corp. Bioacoustic control system, method and apparatus
WO2003007291A2 (en) * 2001-07-13 2003-01-23 Universal Electronics Inc. System and method for using a hand held device to display information
US6990639B2 (en) * 2002-02-07 2006-01-24 Microsoft Corporation System and process for controlling electronic components in a ubiquitous computing environment using multimodal integration
US8424034B2 (en) * 2002-05-03 2013-04-16 Disney Enterprises, Inc. System and method for displaying commercials in connection with an interactive television application

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4745402A (en) 1987-02-19 1988-05-17 Rca Licensing Corporation Input device for a display system using phase-encoded signals
US4796019A (en) 1987-02-19 1989-01-03 Rca Licensing Corporation Input device for a display system
US5598187A (en) * 1993-05-13 1997-01-28 Kabushiki Kaisha Toshiba Spatial motion pattern input system and input method
WO1999022338A1 (en) 1997-10-28 1999-05-06 British Telecommunications Public Limited Company Portable computers
US6249606B1 (en) * 1998-02-19 2001-06-19 Mindmaker, Inc. Method and system for gesture category recognition and training using a feature vector
US6347290B1 (en) * 1998-06-24 2002-02-12 Compaq Information Technologies Group, L.P. Apparatus and method for detecting and executing positional and gesture commands corresponding to movement of handheld computing device
US6603420B1 (en) 1999-12-02 2003-08-05 Koninklijke Philips Electronics N.V. Remote control device with motion-based control of receiver volume, channel selection or other parameters
US20020190947A1 (en) 2000-04-05 2002-12-19 Feinstein David Y. View navigation and magnification of a hand-held device with a display
US6750801B2 (en) * 2000-12-29 2004-06-15 Bellsouth Intellectual Property Corporation Remote control device with directional mode indicator
US7233316B2 (en) * 2003-05-01 2007-06-19 Thomson Licensing Multimedia user interface
US7123180B1 (en) * 2003-07-29 2006-10-17 Nvidia Corporation System and method for controlling an electronic device using a single-axis gyroscopic remote control

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Ari Y. Benbasat et al, An Inertial Measurement Framework for Gesture Recognition and Applications, MIT Media Laboratory, Cambridge, 2001.
Christopher Lee et al, Online, Interactive Learning of Gestures for Human/Robot Interfaces, The Robotics Institute, Carnegie Mellon University, Pittsburgh, IEEE International Conference on Robotics and Automation, Minneapolis, 1996.
H. Baldus et al, Sensor-Based Context Awareness, Nat. Lab. Technical Note 2002/247, Issued Sep. 2002, Koninklijke Philips Electronics N.V.
Ho-Sub Yoon et al, Hand Gesture Recognition Using Combined Features of Location, Angle and Velocity, Pattern Recognition, vol. 34, Issue 7, 2001, pp. 1491-1501.
V.P. Buil et al, Context Aware Personal Remote Control, Nat Lab. Technical Note 2001/533, Issued Apr. 2002, Koninklijke Philips Electronics N.V.

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070200658A1 (en) * 2006-01-06 2007-08-30 Samsung Electronics Co., Ltd. Apparatus and method for transmitting control commands in home network system
US20080059178A1 (en) * 2006-08-30 2008-03-06 Kabushiki Kaisha Toshiba Interface apparatus, interface processing method, and interface processing program
US20120081216A1 (en) * 2007-03-23 2012-04-05 Lee Yu-Tuan Remote-controlled motion apparatus with acceleration self-sense and remote control apparatus therefor
US8126211B2 (en) * 2007-04-26 2012-02-28 Atlab Inc. Pointing device and motion value calculating method thereof
US20080273756A1 (en) * 2007-04-26 2008-11-06 Atlab Inc. Pointing device and motion value calculating method thereof
US20090113354A1 (en) * 2007-10-30 2009-04-30 Samsung Electronics Co., Ltd. Broadcast receiving apparatus and control method thereof
US10044967B2 (en) 2007-10-30 2018-08-07 Samsung Electronics Co., Ltd. Broadcast receiving apparatus and control method thereof
US20090128714A1 (en) * 2007-11-16 2009-05-21 Oki Semiconductor Co., Ltd. Electronic device, remote control device, and remote control system
US8405783B2 (en) * 2007-11-16 2013-03-26 Oki Semiconductor Co., Ltd. Electronic device, remote control device, and remote control system
US20100060569A1 (en) * 2008-09-09 2010-03-11 Lucent Technologies Inc. Wireless remote control having motion-based control functions and method of manufacture thereof
US20120007713A1 (en) * 2009-11-09 2012-01-12 Invensense, Inc. Handheld computer systems and techniques for character and command recognition related to human movements
US9174123B2 (en) * 2009-11-09 2015-11-03 Invensense, Inc. Handheld computer systems and techniques for character and command recognition related to human movements
US20160018902A1 (en) * 2009-11-09 2016-01-21 Invensense, Inc. Handheld computer systems and techniques for character and command recognition related to human movements
US8613666B2 (en) 2010-08-31 2013-12-24 Microsoft Corporation User selection and navigation based on looped motions
WO2013124792A1 (en) 2012-02-23 2013-08-29 Koninklijke Philips N.V. Remote control device
US9715823B2 (en) 2012-02-23 2017-07-25 Koninklijke Philips N.V. Remote control device

Also Published As

Publication number Publication date
JP2007518511A (en) 2007-07-12
US20080252491A1 (en) 2008-10-16
CN104732752A (en) 2015-06-24
CN1910636A (en) 2007-02-07
WO2005071636A1 (en) 2005-08-04
EP1709609A1 (en) 2006-10-11
KR20060126727A (en) 2006-12-08

Similar Documents

Publication Publication Date Title
Patten et al. Audiopad: a tag-based interface for musical performance
US5670987A (en) Virtual manipulating apparatus and method
US8049730B2 (en) Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen
CN100334531C (en) User interface system based on pointing device
US10331228B2 (en) System and method for determining 3D orientation of a pointing device
CN104838352B (en) Action initialization in multi-surface device
US7184592B2 (en) Information processing apparatus, method of controlling the same, and program for causing a computer to execute such a method
JP2006323823A (en) Sound-based virtual keyboard, device and method
US20170235377A1 (en) Systems and methods of creating a realistic grab experience in virtual reality/augmented reality environments
EP2483758B1 (en) System and method for recognizing gestures
Zhang et al. Visual panel: virtual mouse, keyboard and 3D controller with an ordinary piece of paper
US20130010207A1 (en) Gesture based interactive control of electronic equipment
KR20130112061A (en) Natural gesture based user interface methods and systems
US20190091571A1 (en) Semantic gaming and application transformation
CN102566751B (en) Free space pointing devices and method
US9632658B2 (en) Dynamic user interactions for display control and scaling responsiveness of display objects
Ono et al. Touch & activate: adding interactivity to existing objects using active acoustic sensing
US9372544B2 (en) Gesture recognition techniques
US20120226981A1 (en) Controlling electronic devices in a multimedia system through a natural user interface
US9703397B2 (en) High fidelity remote controller device for digital living room
CN102915112B (en) For the system and method for closely motion tracking
EP2426598A2 (en) Apparatus and method for user intention inference using multimodal information
JP2013510381A (en) Handheld computer system and technique for recognizing characters and commands related to human movement
KR101690117B1 (en) Method and system for multimodal and gestural control
CN1554044B (en) Method and apparatus for selecting information in multi-dimensional space

Legal Events

Date Code Title Description
AS Assignment

Owner name: KONINKLIJKE PHILIPS ELECTRONICS, N.V., NETHERLANDS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DERUYTER, BORIS EMMANUEL RACHMUND;BUIL, VINCENTIUS PAULUS;LASHINA, TATIANA A.;AND OTHERS;REEL/FRAME:017956/0244

Effective date: 20040518

AS Assignment

Owner name: NXP B.V., NETHERLANDS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KONINKLIJKE PHILIPS ELECTRONICS N.V.;REEL/FRAME:019719/0843

Effective date: 20070704

Owner name: NXP B.V.,NETHERLANDS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KONINKLIJKE PHILIPS ELECTRONICS N.V.;REEL/FRAME:019719/0843

Effective date: 20070704

STCF Information on status: patent grant

Free format text: PATENTED CASE

FPAY Fee payment

Year of fee payment: 4

AS Assignment

Owner name: BREAKWATERS INNOVATIONS LLC, VIRGINIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NXP B.V.;REEL/FRAME:032642/0564

Effective date: 20131215

AS Assignment

Owner name: NXP B.V., NETHERLANDS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BREAKWATERS INNOVATIONS LLC;REEL/FRAME:035086/0391

Effective date: 20150303

AS Assignment

Owner name: III HOLDINGS 6, LLC, DELAWARE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NXP B.V.;REEL/FRAME:036304/0330

Effective date: 20150730

AS Assignment

Owner name: KONINKLIJKE PHILIPS ELECTRONICS N.V., NETHERLANDS

Free format text: NUNC PRO TUNC ASSIGNMENT;ASSIGNORS:DE RUYTER, BORIS EMMANUEL RACHMUND;TU, JIAWEN W.;BUIL, VINCENTIUS PAULUS;AND OTHERS;SIGNING DATES FROM 20130713 TO 20150901;REEL/FRAME:036528/0899

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552)

Year of fee payment: 8