US20070149282A1 - Interactive gaming method and apparatus with emotion perception ability - Google Patents

Interactive gaming method and apparatus with emotion perception ability Download PDF

Info

Publication number
US20070149282A1
US20070149282A1 US11/519,229 US51922906A US2007149282A1 US 20070149282 A1 US20070149282 A1 US 20070149282A1 US 51922906 A US51922906 A US 51922906A US 2007149282 A1 US2007149282 A1 US 2007149282A1
Authority
US
United States
Prior art keywords
signal
interactive gaming
gaming apparatus
bio
detecting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11/519,229
Inventor
Ying-Ko Lu
Ming-Jye Tsai
Shun-Nan Liou
Cheng-Hung Chang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial Technology Research Institute
Original Assignee
Industrial Technology Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to TW94146650 priority Critical
Priority to TW094146650 priority
Application filed by Industrial Technology Research Institute filed Critical Industrial Technology Research Institute
Assigned to INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE reassignment INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: TSAI, MING-JYE, CHANG, CHENG-HUNG, LIOU, SHUN-NAN, LU, YING-KO
Publication of US20070149282A1 publication Critical patent/US20070149282A1/en
Application status is Abandoned legal-status Critical

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20Input arrangements for video game devices
    • A63F13/21Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/212Input arrangements for video game devices characterised by their sensors, purposes or types using sensors worn by the player, e.g. for measuring heart beat or leg activity
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/02Accessories
    • A63F13/06Accessories using player-operated means for controlling the position of a specific area display
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20Input arrangements for video game devices
    • A63F13/21Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/211Input arrangements for video game devices characterised by their sensors, purposes or types using inertial sensors, e.g. accelerometers or gyroscopes
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/40Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
    • A63F13/42Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
    • A63F13/428Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20Input arrangements for video game devices
    • A63F13/23Input arrangements for video game devices for interfacing with the game device, e.g. specific interfaces between game controller and console
    • A63F13/235Input arrangements for video game devices for interfacing with the game device, e.g. specific interfaces between game controller and console using a wireless connection, e.g. infrared or piconet
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F9/00Games not otherwise provided for
    • A63F9/24Electric games; Games using electronic circuits not otherwise provided for
    • A63F2009/2401Detail of input, input devices
    • A63F2009/2402Input by manual operation
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F9/00Games not otherwise provided for
    • A63F9/24Electric games; Games using electronic circuits not otherwise provided for
    • A63F2009/2401Detail of input, input devices
    • A63F2009/2402Input by manual operation
    • A63F2009/2408Touch-sensitive buttons
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F9/00Games not otherwise provided for
    • A63F9/24Electric games; Games using electronic circuits not otherwise provided for
    • A63F2009/2401Detail of input, input devices
    • A63F2009/243Detail of input, input devices with other kinds of input
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F9/00Games not otherwise provided for
    • A63F9/24Electric games; Games using electronic circuits not otherwise provided for
    • A63F2009/2401Detail of input, input devices
    • A63F2009/2436Characteristics of the input
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1012Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/105Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals using inertial sensors, e.g. accelerometers, gyroscopes
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/20Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterised by details of the game platform
    • A63F2300/204Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterised by details of the game platform the platform being a handheld device

Abstract

The present invention relates to an interactive gaming method and apparatus with emotion perception ability, by which not only gestures of a user can be detected and used as inputs for controlling a game, but also physiological attributes of the user such as heart beats, galvanic skin response (GSR), etc., can be sensed and used as emotional feedbacks of the game affecting the user. According to the disclosed method, the present invention further provides an interactive gaming apparatus that will interpret the signals detected by the inertial sensing module and the bio sensing module and use the interpretation as a basis for evaluating the movements and emotions of a user immediately, and then the evaluation obtained by the interactive gaming apparatus is sent to the gaming platform to be used as feedbacks for controlling the game to interact with the user accordingly. Therefore, by the method and apparatus according to the present invention, not only the harmonics of human motion can be trained to improve, but also the self-control of a user can be enhanced.

Description

    FIELD OF THE INVENTION
  • The present invention relates to a gaming method and apparatus, and more particularly, to an interactive gaming method and apparatus with emotion perception ability, which is capable of interpreting a user's gestures and bio statuses detected respectively by the inertial sensing module and the bio sensing module while using the interpretation as feedbacks for controlling a game to interact with the user accordingly so as to train and improve the motion harmonics and self-control of the user.
  • BACKGROUND OF THE INVENTION
  • With the evolution of video games, the current trend is to enable the game characters of a video game to interact with players through certain input devices of the game platform. The conventional input devices, such as mice, keyboards, or joysticks, etc., simply rely on a cross key and push button arranged thereon for enabling game character movement and command selection.
  • However, as the popular of so-called “virtual reality” (VR) game systems, players are enabled to experience computer-generated environments instead of merely interacting with them over a display screen. Such systems typically require the user to don goggles, through which he or she reacts naturally to the changing virtual environment, generating signals that the computer interprets to determine the state and progress of the presented environment. In these virtual reality game systems, the realistic feeling can be enforced if signals perceivable to the computer can be generated in accordance to the gestures and emotional responses of the user in real time. Moreover, as the signals generated by the combinations of user's gestures and emotional responses have far more varieties than that of button-pressing input devices, the control of the user to video game is greatly enhanced.
  • With this in mind, an improved input device, which is capable of detecting and interpreting a user's external gestures and internal physiological/emotional responses while using the interpretation as feedbacks for controlling a game to interact with the user, are in great need.
  • Many attempts have been made to develop such input devices, one of which is disclosed in U.S. Pat. No. 5,534,917, entitled “VIDEO IMAGE BASED CONTROL SYSTEM”. The control system of the aforesaid patent uses a video camera to capture images of a user, such that the captured imaged are processed and analyzed to be applied in a three-dimensional VR game. Although the abovementioned system can allow a device to be controlled through movement or positioning of a user within the field of view of a camera, which is preferred than those conventional mice, keyboards and joysticks, the type of control offered by the abovementioned system requires a plurality of video cameras to operate simultaneously that the whole system is bulky, difficult to set up and costly. In addition, the abovementioned system has no usefulness in detecting user's physiological/emotional responses while he/she is playing the three-dimensional VR video game.
  • Another attempt is disclosed in U.S. Pat. No. 6,909,420, entitled “DEVICE INDICATING MOVEMENTS FOR SOFTWARE”. The system of the aforesaid patent comprises at least a sleeve for putting on over the joints of a users and at least a movement sensor fixed to each sleeve, by which a movement of the user wearing the sleeves on the limbs thereof can be detected and used as input of a game platform for enabling the user to interact with a game operating on the game platform. However, since the movement sensor used in the abovementioned system is an on/off sensor, the sleeves can not detect sophisticated movements and displacement since the sensor with on two settings, i.e. ON and OFF, is not sufficiently sensitive. In addition, the abovementioned system also has no usefulness in detecting user's physiological/emotional responses.
  • Yet, another attempt is disclosed in U.S. Pat. No. 6,190,314, entitled “COMPUTER INPUT DEVICE WITH BIIOSENSORS FOR SENSING USER EMOTIONS”, as shown in FIG. 1. In FIG. 1, the aforesaid input device 10 is comprised of: a biosensor 101, for detecting and measuring pulses of a user by a means of infrared; a temperature sensor, for measuring temperature variation of the user; and a contact electrode, for measuring galvanic skin response (GSR) of the user; such that the physiological attributes relating to the user's emotional state, including anger, fear, sadness, disgust, joy and surprise, etc., can be recognized and measured. However, the input device 10 is an optical mouse or a mechanical mouse that can only detect user's hand movements on a plane and is connected to a computer 102, it is not operable in space and thus can not be applied in a three-dimensional VR video game
  • From the above description, it is noted that the prior-art interactive control systems are lack of means of using physiological attributes relating to a user's emotional state and the user's gestures as inputs to a video game for enabling the user to interact with game characters of the video game. Therefore, a novel interactive gaming method and apparatus with emotion perception ability is needed.
  • SUMMARY OF THE INVENTION
  • The primary object of the present invention is to provide an interactive gaming method and apparatus with emotion perception ability, by which not only gestures of a user can be detected and used as inputs for controlling a game, but also physiological attributes of the user such as heart beats, pulses, blood oxygen level, general somatic activities, galvanic skin response (GSR), etc., can be sensed and used as emotional feedbacks of the game affecting the user, that is, the interactive gaming apparatus will interpret the detected physiological attributes and gestures of a user of the gaming apparatus, and employ the interpretation as a basis for evaluating the movements and emotions of the user immediately, and then the evaluation obtained by the interactive gaming apparatus is sent to the gaming platform to be used as feedbacks for controlling the game to interact with the user accordingly.
  • Another object of the invention is to provide an interactive gaming method and apparatus with emotion perception ability, which is compact and commercially feasible by the utilization of micro inertial sensing modules, non-invasive bio sensing modules and wireless communication means.
  • Yet, another object of the invention is to provide an interactive gaming method and apparatus with emotion perception ability, capable of using an micro inertial sensing module to detect the displacement/rotation movements of hands holding the gaming apparatus and thus generating electronic signals to be received and processed by a signal processor while enabling the processed signals to be used as input signals of a gaming platform, and therefore, freeing the hands from those button-pressing operations for generating input signals and thus releasing the fingers from pain and fatigue.
  • One further object of the invention is to provide an interactive gaming method and apparatus with emotion perception ability, capable of using an micro inertial sensing module to detect the displacement/rotation movements of hands holding the gaming apparatus and thus generating electronic signals to be received and processed by a signal processor while enabling the processed signals to be used as input signals of a gaming platform, and therefore, increasing the information displaying area of the gaming apparatus since the conventional press buttons can be eliminated from the interactive gaming apparatus of the invention.
  • To achieve the above object, the present invention provides an interactive gaming method with emotion perception ability, which comprises steps of: sensing a movement for generating a gesture signal accordingly while detecting at least a bio signal caused by emotional variation; receiving and processing the gesture signal and the least one bio signal so as to generate a processed signal accordingly; and interpreting the movement and the emotion variation of the user with reference to the processed signal while using the interpretation as basis for evaluating the movements and emotions of the user so as to issue a digital signal accordingly.
  • Preferably, the interactive gaming method with emotion perception ability further comprises a step of: enabling an interactive apparatus to display an interaction with respect to the digital signal. In a preferred aspect, the interactive apparatus can be a device selected from the group consisting of a display and a gaming platform.
  • Preferably, the bio signal can be a signal selected form the group consisting of: a heart rate signal, a blood oxygen level signal, a blood perfusion rate signal, a skin impedance signal, a temperature signal and a moisture signal.
  • Preferably, the gesture signal comprises at least an acceleration signal.
  • Moreover, to achieve the above object, the present invention provides an interactive gaming apparatus with emotion perception ability, which comprises: an inertial sensing module, for sensing a movement and generating a gesture signal accordingly; a bio sensing module, for sensing at least a physiological attribute and generating at least a corresponding bio signal accordingly; and a signal processing unit, electrically connect to the inertial sensing module and the bio sensing module, for receiving and interpreting the gesture signal and the at least one bio signal while using the interpretation as basis for issuing a digital signal accordingly.
  • Preferably, the bio sensing module is a module comprising devices selected form the group consisting of an optical heart rate sensor, an electronic heart rate sensor, an optical blood oxygen level sensor, an optical blood perfusion rate meter, an electronic skin impedance sensor, a temperature sensor and a moisture sensor.
  • Preferably, the interactive gaming apparatus with emotion perception ability of the invention is substantially a device selected from the group consisting of a game wrist sleeve, a game groove, a joystick, and a gamepad.
  • Preferably, the signal processing unit further comprises: a first calculation unit, for receiving and processing the gesture signal to form a first electronic signal accordingly; a second calculation unit, for receiving and processing each bio signal to form a second electronic signal accordingly; and a signal processing/emotion interpretation module, for receiving the first and the second electronic signals while using the received signals as basis for evaluating the movement and emotions of the user so as to issue a digital signal according to the evaluation.
  • Preferably, the inertial sensing module further comprises a first inertial sensing part, for detecting a displacement of a movement moving along a first axis and a rotation of the movement rotating about a second axis.
  • Preferably, the inertial sensing module further comprises a second inertial sensing part, for detecting the displacement moving along the second axis and the rotations rotating about the first axis and a third axis. Moreover, the second inertial sensing part further is comprised of: a primary accelerometer, for detecting the displacement moving along the second axis; and an auxiliary accelerometer, being arranged apart from the primary accelerometer by a specific distance, for enabling the auxiliary accelerometer to detect an acceleration of one degree-of-freedom; wherein the accelerations measured by the primary accelerometer and the auxiliary accelerometer are subjected to a differential operation so as to detect the rotation rotating about the third axis.
  • In a preferred embodiment of the invention, the inertial sensing module further comprises a second inertial sensing part, for detecting the displacement moving along the second axis and the rotation rotating about the first axis and a third axis. Moreover, the second inertial sensing part further is comprised of: an accelerometer, for detecting the displacement moving along the second axis and the rotation rotating about the first axis; and a gyroscope, for detecting the rotation rotating about the third axis.
  • Preferably, the inertial sensing module further comprises a third inertial sensing part, for detecting the rotation rotating about the third axis.
  • In addition, to achieve the above object, the present invention provides an interactive gaming apparatus with emotion perception ability, which comprises:
      • a sensing unit, further comprising:
        • an inertial sensing module for detecting and measuring a movement to generate a gesture signal accordingly; and
        • a bio sensing module, for detecting and measuring at least a physiological attribute to generate at least a bio signal accordingly;
      • a signal processing unit, electrically connect to the inertial sensing module and the bio sensing module, for receiving and interpreting the gesture signal and the at least one bio signal while using the interpretation as basis for issuing a digital signal accordingly; and
      • a gaming platform, capable receiving the digital signal from the signal processing unit while using the received digital signal to control a game played thereon.
  • Preferably, the sensing unit further comprises a wireless transmission module, and the gaming platform further comprises a wireless reception module.
  • Preferably, the sensing unit is substantially a device selected from the group consisting of a game wrist sleeve, a game groove, a joystick, and a gamepad.
  • Preferably, the signal processing unit further comprises: a first calculation unit, for receiving and processing the gesture signal to form a first electronic signal accordingly; and a second calculation unit, for receiving and processing each bio signal to form a second electronic signal accordingly; wherein the first calculation unit can be arranged at a location selected from the group consisting a position on the sensing unit and a position on the gaming platform; and that is the same to the second calculation unit.
  • Preferably, the signal processing unit is electrically connected to a signal processing/emotion interpretation module, whereas the signal processing/emotion interpretation module is arranged at a location selected from the group consisting a position on the sensing unit and a position on the gaming platform.
  • In a preferred aspect of the invention, the signal processing unit further comprises: a first calculation unit, for receiving and processing the gesture signal to form a first electronic signal accordingly; a second calculation unit, for receiving and processing each bio signal to form a second electronic signal accordingly; and a signal processing/emotion interpretation module, for receiving the first and the second electronic signals while using the received signals as basis to issue a digital signal accordingly; wherein the signal processing/emotion interpretation module can be arranged at a location selected from the group consisting a position on the sensing unit and a position on the gaming platform.
  • In addition, the first calculation unit can be arranged at a location selected from the group consisting a position on the sensing unit and a position on the gaming platform, and that is the same to the second calculation unit.
  • Furthermore, to achieve the above object, the present invention provides an interactive gaming apparatus with emotion perception ability, which comprises:
      • an inertial sensing module for detecting and measuring a movement to generate a gesture signal accordingly;
      • a bio sensing module, for detecting and measuring at least a physiological attribute to generate at least a bio signal accordingly;
      • a signal processing unit, electrically connect to the inertial sensing module and the bio sensing module, f or receiving and interpreting the gesture signal and the at least one bio signal while using the interpretation as basis for issuing a digital signal accordingly; and
      • a displaying unit, coupled to the signal processing unit for receiving the digital signal.
  • Preferably, the interactive gaming apparatus of the invention further comprises a networking interface, for enabling the interactive gaming apparatus to connect to a network for downloading and updating softwares resided in the interactive gaming apparatus.
  • Preferably, the interactive gaming apparatus with emotion perception ability of the invention is substantially a mobile phone, a personal digital assistant, or a portable gaming console.
  • Preferably, the interactive gaming apparatus with emotion perception ability of the invention is capable of executing a game software, whereas the game software can be a software selected from the group consisting of a multimedia game, a 2D/3D animation game and a music playing software with tempo/volume control.
  • Preferably, the interactive gaming apparatus of the invention further comprises a memory, for storing the video game and icons of emotional expression.
  • Other aspects and advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the present invention.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a schematic view of a conventional computer input device disclosed in U.S. Pat. No. 6,190,314.
  • FIG. 2 is a flowchart depicting steps of an interactive gaming method with emotion perception ability according to a preferred embodiment of the invention.
  • FIG. 3A is a schematic view of an interactive gaming apparatus with emotion perception ability according to a first preferred embodiment of the invention.
  • FIG. 3B is a schematic view of an interactive gaming apparatus with emotion perception ability according to a second preferred embodiment of the invention.
  • FIG. 4 is a schematic diagram showing the application of the interactive gaming apparatus with emotion perception ability of the first preferred embodiment of the invention.
  • FIG. 5A is a schematic view of an interactive gaming apparatus with emotion perception ability according to a third preferred embodiment of the invention.
  • FIG. 5B is a schematic diagram depicting the operation of the inertial sensing module and the bio sensing module of the third preferred embodiment of the invention.
  • DESCRIPTION OF THE PREFERRED EMBODIMENT
  • For your esteemed members of reviewing committee to further understand and recognize the fulfilled functions and structural characteristics of the invention, several preferable embodiments cooperating with detailed description are presented as the follows.
  • Please refer to FIG. 2, which is a flowchart depicting steps of an interactive gaming method with emotion perception ability according to a preferred embodiment of the invention. The flow of the interactive gaming method 4 of FIG. 2 starts at step 40. At step 40, a movement is sensed for generating a gesture signal accordingly while detecting at least a bio signal caused by emotional variation, and then the flow proceeds to step 41. At step 41, the gesture signal and the least one bio signal are received and processed so as to generate a processed signal accordingly, and then the flow proceeds to step 42. It is noted that the bio signal can be a signal selected form the group consisting of: a heart rate signal, a blood oxygen level signal, a blood perfusion rate signal, a skin impedance signal, a temperature signal and a moisture signal, and the gesture signal is an acceleration signal. At step 42, the movement and the emotion variation are interpreted with reference to the processed signal while using the interpretation as basis to issue a digital signal accordingly, and then the flow proceeds to step 43. At step 43, an interactive apparatus is enabled to display an interaction with respect to the digital signal. In a preferred aspect, the interactive apparatus can be a display, or a gaming platform.
  • By the aforesaid method, an interactive gaming apparatus with emotion perception ability can be provided. Please refer to FIG. 3A, which is a schematic view of an interactive gaming apparatus with emotion perception ability according to a first preferred embodiment of the invention. In FIG. 3A, the interactive gaming apparatus with emotion perception ability 2 is comprised of a sensing unit 21 and a gaming platform 23. The sensing unit 21 can be arranged on a game wrist sleeve, a game groove, a joystick, and a gamepad, which is used for detecting a movement of a user and physiological attributes of the same while playing a video game. Moreover, the sensing unit 21 includes an inertial sensing module 211, a bio sensing module 211 and a signal processing unit 213. The inertial sensing module 211 is used for detecting and measuring accelerations or variations of angular velocity resulting from the movement of the user while generating a gesture signal accordingly. In this preferred embodiment, the inertial sensing module 211 is either being device capable of sensing a movement measured along a specific axis of a plane or along a specific direction define on the plane, or being a device capable of sensing a movement in a space defined by a 3-D Cartesian coordinates of X, Y, Z axes. In addition, the bio sensing module 212 is used for sensing at least a physiological attribute of the user and generating at least a bio signal accordingly; wherein the bio sensing module 212 can be an optical module comprising devices selected form the group consisting of a heart rate sensor, a blood oxygen level sensor, a blood perfusion rate meter, a skin impedance sensor, a temperature sensor and a moisture sensor.
  • The signal processing unit 213 is electrically connected to the inertial sensing module 211 and the bio sensing module 212 for receiving the gesture signal from the inertial sensing module 211 and the bio signal from the bio sensing module 212. In a preferred embodiment of the invention, the signal processing unit 213 further includes a first calculation unit 2131, a second calculation unit 2132 and a signal processing/emotion interpretation module 2133; wherein the first calculation unit 2131 is used for receiving and processing the gesture signal to form a first electronic signal accordingly; and the second calculation unit 2132 is used for receiving and processing each bio signal to form a second electronic signal accordingly. In addition, the sensing unit 21 further comprises a wireless transmission module 214, which is used for encoding the first and the second electronic signals while transmitting the encoded signals to the gaming platform 23.
  • In the first preferred embodiment shown in FIG. 3A, the signal processing/emotion interpretation module 2133 is arranged on the gaming platform 23 such that, as the wireless reception module 231 of the gaming platform 23 receives encoded signals transmitted from the wireless transmission module 24, the received encoded signals are decoded and sent to the signal processing/emotion interpretation module 2133 to be interpreted and used as a basis for evaluating the movements and emotions of a user immediately, and then the obtained evaluation is sent to the gaming platform 23 to be used as feedbacks for controlling a game or a content 232 to interact with the user accordingly, such as controlling the tempo, the speed, the batting average, shooting rate, or size/color of game characters, etc. of the game playing on the gaming platform 23. Moreover, the gaming platform 23 can be a computer, a game console or a multimedia player.
  • Please refer to FIG. 3B, which is a schematic view of an interactive gaming apparatus with emotion perception ability according to a second preferred embodiment of the invention. The configuration of the interactive gaming apparatus with emotion perception ability 2 of FIG. 3B is generally the same as that shown in FIG. 3A, and the only difference is that the signal processing/emotion interpretation module 2133 is arranged on the sensing unit 21 instead of the gaming platform 23. In addition, for simplified the structure of the sensing unit 21 as well as saving the power consumption of the same, it is also feasible to move and arrange the whole signal processing unit 213 on the gaming platform so that the wireless transmission module 214 simply transmits signals detected by the inertial sensing module 211 and the bio sensing module 212 while leaving the entire posterior signal processing to be performed on the gaming platform 23.
  • Please refer to FIG. 4, which is a schematic diagram showing the application of the interactive gaming apparatus with emotion perception ability of the first preferred embodiment of the invention. As shown in FIG. 4, the sensing unit 21 is a wrist sleeve having an inertial sensing module 211, a bio sensing module 212, a signal processing unit 213 and a wireless transmission module 214; wherein the bio sensing module 213 is further composed of a optical bio sensor 2121 and a impedance electrode 2122. The optical bio sensor 2121 is used for measuring physiological attributes, such as blood oxygen level, pulse, heart beat, and blood perfusion rate, etc. When a video game like a shooting game, a game of aero exercising, a ball game, a boxing game or a fighting game of martial art, etc., is played by a user wearing the wrist sleeve, the emotional responses and the gestures of the user will be detected by the bio sensing module 212 and the inertial sensing module 211 and enables the two modules to issue signals, corresponding to the detection, to be received and processed by the signal processing unit 213. Thereafter, the processed signals are sent to the wireless reception module 231 of the gaming platform 23 by the wireless transmission module 214 and further to the signal processing/emotion interpretation module 2133 to be interpreted and used as a basis for evaluating the movements and emotions of a user immediately, and then the obtained evaluation is sent to the gaming platform 23 to be used as feedbacks for controlling a game or a content 232 to interact with the user accordingly.
  • Please refer to FIG. 5A, which is a schematic view of an interactive gaming apparatus with emotion perception ability according to a third preferred embodiment of the invention. As seen in FIG. 4, the housing of the interactive gaming apparatus with emotion perception ability 3 is configured with a displaying unit 33, a networking interface 37 and a bio interface 38. The displaying unit can be a flat panel display of liquid crystal display, or organic light emitting diode panel, but is not limited thereby. The networking interface 37 is used for enabling the interactive gaming apparatus 3 to connect to a network either through a wired manner or a wireless manner so as to download and update softwares resided in the interactive gaming apparatus 3. Moreover, the interactive gaming apparatus 3 further comprises a memory, which is used for storing the video game and icons of emotional expression. It is noted that the interactive gaming apparatus 3 can be applied in a variety of devices, such as a mobile phone, a personal digital assistant (PDA), and a portable game console, and so on.
  • The internal sensing/processing mechanism of the interactive gaming apparatus 3 is shown in FIG. 5B, which shows the operation of the inertial sensing module and the bio sensing module of the third preferred embodiment of the invention. As seen in FIG. 5B, the internal sensing/processing mechanism of the interactive gaming apparatus 3 is arranged on a substrate 30, which is comprised of an inertial sensing module 31, a signal processing unit 31 and a bio sensing module 34. It is noted that the bio sensing module 33 is connected to the bio interface 38 of FIG. 5A, such that physiological attributes of a user can be detected by the bio sensing module 34 though the bio interface 38 as the thumb of the user is engaged with the bio interface 38 while the interactive gaming apparatus 3 is held in the hand of the user.
  • Moreover, the inertial sensing module 31 is an inertial sensor of sensing a least one degree-of-freedom sensibility. In this preferred embodiment, the inertial sensing module 31 is an inertial sensor capable of sensing a movement of six degree-of-freedom, which is comprised of an X-axis inertial sensing part 311, a Y-axis inertial sensing part 312 and a Z-axis inertial sensing part 313. That is, as inertial sensing module 31 is place in a space defined by a 3-D Cartesian coordinates of X, Y, Z axes, a movement of six degree-of-freedom can be sensed by the inertial sensing module 31 and generate a sensing signal accordingly, that the X-axis inertial sensing part 311 is enabled to sense a displacement measured along the X axis and the rotation ωy about the Y axis, and the Y-axis inertial sensing part 312 is enabled to sense a displacement measured along the Y axis and the rotations ωy, ωz respectively about the Y and Z axis, and the Z-axis inertial sensing part 313 is enabled to sense a displacement measured along the Z axis. In addition, the signal processing unit 32 is coupled to the inertial sensing module 31 and the displaying unit (not shown in FIG. 5B), so that the signal processing unit 32 can receive and process the sensing signal and then transmit the processed signal to the displaying unit for enabling the displaying unit to display game characters with respect to the processed signal.
  • In the preferred embodiment of FIG. 5B, the X-axis inertial sensing part 311 and the Z-axis inertial sensing part 313 are respectively an accelerometer, and the Y-axis inertial sensing part 312 is further comprised of: a primary accelerometer 3121, for detecting the displacement moving along the Y axis; and an auxiliary accelerometer 3122, being arranged apart from the primary accelerometer 3121 by a specific distance D, for enabling the auxiliary accelerometer to detect an acceleration of one degree-of-freedom; wherein the accelerations measured by the primary accelerometer 3121 and the auxiliary accelerometer 3122 are subjected to a differential operation so as to detect the rotation rotating about the Z axis. As for the rotation about the X axis can be selected to be sensed by the primary accelerometer 3121 or the auxiliary accelerometer 3122.
  • As the inertial sensing module 31 is subjecting to a movement of a user holding the interactive gaming apparatus 3, the sensing signal is sent to the signal processing unit 32 to be interpreted and used as a basis for evaluating the movement of the user, and then the evaluation is used as feedbacks for controlling the game character 35 of FIG. 5A to interact with the user accordingly. Moreover, the inertial sensing module 31 also can be used for controlling the command selection of the menu 36 shown in FIG. 5A, so that number of conventional press buttons required in the interactive gaming apparatus 3 can be minimized and thus the displaying area are enlarged. Furthermore, the inertial sensing module 31 can enable the interactive gaming apparatus 3 to have more ways of controlling the game character 35. Preferably, the software resided in the interactive gaming apparatus 3 can be a multimedia game, a 2D/3D animation game, or a music playing software with tempo/volume control.
  • Referring to the emotion sensing capability of the interactive gaming apparatus of the invention, as the interactive gaming apparatus 3 of FIG. 5A is held in the both hands of a user, the fingers of the user will engage with the bio interface 38 so that the sensing signal of the bio sensing module 34 is sent to the signal processing unit 32 to be interpreted and used as a basis for evaluating the emotions of the user, and then the evaluation is used as feedbacks for controlling the game character 35 to interact with the user accordingly. In a preferred aspect, the bio sensing module 34 can be employed for measuring emotion indexes or indexes of physiological attribute while the result of the measuring can be displayed on the displaying unit in a graphic manner.
  • In the first and the second embodiments of the invention, the inertial sensing module can be a stand-along X-axis inertial sensing part, a stand-along Y-axis inertial sensing part, or a stand-along Z-axis inertial sensing part. However, also in the first and the second embodiments of the invention, the inertial sensing module can be a module comprising a X-axis inertial sensing part, a Y-axis inertial sensing part, and a Z-axis inertial sensing part, so as to enable the sensing module to sensing a movement of six degree-of-freedom. As the principle of the aforesaid module is known in prior art that it is not described further herein.
  • Although the interactive gaming apparatus 3 of FIG. 5A is applied in a portable game console, it is not limited thereby. That is, the interactive gaming apparatus can be applied in a video game station or control pad thereof, such as joystick, and hand-held controller, etc. Moreover, although the sensing unit of the invention is fitted on a wrist sleeve as shown in FIG. 4, it is not limited thereby and can be fitted to any means worn on elbow, knee, waist, etc., such that all limb and body movements of the user can be detected.
  • To sum up, the present invention provides a compact and commercially feasible interactive gaming apparatus with emotion perception ability, by which not only gestures of a user can be detected and used as inputs for controlling a game, but also physiological attributes of the user can be sensed and used as emotional feedbacks of the game affecting the user, that is, the interactive gaming apparatus will interpret the detected physiological attributes and gestures of a user of the gaming apparatus, and employ the interpretation as a basis for evaluating the movements and emotions of the user immediately, and then the evaluation obtained by the interactive gaming apparatus is sent to the gaming platform to be used as feedbacks for controlling the game to interact with the user accordingly.
  • While the preferred embodiment of the invention has been set forth for the purpose of disclosure, modifications of the disclosed embodiment of the invention as well as other embodiments thereof may occur to those skilled in the art. Accordingly, the appended claims are intended to cover all embodiments which do not depart from the spirit and scope of the invention.

Claims (29)

1. An interactive gaming method with emotion perception ability, comprising steps of:
sensing a movement for generating a gesture signal accordingly while detecting at least a bio signal caused by emotional variation;
receiving and processing the gesture signal and the least one bio signal so as to generate a processed signal accordingly; and
interpreting the movement and the emotion variation with reference to the processed signal while using the interpretation as basis to issue a digital signal accordingly.
2. The interactive gaming method of claim 1, further comprising a step of:
enabling an interactive apparatus to display an interaction with respect to the digital signal.
3. The interactive gaming method of claim 2, wherein the interactive apparatus is a device selected from the group consisting of a display and a gaming platform.
4. The interactive gaming method of claim 1, wherein the bio signal is a signal selected form the group consisting of: a heart rate signal, a blood oxygen level signal, a blood perfusion rate signal, a skin impedance signal, a temperature signal and a moisture signal.
5. The interactive gaming method of claim 1, wherein the gesture signal comprises at least an acceleration signal.
6. An interactive gaming apparatus with emotion perception ability, comprising:
a sensing unit, for detecting and measuring a movement to generate a gesture signal accordingly and for detecting and measuring at least a physiological attribute to generate at least a bio signal accordingly; and
a signal processing unit, electrically connected to the sensing unit, for receiving and interpreting the gesture signal and the at least one bio signal while using the interpretation as basis to issue a digital signal accordingly.
7. The interactive gaming apparatus of claim 6, wherein the sensing unit further comprises:
an inertial sensing module, electrically connected to the signal processing unit, for detecting and measuring the movement so as to generate the gesture signal accordingly; and
a bio sensing module, electrically connected to the signal processing unit, for detecting and measuring at least a physiological attribute to generate the at least one bio signal accordingly.
8. The interactive gaming apparatus of claim 6, further comprising:
a displaying unit, coupled to the signal processing unit for enabling the same to received the digital signal.
9. The interactive gaming apparatus of claim 6, further comprising:
a gaming platform, capable receiving the digital signal while using the received digital signal to interact.
10. The interactive gaming apparatus of claim 6, further comprising:
a displaying unit, coupled to the signal processing unit for enabling the same to received the digital signal; and
a gaming platform, capable receiving the digital signal from the signal processing unit while using the received digital signal to interact.
11. The interactive gaming apparatus of claim 6, wherein the bio sensing module is a module comprising devices selected form the group consisting of an optical heart rate sensor, an electronic heart rate sensor, an optical blood oxygen level sensor, an optical blood perfusion rate meter, an electronic skin impedance sensor, a temperature sensor and a moisture sensor.
12. The interactive gaming apparatus of claim 6, wherein the interactive gaming apparatus is substantially a device selected from the group consisting of a game wrist sleeve, a game groove, a joystick, and a gamepad.
13. The interactive gaming apparatus of claim 6, wherein the interactive gaming apparatus is substantially a device selected from the group consisting of a mobile phone, a personal digital assistant, and a portable game console.
14. The interactive gaming apparatus of claim 6, wherein the signal processing unit further comprises:
a first calculation unit, for receiving and processing the gesture signal to form a first electronic signal accordingly; and
a second calculation unit, for receiving and processing each bio signal to form a second electronic signal accordingly.
15. The interactive gaming apparatus of claim 14, wherein the signal processing unit further comprises:
a signal processing/emotion interpretation module, for receiving the first and the second electronic signals while performing an operation upon the received signals so as to issue a digital signal accordingly.
16. The interactive gaming apparatus of claim 6, wherein inertial sensing module is substantially a first inertial part, capable of detecting a displacement of a movement moving along a first axis and a rotation of the movement rotating about a second axis.
17. The interactive gaming apparatus of claim 6, wherein inertial sensing module is substantially a second inertial part, capable of detecting the displacement moving along a second axis and the rotations rotating about a first axis and a third axis.
18. The interactive gaming apparatus of claim 17, wherein the second inertial sensing part further comprising:
a primary accelerometer, for detecting the displacement moving along the second axis; and
an auxiliary accelerometer, being arranged apart from the primary accelerometer by a specific distance, for enabling the auxiliary accelerometer to detect an acceleration of one degree-of-freedom;
wherein the accelerations measured by the primary accelerometer and the auxiliary accelerometer are subjected to a differential operation so as to detect the rotation rotating about the third axis.
19. The interactive gaming apparatus of claim 17, wherein the second inertial sensing part further comprising:
an accelerometer, for detecting the displacement moving along the second axis and the rotation rotating about the first axis; and
a gyroscope, for detecting the rotation rotating about the third axis.
20. The interactive gaming apparatus of claim 6, wherein inertial sensing module is substantially a third inertial part, capable of detecting the rotation rotating about the a third axis.
21. The interactive gaming apparatus of claim 6, wherein inertial sensing module is a module composed of a first inertial part, a second inertial part and a third inertial part.
22. The interactive gaming apparatus of claim 6, wherein the sensing unit further comprises a wireless transmission module while the gaming platform further comprises a wireless reception module.
23. The interactive gaming apparatus of claim 6, further comprising a networking interface, for enabling the interactive gaming apparatus to connect to a network so as to download and update softwares resided in the interactive gaming apparatus.
24. The interactive gaming apparatus of claim 6, capable of executing a game software.
25. The interactive gaming apparatus of claim 6, further comprising a storage media, for providing memory ability to the interactive gaming apparatus.
26. An interactive gaming apparatus with emotion perception ability, comprising:
an inertial sensing module, for detecting and measuring a movement to generate a gesture signal accordingly;
a bio sensing module, for detecting and measuring at least a physiological attribute to generate at least a bio signal accordingly;
a signal processing unit, electrically connected to the inertial sensing module and the bio sensing module, for receiving and interpreting the gesture signal and the at least one bio signal while using the interpretation as basis to issue a digital signal accordingly; and
a displaying unit, coupled to the signal processing unit for enabling the same to received the digital signal.
27. An interactive gaming apparatus with emotion perception ability, comprising:
a sensing unit, for detecting and measuring a movement to generate a gesture signal accordingly and for detecting and measuring at least a physiological attribute to generate at least a bio signal accordingly; and
a signal processing unit, electrically connected to the sensing unit, for receiving and interpreting the gesture signal and the at least one bio signal sensed while using the interpretation as basis to issue a digital signal according to the evaluation; and
a gaming platform, capable receiving the digital signal.
28. The interactive gaming apparatus of claim 27, wherein the sensing unit further comprises:
an inertial sensing module, electrically connected to the signal processing unit, for detecting and measuring the movement so as to generate the gesture signal accordingly; and
a bio sensing module, electrically connected to the signal processing unit, for detecting and measuring at least a physiological attribute to generate the at least one bio signal accordingly.
29. The interactive gaming apparatus of claim 27, wherein the signal processing unit is electrically connected to a signal processing/emotion interpretation module, and the signal processing/emotion interpretation module is arranged at a location selected from the group consisting a position on the sensing unit and a position on the gaming platform.
US11/519,229 2005-12-27 2006-09-12 Interactive gaming method and apparatus with emotion perception ability Abandoned US20070149282A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
TW94146650 2005-12-27
TW094146650 2005-12-27

Publications (1)

Publication Number Publication Date
US20070149282A1 true US20070149282A1 (en) 2007-06-28

Family

ID=38194575

Family Applications (1)

Application Number Title Priority Date Filing Date
US11/519,229 Abandoned US20070149282A1 (en) 2005-12-27 2006-09-12 Interactive gaming method and apparatus with emotion perception ability

Country Status (2)

Country Link
US (1) US20070149282A1 (en)
TW (1) TWI311067B (en)

Cited By (57)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070162851A1 (en) * 2006-01-10 2007-07-12 Hon Hai Precision Industry Co., Ltd. Input device and method for menu control on a portable computing device
US20080215971A1 (en) * 2007-03-01 2008-09-04 Sony Computer Entertainment America Inc. System and method for communicating with an avatar
US20080293491A1 (en) * 2007-05-25 2008-11-27 Asustek Computer Inc. Game Controller
WO2009042190A1 (en) * 2007-09-25 2009-04-02 Wms Gaming Inc. Accessing wagering game services by aiming handheld device at external device
WO2009134909A2 (en) * 2008-04-29 2009-11-05 Bally Gaming, Inc. Biofeedback for a gaming device, such as an electronic gaming machine (egm)
US20100105478A1 (en) * 2008-10-18 2010-04-29 Hallaian Stephen C Mind-control toys and methods of interaction therewith
US20100323794A1 (en) * 2009-06-18 2010-12-23 Yui-Zhang Su Sensor based human motion detection gaming with false positive detection
US20110007142A1 (en) * 2009-07-09 2011-01-13 Microsoft Corporation Visual representation expression based on player expression
US7892178B1 (en) * 2009-09-28 2011-02-22 Impact Sports Technologies, Inc. Monitoring device for an interactive game
US20110166940A1 (en) * 2010-01-05 2011-07-07 Searete Llc Micro-impulse radar detection of a human demographic and delivery of targeted media content
US20110183757A1 (en) * 2010-01-22 2011-07-28 Nintendo Co., Ltd. Storage medium having game program stored therein, game apparatus, control method, and game system
US20110184248A1 (en) * 2010-01-27 2011-07-28 Nintendo Co., Ltd. Storage medium having stored therein information processing program, information processing device, information processing method, and information processing system
US20110195777A1 (en) * 2010-02-10 2011-08-11 Nintendo Co., Ltd. Storage medium having game program stored thereon, game apparatus, game system, and game processing method
US20110207100A1 (en) * 2008-10-20 2011-08-25 Koninklijke Philips Electronics N.V. Controlling an influence on a user in a rendering environment
US20110260830A1 (en) * 2010-04-22 2011-10-27 Sony Computer Entertainment Inc. Biometric interface for a handheld device
US8116841B2 (en) 2007-09-14 2012-02-14 Corventis, Inc. Adherent device with multiple physiological sensors
WO2012036753A1 (en) * 2010-09-17 2012-03-22 Searete Llc Control of an electronic apparatus using micro-impulse radar
US20120178529A1 (en) * 2009-07-17 2012-07-12 Sony Computer Entertainment Europe Limited User Interface and Method of User Interaction
US8249686B2 (en) 2007-09-14 2012-08-21 Corventis, Inc. Adherent device for sleep disordered breathing
US8374688B2 (en) 2007-09-14 2013-02-12 Corventis, Inc. System and methods for wireless body fluid monitoring
US20130057576A1 (en) * 2011-09-02 2013-03-07 Inventec Corporation Display method and electronic device using the same
US8412317B2 (en) 2008-04-18 2013-04-02 Corventis, Inc. Method and apparatus to measure bioelectric impedance of patient tissue
US8460189B2 (en) 2007-09-14 2013-06-11 Corventis, Inc. Adherent cardiac monitor with advanced sensing capabilities
US8461468B2 (en) 2009-10-30 2013-06-11 Mattel, Inc. Multidirectional switch and toy including a multidirectional switch
US20130225261A1 (en) * 2008-11-19 2013-08-29 Immersion Corporation Method and apparatus for generating mood-based haptic feedback
US20130234933A1 (en) * 2011-08-26 2013-09-12 Reincloud Corporation Coherent presentation of multiple reality and interaction models
US20140018166A1 (en) * 2012-07-16 2014-01-16 Wms Gaming Inc. Position sensing gesture hand attachment
US8684925B2 (en) 2007-09-14 2014-04-01 Corventis, Inc. Injectable device for physiological monitoring
US8718752B2 (en) 2008-03-12 2014-05-06 Corventis, Inc. Heart failure decompensation prediction based on cardiac rhythm
US20140135960A1 (en) * 2012-11-15 2014-05-15 Samsung Electronics Co., Ltd. Wearable device, display device, and system to provide exercise service and methods thereof
US8790259B2 (en) 2009-10-22 2014-07-29 Corventis, Inc. Method and apparatus for remote detection and monitoring of functional chronotropic incompetence
US20140234815A1 (en) * 2013-02-18 2014-08-21 Electronics And Telecommunications Research Institute Apparatus and method for emotion interaction based on biological signals
US20140287387A1 (en) * 2013-03-24 2014-09-25 Emozia, Inc. Emotion recognition system and method for assessing, monitoring, predicting and broadcasting a user's emotive state
US8884813B2 (en) 2010-01-05 2014-11-11 The Invention Science Fund I, Llc Surveillance of stress conditions of persons using micro-impulse radar
US8897868B2 (en) 2007-09-14 2014-11-25 Medtronic, Inc. Medical device automatic start-up upon contact to patient tissue
US8915859B1 (en) * 2004-09-28 2014-12-23 Impact Sports Technologies, Inc. Monitoring device, system and method for a multi-player interactive game
US8965498B2 (en) 2010-04-05 2015-02-24 Corventis, Inc. Method and apparatus for personalized physiologic parameters
US9019149B2 (en) 2010-01-05 2015-04-28 The Invention Science Fund I, Llc Method and apparatus for measuring the motion of a person
US20150116214A1 (en) * 2013-10-29 2015-04-30 Anders Grunnet-Jepsen Gesture based human computer interaction
US9024814B2 (en) 2010-01-05 2015-05-05 The Invention Science Fund I, Llc Tracking identities of persons using micro-impulse radar
US9058716B2 (en) 2011-06-06 2015-06-16 Bally Gaming, Inc. Remote game play in a wireless gaming environment
WO2015087323A1 (en) * 2013-12-09 2015-06-18 Mantisvision Ltd Emotion based 3d visual effects
US20150178915A1 (en) * 2013-12-19 2015-06-25 Microsoft Corporation Tagging Images With Emotional State Information
US9167975B1 (en) * 2010-07-28 2015-10-27 Impact Sports Technologies, Inc. Motion resistant device to monitor heart rate in ambulatory patients
US20160023116A1 (en) * 2014-07-03 2016-01-28 Spitfire Technologies, Llc Electronically mediated reaction game
US9261967B2 (en) 2011-02-21 2016-02-16 Koninklijke Philips N.V. Gesture recognition system
US9411936B2 (en) 2007-09-14 2016-08-09 Medtronic Monitoring, Inc. Dynamic pairing of patients to data collection gateways
WO2016146536A1 (en) * 2015-03-13 2016-09-22 Hanne Storm Method and apparatus for controlling a user's interaction with a non-medical application
US9451897B2 (en) 2009-12-14 2016-09-27 Medtronic Monitoring, Inc. Body adherent patch with electronics for physiologic monitoring
US9563202B1 (en) 2012-06-29 2017-02-07 Monkeymedia, Inc. Remote controlled vehicle with a head-mounted display apparatus
US9579586B2 (en) 2012-06-29 2017-02-28 Monkeymedia, Inc. Remote controlled vehicle with a handheld display device
US9591973B1 (en) 2011-06-13 2017-03-14 Impact Sports Technologies, Inc. Monitoring device with a pedometer
US9629562B1 (en) 2014-07-25 2017-04-25 Impact Sports Technologies, Inc. Mobile plethysmographic device
US10021344B2 (en) 2015-07-02 2018-07-10 Krush Technologies, Llc Facial gesture recognition and video analysis tool
US10051298B2 (en) 1999-04-23 2018-08-14 Monkeymedia, Inc. Wireless seamless expansion and video advertising player
US20180314321A1 (en) * 2017-04-26 2018-11-01 The Virtual Reality Company Emotion-based experience feedback
US10293260B1 (en) * 2015-06-05 2019-05-21 Amazon Technologies, Inc. Player audio analysis in online gaming environments

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI419833B (en) * 2009-01-09 2013-12-21 Hon Hai Prec Ind Co Ltd Sensor assembly
US8373658B2 (en) * 2010-05-24 2013-02-12 Cywee Group Limited Motion sensing system
TWI549728B (en) * 2014-06-27 2016-09-21 國立臺中科技大學 Management method for detecting multi-users' motions and system thereof

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4787051A (en) * 1986-05-16 1988-11-22 Tektronix, Inc. Inertial mouse system
US5453758A (en) * 1992-07-31 1995-09-26 Sony Corporation Input apparatus
US5534917A (en) * 1991-05-09 1996-07-09 Very Vivid, Inc. Video image based control system
US5860935A (en) * 1996-10-29 1999-01-19 Novid Inc. Game apparatus and method for monitoring psycho-physiological responses to questions
US6190314B1 (en) * 1998-07-15 2001-02-20 International Business Machines Corporation Computer input device with biosensors for sensing user emotions
US6375572B1 (en) * 1999-10-04 2002-04-23 Nintendo Co., Ltd. Portable game apparatus with acceleration sensor and information storage medium storing a game progam
US6832318B1 (en) * 1999-01-15 2004-12-14 Sony Corporation Method and apparatus for secure distribution of information recorded on fixed media
US6909420B1 (en) * 1998-12-03 2005-06-21 Nicolas Frederic Device indicating movements for software

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4787051A (en) * 1986-05-16 1988-11-22 Tektronix, Inc. Inertial mouse system
US5534917A (en) * 1991-05-09 1996-07-09 Very Vivid, Inc. Video image based control system
US5453758A (en) * 1992-07-31 1995-09-26 Sony Corporation Input apparatus
US5860935A (en) * 1996-10-29 1999-01-19 Novid Inc. Game apparatus and method for monitoring psycho-physiological responses to questions
US6190314B1 (en) * 1998-07-15 2001-02-20 International Business Machines Corporation Computer input device with biosensors for sensing user emotions
US6909420B1 (en) * 1998-12-03 2005-06-21 Nicolas Frederic Device indicating movements for software
US6832318B1 (en) * 1999-01-15 2004-12-14 Sony Corporation Method and apparatus for secure distribution of information recorded on fixed media
US6375572B1 (en) * 1999-10-04 2002-04-23 Nintendo Co., Ltd. Portable game apparatus with acceleration sensor and information storage medium storing a game progam

Cited By (112)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10051298B2 (en) 1999-04-23 2018-08-14 Monkeymedia, Inc. Wireless seamless expansion and video advertising player
US8915859B1 (en) * 2004-09-28 2014-12-23 Impact Sports Technologies, Inc. Monitoring device, system and method for a multi-player interactive game
US7697274B2 (en) * 2006-01-10 2010-04-13 Hong Fu Jin Precision Industry (Shen Zhen) Co., Ltd. Input device and method for menu control on a portable computing device
US20070162851A1 (en) * 2006-01-10 2007-07-12 Hon Hai Precision Industry Co., Ltd. Input device and method for menu control on a portable computing device
US20080215973A1 (en) * 2007-03-01 2008-09-04 Sony Computer Entertainment America Inc Avatar customization
US20080215679A1 (en) * 2007-03-01 2008-09-04 Sony Computer Entertainment America Inc. System and method for routing communications among real and virtual communication devices
US20080235582A1 (en) * 2007-03-01 2008-09-25 Sony Computer Entertainment America Inc. Avatar email and methods for communicating between real and virtual worlds
US20080214253A1 (en) * 2007-03-01 2008-09-04 Sony Computer Entertainment America Inc. System and method for communicating with a virtual world
US8502825B2 (en) 2007-03-01 2013-08-06 Sony Computer Entertainment Europe Limited Avatar email and methods for communicating between real and virtual worlds
US8788951B2 (en) * 2007-03-01 2014-07-22 Sony Computer Entertainment America Llc Avatar customization
US8425322B2 (en) 2007-03-01 2013-04-23 Sony Computer Entertainment America Inc. System and method for communicating with a virtual world
US20080215972A1 (en) * 2007-03-01 2008-09-04 Sony Computer Entertainment America Inc. Mapping user emotional state to avatar in a virtual world
US20080215971A1 (en) * 2007-03-01 2008-09-04 Sony Computer Entertainment America Inc. System and method for communicating with an avatar
US7979574B2 (en) 2007-03-01 2011-07-12 Sony Computer Entertainment America Llc System and method for routing communications among real and virtual communication devices
US20080293491A1 (en) * 2007-05-25 2008-11-27 Asustek Computer Inc. Game Controller
US8249686B2 (en) 2007-09-14 2012-08-21 Corventis, Inc. Adherent device for sleep disordered breathing
US8897868B2 (en) 2007-09-14 2014-11-25 Medtronic, Inc. Medical device automatic start-up upon contact to patient tissue
US8460189B2 (en) 2007-09-14 2013-06-11 Corventis, Inc. Adherent cardiac monitor with advanced sensing capabilities
US10405809B2 (en) 2007-09-14 2019-09-10 Medtronic Monitoring, Inc Injectable device for physiological monitoring
US10028699B2 (en) 2007-09-14 2018-07-24 Medtronic Monitoring, Inc. Adherent device for sleep disordered breathing
US8790257B2 (en) 2007-09-14 2014-07-29 Corventis, Inc. Multi-sensor patient monitor to detect impending cardiac decompensation
US9770182B2 (en) 2007-09-14 2017-09-26 Medtronic Monitoring, Inc. Adherent device with multiple physiological sensors
US8684925B2 (en) 2007-09-14 2014-04-01 Corventis, Inc. Injectable device for physiological monitoring
US9579020B2 (en) 2007-09-14 2017-02-28 Medtronic Monitoring, Inc. Adherent cardiac monitor with advanced sensing capabilities
US8591430B2 (en) 2007-09-14 2013-11-26 Corventis, Inc. Adherent device for respiratory monitoring
US9538960B2 (en) 2007-09-14 2017-01-10 Medtronic Monitoring, Inc. Injectable physiological monitoring system
US8374688B2 (en) 2007-09-14 2013-02-12 Corventis, Inc. System and methods for wireless body fluid monitoring
US8116841B2 (en) 2007-09-14 2012-02-14 Corventis, Inc. Adherent device with multiple physiological sensors
US9411936B2 (en) 2007-09-14 2016-08-09 Medtronic Monitoring, Inc. Dynamic pairing of patients to data collection gateways
US8285356B2 (en) 2007-09-14 2012-10-09 Corventis, Inc. Adherent device with multiple physiological sensors
US9186089B2 (en) 2007-09-14 2015-11-17 Medtronic Monitoring, Inc. Injectable physiological monitoring system
US8727881B2 (en) 2007-09-25 2014-05-20 Wms Gaming Inc. Accessing wagering game services by aiming handheld device at external device
WO2009042190A1 (en) * 2007-09-25 2009-04-02 Wms Gaming Inc. Accessing wagering game services by aiming handheld device at external device
US8718752B2 (en) 2008-03-12 2014-05-06 Corventis, Inc. Heart failure decompensation prediction based on cardiac rhythm
US8412317B2 (en) 2008-04-18 2013-04-02 Corventis, Inc. Method and apparatus to measure bioelectric impedance of patient tissue
WO2009134909A2 (en) * 2008-04-29 2009-11-05 Bally Gaming, Inc. Biofeedback for a gaming device, such as an electronic gaming machine (egm)
US8308562B2 (en) 2008-04-29 2012-11-13 Bally Gaming, Inc. Biofeedback for a gaming device, such as an electronic gaming machine (EGM)
WO2009134909A3 (en) * 2008-04-29 2010-01-21 Bally Gaming, Inc. Biofeedback for a gaming device, such as an electronic gaming machine (egm)
US20100105478A1 (en) * 2008-10-18 2010-04-29 Hallaian Stephen C Mind-control toys and methods of interaction therewith
US8157609B2 (en) 2008-10-18 2012-04-17 Mattel, Inc. Mind-control toys and methods of interaction therewith
US9612654B2 (en) * 2008-10-20 2017-04-04 Koninklijke Philips N.V. Controlling an influence on a user in a rendering environment
US20110207100A1 (en) * 2008-10-20 2011-08-25 Koninklijke Philips Electronics N.V. Controlling an influence on a user in a rendering environment
US10289201B2 (en) 2008-11-19 2019-05-14 Immersion Corporation Method and apparatus for generating mood-based haptic feedback
US20130225261A1 (en) * 2008-11-19 2013-08-29 Immersion Corporation Method and apparatus for generating mood-based haptic feedback
US9841816B2 (en) * 2008-11-19 2017-12-12 Immersion Corporation Method and apparatus for generating mood-based haptic feedback
US20100323794A1 (en) * 2009-06-18 2010-12-23 Yui-Zhang Su Sensor based human motion detection gaming with false positive detection
WO2011005784A3 (en) * 2009-07-09 2011-05-05 Microsoft Corporation Visual representation expression based on player expression
US8390680B2 (en) 2009-07-09 2013-03-05 Microsoft Corporation Visual representation expression based on player expression
US9519989B2 (en) 2009-07-09 2016-12-13 Microsoft Technology Licensing, Llc Visual representation expression based on player expression
US20110007142A1 (en) * 2009-07-09 2011-01-13 Microsoft Corporation Visual representation expression based on player expression
US9095775B2 (en) 2009-07-17 2015-08-04 Sony Computer Entertainment Europe Limited User interface and method of user interaction
US8562436B2 (en) * 2009-07-17 2013-10-22 Sony Computer Entertainment Europe Limited User interface and method of user interaction
US20120178529A1 (en) * 2009-07-17 2012-07-12 Sony Computer Entertainment Europe Limited User Interface and Method of User Interaction
US7892178B1 (en) * 2009-09-28 2011-02-22 Impact Sports Technologies, Inc. Monitoring device for an interactive game
US8012097B1 (en) 2009-09-28 2011-09-06 Impact Sports Technologies, Inc. Monitoring device for an interactive game
US8002709B1 (en) 2009-09-28 2011-08-23 Impact Sports Technologies, Inc. Monitoring device for an interactive game
US8827915B1 (en) * 2009-09-28 2014-09-09 Impact Sports Technologies, Inc. Monitoring device for an interactive game
US8915858B1 (en) 2009-09-28 2014-12-23 Impact Sports Technologies, Inc. Monitoring device for an interactive game
US8002710B1 (en) 2009-09-28 2011-08-23 Impact Sports Technologies, Inc. Monitoring device for an interactive game
US8790259B2 (en) 2009-10-22 2014-07-29 Corventis, Inc. Method and apparatus for remote detection and monitoring of functional chronotropic incompetence
US9615757B2 (en) 2009-10-22 2017-04-11 Medtronic Monitoring, Inc. Method and apparatus for remote detection and monitoring of functional chronotropic incompetence
US8461468B2 (en) 2009-10-30 2013-06-11 Mattel, Inc. Multidirectional switch and toy including a multidirectional switch
US9451897B2 (en) 2009-12-14 2016-09-27 Medtronic Monitoring, Inc. Body adherent patch with electronics for physiologic monitoring
US20110166940A1 (en) * 2010-01-05 2011-07-07 Searete Llc Micro-impulse radar detection of a human demographic and delivery of targeted media content
US9024814B2 (en) 2010-01-05 2015-05-05 The Invention Science Fund I, Llc Tracking identities of persons using micro-impulse radar
US9019149B2 (en) 2010-01-05 2015-04-28 The Invention Science Fund I, Llc Method and apparatus for measuring the motion of a person
US8884813B2 (en) 2010-01-05 2014-11-11 The Invention Science Fund I, Llc Surveillance of stress conditions of persons using micro-impulse radar
US20110183757A1 (en) * 2010-01-22 2011-07-28 Nintendo Co., Ltd. Storage medium having game program stored therein, game apparatus, control method, and game system
US8449392B2 (en) * 2010-01-22 2013-05-28 Nintendo Co., Ltd. Storage medium having game program stored therein, game apparatus, control method, and game system using a heartbeat for performing a game process in a virtual game world
US20110184248A1 (en) * 2010-01-27 2011-07-28 Nintendo Co., Ltd. Storage medium having stored therein information processing program, information processing device, information processing method, and information processing system
US20110195777A1 (en) * 2010-02-10 2011-08-11 Nintendo Co., Ltd. Storage medium having game program stored thereon, game apparatus, game system, and game processing method
US9604141B2 (en) * 2010-02-10 2017-03-28 Nintendo Co., Ltd. Storage medium having game program stored thereon, game apparatus, game system, and game processing method
US8965498B2 (en) 2010-04-05 2015-02-24 Corventis, Inc. Method and apparatus for personalized physiologic parameters
US9173615B2 (en) 2010-04-05 2015-11-03 Medtronic Monitoring, Inc. Method and apparatus for personalized physiologic parameters
US20110260830A1 (en) * 2010-04-22 2011-10-27 Sony Computer Entertainment Inc. Biometric interface for a handheld device
US9557814B2 (en) * 2010-04-22 2017-01-31 Sony Interactive Entertainment Inc. Biometric interface for a handheld device
US9167975B1 (en) * 2010-07-28 2015-10-27 Impact Sports Technologies, Inc. Motion resistant device to monitor heart rate in ambulatory patients
WO2012036753A1 (en) * 2010-09-17 2012-03-22 Searete Llc Control of an electronic apparatus using micro-impulse radar
US9069067B2 (en) 2010-09-17 2015-06-30 The Invention Science Fund I, Llc Control of an electronic apparatus using micro-impulse radar
US9261967B2 (en) 2011-02-21 2016-02-16 Koninklijke Philips N.V. Gesture recognition system
US9058716B2 (en) 2011-06-06 2015-06-16 Bally Gaming, Inc. Remote game play in a wireless gaming environment
US9898889B2 (en) 2011-06-06 2018-02-20 Bally Gaming, Inc. Remote game play in a wireless gaming environment
US9591973B1 (en) 2011-06-13 2017-03-14 Impact Sports Technologies, Inc. Monitoring device with a pedometer
US9820659B1 (en) 2011-06-13 2017-11-21 Impact Sports Technologies, Inc. Monitoring device with a pedometer
US20130234933A1 (en) * 2011-08-26 2013-09-12 Reincloud Corporation Coherent presentation of multiple reality and interaction models
US9274595B2 (en) 2011-08-26 2016-03-01 Reincloud Corporation Coherent presentation of multiple reality and interaction models
US8963916B2 (en) 2011-08-26 2015-02-24 Reincloud Corporation Coherent presentation of multiple reality and interaction models
US20130057576A1 (en) * 2011-09-02 2013-03-07 Inventec Corporation Display method and electronic device using the same
US9658617B1 (en) 2012-06-29 2017-05-23 Monkeymedia, Inc. Remote controlled vehicle with a head-mounted display
US9919233B2 (en) 2012-06-29 2018-03-20 Monkeymedia, Inc. Remote controlled vehicle with augmented reality overlay
US9563202B1 (en) 2012-06-29 2017-02-07 Monkeymedia, Inc. Remote controlled vehicle with a head-mounted display apparatus
US9612627B2 (en) 2012-06-29 2017-04-04 Monkeymedia, Inc. Head-mounted display apparatus for navigating a virtual environment
US9791897B2 (en) 2012-06-29 2017-10-17 Monkeymedia, Inc. Handheld display device for navigating a virtual environment
US9782684B2 (en) 2012-06-29 2017-10-10 Monkeymedia, Inc. Remote controlled vehicle with a handheld display device
US9656168B1 (en) 2012-06-29 2017-05-23 Monkeymedia, Inc. Head-mounted display for navigating a virtual environment
US9579586B2 (en) 2012-06-29 2017-02-28 Monkeymedia, Inc. Remote controlled vehicle with a handheld display device
US8992324B2 (en) * 2012-07-16 2015-03-31 Wms Gaming Inc. Position sensing gesture hand attachment
US20140018166A1 (en) * 2012-07-16 2014-01-16 Wms Gaming Inc. Position sensing gesture hand attachment
US20140135960A1 (en) * 2012-11-15 2014-05-15 Samsung Electronics Co., Ltd. Wearable device, display device, and system to provide exercise service and methods thereof
US20140234815A1 (en) * 2013-02-18 2014-08-21 Electronics And Telecommunications Research Institute Apparatus and method for emotion interaction based on biological signals
US20140287387A1 (en) * 2013-03-24 2014-09-25 Emozia, Inc. Emotion recognition system and method for assessing, monitoring, predicting and broadcasting a user's emotive state
US20150116214A1 (en) * 2013-10-29 2015-04-30 Anders Grunnet-Jepsen Gesture based human computer interaction
US9304597B2 (en) * 2013-10-29 2016-04-05 Intel Corporation Gesture based human computer interaction
WO2015087323A1 (en) * 2013-12-09 2015-06-18 Mantisvision Ltd Emotion based 3d visual effects
US20150178915A1 (en) * 2013-12-19 2015-06-25 Microsoft Corporation Tagging Images With Emotional State Information
US20160023116A1 (en) * 2014-07-03 2016-01-28 Spitfire Technologies, Llc Electronically mediated reaction game
US9629562B1 (en) 2014-07-25 2017-04-25 Impact Sports Technologies, Inc. Mobile plethysmographic device
WO2016146536A1 (en) * 2015-03-13 2016-09-22 Hanne Storm Method and apparatus for controlling a user's interaction with a non-medical application
US10293260B1 (en) * 2015-06-05 2019-05-21 Amazon Technologies, Inc. Player audio analysis in online gaming environments
US10021344B2 (en) 2015-07-02 2018-07-10 Krush Technologies, Llc Facial gesture recognition and video analysis tool
US20180314321A1 (en) * 2017-04-26 2018-11-01 The Virtual Reality Company Emotion-based experience feedback
US10152118B2 (en) * 2017-04-26 2018-12-11 The Virtual Reality Company Emotion-based experience freedback

Also Published As

Publication number Publication date
TW200724204A (en) 2007-07-01
TWI311067B (en) 2009-06-21

Similar Documents

Publication Publication Date Title
US10238978B2 (en) Game operating device
US8576171B2 (en) Systems and methods for providing haptic feedback to touch-sensitive input devices
CN105264460B (en) Hologram object is fed back
CN100397304C (en) Hand-held computer interactive device
CN101837191B (en) Game controller and game system
JP4884867B2 (en) Information processing apparatus and information processing program
US20110212781A1 (en) Video game using dual motion sensing controllers
US8259072B2 (en) Input control apparatus and an interactive system using the same
JP3753744B2 (en) Force feedback interface with isotonic and isometric functions
JP5669336B2 (en) 3D viewpoint and object designation control method and apparatus using pointing input
US8870655B2 (en) Wireless game controllers
US20100004896A1 (en) Method and apparatus for interpreting orientation invariant motion
KR20120026072A (en) Gesture shortcuts
KR100948095B1 (en) Motion-input device for a computing terminal and method of its operation
US8308563B2 (en) Game system and storage medium having game program stored thereon
US20080268956A1 (en) Game system and information processing system
US20140313134A1 (en) Image processing apparatus and storage medium storing image processing program
US9327192B2 (en) Game system and storage medium storing game program
US20120157263A1 (en) Multi-user smartglove for virtual environment-based rehabilitation
KR101169813B1 (en) Game system and storage medium having game program stored thereon
TWI408572B (en) Information processing device, input device, information processing system, information processing method and computer readable storage medium
EP2016360B1 (en) System and method for detecting moment of impact and/or strength of a swing based on accelerometer data
US7833099B2 (en) Game apparatus and recording medium recording game program for displaying a motion matching a player's intention when moving an input device
Wingrave et al. The wiimote and beyond: Spatially convenient devices for 3d user interfaces
US20150070274A1 (en) Methods and systems for determining 6dof location and orientation of head-mounted display and associated user movements

Legal Events

Date Code Title Description
AS Assignment

Owner name: INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE, TAIWAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LU, YING-KO;TSAI, MING-JYE;LIOU, SHUN-NAN;AND OTHERS;REEL/FRAME:018298/0186;SIGNING DATES FROM 20060306 TO 20060309

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION