JP2009535173A - Three-dimensional input control system, method, and apparatus - Google Patents

Abstract

A game controller, a tracking device, and an associated device used when acquiring information for controlling execution of a game program by a processor to enable a user to play an interactive game. Disclosed. A method for tracking a video game controller and providing input to the system is also disclosed.
[Selection] Figure 5A

Description

  The present invention relates generally to computer entertainment systems, and more specifically to the operation of a controller of such a computer entertainment system by a user.

  Computer entertainment systems typically include handheld controllers, game controllers, or other controllers. A user or player uses a controller to send commands or other instructions to the entertainment system to control the video game or other simulation being played. For example, the controller may be provided with an operation unit such as a joystick operated by the user. The manipulated variable of the joystick is converted from an analog value to a digital value and transmitted to the main frame of the game machine. The controller may be provided with buttons that can be operated by the user.

  The present invention has been developed with respect to these and other background information elements.

1 is a diagram illustrating a video game system that operates in accordance with an embodiment of the present invention. FIG. It is a perspective view of the controller concerning an embodiment of the invention. It is a three-dimensional schematic diagram showing an accelerometer used in the controller according to the embodiment of the present invention. FIG. 3B is a schematic cross-sectional view showing a state where the accelerometer of FIG. 3A is rotated around a pitch or roll axis. FIG. 3B is a schematic cross-sectional view showing a state in which translational acceleration is applied to the accelerometer of FIG. 3A. FIG. 3B is a schematic plan view showing a state where rotational acceleration around the yaw axis is given to the accelerometer of FIG. 3A. FIG. 3B is a schematic plan view showing a state where rotational acceleration around the yaw axis is given to the accelerometer of FIG. 3A. FIG. 4 is a three-dimensional schematic diagram illustrating correction of a direction-dependent zero point accelerometer signal according to an embodiment of the present invention. It is a block diagram of a part of the video game system of FIG. FIG. 5 is a flow diagram of a method for tracking a controller of a video game system according to an embodiment of the present invention. FIG. 3 is a flow diagram illustrating a method for utilizing position and / or orientation information during game play in a video game system according to an embodiment of the present invention. 1 is a block diagram illustrating a video game system according to an embodiment of the present invention. It is a block diagram of the implementation by the cell processor of the video game system which concerns on embodiment of this invention.

Priority claim This application is based on US patent application Ser. No. 11 / 381,729 (inventor: Shadon Mao, title of invention: “miniature microphone array”, agent case number: SCEA05062US00, filing date: May 2006. 4th), US patent application 11 / 381,728 (inventor: Shadon Mao, title of invention: "Echo and noise cancellation", agent case number: SCEA05064US00, filing date: May 4, 2006), US Patent application 11 / 381,725 (inventor: Shadon Mao, title of invention: “target speech detection method and apparatus”, agent case number: SCEA05072US00, filing date: May 4, 2006), US patent application 11 / 381,727 (inventor: Shadon Mao, title of invention: “electronics by far-field microphone on console Device Noise Reduction ”, Agent Case Number: SCEA05073US00, Filing Date: May 4, 2006, US Patent Application 11 / 381,724 (Inventor: Shadon Mao, Title of Invention:“ Target Voice Detection and Characters ” "Method and apparatus for tectarization", agent case number: SCEA05079US00, filing date: May 4, 2006, US patent application 11 / 381,721 (inventor: Shadon Mao, title of invention: "computer dialogue All rights are hereby incorporated by reference: “Selective sound source listening in conjunction with processing”, agent case number: SCEA04005JUMBOUS, filing date: May 4, 2006).

  This application is further related to US patent application 11 / 382,031 (inventor: Gary Zalewski et al., Title of invention: “multi-input game control mixer”, agent case number: SCEA06MXR1, filed May 6, 2006). US patent application 11 / 382,032 (inventor: Gary Zalewski et al., Title of invention: "system for tracking user actions in the environment", agent case number: SCEA06MXR2, filed May 6, 2006. ) And are incorporated herein by reference.

  This application is further related to co-pending US patent application 11 / 418,988 (inventor: Shadon Mao, title of invention: “Method and apparatus for adjusting listening area for obtaining sound”, agent Case number: SCEA-00300, filing date: May 4, 2006), the entire disclosure of which is incorporated herein by reference. This application further includes co-pending U.S. patent application 11 / 418,989 (inventor: Shadon Mao, title of invention: "Method and apparatus for acquiring audio signals based on visual images", agent case. No .: SCEA-00400, filing date: May 4, 2006), the entire disclosure of which is incorporated herein by reference. This application is further related to co-pending U.S. Patent Application 11 / 429,047 (inventor: Shadon Mao, title of invention: “Method and Apparatus for Acquiring Audio Signal Based on Signal Position”, Attorney (Case Number: SCEA-50050, filing date: May 4, 2006), the entire disclosure of which is incorporated herein by reference. This application is further related to co-pending US patent application 11 / 429,133 (inventor: Richard L. Marks, title: “selective sound source listening in conjunction with computer interaction processing”, agent case number: SCEA04005US01. -SONYP045, filing date: May 4, 2006), the entire disclosure of which is incorporated herein by reference. This application is further related to co-pending U.S. Patent Application 11 / 429,414 (inventor: Richard L. Marks, title of invention: "input device for computer image and sound intensity processing and interface with computer program") , Agent case number: SONYP052, filing date: May 4, 2006), the entire disclosure of which is incorporated herein by reference.

  This application is further related to co-pending US patent application 11 / 382,033 (invention name: “3D input control system, method and apparatus”, agent case number: SCEA06INRT1, filing date: May 6, 2006). ), The entire disclosure of which is incorporated herein by reference. This application is further related to co-pending US patent application 11 / 382,035 (Title of Invention: “Inertically Traceable Handheld Computer”, Agent Case Number: SCEA06INRT2, Filing Date: May 6, 2006) The entire disclosure of which is hereby incorporated by reference. This application is further related to co-pending U.S. Patent Application 11 / 382,036 (Title of Invention: “Method and Apparatus for Applying Gearing Effect to Visual Tracking”, Agent Case Number: SONYP058A, Filing Date: 2006 The disclosure of which is hereby incorporated by reference. This application is further related to co-pending US patent application 11 / 382,041 (Title of Invention: “Method and Apparatus for Applying Gearing Effect to Inertia Tracking”, Attorney Case Number: SONYP058B, Filing Date: 2006 The disclosure of which is hereby incorporated by reference. This application is further related to co-pending US patent application 11 / 382,038 (Title of Invention: “Method and Apparatus for Applying Gearing Effect to Acoustic Tracking”, Agent Case Number: SONYP058C, Filing Date: 2006 The disclosure of which is hereby incorporated by reference. This application further includes co-pending US patent application 11 / 382,040 (Title of Invention: “Method and Apparatus for Applying Gearing Effects to Multi-Channel Mixed Inputs”, Attorney Case Number: SONYP058D, Filing Date. : May 6, 2006), the entire disclosure of which is incorporated herein by reference. This application is further related to co-pending US patent application 11 / 382,034 (invention name: “mechanism for detecting and tracking user operation of game controller body”, agent case number: SCEA05082US00, filing date: 2006). The disclosure of which is hereby incorporated by reference. This application is further related to co-pending U.S. Patent Application 11 / 382,037 (Title of Invention: "Mechanism for Converting Handheld Computer Motion into Input to System", Agent Case Number: 86324, Filing Date: (6 May 2006), the entire disclosure of which is incorporated herein by reference. This application is further related to co-pending US patent application 11 / 382,043 (Title of Invention: “Detectable and Traceable Handheld Computer”, Agent Case Number: 86325, Filing Date: May 6, 2006) The entire disclosure of which is hereby incorporated by reference. This application is further related to co-pending U.S. Patent Application 11 / 382,039 (Title: “Method for Mapping Handheld Computer Movements to Game Commands”, Agent Case Number: 86326, Filing Date: 2006) May 6), the entire disclosure of which is incorporated herein by reference. This application is further related to co-pending U.S. patent application 29 / 259,349 (invention name: “controller with trademark infrared port”, agent case number: SCEA06007US00, filing date: May 6, 2006) The entire disclosure of which is hereby incorporated by reference. This application is further related to co-pending U.S. Patent Application 29 / 259,350 (Invention name: "Controller with tracking sensor (TM)", agent case number: SCEA06008US00, filing date: May 6, 2006) The entire disclosure of which is hereby incorporated by reference. This application further gives priority to co-pending US patent application 60 / 798,031 (Title of Invention: “Dynamic Object Interface”, Agent Case Number: SCEA06009US00, Filing Date: May 6, 2006). All the disclosures of which are hereby incorporated by reference. This application is further related to co-pending US patent application 29 / 259,348 (Title of Invention: "Tracked Controller Device (TM)", Agent Case Number: SCEA06010US00, Filing Date: May 6, 2006) The entire disclosure of which is hereby incorporated by reference.

-Cross-reference to related applications This application is based on U.S. Patent Application 10 / 207,677 (Title of Invention: "Man Machine Interface Using Deformable Device", Filing Date: July 27, 2002), US Patent Application 10 / 650,409 (Title of Invention: “Voice Input System”, Filing Date: August 27, 2003), US Patent Application 10 / 663,236 (Title of Invention: “Tracked Head Movements” Method and apparatus for adjusting the field of view of a displayed scene according to US patent application Ser. No. 10 / 759,782 (Title of Invention: “Method for optical input device” And device ", filing date: January 16, 2004), U.S. patent application 10 / 820,469 (invention name:" method and device for detecting and eliminating sound disturbance ", filing date: April 2004) 7th ), U.S. Patent Application No. 11 / 301,673 (Title of Invention: “Method for Realizing Pointing Interface via Camera Tracking Using Relative Position of Head and Hand”, Filing Date: December 12, 2005) , U.S. Patent Application 60 / 718,145 (Title of Invention: “Examples of Audio, Video, Simulation, and User Interface”, filing date: September 15, 2005), the entire disclosure of which is here Incorporated as a reference.

DESCRIPTION OF SPECIFIC EMBODIMENTS The following detailed description includes a number of specific details for purposes of illustration, but it will be apparent to those skilled in the art that many variations and permutations to the details below are within the scope of the invention. It is being recognized. Accordingly, the embodiments of the invention described below are described without losing the generality of the claimed invention and without adding limitations.

  Various embodiments of the methods, apparatus, mechanisms and systems described herein provide for detection, acquisition, and tracking of movement, movement, and / or operation by a user of the entire controller. The detected movement, movement, and / or manipulation of the entire controller by the user may be used as further commands to control various aspects of the game being played or other simulations.

  The detection and tracking of the operation by the user of the game controller main body can be realized by various methods. For example, in one embodiment, inertial sensors such as accelerometers or gyroscopes can be used with a computer entertainment system to detect movements of the handheld controller body and convert them into actions in the game. Inertial sensors may be used to detect many different types of movements of the controller, such as up / down movement, twisting, left / right movement, pulling, swinging motion, sticking out, and the like. These movements may be associated with various commands in order to translate into actions in the game.

  The detection and tracking of game controller operations by the user can, for example, engage with swords and light sabers, use sticks to trace the shape of items, fight in various sports competitions, on-screen battles and other encounters The present invention can be used to realize various types of games, simulations, and the like that enable the user to fight in battles.

  FIG. 1 illustrates a system 100 that operates in accordance with an embodiment of the present invention. As shown, the computer entertainment console 102 may be connected to a television or other video display device 104 for displaying images of a video game or other simulation. The game or other simulation may be stored on a DVD, CD, flash memory, USB memory, or other storage medium 106 inserted into the console 102. A user or player 108 operates a game controller 110 to control a video game or other simulation. In FIG. 2, the game controller 110 includes an inertial sensor 112 that generates a signal in response to the position, movement, direction, or change in direction of the game controller 110. In addition to the inertial sensor, the game controller 110 may include a conventional control input device such as a joystick 111, buttons 113, R1, and L1.

  In operation, the user 108 physically moves the controller 110. For example, the user 108 may move the controller 110 in any direction, for example, up, down, left, and right, twist, rotate, shake, pull, or poke. These movements of the controller 110 themselves may be detected and acquired by the camera 114 for tracking through analysis of signals from the inertial sensor 112 in the manner described below.

  Referring again to FIG. 1, the system 100 may optionally include a camera or other video acquisition device 114. The camera may be provided at a position where the controller 110 is within the field of view 116 thereof. Analysis of the image from the video acquisition device 114 may be used in connection with analysis of data from the inertial sensor 112. As shown in FIG. 2, the controller 110 may be selectively provided with light sources such as LEDs 202, 204, 206, 208 to facilitate tracking through video analysis. Such video analysis aimed at tracking the controller 110 is described, for example, in US patent application Ser. No. 11 / 382,034 (Title for Invention: “Mechanism for Detecting and Tracking User Operation of Game Controller Body”. ", Agent case number: SCEA05082US00), incorporated herein by reference. Console 102 may include a microphone array 118. The controller 110 may include an acoustic signal generator 210 (eg, a speaker) to provide a sound source for facilitating acoustic tracking and acoustic signal processing of the controller 110 by the microphone array 118. That technique is described, for example, in US patent application Ser. No. 11 / 381,724, incorporated herein by reference.

  In general, signals from inertial sensors are used to generate position and orientation data for controller 110. These data are used to calculate a number of physical aspects of the movement of the controller 110, eg, any telemetry data of the controller 110 such as acceleration and velocity along any axis, tilt, gradient, yaw, rotation, etc. May be. Here, telemetry generally refers to telemetry and reporting of the information of interest to a system or system designer or operator.

  By detecting and tracking the movement of the controller 110, it can be determined whether a predefined movement of the controller 110 has been performed. That is, a specific movement pattern or gesture of the controller 110 can be defined in advance and used as an input command for a game or other simulation. For example, a gesture that pushes down the controller 110 can be defined as one command, a gesture that twists the controller 110 can be defined as another command, and a gesture that shakes the controller 110 can be defined as another command. In this way, using the method in which the user 108 physically moves the controller 110 as an input to control the game can provide a more exciting and entertaining experience for the user.

  The inertial sensor 112 may be an accelerometer, for example, but is not limited thereto. FIG. 3A shows an example of an accelerometer 300 in the form of a simple mass 302 that is elastically coupled to the frame 304 at four points, eg, by springs 306, 308, 310, 312. The pitch axis and roll axis (indicated by X and Y, respectively) lie on a plane that intersects the frame. The yaw axis Z is a direction perpendicular to a plane including the pitch axis X and the roll axis Y. The frame 304 may be mounted on the controller 110 in any suitable manner. As frame 304 (and joystick controller 110) is accelerated and / or rotated, mass 302 is displaced relative to frame 304 and springs 306, 308, 310, 312 are subject to translational and / or rotational acceleration. Stretch depending on value and direction and / or pitch and / or roll and / or yaw angle. The amount of displacement of the mass 302 and / or the amount of expansion or contraction of the springs 306, 308, 310, 312 can be sensed by suitable sensors 314, 316, 318, 320, for example, and pitch and / or roll in a known or determinable manner. It is converted into a signal that depends on the acceleration.

  There are many different ways to track the position of the mass and / or the force applied to it. These methods include resistive strain gauge materials, optical sensors, magnetic sensors, Hall effect devices, piezoelectric devices, capacitive sensors, and the like. Embodiments of the present invention may include any number and type of sensors, and may include combinations of multiple types of sensors. Sensors 314, 316, 318, 320 may be gap electrodes disposed on mass 302. The capacitance between the mass and each electrode changes as the mass position changes relative to each electrode. Each electrode may be connected to a circuit that generates a signal related to the capacitance (and proximity) of the mass 302 to the electrode. Further, the springs 306, 308, 310, 312 may include resistive strain gauge sensors that generate signals related to spring expansion and contraction.

  In some embodiments, the frame 304 may be a gimbal that is mounted on the controller 110 in order for the accelerometer 300 to maintain a fixed orientation with respect to the pitch and / or roll and / or yaw axes. According to this, it is possible to map the X, Y, and Z axes of the controller directly to the corresponding axes in the real world without considering the inclination of the controller axis with respect to the coordinate axes in the real world.

  3B-3D show examples of different expansion and contraction of springs 306, 308, 310, 312 under different conditions of acceleration and / or rotation. Specifically, FIG. 3B shows a situation where the frame 304 has rotated about the Y axis. Due to the weight acting on the mass 302, the springs 306, 310 extend and the mass 302 approaches the sensors 314, 318 and moves away from the sensors 316, 320. Rotating in the opposite direction about the Y (roll) axis similarly causes the springs 306, 310 to expand, but the mass approaches the sensors 316, 320 and away from the sensors 314, 318. Similarly, rotation about the X (pitch) axis causes the springs 308, 312 to expand and the mass approaches the sensor 314, 316 and away from the sensor 318, 320, depending on the direction of rotation.

  FIG. 3C shows a situation in which the frame 304 is rapidly accelerated downward (indicated by an arrow) while maintaining a horizontal plane. In this situation, all four springs 306, 308, 310, 312 are extended and the mass approaches all four sensors 314, 316, 318, 320. In FIG. 3D, the frame 304 is accelerated to the left (indicated by an arrow) while maintaining a horizontal plane. In this situation, the springs 306, 308, and 312 are extended and the spring 310 is compressed. The mass 302 moves away from the sensors 314, 318 and approaches the sensors 316, 320. FIG. 3E shows that the frame 304 is given angular acceleration about the Z (yaw) axis, all four springs 306, 308, 310, 312 are extended, and the mass 302 is four sensors 314, 316, 318, 320. The situation that keeps away from everything. As can be seen from FIGS. 3B-3E, the motion and / or direction of different frames 304 produces a particular signal combination, so by analyzing the signal combination, the direction and / or frame 304 (and controller 110) Or the movement can be determined.

In the absence of an external force acting on the mass 302, the displacement along the Z axis from the stable position of the mass 302 is approximately proportional to the acceleration along the Z axis. The detectors 314, 316, 318, 320 produce a signal that is proportional to the displacement of the mass 302 and thus proportional to the acceleration of the frame 304 (and the controller 110) along the Z axis. Similarly, signals from sensors may be used to estimate acceleration along the X and Y axes. It should be noted here that if gravity acts on the mass 302, the sensors 314, 316, 318, 320 will generate a non-zero signal. For example, in steady state, when no pitch or roll is applied to the joystick controller, the Z axis coincides with the vertical axis (determined by gravity). Gravity moves mass 302 from the assumed position in the absence of gravity. As a result of the displacement, the sensor generates a signal V ₀ is not zero. Here, this is called a “zero point” acceleration signal. The zero point acceleration signal V ₀ is typically subtracted from the accelerometer signal V before analyzing the raw signal from the sensors 314, 316, 318, 320.

If the frame 304 (and controller 110) if maintained in the same direction with respect to pitch and roll the zero-point acceleration signal V ₀ is constant. However, the zero point acceleration signal V ₀ depends on the pitch and the amount of rotation about the roll axis. Embodiments of the present invention takes into account the effects of pitch and roll with respect to the zero-point acceleration signal V _0. For example, FIG. 4 shows the situation for a single axis accelerometer 400 having a mass 402 forced to move in a tube 404 along the Z axis. Spring 406 connects to mass 402 at the bottom of tube 404. The sensor 408 is, for example, the above-described capacitance sensor. The pitch and roll of the tube 404, the tube axis Z is vertical Z 'inclined angle θ with respect to a (shown by a broken line), "rotated" zero-point acceleration signal V _0' is the following to V ₀ and θ It seems to be related.
V ₀ '= V ₀ cos θ
Note that in the extreme case of θ = 90 °, V ₀ ′ = 0.

  The angle θ generally depends on the pitch and roll angle. These may be determined by signals from separate sensors. The unit vector z along the tube axis Z may consist of a known absolute value of the pitch and roll relative to a known initial direction, such as the direction in which the axis of the accelerometer coincides with the unit vector z ′ along the vertical axis. . The initial direction may be any direction of the joystick controller that generates an invariant signal from the accelerometer 400. The inner product of the unit vectors z and z 'is given by the cosine of the angle θ between them. This inner product may be multiplied by the zero point signal V 0 and subtracted from the acceleration signal obtained from the sensor 408 to provide the desired correction factor.

  In the sensor of the present embodiment, various types of inertial sensor devices provide information on six degrees of freedom (for example, translation in the X, Y, and Z directions and rotation about the X, Y, and Z axes). May be used for Examples of suitable inertial sensors to provide six degrees of freedom information include accelerometers of the type shown in FIG. 3A, one or more single axis accelerometers, mechanical gyroscopes, ring laser gyroscopes, or the like Includes two or more combinations.

  The signal from the sensor may be analyzed to determine the movement and / or direction of the controller 110 during video game play according to the inventive method. Such a method may be implemented as a series of processors capable of executing program code instructions stored in a processor readable medium and executed on a digital processor. For example, as shown in FIG. 5A, the video game system 100 may include a processor 502 on the console 102. The processor 502 may be any suitable digital processor unit such as a microprocessor of the type commonly used in video game consoles. The processor may realize the inertia analysis unit 504 by executing a processor-readable instruction. Some of the instructions may be stored in the memory 506. In another example, the inertia analysis unit 504 may be realized by hardware such as an ASIC. The hardware of such an analysis unit may be arranged in the controller 110 or the console 102, or may be arranged apart from other places. In a hardware implementation, the analysis unit 504 may be programmable in response to external signals from the processor 502, other remotely located signal sources connected by USB cable, wireless, or network.

  The inertial analysis unit 504 may include a command that analyzes the signal generated by the inertial sensor 112 and uses information regarding the position and / or orientation of the controller 110. For example, as shown in the flow diagram 510 of FIG. 5B, the signal may be generated by the inertial sensor 112 as shown in block 512. As indicated at block 514, the inertial sensor signal may be analyzed to determine information regarding the position and / or orientation of the controller 110. The position / and / or orientation information may be utilized during video game play in the system 100, as shown in block 516.

  In certain embodiments, position and / or orientation information may be used in connection with gestures made by the user 108 during game play. As shown in the flow diagram 520 of FIG. 5C, the path of the controller 110 may be tracked using position and / or orientation information, as shown in block 522. In a non-limiting example, the path may include a series of points that indicate the position of the center of gravity of the controller with respect to a certain coordinate system. Each position point may be represented by one or more coordinates such as X, Y, and Z coordinates in an orthogonal coordinate system. In order to be able to monitor both the shape of the path and the progress of the controller along the path, time may be associated with each point on the path. Further, each point on the path may be associated with data indicating the direction of the controller, eg, one or more rotation angles with respect to the center of gravity of the controller. Furthermore, each point on the path may be related to the speed and acceleration of the center of gravity of the controller, the angular speed and acceleration with respect to the center of gravity of the controller,

  As shown in block 524, the tracked path is compared with one or more stored paths corresponding to known and / or pre-recorded gestures 508 associated with the status of the video game being played. Also good. The analysis unit 504 may be configured to recognize a user or process a sound-authenticated gesture or the like. For example, the user may be identified by the analysis unit 504 using a gesture and that the gesture can identify the user. Such a specific gesture may be recorded and included in a gesture 508 pre-recorded in the memory 506. In the recording process, sounds generated during gesture recording may be further stored. The detected environment is sampled and processed by the multi-channel analyzer. The processing unit may refer to the gesture model to determine, authenticate, and / or identify a user or object with high accuracy and performance based on voice or acoustic patterns.

  As shown in FIG. 5A, gestures may be stored in memory 506. Non-limiting examples of gestures include, for example, throwing an object such as a ball, shaking an object such as a bat or golf club, moving a manual pump, opening or closing a door or window, rotating a steering wheel or other vehicle control means , Martial arts action such as punch, sanding, waxing, waxing, painting the house, shaking, rattling, turning, throwing football, turning the knob, 3D mouse action, scrolling action, known contour Motion, any recordable motion, back-and-forth motion along any vector, such as inflating the tire, but with any rotation in space, motion along the path, recorded within the noise level Have any exact stop and start times based on user actions that can be tracked and repeated Action, including action to put a key. Each of these gestures may be pre-recorded from the pass data and stored as a time based model. The comparison between the pass and the stored gesture may begin with a stable state assumption, and if the pass goes out of the stable state, the stored gesture may be compared with the pass through an erasure process. If there is no match at block 526, the analyzer 504 may continue tracking the controller's path at block 522. If there is a sufficient match between the pass (or part of it) and the stored gesture, the game state may be changed as shown at 528. Changing the game state includes, but is not limited to, interrupting, sending control signals, changing variables, and the like.

  Here is an example where this can happen. If it is determined that the controller 110 is out of the stable state of the path, the analysis unit 504 tracks the operation of the controller 110. As long as the path of the controller 110 satisfies the path defined in the stored gesture model 508, these gestures may “hit”. If the controller 110 path deviates from any gesture model 508 (with a set noise tolerance), that gesture model is removed from the hit list. Each gesture reference model includes a time base in which gestures are recorded. The analysis unit 504 compares the controller pass data with the stored gesture 508 at an appropriate time index. The clock is reset by the occurrence of a steady state condition. When out of steady state (ie, when an action is tracked beyond the noise threshold), the hit list is populated with all possible gesture models. The clock is started and the controller movement is compared to the hit list. Again, the comparison is made over time. When any gesture in the hit list reaches the end of the gesture, it becomes a hit.

  In an embodiment, the analysis unit 504 may notify the game program when a specific event occurs. Examples of such events include:

-Zero acceleration point arrival interrupt (X, Y, and / or Z axis): In a certain game situation, the analysis unit 504 notifies or interrupts a routine in the game program when the acceleration of the controller reaches an inflection point. You may spend. For example, the user 108 may use the controller 110 to control a game avatar that represents a quarterback in a football simulation game. The analysis unit 504 may track a controller representing football via a path generated from the signal from the inertial sensor 112. A specific change in the acceleration of the controller 110 may signal a football release. In this regard, the analyzer can trigger another routine in a program, such as a physics simulation package, that simulates a football trajectory based on the controller's position at release and / or speed and / or direction. Also good.

-Interruption of a recognized new gesture.

  Further, the analysis unit 504 may be set by one or more inputs. Examples of such inputs include:

-Noise level setting (X, Y or Z axis). The noise level may be a reference tolerance used when analyzing a user's hand trembling in a game.

・ Sampling rate setting. Here, the sampling rate refers to the frequency with which the analysis unit 504 samples a signal from the inertial sensor. The sampling rate may be set to oversample or average the signal.

・ Gearing setting. Here, gearing generally refers to the ratio between the movement of the controller and the movement that occurs in the game. An example of such “gearing” in video game control is described in US patent application 11 / 382,040 (filed May 7, 2006, agent case number: SONYP058D), which is incorporated herein by reference. Incorporated.

・ Mapping chain settings. Here, the mapping chain refers to gesture model mapping. Gesture model mapping may be adapted to a specific input channel (eg, path data generated only from inertial sensor signals), or a mixed channel generated in the mixing section. Three input channels may be provided by two or more different analyzers similar to inertial analyzer 504. These include, among other things, the inertial analysis section 504 described herein, eg, US patent application 11 / 382,034 (invention name: “mechanism for detecting and tracking user operation of game controller body”, agent case number). : SCEA05082US00, incorporated herein by reference), for example, an audio analyzer described in US patent application 11 / 381,721 (incorporated herein by reference). A mapping chain may be incorporated in the analysis unit. The mapping chain may be swapped out by the game during the game and set for the analysis unit and the mixing unit.

  Referring again to FIG. 5B, those skilled in the art will recognize that there are many ways to generate a signal from the inertial sensor 112 at block 512. Some examples are described above in connection with FIGS. 3a-3E. Referring to block 514, there are many ways to analyze the detection signal generated in block 512 to obtain information related to the position and / or orientation of the controller 110. For example, the position and / or direction information may include the following parameters individually or in any combination.

• Controller direction. The direction of the controller 110 may be expressed in radians, for example, with respect to pitch, roll, or yaw angle with respect to a reference direction. The rate of change in the direction of the controller (eg, angular velocity or angular acceleration) may be included in the position and / or direction information. If the inertial sensor 112 includes a gyroscope sensor, controller orientation information may be obtained directly in the form of one or more output values proportional to pitch, roll, or yaw angle.

The position of the controller (for example, the orthogonal coordinates X, Y, Z of the controller 110 in a certain reference frame).

• Controller X-axis speed.

• Y-axis speed of the controller.

・ Z-axis speed of the controller.

• X-axis acceleration of the controller.

• Y-axis acceleration of the controller.

• Z-axis acceleration of the controller.

  Regarding the position, velocity, and acceleration, the position and / or direction information may be expressed in a coordinate system other than the orthogonal coordinate system. For example, cylindrical coordinates or old coordinates may be used for position, velocity and acceleration. Acceleration information about the X, Y, and Z axes may be obtained directly from a sensor in the form of an accelerometer, eg, as described above with reference to FIGS. 3A-3E. The acceleration in the X, Y, and Z directions may be integrated over time from some initial time to determine the change in velocity in the X, Y, and Z directions. These speeds may be calculated by adding the change in speed to the known speeds in the X, Y, and Z directions at the initial time. The velocity in the X, Y and Z directions may be integrated over time to determine the displacement of the controller in the X, Y and Z directions. The positions in the X, Y, and Z directions may be determined by adding displacement to the known positions in the X, Y, and Z directions at the initial time.

-Stable state (Y / N). This special information indicates whether the controller is in a stable state. This information may be defined at any location and is subject to change. In a preferred embodiment, the steady state position may be a state where the controller is held in a higher or lower level direction, approximately at the same height as the user's waist.

The time from the last stable state is generally data related to the time elapsed since the last stable state was detected. This time determination may be calculated at the actual time, as described above, or at the processor frequency or sampling period. The time data from the last steady state is important for resetting the controller tracking with respect to the initial position to ensure the accuracy of the character or object mapping in the game environment. This data is also important for determining possible actions or gestures that will be subsequently and exclusively executed in the gaming environment.

  The last recognized gesture is generally the gesture recognized last by the gesture recognition unit 505 realized by hardware or software. The identification of the last recognized gesture is important with respect to the fact that the previous gesture may be related to a gesture that may subsequently be recognized or other actions performed in the gaming environment.

・ The last time the gesture was recognized.

  The above output may be sampled at an arbitrary timing by a game program or software.

  According to an embodiment of the present invention, a video game system and method of the type described above is implemented as shown in FIG. Video game system 600 may include a processor 601 and memory 602 (eg, RAM, DRAM, ROM, etc.). Furthermore, the video game system 600 may include a plurality of processors 601 when parallel processing is implemented. Memory 602 includes data and game program code 604 including portions configured as described above. In particular, the memory 602 may include inertial signal data 606 including path information stored in the controller described above. The memory 602 may further include stored gesture data 608, such as data indicating one or more gestures associated with the game program 604.

  The system 600 may further include known support functions such as input / output (I / O) elements 611, a power supply (P / S) 612, a clock (CLK) 613, and a cache 614. The device 600 may include a mass storage device 615, such as a disk drive, CD-ROM drive, tape drive, etc., for storing programs and / or data. The controller may further include a display unit 616 and a user interface unit 618 to facilitate interaction between the controller 600 and the user. The display unit 616 may be in the form of a cathode ray tube (CRT) or flat display that displays text, numbers, display symbols, or images. User interface 618 may include a keyboard, mouse, joystick, light pen, or other device. In addition, the user interface 618 may include a microphone, video camera, or other signal conversion device to directly acquire the signal to be analyzed. As shown in FIG. 6, processor 601, memory 602, and other system 600 components may exchange signals (eg, code instructions and data) with each other via system bus 620.

  Microphone array 622 may be connected to system 600 via input / output function 611. The microphone array may include about 2 to 8, preferably about 4, microphones, and adjacent microphones may be separated by a distance of about 4 centimeters or less, preferably about 1 to 2 centimeters. The microphones in array 622 are preferably omnidirectional microphones. An optional image acquisition unit 623 (for example, a digital camera) may be connected to the apparatus 600 via the input / output function 611. One or more pointing actuators (P / A) 625 mechanically connected to the camera may exchange signals with the processor 601 via the input / output function 611.

  As used herein, the term input / output generally refers to any program, operation, or device that transfers data from or to system 600 and from or to a peripheral device. All data may be output from one device and considered to be input to another device. Peripheral devices include not only devices capable of both input and output, such as a writable CD-ROM, but also input devices such as a keyboard and a mouse, and output devices such as a printer. The term “peripheral device” is not only an internal device such as a CD-ROM drive, a CD-R drive, an internal modem, or other peripheral devices such as a flash memory reader / writer, a hard disk, but also a mouse, keyboard, printer, Also includes external devices such as a monitor, microphone, game controller, camera, external Zip drive, or scanner.

  In some embodiments of the present invention, the device 600 may be a video game unit that includes a controller 630 connected to the processor via an input / output function 611 by wire (eg, USB cable) or wirelessly. In some embodiments, joystick controller 630 may be wearable on the user's body. The controller 630 may have an analog joystick 631 and conventional buttons 633 that provide control signals commonly used during video game play. Such video games may be obtained by instructions from a program 604 stored in other processor readable media such as data readable by the processor and / or associated with memory 602, mass storage 615, etc. It may be realized.

  The joystick 631 is generally configured to show an operation along the X axis when the stick is moved left and right, and to show an operation along the Y axis when the stick is moved back and forth or up and down. In a joystick configured for three-dimensional motion, twisting the stick left (counterclockwise) or right (clockwise) may indicate motion along the Z axis. These three X, Y, and Z axes are often referred to as roll, pitch, and yaw, respectively, particularly in connection with airplanes.

  In addition to conventional features, the controller 630 may include one or more inertial sensors 632 that provide position and / or orientation information to the processor 601 via inertial signals. The direction information may include angle information such as tilt, roll, or yaw of the controller 630. For example, inertial sensor 632 may include any number and / or combination of accelerometers, gyroscopes, or tilt sensors. In a preferred embodiment, the inertial sensor 632 includes a tilt sensor for detecting the direction of the joystick controller relative to the tilt and roll axis, a first accelerometer for detecting acceleration along the yaw axis, and a yaw axis. A second accelerometer for detecting angular acceleration with respect to. The accelerometer may be implemented, for example, as a MEMS device that includes a mass mounted by one or more springs and a sensor for detecting mass displacement in one or more directions. A signal from a sensor that depends on the displacement of the mass may be used to determine the acceleration of the joystick controller 630. Such a technique may be realized by an instruction from the game program 604 stored in the memory 602 and executed by the processor 601.

  For example, an accelerometer suitable for the inertial sensor 632 may be a simple mass that is elastically coupled to the frame at three or four points, for example by a spring. The pitch and roll axes lie in a plane that intersects the frame mounted on the joystick controller 630. As the frame (and joystick controller 630) rotates about the pitch and roll axis, the mass is displaced under the influence of gravity and the springs expand and contract depending on the pitch and / or roll axis. Mass displacement is detected and converted into a signal that depends on the pitch and / or the amount of roll. Angular acceleration around the yaw axis or linear acceleration along the yaw axis is also detected and produces a characteristic pattern of spring expansion or contraction or mass movement that translates into a signal that depends on the amount of pitch and / or roll. Sometimes. Such an accelerometer can measure the tilt around the yaw axis, the roll angular acceleration, and the linear acceleration along the yaw axis by tracking the movement of the mass or the stretching force of the spring. There are many methods for tracking the position of the mass and / or the force exerted on it, such as resistive strain gauge materials, optical sensors, magnetic sensors, Hall effect devices, piezoelectric devices, capacitive sensors, and the like. In an embodiment, the inertial sensor 632 may be detachably mounted on the “main body” of the joystick controller 630.

  In addition, joystick controller 630 may include one or more light sources 634, such as light emitting diodes (LEDs). The light source 634 may be used to distinguish the controller from other controllers. For example, this can be accomplished by blinking or sustaining the LED pattern code with one or more LEDs. For example, five LEDs may be provided in the controller 630 in a straight line or a two-dimensional pattern. Although it is preferred that the LEDs be arranged linearly, the LEDs are rectangular or arched to facilitate determination of the LED image plane when analyzing the image of the LED pattern acquired by the image acquisition unit 623. You may arrange in a pattern. Further, the LED pattern code may be used to determine the position of the joystick controller 630 during game play. For example, the LEDs help to identify controller tilt, yaw, and roll. This detection pattern is useful for improving the user's feeling of use in a game such as an aircraft flight game. The image acquisition unit 623 may acquire an image including the joystick controller 630 and the light source 634. By analyzing such an image, the position and / or orientation of the joystick controller can be determined. Such analysis may be implemented by program code instructions 604 stored in memory 602 and executed by processor 601. In order to facilitate the acquisition of the image of the light source 634 by the image acquisition unit 623, the light source 634 may be disposed on two or more different sides of the joystick controller 630, for example, front and back (shown in dashed lines). With such an arrangement, the image acquisition unit 623 can acquire an image of the light source 634 even if the direction of the joystick controller 630 is different depending on how the joystick controller 630 is gripped by the user. .

  Furthermore, the light source 634 may provide a telemetry signal to the processor 601 in a manner such as pulse code, amplitude modulation, or frequency modulation. Such a telemetry signal may indicate which joystick button was pressed and / or how hard the button was pressed. The telemetry signal may be encoded into the optical signal by pulse code, pulse width modulation, frequency modulation, luminous intensity (amplitude) modulation, and the like. The processor 601 may decode the telemetry signal from the optical signal and execute a game command in accordance with the decoded telemetry signal. The telemetry signal may be decoded from an analysis of the image of the joystick controller 630 acquired by the image acquisition unit 623. Alternatively, the apparatus 600 may include another optical sensor provided to receive a telemetry signal from the light source 634. The use of LEDs to determine intensity in conjunction with a computer program is described, for example, in US patent application 11 / 429,414 (inventor: Richard L. Marks et al., Title of invention: “Intensity in conjunction with computer program. And computer image / sound processing of input device ”, agent case number: SONYP052), which is incorporated herein by reference. Further, analysis of the image including the light source 634 may be used for both telemetry and determination of the position and / or orientation of the joystick controller 630. Such a technique may be realized by an instruction of a program 604 stored in the memory 602 and executed by the processor 601.

  The processor 601 detects the optical signal from the light source 634 detected by the image acquisition unit 623 and / or the sound detected by the microphone array 622 to estimate information about the position and / or orientation of the controller 630 and / or its user. The inertial signal from the inertial sensor 632 may be used in combination with the sound source position and feature information from the signal. For example, in conjunction with microphone array 622 to detect light source position and features to track voice movement while joystick controller movement is tracked independently (by inertial sensor 632 and / or light source 634). A “sonic radar” may be used. In the acoustic radar, a pre-calibrated listening area is selected at run time, and sound emitted from a sound source outside the pre-calibrated listening area is removed. The pre-calibrated listening area may include a listening area corresponding to the focal volume or field of view of the image acquisition unit 623. An example of an acoustic radar is US patent application 11 / 381,724 (inventor: Shadon Mao, title of invention: "Method and apparatus for detecting and characterizing speech of interest", filing date: May 4, 2006. ), Which is incorporated herein by reference. Any number of different combinations of different aspects of providing control signals to the processor 601 may be used in connection with embodiments of the present invention. Such a technique may be implemented by program code instructions 604 stored in memory 602 and executed by processor 601, selecting a pre-calibrated listening area at runtime and from a sound source outside the pre-calibrated listening area. One or more instructions may be included that instruct the one or more processors to remove the spoken voice. The pre-calibrated listening area may include a listening area corresponding to the focal volume or field of view of the image acquisition unit 623.

Program 604 generates discrete time domain input signals xm (t) from microphones M0-MM of microphone array 622, determines the listening area, and separates finite impulse responses from input signal xm (t). One or more instructions may be included to instruct one or more processors to use the listening area for semi-blind source separation to select filter coefficients. The program 604 may include instructions for applying one or more partial delays to a selected input signal xm (t) other than the input signal x0 (t) from the reference microphone M0. Each partial delay may be selected to optimize the signal for the noise ratio of the discrete time domain output signal y (t) from the microphone array. The partial delay may be selected so that the signal from the reference microphone M0 is first in time compared to signals from other microphones in the array. Program 604 may include instructions for introducing a partial time delay Δ into the output signal y (t) of the microphone array as follows.
y (t + Δ) = x (t + Δ) * b0 + x (t-1 + Δ) * b1 + x (t-2 + Δ) * b2 + ... + x (t-N + Δ) bN
However, while Δ is between 0 and ± 1, examples of such techniques are described in US patent application 11 / 381,729 (inventor: Shadon Mao, title of invention: “miniature microphone array”, filing date: 2006). May 4), incorporated herein by reference.

  The program 604 may include one or more instructions that, when executed, cause the system 600 to select a pre-calibrated listening area that includes a sound source. Such an instruction may cause the device to determine whether the sound source is within the initial region or on a particular side of the initial region. If the sound source is not within the default area, the instruction may select a different area on a particular side of the default area at execution time. Different regions may be characterized by the attenuation of the input signal closest to the optimum value. These instructions may calculate the attenuation of the input signal from the microphone array 622 and the attenuation to the optimum value at execution time. The instruction may cause the apparatus 600 to determine an attenuation value of the input signal in one or more regions at execution time and to select the region where the attenuation is closest to the optimal value. Examples of such techniques are described in US patent application Ser. No. 11 / 381,725 (inventor: Shadon Mao, title of invention: “object speech detection method and apparatus”, filing date: May 4, 2006). And incorporated herein by reference.

  The signal from inertial sensor 632 provides part of the tracking information input, and the signal generated by image acquisition unit 623 from tracking one or more light sources 634 may provide another part of the tracking information input. Good. Such a “mixed system” signal may be used, for example, in a football-type video game, when the quarterback moves the head to the left and fakes, then throws the ball to the right. Specifically, the game player holding the controller 630 may generate a sound while moving his head to the left and swinging the controller to the right to throw it as if it is football. A microphone array 622 associated with the “acoustic radar” program code can track the user's voice. The image acquisition unit 623 can track the movement of the user's head or track other commands that do not require the use of voice or a controller. Sensor 632 may track the movement of a joystick controller (representing football). The image acquisition unit 623 may further track the light source 634 on the controller 632. The user can release the “ball” when the acceleration of the joystick controller 630 reaches a certain amount and / or direction, or when a key command is generated by pressing a button on the controller 630.

  In some embodiments of the present invention, inertia signals from, for example, accelerometers or gyroscopes may be used to determine the position of controller 630. Specifically, the acceleration signal from the accelerometer may be integrated over time to determine the change in velocity, and the velocity may be integrated over time to determine the change in position. If the initial position and velocity values at a certain point in time are known, the absolute position can be determined using these values and the amount of change in velocity and position. Position determination using the inertial sensor can be performed at a higher speed than using the image acquisition unit 623 and the light source 634. However, the inertial sensor 632 accumulates errors over time, and the position of the joystick 631 calculated from the inertial signal. It is susceptible to a kind of error called “drift” that causes a discrepancy D between the joystick controller 630 and the actual position of the joystick controller 630 (shown in phantom). Embodiments of the present invention allow many ways to deal with such errors.

  For example, the drift can be manually canceled by resetting the initial position of the controller 630 to be equal to the current calculated position. The user can use one or more buttons on the controller 630 as a trigger for a command to reset the initial position. Alternatively, image-based drift may be performed by resetting the current position with reference to a position determined from an image acquired from the image acquisition unit 623. Such image-based drift compensation may be performed manually, such as when the user activates one or more buttons of the joystick controller 630. Alternatively, image-based drift compensation may be performed automatically, for example periodically or in response to game play. Such a technique may be realized by program code instructions 604 stored in the memory 602 and executed by the processor 601.

  In certain embodiments, it is desirable to correct erroneous data in the inertial sensor signal. For example, the signal from the inertial sensor 632 may be oversampled, and a moving average (sliding average) may be calculated from the oversampled signal to remove erroneous data from the inertial sensor signal. In certain situations, it may be desirable to oversample the signal, remove high and / or low values from the subset of data points, and calculate a moving average from the remaining data points. Furthermore, data oversampling and handling techniques may be used to adjust the signal from the inertial sensor to remove or reduce the importance of erroneous data. The choice of technology may depend on the type of signal, the operation to be performed on the signal, the type of game play, or a combination of two or more of these. These techniques may be realized by instructions of a program 604 stored in the memory 602 and executed by the processor 601.

  As described above, the processor 601 may analyze the inertial signal data 606 according to the data 606 and the program code instructions of the program 604 stored and acquired in the memory 602 and executed by the processor module 601. Good. Some of the code for program 604 may conform to any of a number of different programming languages, such as assembler, C ++, JAVA, or many other languages. The processor module 601 constitutes a general purpose computer. It becomes a specific purpose computer when executing a program such as program code 604. Here, the case where the program code 604 is realized as software executed on a general-purpose computer has been described. However, the task management method can also be realized using hardware such as an ASIC and other hardware circuits. That is understood by the contractor. Similarly, it is understood that some or all of the embodiments of the present invention can be realized by software, hardware, or a combination thereof.

  In certain embodiments, program code 604 may include a set of processor readable instructions that implement methods having features similar to method 510 of FIG. 5B and method 520 of FIG. 5C or a combination of two or more thereof. . The program code 604 generally instructs one or more processors to analyze the signal from the inertial sensor 632 to generate position and / or orientation information and use that information during video game play. May also include instructions.

  The program code 604, when executed, causes the image acquisition unit 623 to monitor the field of view in front of the image acquisition unit 623, identify one or more light sources 634 within the field of view, detect changes in light emitted from the light sources 634, and change the changes. It may further include a processor readable instruction including one or more instructions that cause an input command to the processor 601 when detected. The use of LEDs in conjunction with an image acquisition device to trigger an action in a game controller is described in US patent application 10 / 759,782 (inventor: Richard L. Marks, filing date: January 16, 2004, Name: “Methods and Apparatus for Optical Input Devices”), incorporated herein by reference.

  Program code 604 includes one or more instructions that use signals from the inertial sensor at run time and signals generated from the image acquisition unit by tracking one or more light sources as input to the game system as described above. It may further include processor readable instructions. Program code 604 may further include processor-readable instructions that include one or more instructions that compensate for drift in inertial sensor 632 during execution.

  In the embodiment of the present invention, an example regarding the video game controller 630 has been described. However, the embodiment of the present invention including the system 600 can be applied to a body operated by a user, a modeled object, a knob, a structure, and the like. The inertia detection function and the inertial sensor signal transmission function may be used wirelessly or in another method.

  For example, embodiments of the present invention may be executed on a parallel processing system. Such parallel processing systems typically include two or more processor elements configured to execute portions of a program in parallel on separate processors. As a non-limiting example, FIG. 7 shows a type of cell processor 700 according to an embodiment of the present invention. The cell processor 700 may be used as the processor of FIG. 6 or may be used as the processor 502 of FIG. 5A. In the example illustrated in FIG. 7, the cell processor 700 includes a main memory 702, a power processor element (PPE) 704, and a plurality of synergistic processor elements (SPE) 706. In the example shown in FIG. 7, the cell processor 700 includes one PPE 704 and eight SPEs 706. In such a configuration, seven SPEs 706 may be used for parallel processing, and one may be reserved as a backup when any of the other seven fails. Alternatively, the cell processor may include a plurality of groups of PPE (PPE groups) and a plurality of groups of SPEs (SPE groups). In this case, hardware resources may be shared among units in the group. However, SPE and PPE must be considered software as independent elements. The embodiment of the present invention is not limited to being used by the configuration shown in FIG.

  Main memory 702 typically includes not only special purpose hardware registers or arrays used for functions such as system configuration, data transfer synchronization, memory mapped I / O, and I / O subsystems. Including general-purpose and non-volatile storage devices. In the embodiment of the present invention, the video game program 703 may be resident in the main memory 702. The memory 702 may include signal data 709. Video program 703 may include an analyzer or some combination thereof configured as described above in connection with FIGS. 5A, 5B, or 5C. The program 703 may be executed on the PPE. The program 703 may be divided into a plurality of signal processing tasks that can be executed on the SPE and / or PPE.

  For example, the PPE 704 may be a 64-bit PPU (PowerPC Processor Unit) in which the caches L1 and L2 are combined. The PPE 704 is a general-purpose processing unit that can access system management resources such as a memory protection table. Hardware resources may be explicitly mapped into the real address space so that the PPE can reference them. Thus, the PPE can address any of these resources with an appropriate effective address value. The main function of the PPE 704 is task management and assignment for the SPE 706 of the cell processor 706.

  Although only one PPE is shown in FIG. 7, in a cell processor implementation such as a cell broadband engine architecture (CBEA), the cell processors 700 are grouped into one or more PPE groups. You may have more than one PPE. These PPE groups may share access to the main memory 702. Further, the cell processor 700 may include two or more SPE groups. SPE groups may also share access to main memory 702. Such a configuration is within the scope of the present invention.

  Each SPE 706 includes a SPU (synergistic processor unit) and its own local storage area LS. The local storage area LS may include one or more divided memory areas, each associated with a particular SPU. Each SPU may be configured to execute only instructions (including data load and data store instructions) from within the local storage area associated with it. In such a configuration, data transfer between the local storage area LS and other configurations of the system 700 is a memory flow controller (MFC) for transferring to or from the local storage area (in an individual SPE) May be executed by issuing a direct memory access (DMA) command from The SPU is a computational unit that is less complex than the PPE 704 in that it does not perform system management functions. SPUs generally have the ability to process multiple data simultaneously (SIMD) with a single instruction, typically processing data to perform assigned tasks, and any requested Initiate data transfer (assuming access to properties set by PPE). The purpose of the SPU is to enable applications that require higher computational unit density and can efficiently use the provided instruction set. The large number of SPEs in the system managed by PPE 704 allows for cost effective processing across a wide range of applications.

  Each SPE 706 may include a dedicated memory flow controller (MFC) that includes a memory management unit capable of holding and processing memory protection information and access permission information. The MFC provides the primary method for data transfer, protection, and synchronization between the main storage of the cell processor and the local storage of the SPE. The MFC command represents a transfer to be performed. A command for transferring data is also called an MFC direct memory access (DMA) command (MFCDMA command).

  Each MFC supports multiple DMA transfers simultaneously, and can hold and process multiple MFC commands. Each MFC / DMA data transfer command request may include both a local storage address (LSA) and an effective address (EA). The local storage address may directly address only the local storage area of the associated SPE. The effective address may have a more general application, for example, may refer to the main storage including all SPE local storage areas as long as it is aliased to the real address space.

  To facilitate communication between SPEs 706 and / or between SPEs 706 and PPEs 704, SPEs 706 and PPEs 704 may include signal notification registers related to signaling events. PPE 704 and SPE 706 may be connected by a star topology where PPE 704 functions as a router for sending messages to SPE 706. Alternatively, each SPE 706 and PPE 704 may have a one-way signal notification register referred to as a mailbox. Mailboxes may be used for operating system (OS) synchronization.

  Cell processor 700 may include an input / output (I / O) function 708 that allows cell processor 700 to interface with peripheral devices such as microphone array 712, image acquisition unit 713, and game controller 730. The game controller unit may include an inertial sensor 732 and a light source 734. Furthermore, the element interconnection bus 710 may connect the various components described above. Each SPE and PPE can access the bus 710 via the bus interface unit BIU. The cell processor 700 controls the flow of data between the I / O 708 and the bus 710, and the memory interface controller MIC that controls the flow of data between the bus 710 and the main memory 710, as typically found in the processor. The bus interface controller BIC may further include two controllers. The requirements for MIC, BIC, BIU and bus 710 can vary widely in different implementations, but their functionality and circuitry for implementation are well known to those skilled in the art.

  The cell processor 700 may further include an internal interrupt controller IIC. The IIC component manages the priority of interrupts transmitted to the PPE. IIC can handle interrupts from other components of the cell processor 700 without using the main system interrupt controller. The IIC may be considered a second level controller. The main system interrupt controller may handle interrupts from outside the cell processor.

  In embodiments of the present invention, certain calculations such as the partial delay described above may be performed in parallel using PPE 704 and / or one or more SPEs 706. Each partial delay calculation may be performed as a task divided into one or more so that a different SPE 706 can be executed.

  While the above is a complete description of the preferred embodiment of the invention, various alternatives, modifications and equivalents may be used. The scope of the invention should, therefore, be determined not with reference to the above description, but instead should be determined with reference to the appended claims along with their full scope of equivalents. All features described herein may be combined with all other features described herein, whether preferred or not. In the claims, what follows an indefinite article refers to the quantity of one or more items, unless expressly specified otherwise. The appended claims should not be construed to include means plus function limitations, unless explicitly limited by use of the phrase “means for”.

Claims (66)

A method for tracking a controller of a video game system, comprising:
Generating one or more signals from inertial sensors located on a controller of the video game system;
Analyzing the one or more signals to determine position and / or orientation information of the controller;
Utilizing the position and / or orientation information during video game play in the video game system;
A method comprising the steps of:   The method of claim 1, wherein the one or more signals include a signal related to a translational acceleration of the controller in one or more directions.   Analyzing the one or more signals includes integrating a translation acceleration signal or data generated from the translation acceleration signal over time to generate a velocity signal. Item 2. The method according to Item 1.   The step of analyzing the one or more signals includes second-order integration with respect to time of the translation acceleration signal or data generated from the translation acceleration signal to generate a displacement signal. The method according to claim 1.   The method of claim 1, wherein the one or more signals include signals related to rotation of the controller with respect to pitch or yaw axis.   Utilizing the position and / or orientation information during the video game play in the video game system includes determining the controller path from the position and / or orientation information. The method of claim 1.   The method of claim 6, further comprising comparing the controller path to one or more known gestures.   8. The method of claim 7, further comprising changing the state of the video game if the controller path or a portion thereof matches one or more known gestures. A method used to provide input to a system,
Generating one or more signals from inertial sensors located on a controller of the system;
Analyzing the one or more signals to determine position and / or orientation information of the controller;
Comparing the determined position and / or direction information with predetermined position information associated with one or more commands;
Changing the state of the system if the determined position and / or direction information matches the default position information for the command;
A method comprising the steps of: A game controller,
The body,
At least one input device attached to the body operable by a user to register input from the user;
An inertial sensor capable of generating information for quantifying the movement of the main body in space;
A game controller comprising: A signal encoder;
An infrared signal transmitter capable of transmitting an infrared signal via the atmosphere using the signal from the signal encoder;
The signal encoder encodes a signal with a selected one of a plurality of signal codes for reception by an electronic device having an infrared receiver and a signal decoder operable with the selected one signal code. The game controller of claim 10, wherein the game controller is programmable. The body includes a housing that can be gripped by hand,
The game controller according to claim 10, wherein the input device includes an element movable by the user with respect to the main body of the game controller to register an input from the user.   The game controller according to claim 12, wherein the housing includes a grip that can be gripped by a hand.   The game controller according to claim 10, wherein the main body of the game controller is attachable to a user's body.   15. The inertial sensor can generate information for quantifying a first component of movement of the body along a first axis. The listed game controller.   16. The information of claim 15, wherein the inertial sensor is capable of generating information for quantifying a second component of the movement along a second axis that is orthogonal to the first axis. Game controller.   The inertial sensor is capable of generating information for quantifying a third component of the movement along a third axis orthogonal to the first axis and the second axis. The game controller according to claim 16.   The game controller according to claim 15, wherein the inertial sensor includes at least one accelerometer.   The game controller according to claim 15, wherein the inertial sensor includes at least one mechanical gyroscope.   The game controller of claim 19, wherein the inertial sensor includes at least one laser gyroscope.   The game controller according to claim 10, wherein the inertial sensor can generate information for quantifying the movement of the main body in at least three degrees of freedom.   The game controller of claim 21, wherein the at least three degrees of freedom include pitch, yaw and roll.   The game controller of claim 21, wherein the at least three degrees of freedom include an x-axis, a y-axis, and a z-axis that are orthogonal to each other.   The game controller according to claim 23, wherein the inertial sensor can quantify the movement in the three degrees of freedom and the six degrees of freedom including pitch, yaw, and roll.   16. A series of samples indicating the acceleration of the body along at least one axis at different points in time can further be obtained from the information generated by the inertial sensor. 25. The game controller according to any one of 24 to 24.   26. The game controller of claim 25, further comprising a processor capable of determining the speed of the main body using the series of samples.   27. The game controller of claim 26, wherein the processor is capable of determining the velocity by integrating acceleration values obtained from the series of samples over a time interval.   The processor can determine a displacement in the space of the body by integrating acceleration values obtained from the series of samples over a time interval and integrating the integrated result over a time interval. The game controller according to claim 26.   29. The game controller of claim 28, wherein the processor is capable of determining the displacement relative to a previously determined position to determine a current position in the body space. A game controller according to claim 25,
A processor capable of executing a program for providing an interactive game that can be played by the user according to an input via the input device operable by the user;
The apparatus, wherein the processor is capable of determining the speed of the body using the series of samples.   31. The apparatus of claim 30, wherein the processor is capable of determining the velocity by integrating acceleration values obtained from the series of samples over a time interval.   The processor is capable of determining a displacement in the space of the main body by integrating acceleration values obtained from the series of samples over a time interval and integrating the integrated result. 30. Apparatus according to 30.   31. The apparatus of claim 30, wherein the processor is capable of determining a position of the body in space by determining the displacement relative to a previously determined position. A tracking device used when obtaining information for controlling execution by a processor of a game program for enabling a user to play an interactive game,
The body,
An inertial sensor capable of generating information for quantifying the movement of the main body in space;
A tracking device comprising: The main body can be mounted on a game controller,
The game controller
The game controller itself,
The tracking device according to claim 34, comprising: at least one input device attached to the game controller main body operable by a user to register input from the user.   36. A device comprising the tracking device and game controller of claim 35.   The tracking device according to claim 34, wherein the main body of the tracking device is attachable to a user's body.   38. The inertial sensor is capable of generating information for quantifying a first component of movement of the body along a first axis. Tracking device.   39. The information of claim 38, wherein the inertial sensor is capable of generating information for quantifying a second component of the movement along a second axis that is orthogonal to the first axis. Tracking device.   The inertial sensor is capable of generating information for quantifying a third component of the movement along a third axis orthogonal to the first axis and the second axis. 40. A tracking device according to claim 39.   41. A tracking device according to any of claims 38 to 40, wherein the inertial sensor includes at least one accelerometer.   41. A tracking device according to any of claims 38 to 40, wherein the inertial sensor comprises at least one mechanical gyroscope.   43. The tracking device of claim 42, wherein the inertial sensor includes at least one laser gyroscope.   The tracking device according to claim 34, wherein the inertial sensor is capable of generating information for quantifying the movement of the body in at least three degrees of freedom.   45. The tracking device of claim 44, wherein the at least three degrees of freedom include pitch, yaw and roll.   46. The tracking device according to claim 45, wherein the at least three degrees of freedom include an x-axis, a y-axis, and a z-axis that are orthogonal to each other.   The tracking device according to claim 46, wherein the inertial sensor is capable of quantifying the movement in the three degrees of freedom and the six degrees of freedom including pitch, yaw, and roll.   40. It is further possible to obtain from the information generated by the inertial sensor a series of samples showing acceleration of the body along at least one axis at different points in time. 48. The tracking device according to any one of.   49. The tracking device of claim 48, further comprising a processor capable of determining the speed of the body using the series of samples.   50. The tracking device of claim 49, wherein the processor is capable of determining the velocity by integrating acceleration values obtained from the series of samples over a time interval.   The processor can determine a displacement in the space of the body by integrating acceleration values obtained from the series of samples over a time interval and integrating the integrated result over a time interval. 49. A tracking device according to claim 48.   52. The tracking device of claim 51, wherein the processor is capable of determining the displacement relative to a previously determined position to determine a current position in the body space. 49. A tracking device according to claim 48,
The apparatus further comprising a processor capable of executing a program for providing an interactive game that can be played by a user in accordance with an input acquired by processing information generated by the inertial sensor.   37. The communication interface according to claim 34, further comprising a communication interface capable of executing digital communication with either the processor or the game controller or both of the processor and the game controller. Tracking device.   55. The tracking device according to claim 54, wherein the communication interface includes a universal asynchronous receiver transmitter (UART).   The general-purpose asynchronous transmission / reception circuit can execute at least one of reception of a control signal for controlling the operation of the tracking device and transmission of a signal from the tracking device for communication with another device. 56. The tracking device according to claim 55.   56. The tracking device according to claim 54 or 55, wherein the communication interface includes a universal serial bus (USB) controller.   The general-purpose serial bus controller can execute at least one of reception of a control signal for controlling the operation of the tracking device and transmission of a signal from the tracking device for communication with another device. 58. A tracking device according to claim 57.   54. The apparatus of claim 53, wherein the processor is capable of determining the speed of the body using the series of samples.   54. The apparatus of claim 53, wherein the processor is capable of determining the velocity by integrating acceleration values obtained from the series of samples over a time interval.   The processor can determine a displacement in the space of the body by integrating acceleration values obtained from the series of samples over a time interval and integrating the integrated result over a time interval. 54. The apparatus of claim 53.   54. The apparatus of claim 53, wherein the processor is capable of determining a position of the body in space by determining the displacement relative to a previously determined position.   30. The game controller according to claim 10, wherein the inertial sensor is detachably mounted on the main body.   The apparatus according to any one of claims 30 to 33, wherein the inertial sensor is detachably mounted on the main body.   The tracking device according to any one of claims 34 to 52 and 54 to 62, wherein the inertial sensor is detachably mounted on the main body.   37. The tracking device according to claim 35 or 36, wherein the inertial sensor is detachably mounted on the controller main body.

Images