WO2004072909A1 - Method and apparatus for real time motion capture - Google Patents

Method and apparatus for real time motion capture Download PDF

Info

Publication number
WO2004072909A1
WO2004072909A1 PCT/US2003/031502 US0331502W WO2004072909A1 WO 2004072909 A1 WO2004072909 A1 WO 2004072909A1 US 0331502 W US0331502 W US 0331502W WO 2004072909 A1 WO2004072909 A1 WO 2004072909A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
display screen
depth image
tracked
control
Prior art date
Application number
PCT/US2003/031502
Other languages
English (en)
French (fr)
Inventor
Richard L. Marks
Original Assignee
Sony Computer Entertainment Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Computer Entertainment Inc. filed Critical Sony Computer Entertainment Inc.
Priority to JP2004568305A priority Critical patent/JP4723863B2/ja
Priority to AU2003279810A priority patent/AU2003279810B2/en
Priority to DE60329424T priority patent/DE60329424D1/de
Priority to AT03773138T priority patent/ATE443903T1/de
Priority to EP03773138A priority patent/EP1593096B1/en
Publication of WO2004072909A1 publication Critical patent/WO2004072909A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G06T2207/30208—Marker matrix
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2215/00—Indexing scheme for image rendering
    • G06T2215/16—Using real world measurements to influence rendering

Definitions

  • This invention relates generally to video processing and more particularly to tracking depth of an image and/or a marker associated with the image to provide real time motion capture for video game applications.
  • FIG. 1 is a schematic diagram of a person having a plurality of balls distributed over their body to store the persons moves. Person 100 has a plurality of balls 102 distributed over their body.
  • Person 100 will then perform some activity which is captured by a plurality of cameras 104.
  • the captured video data may then be edited as desired to define video data 106.
  • Video data 106 is then stored in database 108 for later retrieval for insertion into a movie.
  • the scheme described with respect to Figure 1 works well for a controlled environment, such as movie editing, where real time motion capture is not needed.
  • the person's movements are captured and stored, the person only has to wear the balls once.
  • the motion capture as described with respect to Figure 1 is used in a passive sense, i.e., editors insert the stored movement into a movie while editing the movie.
  • the captured movement does not control any aspects of the movie and is inserted into a movie at a later time.
  • a video game i.e., captured motion is not used to control any aspects of a video game.
  • the complex computational capabilities and the need for point by point scanning has prohibited the use of captured motion to act as a control in real time.
  • a method for real time motion capture for control of a video game character is provided. The method initiates with defining a model of a control object. Then, a location of a marker on the model is identified. Next, movement associated with the control object is captured. Then, the movement associated with the control object is interpreted to change a position of the model. Next, movement of the character being presented on the display screen is controlled according to the change of position of the model.
  • a method for controlling an object presented on a display screen in communication with a computing device through real time motion capture begins with identifying a depth image associated with an object being tracked. Then, a model associated with both an object presented on a display screen and the object being tracked is identified. Next, the model is fit to the depth image to capture motion associated with the object being tracked. Then, the object presented on the display screen is controlled in real time according to the fitting of the model to the depth image.
  • a method for controlling movements of an image presented on a display screen through real time motion capture begins with defining a model of a person. Then, a location of a marker on the model of the person is identified. Next, a depth image corresponding to a portion of the model of the person is provided. The portion of the model includes the marker. Then, the location of the marker is associated with a point on the depth image. Next, the portion of the model is positioned based upon a configuration of the depth image. Then, a video character displayed on a display screen is controlled according to the positioning of the portion of the model.
  • a computer readable media having program instructions for controlling an object presented on a display screen, in communication with a computing device, through real time motion capture.
  • the computer readable media includes program instructions for identifying a depth image of an object being tracked and program instructions for identifying a model associated with both an object presented on a display screen and the object being tracked.
  • Program instructions for fitting the model to the depth image to capture motion associated with the object being tracked are included.
  • Program instructions for controlling the object presented on the display screen in real time according to the fitting of the model to the depth image are provided.
  • a system enabling control of an object through real time motion capture associated with a control image includes a computing device.
  • a display screen in communication with the computing device is provided.
  • the display screen is configured to display an image of an object, where data corresponding to the image of the object is provided to the display screen by the computing device.
  • a video capture device in communication with the computing device is includes.
  • the video capture device is enabled to track a control object.
  • the video capture device is configured to translate motion associated with the control object to control motion of the object being displayed on the display screen as the control object moves.
  • a system enabling control of video character through real time motion capture associated with a control image includes a computing device. Means for displaying an image of a video character from data received by the computing device and means for capturing a depth image associated with a control object are provided. Means for fitting a model to the depth image to define movement of the control object are included. Means for translating the movement of the control object to control motion associated with the video character on a display screen as the control object moves are also included.
  • Figure 1 is a schematic diagram of a person having a plurality of balls distributed over their body to store the persons moves.
  • Figure 2 is a simplified schematic diagram illustrating real time motion capture used to control a character of a video game in accordance with one embodiment of the invention.
  • Figure 3 is a schematic diagram of a model of a person where the model includes markers depicting joint angles in accordance with one embodiment of the invention.
  • Figures 4A through 4C illustrate exemplary patterns that may be worn by the control object as a marker in accordance with one embodiment of the invention.
  • Figure 5 is a schematic diagram of real time motion capture of a control object being used for controlling movement associated with a character presented on a display screen in accordance with one embodiment of the invention.
  • Figure 6A is a schematic diagram of a bodysuit having markers included in the bodysuit in accordance with one embodiment of the invention.
  • Figure 6B is an alternative embodiment to the bodysuit of Figure 6A.
  • Figure 7 is a schematic diagram illustrating an optimization technique for monitoring an image frame for markers in accordance with one embodiment of the invention.
  • Figure 8 is a schematic diagram of a model depicting regions filled in by a depth image in accordance with one embodiment of the invention.
  • Figure 9 is a block diagram of an exemplary user input system for interaction with an object on a graphical display that can be used to implement embodiments of the present invention.
  • Figure 10 illustrates an input system for user interaction with an object on a graphical display, according to embodiments of the present invention.
  • FIG. 11 is a simplified block diagram of a computer processing system configured to implement the embodiments of the invention described herein
  • Figure 12 is a flowchart diagram of the method operations for real time motion capture for controlling a character presented on a display screen in accordance with one embodiment of the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • the embodiments of the present invention provide a system and method for allowing real time motion capture to control a video game character or object.
  • the movement of a person playing a video game is used to control a character on a display screen associated with the video game.
  • real time motion capture of the person playing the video game is used to control the character on the display screen.
  • the raw movement data of the person playing the video game is captured and used for control purposes in real time, as opposed to movement capture that is edited offline and then inserted into a movie at a later point in time.
  • the movement of a human character presented on a display screen is controlled by the movement of a user or a person acting as a control object.
  • the movement of the user may be captured through a video capture device, also referred to as an image capture device, such as a web cam or some other suitable type of camera.
  • the video capture device is configured to capture a depth image of the control object.
  • the depth image of the control object provides data associated with a distance or depth associated with each pixel relative to the video capture device. Therefore, a skeleton associated with the control object and defining the location of joint angles may be mapped to a human character or even a non-human character.
  • markers worn by the control object may be used to indicate the location of joint angles. The depth images are combined with the skeleton using the locations defined by the markers to create a control object.
  • FIG. 1 is a simplified schematic diagram illustrating real time motion capture used to control a character of a video game in accordance with one embodiment of the invention.
  • Motion capture input 120 is stored in motion capture database 124.
  • Control object motion 122 is captured through a capture device, such as a web cam, and associated with corresponding motion capture input 120 stored in database 124. The associated motion capture input is then presented on a display screen as represented as output motion from data base 126.
  • control object motion 122 may capture some motion or activity of a person playing a video game.
  • motion capture database 124 stores a plurality of motion capture input files 120 that capture the motion of a professional athlete. Accordingly, control object motion 122 is associated with a similar motion capture input 120 that corresponds to the control object motion and the similar motion capture input is presented on a display screen in real time.
  • control object motion 122 is used as constraint information.
  • the constraint information drives the animation of a character image, therefore, control object motion 122 directly controls the motion of a game character.
  • a physical simulation system may accomplish this feature.
  • the game character mirrors the motion of the control object.
  • certain movement of the control object is mapped to cause a different movement of the game character.
  • control object movement when the control object moves a limb, arm or leg, the limb movement may be mapped to cause the eyebrows of a video game character to move. It should be appreciated that any type of control object movement may be mapped to any different type of movement of a character image, such as a video game character.
  • control object motion 122 may depict a person playing a basketball video game where the person performs a shooting motion as depicted in block 122a.
  • the shooting motion of block 122a is associated with a shooting motion of a professional basketball player stored in database 124.
  • the shooting motion of the professional basketball player is displayed on a viewable screen as depicted in block 126a. Therefore, the motion capture of the person playing the video game is used to control, in real time, the professional character performing a similar motion.
  • the basketball example is shown for illustrative purposes and not meant to be limiting. That is, the motion of the person playing the video game may correspond to any sport motion.
  • the motion of the person playing the video game may be associated with non- sport activities.
  • the captured motion of the person playing the video game may be used to control movements of an animal or other non-human living object being displayed by the video game or even an inanimate object.
  • FIG. 3 is a schematic diagram of a model of a person where the model includes markers depicting joint angles in accordance with one embodiment of the invention.
  • the model is a skeleton image.
  • a skeleton image refers to any model of a structure that is tracked and is not limited to an articulated model where the model is rigid but has joints.
  • the skeleton image may be defined with varying precision, e.g., a variety of joint constraints. Of course, the more joints and limbs associated with the skeleton image or the model, correlates to more data required to be tracked.
  • markers 140a-140i are distributed over skeleton image 142.
  • Markers 140a and 140d correspond to the wrist location
  • markers 140b and 140c correspond to an elbow location
  • marker 140e corresponds to the torso
  • Markers 140g and 140f correspond to the knees
  • markers 140h and 140i correspond to the ankles.
  • headphones 141 can be used as a marker indicating a position of a head of the control object.
  • a headband may be used as a marker.
  • ear insertable devices may be. provided where the ear insertable devices act as a marker and provide sound for the control object.
  • a video capture device e.g., camera
  • the markers may be configured as straps a person can wear or the markers may be incorporated into the fabric of the person, i.e. control object.
  • Software can then analyze the captured image of the person with the markers to create skeleton image 142 with known location of the joint angles as provided by markers 140a- 140i.
  • the video capture device is configured to provide a depth image that can be used to fill in portion of the skeleton image and position the skeleton image in three dimensional space.
  • markers 140a-140i provide data as to a starting location of an appendage or limb and the depth image from the video capture device can fill in the appendage or limb in three dimensional space.
  • appendage and limb are not meant to be limiting as a person, i.e., control object, may be controlling an object that is also captured by the depth image.
  • the appendage or limb may include the objects being controlled by the person acting as a control object.
  • the video capture device does not have depth capturing capability and the markers will indicate a known location in space for a starting point of an appendage or limb.
  • an arm, hand, leg, foot, or some other appendage may be filled in from a database storing a typical configuration of the associated appendage.
  • a depth image of the control object can be taken prior to starting the game and the depth image may be stored in a database as a model to be used as needed for the control of a video game character in real time.
  • a video capture device is used to capture motion of the person playing the video game without markers.
  • certain assumptions about the location of the object e.g., which hand, foot, etc., are from the right side or left side are made in order to translate the captured motion for control of a video game character.
  • the captured motion is used to control motion associated with a character of a video game in real time.
  • the markers may take on various forms. For example, material having a certain shape, color, pattern, reflective capability, or some other distinguishing quality so that a video capture device can identify a point in space by the marker may be used.
  • retro-reflective material may be used to provide a distinguishing reflective capability.
  • a light associated with the video capture device may be used in combination with the retro-reflective tape to provide the location in space of the starting point for a particular appendage of the control object.
  • the markers may take the form of blinking lights.
  • the lights could be non- visible light, such as infrared light. The lights may blink at a set frequency where the set frequency corresponds to a particular person or team.
  • Figures 4A through 4C illustrate exemplary patterns that may be worn by the control object as a marker in accordance with one embodiment of the invention.
  • the exemplary patterns include a striped pattern of Figure 4A, a block pattern of Figure 4B and a crosshatching pattern of Figure 4C.
  • the patterns illustrated in Figures 4A-4C are not meant to be limiting as any type of pattern capable of being recognized by a video capture device may be used.
  • the different patterns may be used to differentiate between right and left limbs in one embodiment.
  • marker 140d of Figure 3 may include the pattern of Figure 4A
  • marker 140a of Figure 3 may include the pattern of Figure 4B. Accordingly, the right and left wrists will be differentiated.
  • shapes, colors, etc. may be used in conjunction with the patterns to define a marker. Thus, should the pattern become deformed through movement, a backup indicator is still available for the marker.
  • FIG. 5 is a schematic diagram of real time motion capture of a control object being used for controlling movement associated with a character presented on a display screen in accordance with one embodiment of the invention.
  • Skeleton image 142 corresponding to a person playing a video game, is tracked by video capture device 148.
  • video capture device 148 is configured to capture depth data for each pixel.
  • depth cameras offered by 3DV STSTEMS or CANESTA may be employed as video capture device 148 for the embodiments described herein that capture depth data.
  • camera 148 includes light 150.
  • light 150 may be an infrared light.
  • light 150 is blinking so that it is possible to differentiate between permanent lights in a scene and markers to be tracked, i.e., the reflection of the blinking light from the retro-reflective tape.
  • the retro- reflective tape reflects all of the light from light 150, therefore, the retro-reflective tape will be a bright spot in the image data provided by video capture device 148.
  • the bright spot or void will indicate a starting point for an appendage associated with the marker.
  • marker 140d is associated with a right wrist and would indicate the starting point for a depth image including the right wrist, which could include a hand or a forearm.
  • the hand could be controlling an object as described above.
  • camera 148 may include several blinking lights of different colors.
  • the retro-reflective tape, or any suitable reflective marker may be designed to reflect different colors, thereby allowing for differentiation between a number of reflective markers.
  • Marker 140c of Figure 5 indicates the position of the torso. As mentioned above, any number of markers may be worn by the control object at defined locations. It will be apparent to one skilled in the art that the capture device can identify the particular marker as being associated with a particular appendage, i.e., wrist or torso with respect to Figure 5. Where video capture device 148 is configured to provide a depth image, the video capture device provides the data to fill in the rest of the body for skeleton image 140 with the depth information as described with reference to Figure 8.
  • Camera 148 includes microprocessor 152 configured to combine the depth information with the marker location to generate and track the image of the control object, i.e., person playing the video game. Microprocessor 152 may perform processing functions as described with reference to Figure 9.
  • FIG. 148 is in communication with computing device 154, which in turn is in communication with display device 156.
  • computing device 154 is a video game console such as the "PLAYSTATION 2"® manufactured by Sony Computer Entertainment Inc.
  • light 150 may be configured to emit light in the infrared spectrum to minimize the visible light directed at the control object.
  • Figure 6A is a schematic diagram of a bodysuit having markers included in the bodysuit in accordance with one embodiment of the invention. Markers 162a through 162i may be integrated into bodysuit 160 at joint locations corresponding to the locations discussed with reference to Figure 3.
  • torso marker 162e may be encoded with data that identifies the person wearing the suit with a certain sports figure, celebrity, team identification, etc.
  • a marker may be encoded to provide the person wearing the suit with enhanced features during a video game.
  • the bodysuit may provide the user extra protection through more body armor or other enhancements that would entice consumers to purchase the bodysuits for use with a video game.
  • Figure 6B is an alternative embodiment to the bodysuit of Figure 6A.
  • micro fibers of retro-reflective material are woven into bodysuit, i.e., incorporated throughout the fabric of bodysuit 160.
  • the retro-reflective material is distributed throughout the bodysuit.
  • the body suit would appear as a maximum to a depth camera, but would be preferable for the embodiment where the camera is not enabled to capture depth data.
  • FIG. 7 is a schematic diagram illustrating an optimization technique for monitoring an image frame for markers in accordance with one embodiment of the invention.
  • region 170 which defines an area around skeleton image 142.
  • FIG. 8 is a schematic diagram of a model depicting regions filled in by a depth image in accordance with one embodiment of the invention.
  • a video capture device e.g., camera
  • the captured image of the person is analyzed with the markers to create skeleton image 142, also referred to as a model, with known location of the joint angles as provided by markers 140a-140i.
  • the person wearing the markers stands in front of the video capture device to define a model or body mesh of the control object that is stored as mentioned above. That is, a calibration step is performed initially to define a model depth image for the control object. Then, the depth image or portions of the depth image are used to fill in skeleton 142 to create a three dimensional image.
  • the respective marker defines a position for the corresponding depth image portion to be located.
  • left and right forearms are be defined in regions 180c and 180e, respectively.
  • Upper torso and lower torso are defined in regions 180d and 180b, respectively, while left shin and right shin are defined in regions 180a and 180f, respectively.
  • hands and feet are defined below respective markers 140a, 140d, 140i and 140h.
  • the markers define a starting location and the depth image is used to fill in the remainder to provide a three dimensional control object for controlling the motion of a video game character in real time.
  • Figure 9 is a block diagram of an exemplary user input system for interaction with an object on a graphical display that can be used to implement embodiments of the present invention.
  • the user input system is comprised of a video capture device 200, an input image processor 202, an output image processor 204, and a video display device 206.
  • Video capture device 200 may be any device capable of capturing sequences of video images, and, in one embodiment, is a digital video camera (such as a "web-cam"), or similar image capturing device. As mentioned above, the video capture device may be configured to provide depth image.
  • Input image processor 202 translates the captured video images of the control object into signals that are delivered to an output image processor.
  • input image processor 202 is programmed to isolate the control object from the background in the captured video image through the depth information and generate an output signal responsive to the position and/or movement of the control object.
  • the output image processor 106 is programmed to effect translational and/or rotational movement of an object on the video display device 108 in response to signals received from the input image processor 104.
  • These and additional aspects of the present invention may be implemented by one or more processors which execute software instructions.
  • a single processor executes both input image processing and output image processing as illustrated in Figure 5.
  • the processing operations are shown as being divided between an input image processor 202 and an output image processor 204. It should be noted that the invention is in no way to be interpreted as limited to any special processor configuration, such as more than one processor.
  • the multiple processing blocks shown in Figure 9 are shown only for convenience of description.
  • Figure 10 illustrates an input system for user interaction with an object on a graphical display, according to embodiments of the present invention.
  • the input system environment includes control object 212, video capture device 214, video display device 216, and console 208 containing the processor functionality, such as a video game machine.
  • Control object 212 in the input system environment should be located within the field of view 210 of the video capture device 214.
  • Processing system 208 can be implemented by an entertainment system or game console, such as a Sony® PlaystationTM II or Sony® PlaystationTM I type of processing and computer entertainment system.
  • processing system 208 can be implemented in other types of computer systems, such as personal computers, workstations, laptop computers, wireless computing devices, or any other type of computing device that is capable of receiving and processing graphical image data.
  • control object 212 may incorporate markers as described above and/or video cap[ture device 214 may include depth capturing capability.
  • FIG 11 is a simplified block diagram of a computer processing system configured to implement the embodiments of the invention described herein.
  • the processing system may represent a computer-based entertainment system embodiment that includes central processing unit (“CPU") 224 coupled to main memory 220 and graphical processing unit (“GPU") 226.
  • CPU 224 is also coupled to Input/Output Processor ('TOP") Bus 228.
  • GPU 226 includes an internal buffer for fast processing of pixel based graphical data.
  • GPU 226 can include an output processing portion or functionality to convert the image data processed into standard television signals, for example NTSC or PAL, for transmission to display device 227 connected external to the entertainment system or elements thereof.
  • data output signals can be provided to a display device other than a television monitor, such as a computer monitor, LCD (Liquid Crystal Display) device, or other type of display device.
  • IOP bus 228 couples CPU 224 to various input/output devices and other busses or device. IOP bus 228 is connected to input/output processor memory 230, controller 232, memory card 234, Universal Serial Bus (USB) port 236, --EEE1394 (also known as a Firewire interface) port 238, and bus 250. Bus 250 couples several other system components to CPU 224, including operating system (“OS”) ROM 240, flash memory 242, sound processing unit (“SPU”) 244, optical disc controlling unit 246, and hard disk drive (“HDD”) 248.
  • OS operating system
  • SPU sound processing unit
  • HDD hard disk drive
  • the video capture device can be directly connected to IOP bus 228 for transmission therethrough to CPU 224; where, data from the video capture device can be used to change or update the values used to generate the graphics images in GPU 226.
  • embodiments of the present invention can use any suitable image processing configurations and techniques known in the art.
  • Programs or computer instructions embodying aspects of the present invention can be provided by several different methods.
  • the user input method for interaction with graphical images can be provided in the form of a program stored in HDD 248, flash memory 242, OS ROM 240, or on memory card 232.
  • the program can be downloaded to the processing unit through one or more input ports coupled to CPU 224.
  • Embodiments of the present invention also contemplate distributed image processing configurations.
  • the invention is not limited to the captured image and display image processing taking place in one or even two locations, such as in the CPU or in the CPU and one other element.
  • the input image processing can just as readily take place in an associated CPU, processor or device that can perform processing; essentially all of image processing can be distributed throughout the interconnected system.
  • the present invention is not limited to any specific image processing hardware circuitry and/or software.
  • the embodiments described herein are also not limited to any specific combination of general hardware circuitry and/or software, nor to any particular source for the instructions executed by processing components.
  • FIG. 12 is a flowchart diagram of the method operations for real time motion capture for controlling a character presented on a display screen in accordance with one embodiment of the invention.
  • the method initiates with operation 260 where a skeleton image or a model of a control object is defined.
  • the control object may be a person playing a video game associated with the character presented on the display screen. It should be appreciated that the person may stand in front of a video capture device within the field of view of the video capture device in order to define a model of the person as described above.
  • the skeleton image or model may be defined through a self calibration process where the person stands in front of the video capture device.
  • the method then advances to operation 262 where a location of a marker on the skeleton image or model is identified.
  • the markers may indicate the location of joint angles on the skeleton image or model as described with reference to Figures 3, 5 and 8. It should be appreciated that the markers may take on the various formats as described herein.
  • the method of Figure 12 then proceeds to operation 264 where movement associated with the control object is captured.
  • the movement is captured through a camera configured to provide a depth image, i.e., image data associated with a z axis as well as an x axis and y axis.
  • the movement is captured in two dimensions by a digital video camera, such as a web cam.
  • the method then moves to operation 266, where in response to the movement of the control object, a position of the skeleton image or model is changed.
  • the skeleton image is moved to correspond with the movement of the control object. It should be appreciated that where a depth camera is used as a video capture device, a depth image corresponding to the skeleton image may be provided here. The skeleton image is positioned to correspond with the position of the depth image.
  • the skeleton image may be thought of as a rag doll that is positioned to correspond with the position of the depth image.
  • the depth image tracks the movement of the control object, in turn, the skeleton image is also similarly moved to follow the depth image.
  • the movement of the control object is repeated by the skeleton image.
  • the markers may be used without the depth camera.
  • the depth camera may be used without the markers. It should be appreciated that the depth camera captures the three dimensional data, however, the depth camera does not have the knowledge to map the three dimensional data to the skeleton.
  • the markers provide known data points in space, therefore, the markers enable the depth data to be mapped to the skeleton.
  • the markers eliminate the need for the assumptions as an actual point, such as a joint angle, is labeled by the marker and the corresponding depth data can be filled in from the point identified by the marker.
  • the method then advances to operation 268 where the movement of the character being presented on the display screen is controlled according to the change of position of the skeleton.
  • the movement of the character is being controlled in real time. That is, as the control object moves, the skeleton is positioned accordingly through the use of image data having depth information and/or markers.
  • a system enabling control of an object through real time motion capture associated with a control image consisting of various components defined in terms of the functions of each of the components.
  • the system includes a computing device, e.g., a game console.
  • the system may include means for displaying an image of a display object from data received by the computing device.
  • the means for displaying an image mat be any suitable display screen including a television monitor.
  • Means for capturing a depth image associated with a control object are included.
  • the means for capturing a depth image may be provided by a camera enabled to provide depth data, such as the cameras available from 3DV SYSTEMS or CANESTA mentioned above.
  • Means for fitting a skeleton image or model to the depth image to define movement of the control object are provided.
  • a microprocessor may provide the means for fitting the skeleton image to the depth image to define movement of the control object.
  • the processing may be provided through one or more microprocessors.
  • a microprocessor such as a graphics processing unit, can accomplish the means for translating the movement of the control object to control motion associated with the display object.
  • the microprocessors for performing the above described functionality can be included in a chipset.
  • the above described invention describes a method and a system for providing real time motion capture for controlling a character of a video game.
  • the embodiments described above allow for real time motion capture which may be used to control a character or object of a video game.
  • the ability to track depth combined with a marker can provide a starting point for identifying the limbs/body of a person (control object) being tracked.
  • the person can then be filled in to resemble a true character or person.
  • a skeleton image associated with the depth image can be filled in from a database having the depth images or from depth images captured through the video capture device.
  • Various forms of markers may be used in conjunction with a video capture device that may or may not capture depth images.
  • Examples can include localized placement of wearable retro-reflective tape, e.g., on the wrists, knees, head, etc.
  • body suits or clothes incorporating different types of markers may be provided.
  • a reference to a database of movements may locate a corresponding movement in the database. For example, if the user of game selects a particular character, such as a famous ball player, the movement may be like that performed by the actual famous ball player. These movements would be stored in the database. The obtained database movement will thus be used to control the character (person/skeleton) that is part of the game.
  • a camera or hardware configured to track the depth can include a processor, special circuitry or a DSP that is configured to do the filling in, the database look ups and translate the user's actions into "onscreen" activity.
  • the term skeleton image may be defined broadly to include any model of any object being tracked, whether the object is a human object, an animated object or an inanimate object.
  • the invention may employ various computer-implemented operations involving data stored in computer systems. These operations include operations requiring physical manipulation of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. Further, the manipulations performed are often referred to in terms, such as producing, identifying, determining, or comparing.
  • the above described invention may be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers and the like.
  • the invention may also be practiced in distributing computing environments where tasks are performed by remote processing devices that are linked through a communications network.
  • the invention can also be embodied as computer readable code on a computer readable medium.
  • the computer readable medium is any data storage device that can store data which can be thereafter read by a computer system. Examples of the computer readable medium include hard drives, network attached storage (NAS), read-only memory, random-access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices.
  • the computer readable medium can also be distributed over a network coupled computer system so that the computer readable code is stored and executed in a distributed fashion.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Signal Processing (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
  • Processing Of Color Television Signals (AREA)
  • Closed-Circuit Television Systems (AREA)
PCT/US2003/031502 2003-02-11 2003-10-06 Method and apparatus for real time motion capture WO2004072909A1 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP2004568305A JP4723863B2 (ja) 2003-02-11 2003-10-06 リアルタイムのモーション・キャプチャ方法および装置
AU2003279810A AU2003279810B2 (en) 2003-02-11 2003-10-06 Method and apparatus for real time motion capture
DE60329424T DE60329424D1 (de) 2003-02-11 2003-10-06 Verfahren und vorrichtung zur bewegungserfassung in echtzeit
AT03773138T ATE443903T1 (de) 2003-02-11 2003-10-06 Verfahren und vorrichtung zur bewegungserfassung in echtzeit
EP03773138A EP1593096B1 (en) 2003-02-11 2003-10-06 Method and apparatus for real time motion capture

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/365,120 US9177387B2 (en) 2003-02-11 2003-02-11 Method and apparatus for real time motion capture
US10/365,120 2003-02-11

Publications (1)

Publication Number Publication Date
WO2004072909A1 true WO2004072909A1 (en) 2004-08-26

Family

ID=32824565

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2003/031502 WO2004072909A1 (en) 2003-02-11 2003-10-06 Method and apparatus for real time motion capture

Country Status (10)

Country Link
US (3) US9177387B2 (zh)
EP (1) EP1593096B1 (zh)
JP (1) JP4723863B2 (zh)
KR (1) KR100965348B1 (zh)
CN (1) CN100409261C (zh)
AT (1) ATE443903T1 (zh)
AU (1) AU2003279810B2 (zh)
DE (1) DE60329424D1 (zh)
TW (1) TWI274295B (zh)
WO (1) WO2004072909A1 (zh)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006108279A1 (en) * 2005-04-11 2006-10-19 Martin Lizee Method and apparatus for virtual presence
WO2006112308A1 (ja) * 2005-04-15 2006-10-26 The University Of Tokyo モーションキャプチャシステム、及びモーションキャプチャシステムにおける特徴点の三次元再構成法
US8897495B2 (en) 2009-10-07 2014-11-25 Microsoft Corporation Systems and methods for tracking a model
US9522328B2 (en) 2009-10-07 2016-12-20 Microsoft Technology Licensing, Llc Human tracking system
US9782660B2 (en) 2007-11-30 2017-10-10 Nike, Inc. Athletic training system and method
US10412283B2 (en) 2015-09-14 2019-09-10 Trinamix Gmbh Dual aperture 3D camera and method using differing aperture areas
US10775505B2 (en) 2015-01-30 2020-09-15 Trinamix Gmbh Detector for an optical detection of at least one object
US10823818B2 (en) 2013-06-13 2020-11-03 Basf Se Detector for optically detecting at least one object
US10890491B2 (en) 2016-10-25 2021-01-12 Trinamix Gmbh Optical detector for an optical detection
US10948567B2 (en) 2016-11-17 2021-03-16 Trinamix Gmbh Detector for optically detecting at least one object
US10955936B2 (en) 2015-07-17 2021-03-23 Trinamix Gmbh Detector for optically detecting at least one object
US11041718B2 (en) 2014-07-08 2021-06-22 Basf Se Detector for determining a position of at least one object
US11060922B2 (en) 2017-04-20 2021-07-13 Trinamix Gmbh Optical detector
US11067692B2 (en) 2017-06-26 2021-07-20 Trinamix Gmbh Detector for determining a position of at least one object
US11125880B2 (en) 2014-12-09 2021-09-21 Basf Se Optical detector
US11211513B2 (en) 2016-07-29 2021-12-28 Trinamix Gmbh Optical sensor and detector for an optical detection
US11428787B2 (en) 2016-10-25 2022-08-30 Trinamix Gmbh Detector for an optical detection of at least one object
US11860292B2 (en) 2016-11-17 2024-01-02 Trinamix Gmbh Detector and methods for authenticating at least one object

Families Citing this family (227)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8617008B2 (en) 2001-09-12 2013-12-31 Pillar Vision, Inc. Training devices for trajectory-based sports
US8409024B2 (en) 2001-09-12 2013-04-02 Pillar Vision, Inc. Trajectory detection and feedback system for golf
US10360685B2 (en) * 2007-05-24 2019-07-23 Pillar Vision Corporation Stereoscopic image capture with performance outcome prediction in sporting environments
US6990639B2 (en) 2002-02-07 2006-01-24 Microsoft Corporation System and process for controlling electronic components in a ubiquitous computing environment using multimodal integration
US7646372B2 (en) 2003-09-15 2010-01-12 Sony Computer Entertainment Inc. Methods and systems for enabling direction detection when interfacing with a computer program
US8947347B2 (en) 2003-08-27 2015-02-03 Sony Computer Entertainment Inc. Controlling actions in a video game unit
US8797260B2 (en) 2002-07-27 2014-08-05 Sony Computer Entertainment Inc. Inertially trackable hand-held controller
US7850526B2 (en) * 2002-07-27 2010-12-14 Sony Computer Entertainment America Inc. System for tracking user manipulations within an environment
US9174119B2 (en) 2002-07-27 2015-11-03 Sony Computer Entertainement America, LLC Controller for providing inputs to control execution of a program when inputs are combined
US8160269B2 (en) 2003-08-27 2012-04-17 Sony Computer Entertainment Inc. Methods and apparatuses for adjusting a listening area for capturing sounds
US7803050B2 (en) * 2002-07-27 2010-09-28 Sony Computer Entertainment Inc. Tracking device with sound emitter for use in obtaining information for controlling game program execution
US8570378B2 (en) 2002-07-27 2013-10-29 Sony Computer Entertainment Inc. Method and apparatus for tracking three-dimensional movements of an object using a depth sensing camera
US8313380B2 (en) * 2002-07-27 2012-11-20 Sony Computer Entertainment America Llc Scheme for translating movements of a hand-held controller into inputs for a system
US7918733B2 (en) * 2002-07-27 2011-04-05 Sony Computer Entertainment America Inc. Multi-input game control mixer
US7854655B2 (en) 2002-07-27 2010-12-21 Sony Computer Entertainment America Inc. Obtaining input for controlling execution of a game program
US8139793B2 (en) 2003-08-27 2012-03-20 Sony Computer Entertainment Inc. Methods and apparatus for capturing audio signals based on a visual image
US8686939B2 (en) 2002-07-27 2014-04-01 Sony Computer Entertainment Inc. System, method, and apparatus for three-dimensional input control
US9393487B2 (en) 2002-07-27 2016-07-19 Sony Interactive Entertainment Inc. Method for mapping movements of a hand-held controller to game commands
US8233642B2 (en) 2003-08-27 2012-07-31 Sony Computer Entertainment Inc. Methods and apparatuses for capturing an audio signal based on a location of the signal
US7665041B2 (en) * 2003-03-25 2010-02-16 Microsoft Corporation Architecture for controlling a computer using hand gestures
US8745541B2 (en) 2003-03-25 2014-06-03 Microsoft Corporation Architecture for controlling a computer using hand gestures
US7038661B2 (en) * 2003-06-13 2006-05-02 Microsoft Corporation Pointing device and cursor for use in intelligent computing environments
US20070223732A1 (en) * 2003-08-27 2007-09-27 Mao Xiao D Methods and apparatuses for adjusting a visual image based on an audio signal
US7874917B2 (en) * 2003-09-15 2011-01-25 Sony Computer Entertainment Inc. Methods and systems for enabling depth and direction detection when interfacing with a computer program
US7663689B2 (en) * 2004-01-16 2010-02-16 Sony Computer Entertainment Inc. Method and apparatus for optimizing capture device settings through depth information
US10277290B2 (en) 2004-04-02 2019-04-30 Rearden, Llc Systems and methods to exploit areas of coherence in wireless systems
US8542763B2 (en) 2004-04-02 2013-09-24 Rearden, Llc Systems and methods to coordinate transmissions in distributed wireless systems via user clustering
US8654815B1 (en) 2004-04-02 2014-02-18 Rearden, Llc System and method for distributed antenna wireless communications
US10425134B2 (en) 2004-04-02 2019-09-24 Rearden, Llc System and methods for planned evolution and obsolescence of multiuser spectrum
US9826537B2 (en) 2004-04-02 2017-11-21 Rearden, Llc System and method for managing inter-cluster handoff of clients which traverse multiple DIDO clusters
US9819403B2 (en) 2004-04-02 2017-11-14 Rearden, Llc System and method for managing handoff of a client between different distributed-input-distributed-output (DIDO) networks based on detected velocity of the client
US8681100B2 (en) 2004-07-30 2014-03-25 Extreme Realty Ltd. Apparatus system and method for human-machine-interface
US8872899B2 (en) * 2004-07-30 2014-10-28 Extreme Reality Ltd. Method circuit and system for human to machine interfacing by hand gestures
US8114172B2 (en) 2004-07-30 2012-02-14 Extreme Reality Ltd. System and method for 3D space-dimension based image processing
US8432390B2 (en) * 2004-07-30 2013-04-30 Extreme Reality Ltd Apparatus system and method for human-machine interface
US8560972B2 (en) * 2004-08-10 2013-10-15 Microsoft Corporation Surface UI for gesture-based interaction
US20060055706A1 (en) * 2004-09-15 2006-03-16 Perlman Stephen G Apparatus and method for capturing the motion of a performer
US7554549B2 (en) * 2004-10-01 2009-06-30 Sony Corporation System and method for tracking facial muscle and eye motion for computer graphics animation
US20060072009A1 (en) * 2004-10-01 2006-04-06 International Business Machines Corporation Flexible interaction-based computer interfacing using visible artifacts
US20060192852A1 (en) * 2005-02-09 2006-08-31 Sally Rosenthal System, method, software arrangement and computer-accessible medium for providing audio and/or visual information
GB2438783B8 (en) * 2005-03-16 2011-12-28 Lucasfilm Entertainment Co Ltd Three-dimensional motion capture
US20110256914A1 (en) * 2005-07-25 2011-10-20 Ahdoot Ned M Interactive games with prediction and plan with assisted learning method
US20070030343A1 (en) * 2005-08-06 2007-02-08 Rohde Mitchell M Interactive, video-based content for theaters
US8054312B2 (en) * 2005-08-26 2011-11-08 Sony Corporation Material for motion capture costumes and props
US8780119B2 (en) * 2005-08-26 2014-07-15 Sony Corporation Reconstruction render farm used in motion capture
US8659668B2 (en) 2005-10-07 2014-02-25 Rearden, Llc Apparatus and method for performing motion capture using a random pattern on capture surfaces
US9046962B2 (en) 2005-10-31 2015-06-02 Extreme Reality Ltd. Methods, systems, apparatuses, circuits and associated computer executable code for detecting motion, position and/or orientation of objects within a defined spatial region
US20070285554A1 (en) 2005-10-31 2007-12-13 Dor Givon Apparatus method and system for imaging
US8224025B2 (en) 2005-12-23 2012-07-17 Sony Corporation Group tracking in motion capture
US7570250B2 (en) * 2006-05-04 2009-08-04 Yi-Ming Tseng Control device including a ball that stores data
US7548272B2 (en) * 2006-06-07 2009-06-16 Onlive, Inc. System and method for performing motion capture using phosphor application techniques
US7667767B2 (en) * 2006-06-07 2010-02-23 Onlive, Inc. System and method for three dimensional capture of stop-motion animated characters
US7567293B2 (en) * 2006-06-07 2009-07-28 Onlive, Inc. System and method for performing motion capture by strobing a fluorescent lamp
CA2657309C (en) * 2006-06-07 2018-07-10 Onlive, Inc. System and method for performing motion capture using phosphor application techniques
WO2008008847A2 (en) * 2006-07-11 2008-01-17 Sony Corporation Using quantum nanodots in motion pictures or video games
US8888592B1 (en) 2009-06-01 2014-11-18 Sony Computer Entertainment America Llc Voice overlay
US7907117B2 (en) 2006-08-08 2011-03-15 Microsoft Corporation Virtual controller for visual displays
US7725547B2 (en) * 2006-09-06 2010-05-25 International Business Machines Corporation Informing a user of gestures made by others out of the user's line of sight
US8310656B2 (en) * 2006-09-28 2012-11-13 Sony Computer Entertainment America Llc Mapping movements of a hand-held controller to the two-dimensional image plane of a display screen
USRE48417E1 (en) 2006-09-28 2021-02-02 Sony Interactive Entertainment Inc. Object direction using video input combined with tilt angle information
US8781151B2 (en) * 2006-09-28 2014-07-15 Sony Computer Entertainment Inc. Object detection using video input combined with tilt angle information
US20080100731A1 (en) * 2006-10-30 2008-05-01 Jerry Moscovitch System and Method for Producing and Displaying Images
US20080170750A1 (en) * 2006-11-01 2008-07-17 Demian Gordon Segment tracking in motion picture
US8330823B2 (en) * 2006-11-01 2012-12-11 Sony Corporation Capturing surface in motion picture
GB0622451D0 (en) * 2006-11-10 2006-12-20 Intelligent Earth Ltd Object position and orientation detection device
US8542236B2 (en) * 2007-01-16 2013-09-24 Lucasfilm Entertainment Company Ltd. Generating animation libraries
US8130225B2 (en) * 2007-01-16 2012-03-06 Lucasfilm Entertainment Company Ltd. Using animation libraries for object identification
US8199152B2 (en) * 2007-01-16 2012-06-12 Lucasfilm Entertainment Company Ltd. Combining multiple session content for animation libraries
CA2717485A1 (en) * 2007-03-02 2008-09-12 Organic Motion System and method for tracking three dimensional objects
EP2147393A4 (en) * 2007-04-15 2012-12-05 Extreme Reality Ltd DEVICE, SYSTEM AND METHOD FOR A HUMAN MACHINE INTERFACE
US8408982B2 (en) 2007-05-24 2013-04-02 Pillar Vision, Inc. Method and apparatus for video game simulations using motion capture
TWI377055B (en) * 2007-08-10 2012-11-21 Ind Tech Res Inst Interactive rehabilitation method and system for upper and lower extremities
US20090062693A1 (en) * 2007-08-29 2009-03-05 Lancastria Limited System for determining individual user anthropometric characteristics related to mattress preference
WO2009035705A1 (en) * 2007-09-14 2009-03-19 Reactrix Systems, Inc. Processing of gesture-based user interactions
US8159682B2 (en) 2007-11-12 2012-04-17 Intellectual Ventures Holding 67 Llc Lens system
US8144153B1 (en) 2007-11-20 2012-03-27 Lucasfilm Entertainment Company Ltd. Model production for animation libraries
US8147339B1 (en) 2007-12-15 2012-04-03 Gaikai Inc. Systems and methods of serving game video
US8613673B2 (en) 2008-12-15 2013-12-24 Sony Computer Entertainment America Llc Intelligent game loading
US8968087B1 (en) 2009-06-01 2015-03-03 Sony Computer Entertainment America Llc Video game overlay
DE102007062843A1 (de) * 2007-12-21 2009-06-25 Amedo Smart Tracking Solutions Gmbh Verfahren zur Bewegungserfassung
US9035876B2 (en) 2008-01-14 2015-05-19 Apple Inc. Three-dimensional user interface session control
US8933876B2 (en) 2010-12-13 2015-01-13 Apple Inc. Three dimensional user interface session control
US8024316B2 (en) 2008-01-30 2011-09-20 Google Inc. Providing content using stored query information
US8259163B2 (en) 2008-03-07 2012-09-04 Intellectual Ventures Holding 67 Llc Display with built in 3D sensing
US8368753B2 (en) * 2008-03-17 2013-02-05 Sony Computer Entertainment America Llc Controller with an integrated depth camera
CN101618277A (zh) * 2008-07-04 2010-01-06 英属维京群岛速位互动股份有限公司 可感测使用者身体影像的电子游戏操控装置及方法
GB2465538B (en) 2008-08-01 2013-03-13 Sony Corp Method and apparatus for generating an event log
EP2342642A1 (en) * 2008-09-04 2011-07-13 Extreme Reality Ltd. Method system and software for providing image sensor based human machine interfacing
WO2010046901A2 (en) 2008-10-24 2010-04-29 Extreme Reality Ltd. A method system and associated modules and software components for providing image sensor based human machine interfacing
US8840476B2 (en) 2008-12-15 2014-09-23 Sony Computer Entertainment America Llc Dual-mode program execution
US8926435B2 (en) 2008-12-15 2015-01-06 Sony Computer Entertainment America Llc Dual-mode program execution
US9142024B2 (en) 2008-12-31 2015-09-22 Lucasfilm Entertainment Company Ltd. Visual and physical motion sensing for three-dimensional motion capture
US20120202569A1 (en) * 2009-01-13 2012-08-09 Primesense Ltd. Three-Dimensional User Interface for Game Applications
US8577085B2 (en) * 2009-01-30 2013-11-05 Microsoft Corporation Visual target tracking
US8295546B2 (en) 2009-01-30 2012-10-23 Microsoft Corporation Pose tracking pipeline
US8294767B2 (en) * 2009-01-30 2012-10-23 Microsoft Corporation Body scan
US8577084B2 (en) * 2009-01-30 2013-11-05 Microsoft Corporation Visual target tracking
US8588465B2 (en) * 2009-01-30 2013-11-19 Microsoft Corporation Visual target tracking
US8682028B2 (en) * 2009-01-30 2014-03-25 Microsoft Corporation Visual target tracking
US8565476B2 (en) * 2009-01-30 2013-10-22 Microsoft Corporation Visual target tracking
US8267781B2 (en) * 2009-01-30 2012-09-18 Microsoft Corporation Visual target tracking
US8487938B2 (en) * 2009-01-30 2013-07-16 Microsoft Corporation Standard Gestures
US8565477B2 (en) * 2009-01-30 2013-10-22 Microsoft Corporation Visual target tracking
WO2010103482A2 (en) * 2009-03-13 2010-09-16 Primesense Ltd. Enhanced 3d interfacing for remote devices
US8988437B2 (en) 2009-03-20 2015-03-24 Microsoft Technology Licensing, Llc Chaining animations
CN101533528B (zh) * 2009-04-18 2014-11-26 大连大学 基于模块分段线性模型的光学运动捕捉数据处理方法
US8181123B2 (en) * 2009-05-01 2012-05-15 Microsoft Corporation Managing virtual port associations to users in a gesture-based computing environment
US20100277470A1 (en) * 2009-05-01 2010-11-04 Microsoft Corporation Systems And Methods For Applying Model Tracking To Motion Capture
US9898675B2 (en) 2009-05-01 2018-02-20 Microsoft Technology Licensing, Llc User movement tracking feedback to improve tracking
US9377857B2 (en) * 2009-05-01 2016-06-28 Microsoft Technology Licensing, Llc Show body position
US9015638B2 (en) * 2009-05-01 2015-04-21 Microsoft Technology Licensing, Llc Binding users to a gesture based system and providing feedback to the users
US8334872B2 (en) * 2009-05-29 2012-12-18 Two Pic Mc Llc Inverse kinematics for motion-capture characters
US8803889B2 (en) * 2009-05-29 2014-08-12 Microsoft Corporation Systems and methods for applying animations or motions to a character
US8418085B2 (en) * 2009-05-29 2013-04-09 Microsoft Corporation Gesture coach
US8320619B2 (en) * 2009-05-29 2012-11-27 Microsoft Corporation Systems and methods for tracking a model
US20100306685A1 (en) * 2009-05-29 2010-12-02 Microsoft Corporation User movement feedback via on-screen avatars
US8379101B2 (en) 2009-05-29 2013-02-19 Microsoft Corporation Environment and/or target segmentation
US20100302365A1 (en) * 2009-05-29 2010-12-02 Microsoft Corporation Depth Image Noise Reduction
US20100302253A1 (en) * 2009-05-29 2010-12-02 Microsoft Corporation Real time retargeting of skeletal data to game avatar
US8506402B2 (en) 2009-06-01 2013-08-13 Sony Computer Entertainment America Llc Game execution environments
US20100311512A1 (en) * 2009-06-04 2010-12-09 Timothy James Lock Simulator with enhanced depth perception
US9218126B2 (en) 2009-09-21 2015-12-22 Extreme Reality Ltd. Methods circuits apparatus and systems for human machine interfacing with an electronic appliance
US8878779B2 (en) 2009-09-21 2014-11-04 Extreme Reality Ltd. Methods circuits device systems and associated computer executable code for facilitating interfacing with a computing platform display screen
US8867820B2 (en) 2009-10-07 2014-10-21 Microsoft Corporation Systems and methods for removing a background of an image
US8963829B2 (en) 2009-10-07 2015-02-24 Microsoft Corporation Methods and systems for determining and tracking extremities of a target
US20110164032A1 (en) * 2010-01-07 2011-07-07 Prime Sense Ltd. Three-Dimensional User Interface
US8284157B2 (en) * 2010-01-15 2012-10-09 Microsoft Corporation Directed performance in motion capture system
US8334842B2 (en) 2010-01-15 2012-12-18 Microsoft Corporation Recognizing user intent in motion capture system
JP5491217B2 (ja) * 2010-01-27 2014-05-14 株式会社バンダイナムコゲームス プログラム、情報記憶媒体、ゲームシステム
US8659658B2 (en) * 2010-02-09 2014-02-25 Microsoft Corporation Physical interaction zone for gesture-based user interfaces
US8633890B2 (en) 2010-02-16 2014-01-21 Microsoft Corporation Gesture detection based on joint skipping
US9124651B2 (en) 2010-03-30 2015-09-01 Microsoft Technology Licensing, Llc Controlling media consumption privacy settings
US9646340B2 (en) 2010-04-01 2017-05-09 Microsoft Technology Licensing, Llc Avatar-based virtual dressing room
US9098873B2 (en) * 2010-04-01 2015-08-04 Microsoft Technology Licensing, Llc Motion-based interactive shopping environment
US11117033B2 (en) 2010-04-26 2021-09-14 Wilbert Quinc Murdock Smart system for display of dynamic movement parameters in sports and training
JP5148660B2 (ja) * 2010-06-11 2013-02-20 株式会社バンダイナムコゲームス プログラム、情報記憶媒体及び画像生成システム
US8676591B1 (en) 2010-08-02 2014-03-18 Sony Computer Entertainment America Llc Audio deceleration
US10039978B2 (en) 2010-09-13 2018-08-07 Sony Interactive Entertainment America Llc Add-on management systems
CN101930628A (zh) * 2010-09-21 2010-12-29 北京大学 基于单目摄像机和多平面镜折反射装置的运动捕捉方法
US9477302B2 (en) * 2012-08-10 2016-10-25 Google Inc. System and method for programing devices within world space volumes
US20120150650A1 (en) * 2010-12-08 2012-06-14 Microsoft Corporation Automatic advertisement generation based on user expressed marketing terms
US8872762B2 (en) 2010-12-08 2014-10-28 Primesense Ltd. Three dimensional user interface cursor control
US9171264B2 (en) 2010-12-15 2015-10-27 Microsoft Technology Licensing, Llc Parallel processing machine learning decision tree training
US8488888B2 (en) * 2010-12-28 2013-07-16 Microsoft Corporation Classification of posture states
KR101758058B1 (ko) 2011-01-20 2017-07-17 삼성전자주식회사 깊이 정보를 이용한 카메라 모션 추정 방법 및 장치, 증강 현실 시스템
US20140031123A1 (en) * 2011-01-21 2014-01-30 The Regents Of The University Of California Systems for and methods of detecting and reproducing motions for video games
WO2012098534A1 (en) 2011-01-23 2012-07-26 Extreme Reality Ltd. Methods, systems, devices and associated processing logic for generating stereoscopic images and video
CN106984041B (zh) * 2011-02-11 2021-07-06 漳州市舟锋电子科技有限公司 一种人机互动控制系统
US8761437B2 (en) * 2011-02-18 2014-06-24 Microsoft Corporation Motion recognition
US9533228B2 (en) 2011-03-28 2017-01-03 Brian M. Dugan Systems and methods for fitness and video games
US9610506B2 (en) 2011-03-28 2017-04-04 Brian M. Dugan Systems and methods for fitness and video games
KR101804848B1 (ko) 2011-04-22 2017-12-06 삼성전자주식회사 비디오 객체 탐색 장치, 비디오 객체 변형 장치 및 그 방법
US8620113B2 (en) 2011-04-25 2013-12-31 Microsoft Corporation Laser diode modes
US10671841B2 (en) 2011-05-02 2020-06-02 Microsoft Technology Licensing, Llc Attribute state classification
US8760395B2 (en) 2011-05-31 2014-06-24 Microsoft Corporation Gesture recognition techniques
US8845431B2 (en) * 2011-05-31 2014-09-30 Microsoft Corporation Shape trace gesturing
US8740702B2 (en) * 2011-05-31 2014-06-03 Microsoft Corporation Action trigger gesturing
US8657683B2 (en) * 2011-05-31 2014-02-25 Microsoft Corporation Action selection gesturing
US9724600B2 (en) * 2011-06-06 2017-08-08 Microsoft Technology Licensing, Llc Controlling objects in a virtual environment
US9377865B2 (en) 2011-07-05 2016-06-28 Apple Inc. Zoom-based gesture user interface
US9459758B2 (en) 2011-07-05 2016-10-04 Apple Inc. Gesture-based interface with enhanced features
US8881051B2 (en) 2011-07-05 2014-11-04 Primesense Ltd Zoom-based gesture user interface
US8948447B2 (en) * 2011-07-12 2015-02-03 Lucasfilm Entertainment Companyy, Ltd. Scale independent tracking pattern
US9030498B2 (en) 2011-08-15 2015-05-12 Apple Inc. Combining explicit select gestures and timeclick in a non-tactile three dimensional user interface
US9218063B2 (en) 2011-08-24 2015-12-22 Apple Inc. Sessionless pointing user interface
KR20130030117A (ko) * 2011-09-16 2013-03-26 한국전자통신연구원 실시간 캐릭터 애니메이션에서의 발 미끄러짐 제거를 위한 캐릭터 영상 처리 장치 및 방법
CN103945931B (zh) 2011-09-26 2017-03-22 基因技术股份公司 高效的小体积核酸合成
US20130093751A1 (en) * 2011-10-12 2013-04-18 Microsoft Corporation Gesture bank to improve skeletal tracking
US9508176B2 (en) 2011-11-18 2016-11-29 Lucasfilm Entertainment Company Ltd. Path and speed based character control
US8635637B2 (en) 2011-12-02 2014-01-21 Microsoft Corporation User interface presenting an animated avatar performing a media reaction
US9100685B2 (en) 2011-12-09 2015-08-04 Microsoft Technology Licensing, Llc Determining audience state or interest using passive sensor data
US8811938B2 (en) 2011-12-16 2014-08-19 Microsoft Corporation Providing a user interface experience based on inferred vehicle state
US9229534B2 (en) 2012-02-28 2016-01-05 Apple Inc. Asymmetric mapping for tactile and non-tactile user interfaces
US8898687B2 (en) 2012-04-04 2014-11-25 Microsoft Corporation Controlling a media program based on a media reaction
CA2775700C (en) 2012-05-04 2013-07-23 Microsoft Corporation Determining a future portion of a currently presented media program
US9170667B2 (en) * 2012-06-01 2015-10-27 Microsoft Technology Licensing, Llc Contextual user interface
WO2014010004A1 (ja) * 2012-07-13 2014-01-16 株式会社ソニー・コンピュータエンタテインメント 入力装置、情報処理システム、情報処理装置、および情報処理方法
US20140018169A1 (en) * 2012-07-16 2014-01-16 Zhong Yuan Ran Self as Avatar Gaming with Video Projecting Device
JP2014068714A (ja) * 2012-09-28 2014-04-21 Kitasato Institute 関節角度測定システム
JP5620449B2 (ja) * 2012-09-28 2014-11-05 エクストリーム リアリティー エルティーディー． 人−機械インターフェース装置システム及び方法
US11189917B2 (en) 2014-04-16 2021-11-30 Rearden, Llc Systems and methods for distributing radioheads
US10488535B2 (en) 2013-03-12 2019-11-26 Rearden, Llc Apparatus and method for capturing still images and video using diffraction coded imaging techniques
US9973246B2 (en) 2013-03-12 2018-05-15 Rearden, Llc Systems and methods for exploiting inter-cell multiplexing gain in wireless cellular systems via distributed input distributed output technology
US9923657B2 (en) 2013-03-12 2018-03-20 Rearden, Llc Systems and methods for exploiting inter-cell multiplexing gain in wireless cellular systems via distributed input distributed output technology
US10547358B2 (en) 2013-03-15 2020-01-28 Rearden, Llc Systems and methods for radio frequency calibration exploiting channel reciprocity in distributed input distributed output wireless communications
US8948457B2 (en) 2013-04-03 2015-02-03 Pillar Vision, Inc. True space tracking of axisymmetric object flight using diameter measurement
US9063578B2 (en) * 2013-07-31 2015-06-23 Microsoft Technology Licensing, Llc Ergonomic physical interaction zone cursor mapping
JP2015088096A (ja) * 2013-11-01 2015-05-07 株式会社ソニー・コンピュータエンタテインメント 情報処理装置および情報処理方法
CN103777753A (zh) * 2013-11-12 2014-05-07 广州新节奏智能科技有限公司 新型便携式肢体感应操控装置及其应用方法
US20160320489A1 (en) * 2013-12-18 2016-11-03 Basf Se Target device for use in optical detection of an object
US9536138B2 (en) 2014-06-27 2017-01-03 Microsoft Technology Licensing, Llc Dynamic remapping of components of a virtual skeleton
US9690984B2 (en) * 2015-04-14 2017-06-27 Microsoft Technology Licensing, Llc Two-dimensional infrared depth sensing
US9836118B2 (en) 2015-06-16 2017-12-05 Wilson Steele Method and system for analyzing a movement of a person
GB201520367D0 (en) * 2015-11-19 2016-01-06 Bespoke Vr Ltd Editing interactive motion capture data for creating the interaction characteristics of non player characters
US10403019B2 (en) * 2015-12-16 2019-09-03 Lucasfilm Entertainment Company Multi-channel tracking pattern
US10717001B2 (en) * 2016-03-25 2020-07-21 Zero Latency PTY LTD System and method for saving tracked data in the game server for replay, review and training
CN105872477B (zh) * 2016-05-27 2018-11-23 北京旷视科技有限公司 视频监控方法和视频监控系统
FR3054358B1 (fr) * 2016-07-25 2018-08-31 Commissariat Energie Atomique Procede et systeme de reconstruction de posture par suivi spatial du haut d'un corps et suivi du bas du corps au moyen d'un tapis de detection
US10751609B2 (en) 2016-08-12 2020-08-25 Zero Latency PTY LTD Mapping arena movements into a 3-D virtual world
CN106710351A (zh) * 2016-12-28 2017-05-24 新支点数字科技（宜昌）有限公司 虚拟实境警务培训装置及方法
CN111954860B (zh) 2018-01-21 2024-05-24 斯塔特斯公司 对细粒度对抗性多队员运动进行预测的系统和方法
WO2019144143A1 (en) 2018-01-21 2019-07-25 Stats Llc Method and system for interactive, interpretable, and improved match and player performance predictions in team sports
US11521326B2 (en) 2018-05-23 2022-12-06 Prove Labs, Inc. Systems and methods for monitoring and evaluating body movement
US11086124B2 (en) 2018-06-13 2021-08-10 Reavire, Inc. Detecting velocity state of a device
GB201813450D0 (en) * 2018-08-17 2018-10-03 Hiltermann Sean Augmented reality doll
CN110942479B (zh) * 2018-09-25 2023-06-02 Oppo广东移动通信有限公司 虚拟对象控制方法、存储介质及电子设备
CN110941977A (zh) * 2018-09-25 2020-03-31 Oppo广东移动通信有限公司 图像处理方法、装置、存储介质及电子设备
CN109407840A (zh) * 2018-10-18 2019-03-01 看见故事（苏州）影视文化发展有限公司 一种动作捕捉技术的视角转换方法
TWI715903B (zh) * 2018-12-24 2021-01-11 財團法人工業技術研究院 動作追蹤系統及方法
JP2022501732A (ja) * 2019-01-18 2022-01-06 北京市商▲湯▼科技▲開▼▲發▼有限公司Beijing Sensetime Technology Development Co., Ltd. 画像処理方法及び装置、画像デバイス並びに記憶媒体
KR20200089826A (ko) 2019-01-18 2020-07-28 (주)모션테크놀로지 모션 캡쳐 데이터 정밀도 향상을 위한 시스템 및 방법
CN111460871B (zh) 2019-01-18 2023-12-22 北京市商汤科技开发有限公司 图像处理方法及装置、存储介质
CN113544697A (zh) 2019-03-01 2021-10-22 斯塔特斯公司 用数据和身体姿态分析运动表现以对表现进行个性化预测
US11238604B1 (en) * 2019-03-05 2022-02-01 Apple Inc. Densifying sparse depth maps
US11287505B2 (en) 2019-05-13 2022-03-29 Cast Group Of Companies Inc. Electronic tracking device and related system
JP7285703B2 (ja) * 2019-06-17 2023-06-02 株式会社ソニー・インタラクティブエンタテインメント ロボット制御システム
US11599257B2 (en) * 2019-11-12 2023-03-07 Cast Group Of Companies Inc. Electronic tracking device and charging apparatus
US20230012053A1 (en) * 2019-12-24 2023-01-12 Sony Group Corporation Information processing device and information processing method
CN111260967A (zh) * 2020-02-25 2020-06-09 湖南科技学院 一种基于图像处理的武术教学用姿势纠正系统
US11011071B1 (en) 2020-03-30 2021-05-18 Mobilizar Technologies Pvt Ltd Interactive learning system and a method
WO2021247371A1 (en) 2020-06-05 2021-12-09 Stats Llc System and method for predicting formation in sports
US11232595B1 (en) 2020-09-08 2022-01-25 Weta Digital Limited Three-dimensional assembly for motion capture calibration
US11282233B1 (en) * 2020-09-08 2022-03-22 Weta Digital Limited Motion capture calibration
CN111968207B (zh) * 2020-09-25 2021-10-29 魔珐(上海)信息科技有限公司 动画生成方法、装置、系统及存储介质
EP4222575A1 (en) * 2020-10-01 2023-08-09 Stats Llc Prediction of nba talent and quality from non-professional tracking data
CA3222789A1 (en) * 2021-05-27 2022-12-01 Ai Thinktank Llc 3d avatar generation and robotic limbs using biomechanical analysis
US11615648B2 (en) 2021-05-28 2023-03-28 Sportsbox.ai Inc. Practice drill-related features using quantitative, biomechanical-based analysis

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5563988A (en) * 1994-08-01 1996-10-08 Massachusetts Institute Of Technology Method and system for facilitating wireless, full-body, real-time user interaction with a digitally represented visual environment
US6195104B1 (en) * 1997-12-23 2001-02-27 Philips Electronics North America Corp. System and method for permitting three-dimensional navigation through a virtual reality environment using camera-based gesture inputs
US6441825B1 (en) * 1999-10-04 2002-08-27 Intel Corporation Video token tracking system for animation
US20030020718A1 (en) * 2001-02-28 2003-01-30 Marshall Carl S. Approximating motion using a three-dimensional model

Family Cites Families (321)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US195104A (en) * 1877-09-11 Improvement in fire-place stoves
US3943277A (en) * 1969-02-20 1976-03-09 The United States Of America As Represented By The Secretary Of The Navy Digital memory area correlation tracker
US4313227A (en) * 1979-01-29 1982-01-26 Texas Instruments Incorporated Light energy information transmission system
US4263504A (en) * 1979-08-01 1981-04-21 Ncr Corporation High density matrix code
US6772057B2 (en) 1995-06-07 2004-08-03 Automotive Technologies International, Inc. Vehicular monitoring systems using image processing
US4565999A (en) * 1983-04-01 1986-01-21 Prime Computer, Inc. Light pencil
US4558864A (en) 1984-06-13 1985-12-17 Medwedeff Marion C Handgrip exercising, computer game controller
US5195179A (en) * 1986-01-29 1993-03-16 Hitachi, Ltd. Coordinate input apparatus
US4843568A (en) 1986-04-11 1989-06-27 Krueger Myron W Real time perception of and response to the actions of an unencumbered participant/user
US4787051A (en) 1986-05-16 1988-11-22 Tektronix, Inc. Inertial mouse system
JPS6347616A (ja) * 1986-08-15 1988-02-29 Ricoh Co Ltd 移動量測定方法
EP0348430A4 (en) 1987-02-04 1992-08-19 Mayo Foundation For Medical Education And Research Joystick apparatus having six degrees freedom of motion
US4802227A (en) * 1987-04-03 1989-01-31 American Telephone And Telegraph Company Noise reduction processing arrangement for microphone arrays
GB2206716A (en) 1987-07-06 1989-01-11 Gen Electric Plc Apparatus for monitoring the presence or movement of an object
US4963858A (en) 1987-09-08 1990-10-16 Chien Fong K Changeable input ratio mouse
IT1219405B (it) 1988-06-27 1990-05-11 Fiat Ricerche Procedimento e dispositivo per la visione strumentale in condizioni di scarsa visibilita in particolare per la guida nella nebbia
JPH05501020A (ja) 1988-12-20 1993-02-25 ミート　リサーチ　コーポレーション 光学的に読取り可能なコード化された物標
US5034986A (en) 1989-03-01 1991-07-23 Siemens Aktiengesellschaft Method for detecting and tracking moving objects in a digital image sequence having a stationary background
US5055840A (en) 1990-01-16 1991-10-08 Carroll Touch Incorporated Infrared touch input device and light emitted activation circuit
US5128671A (en) 1990-04-12 1992-07-07 Ltv Aerospace And Defense Company Control device having multiple degrees of freedom
DE69016463T2 (de) 1990-05-01 1995-09-07 Wang Laboratories Handfreie hardwaretastatur.
US5662111A (en) * 1991-01-28 1997-09-02 Cosman; Eric R. Process of stereotactic optical navigation
US5485273A (en) * 1991-04-22 1996-01-16 Litton Systems, Inc. Ring laser gyroscope enhanced resolution system
US5534917A (en) 1991-05-09 1996-07-09 Very Vivid, Inc. Video image based control system
US5144594A (en) 1991-05-29 1992-09-01 Cyber Scientific Acoustic mouse system
US5455685A (en) 1991-09-04 1995-10-03 Fuji Photo Film Co., Ltd. Video camera exposure control apparatus for controlling iris diaphragm and automatic gain control operating speed
US5889670A (en) * 1991-10-24 1999-03-30 Immersion Corporation Method and apparatus for tactilely responsive user interface
US5444462A (en) 1991-12-16 1995-08-22 Wambach; Mark L. Computer mouse glove with remote communication
US5453758A (en) 1992-07-31 1995-09-26 Sony Corporation Input apparatus
US5790834A (en) 1992-08-31 1998-08-04 Intel Corporation Apparatus and method using an ID instruction to identify a computer microprocessor
JP3244798B2 (ja) 1992-09-08 2002-01-07 株式会社東芝 動画像処理装置
US5982352A (en) * 1992-09-18 1999-11-09 Pryor; Timothy R. Method for providing human input to a computer
US7098891B1 (en) 1992-09-18 2006-08-29 Pryor Timothy R Method for providing human input to a computer
US5394168A (en) * 1993-01-06 1995-02-28 Smith Engineering Dual-mode hand-held game controller
US5335011A (en) 1993-01-12 1994-08-02 Bell Communications Research, Inc. Sound localization system for teleconferencing using self-steering microphone arrays
EP0613294B1 (en) 1993-02-24 1998-10-28 Matsushita Electric Industrial Co., Ltd. Gradation correction device and image sensing device therewith
US5435554A (en) 1993-03-08 1995-07-25 Atari Games Corporation Baseball simulation system
JP3679426B2 (ja) 1993-03-15 2005-08-03 マサチューセッツ・インスティチュート・オブ・テクノロジー 画像データを符号化して夫々がコヒーレントな動きの領域を表わす複数の層とそれら層に付随する動きパラメータとにするシステム
US5677710A (en) 1993-05-10 1997-10-14 Apple Computer, Inc. Recognition keypad
US5297061A (en) * 1993-05-19 1994-03-22 University Of Maryland Three dimensional pointing device monitored by computer vision
US5959596A (en) 1993-06-24 1999-09-28 Nintendo Co., Ltd. Airline-based video game and communications system
US5473701A (en) 1993-11-05 1995-12-05 At&T Corp. Adaptive microphone array
JP2552427B2 (ja) * 1993-12-28 1996-11-13 コナミ株式会社 テレビ遊戯システム
FR2714502A1 (fr) 1993-12-29 1995-06-30 Philips Laboratoire Electroniq Procédé et dispositif de traitement d'image pour construire à partir d'une image source une image cible avec changement de perspective.
US5611000A (en) * 1994-02-22 1997-03-11 Digital Equipment Corporation Spline-based image registration
JPH086708A (ja) * 1994-04-22 1996-01-12 Canon Inc 表示装置
US5543818A (en) 1994-05-13 1996-08-06 Sony Corporation Method and apparatus for entering text using an input device having a small number of keys
US5846086A (en) 1994-07-01 1998-12-08 Massachusetts Institute Of Technology System for human trajectory learning in virtual environments
US5528265A (en) 1994-07-18 1996-06-18 Harrison; Simon J. Orientation-operated cursor control device
SE504846C2 (sv) 1994-09-28 1997-05-12 Jan G Faeger Styrutrustning med ett rörligt styrorgan
JP3270643B2 (ja) 1994-12-22 2002-04-02 キヤノン株式会社 指示位置検出方法及び装置
US5929444A (en) 1995-01-31 1999-07-27 Hewlett-Packard Company Aiming device using radiated energy
US5568928A (en) 1995-02-01 1996-10-29 Exertron, Inc. Video game controller for use with an exercise apparatus
US5638228A (en) 1995-02-14 1997-06-10 Iomega Corporation Retroreflective marker for data storage cartridge
US5930741A (en) 1995-02-28 1999-07-27 Virtual Technologies, Inc. Accurate, rapid, reliable position sensing using multiple sensing technologies
US5583478A (en) 1995-03-01 1996-12-10 Renzi; Ronald Virtual environment tactile system
US5900863A (en) * 1995-03-16 1999-05-04 Kabushiki Kaisha Toshiba Method and apparatus for controlling computer without touching input device
US5704836A (en) * 1995-03-23 1998-01-06 Perception Systems, Inc. Motion-based command generation technology
WO1996034332A1 (fr) * 1995-04-28 1996-10-31 Matsushita Electric Industrial Co., Ltd. Dispositif d'interface
US5706364A (en) * 1995-04-28 1998-01-06 Xerox Corporation Method of producing character templates using unsegmented samples
US5913727A (en) 1995-06-02 1999-06-22 Ahdoot; Ned Interactive movement and contact simulation game
US5649021A (en) 1995-06-07 1997-07-15 David Sarnoff Research Center, Inc. Method and system for object detection for instrument control
IL114278A (en) 1995-06-22 2010-06-16 Microsoft Internat Holdings B Camera and method
EP0886790B1 (en) * 1995-06-22 2006-03-01 3DV Systems Ltd. Telecentric 3d camera and method
CN1154910C (zh) 1995-07-26 2004-06-23 蒂吉通信系统公司 压缩键盘的明义系统
US6311214B1 (en) * 1995-07-27 2001-10-30 Digimarc Corporation Linking of computers based on optical sensing of digital data
US5768415A (en) 1995-09-08 1998-06-16 Lucent Technologies Inc. Apparatus and methods for performing electronic scene analysis and enhancement
US5611731A (en) * 1995-09-08 1997-03-18 Thrustmaster, Inc. Video pinball machine controller having an optical accelerometer for detecting slide and tilt
US5850222A (en) 1995-09-13 1998-12-15 Pixel Dust, Inc. Method and system for displaying a graphic image of a person modeling a garment
US5818424A (en) 1995-10-19 1998-10-06 International Business Machines Corporation Rod shaped device and data acquisition apparatus for determining the position and orientation of an object in space
US6281930B1 (en) 1995-10-20 2001-08-28 Parkervision, Inc. System and method for controlling the field of view of a camera
US5963250A (en) 1995-10-20 1999-10-05 Parkervision, Inc. System and method for controlling the field of view of a camera
US5719561A (en) 1995-10-25 1998-02-17 Gilbert R. Gonzales Tactile communication device and method
US6430997B1 (en) * 1995-11-06 2002-08-13 Trazer Technologies, Inc. System and method for tracking and assessing movement skills in multidimensional space
US5870100A (en) * 1995-11-22 1999-02-09 Compaq Computer Corporation Filling of graphical regions
WO1997020305A1 (en) 1995-11-30 1997-06-05 Virtual Technologies, Inc. Tactile feedback man-machine interface device
JP3014313B2 (ja) 1995-12-25 2000-02-28 富士通テン株式会社 エアバッグの衝突判別装置
JP4079463B2 (ja) * 1996-01-26 2008-04-23 ソニー株式会社 被写体検出装置および被写体検出方法
US6049619A (en) * 1996-02-12 2000-04-11 Sarnoff Corporation Method and apparatus for detecting moving objects in two- and three-dimensional scenes
JP2891159B2 (ja) 1996-02-14 1999-05-17 日本電気株式会社 多眼画像からの物体検出方式
RU2069885C1 (ru) 1996-03-01 1996-11-27 Йелстаун Корпорейшн Н.В. Способ наблюдения объектов при пониженной освещенности и устройство для его осуществления
JPH09244793A (ja) * 1996-03-04 1997-09-19 Alps Electric Co Ltd 入力装置
EP0835676B1 (en) 1996-03-05 2004-10-13 Sega Enterprises, Ltd. Controller and extension unit for controller
WO1997039416A2 (en) * 1996-04-02 1997-10-23 Cognex Corporation Image formation apparatus for viewing indicia on a planar specular substrate
US5937081A (en) 1996-04-10 1999-08-10 O'brill; Michael R. Image composition system and method of using same
US5923318A (en) 1996-04-12 1999-07-13 Zhai; Shumin Finger manipulatable 6 degree-of-freedom input device
US5917493A (en) 1996-04-17 1999-06-29 Hewlett-Packard Company Method and apparatus for randomly generating information for subsequent correlating
US6151009A (en) 1996-08-21 2000-11-21 Carnegie Mellon University Method and apparatus for merging real and synthetic images
US6400374B2 (en) 1996-09-18 2002-06-04 Eyematic Interfaces, Inc. Video superposition system and method
US5930383A (en) 1996-09-24 1999-07-27 Netzer; Yishay Depth sensing camera systems and methods
US5832931A (en) * 1996-10-30 1998-11-10 Photogen, Inc. Method for improved selectivity in photo-activation and detection of molecular diagnostic agents
NL1004648C2 (nl) * 1996-11-11 1998-05-14 Johan Michiel Schaaij Computerspelsysteem.
US5914723A (en) 1996-12-30 1999-06-22 Sun Microsystems, Inc. Method and system for converting images in computer systems
US6243491B1 (en) 1996-12-31 2001-06-05 Lucent Technologies Inc. Methods and apparatus for controlling a video system with visually recognized props
US5850473A (en) 1997-01-07 1998-12-15 Lucent Technologies Inc. Method and apparatus for compensating for color variation in a video system
US6021219A (en) * 1997-01-07 2000-02-01 Lucent Technologies Inc. Methods and apparatus for distinguishing among several visual patterns
US5796354A (en) 1997-02-07 1998-08-18 Reality Quest Corp. Hand-attachable controller with direction sensing
US5993314A (en) 1997-02-10 1999-11-30 Stadium Games, Ltd. Method and apparatus for interactive audience participation by audio command
US6009210A (en) 1997-03-05 1999-12-28 Digital Equipment Corporation Hands-free interface to a virtual reality environment using head tracking
AU2228197A (en) 1997-03-07 1998-09-22 3Dv Systems Ltd. Optical shutter
US6061055A (en) 1997-03-21 2000-05-09 Autodesk, Inc. Method of tracking objects with an imaging device
US6144367A (en) 1997-03-26 2000-11-07 International Business Machines Corporation Method and system for simultaneous operation of multiple handheld control devices in a data processing system
US6088042A (en) * 1997-03-31 2000-07-11 Katrix, Inc. Interactive motion data animation system
US6587573B1 (en) 2000-03-20 2003-07-01 Gentex Corporation System for controlling exterior vehicle lights
US8120652B2 (en) * 1997-04-02 2012-02-21 Gentex Corporation System for controlling vehicle equipment
US6215898B1 (en) * 1997-04-15 2001-04-10 Interval Research Corporation Data processing system and method
WO1998048571A1 (en) 1997-04-23 1998-10-29 Thomson Consumer Electronics, Inc. Control of video level by region and content of information displayed
US6809776B1 (en) 1997-04-23 2004-10-26 Thomson Licensing S.A. Control of video level by region and content of information displayed
US6428411B1 (en) 1997-05-02 2002-08-06 Konami Co., Ltd. Volleyball video game system
JPH10334270A (ja) * 1997-05-28 1998-12-18 Mitsubishi Electric Corp 動作認識装置及び動作認識プログラムを記録した記録媒体
US6075895A (en) * 1997-06-20 2000-06-13 Holoplex Methods and apparatus for gesture recognition based on templates
DE69811050T2 (de) 1997-07-29 2003-11-06 Koninkl Philips Electronics Nv Rekonstruktionsverfahren, Vorrichtung und Dekodierungssystem für dreidimensionalen Szenen.
US6044181A (en) * 1997-08-01 2000-03-28 Microsoft Corporation Focal length estimation method and apparatus for construction of panoramic mosaic images
US6720949B1 (en) 1997-08-22 2004-04-13 Timothy R. Pryor Man machine interfaces and applications
US20020036617A1 (en) * 1998-08-21 2002-03-28 Timothy R. Pryor Novel man machine interfaces and applications
US6243074B1 (en) 1997-08-29 2001-06-05 Xerox Corporation Handedness detection for a physical manipulatory grammar
US6297838B1 (en) 1997-08-29 2001-10-02 Xerox Corporation Spinning as a morpheme for a physical manipulatory grammar
US6072494A (en) 1997-10-15 2000-06-06 Electric Planet, Inc. Method and apparatus for real-time gesture recognition
AU1099899A (en) 1997-10-15 1999-05-03 Electric Planet, Inc. Method and apparatus for performing a clean background subtraction
US6101289A (en) 1997-10-15 2000-08-08 Electric Planet, Inc. Method and apparatus for unencumbered capture of an object
US6031934A (en) * 1997-10-15 2000-02-29 Electric Planet, Inc. Computer vision system for subject characterization
JP3812092B2 (ja) * 1997-10-27 2006-08-23 ソニー株式会社 姿勢検出装置及び方法
WO1999026198A2 (en) 1997-11-14 1999-05-27 National University Of Singapore System and method for merging objects into an image sequence without prior knowledge of the scene in the image sequence
JPH11154240A (ja) * 1997-11-20 1999-06-08 Nintendo Co Ltd 取込み画像を用いて画像を作成するための画像作成装置
US6677987B1 (en) * 1997-12-03 2004-01-13 8×8, Inc. Wireless user-interface arrangement and method
US6160540A (en) 1998-01-12 2000-12-12 Xerox Company Zoomorphic computer user interface
US6134346A (en) 1998-01-16 2000-10-17 Ultimatte Corp Method for removing from an image the background surrounding a selected object
JP4195189B2 (ja) 1998-02-08 2008-12-10 スリーディーヴィー システムズ リミテッド 大開口光画像シャッタ
US6115052A (en) * 1998-02-12 2000-09-05 Mitsubishi Electric Information Technology Center America, Inc. (Ita) System for reconstructing the 3-dimensional motions of a human figure from a monocularly-viewed image sequence
US6037942A (en) * 1998-03-10 2000-03-14 Magellan Dis, Inc. Navigation system character input device
JPH11265249A (ja) 1998-03-17 1999-09-28 Toshiba Corp 情報入力装置および情報入力方法および記録媒体
US6411392B1 (en) 1998-04-15 2002-06-25 Massachusetts Institute Of Technology Method and apparatus for data hiding in printed images
US6393142B1 (en) 1998-04-22 2002-05-21 At&T Corp. Method and apparatus for adaptive stripe based patch matching for depth estimation
US6173059B1 (en) * 1998-04-24 2001-01-09 Gentner Communications Corporation Teleconferencing system with visual feedback
JP2000033184A (ja) * 1998-05-14 2000-02-02 Masanobu Kujirada 全身動作入力型のゲ―ム及びイベント装置
US6593956B1 (en) 1998-05-15 2003-07-15 Polycom, Inc. Locating an audio source
US6473516B1 (en) 1998-05-22 2002-10-29 Asa Systems, Inc. Large capacity steganography
JP3646969B2 (ja) 1998-05-25 2005-05-11 富士通株式会社 ３次元画像表示装置
JP3841132B2 (ja) 1998-06-01 2006-11-01 株式会社ソニー・コンピュータエンタテインメント 入力位置検出装置及びエンタテインメントシステム
FR2780176B1 (fr) 1998-06-17 2001-01-26 Gabriel Guary Pistolet de tir pour jeu video
US6513160B2 (en) * 1998-06-17 2003-01-28 Opentv, Inc. System and method for promoting viewer interaction in a television system
IL125221A0 (en) * 1998-07-06 1999-03-12 Toy Control Ltd Motion activation using passive sound source
US6198485B1 (en) * 1998-07-29 2001-03-06 Intel Corporation Method and apparatus for three-dimensional input entry
US6801637B2 (en) * 1999-08-10 2004-10-05 Cybernet Systems Corporation Optical body tracker
US6681031B2 (en) * 1998-08-10 2004-01-20 Cybernet Systems Corporation Gesture-controlled interfaces for self-service machines and other applications
US6256398B1 (en) 1998-08-22 2001-07-03 Kenneth H. P. Chang Encoding and decoding a message within an image
US6970183B1 (en) 2000-06-14 2005-11-29 E-Watch, Inc. Multimedia surveillance and monitoring system including network configuration
US6621938B1 (en) 1998-09-18 2003-09-16 Fuji Photo Film Co., Ltd. Image capture apparatus and method
US6184863B1 (en) * 1998-10-13 2001-02-06 The George Washington University Direct pointing apparatus and method therefor
DE19849515C1 (de) 1998-10-19 2000-12-28 Gerhard Wergen Verfahren zur Übergabe von Zeichen insbesondere an einen Computer und Eingabevorrichtung unter Einsatz dieses Verfahrens
US6307568B1 (en) 1998-10-28 2001-10-23 Imaginarix Ltd. Virtual dressing over the internet
US6712703B2 (en) * 1998-11-19 2004-03-30 Nintendo Co., Ltd. Video game apparatus and information storage medium for video game
JP2000172431A (ja) 1998-12-09 2000-06-23 Sony Corp 情報入力装置及びゲーム装置
JP2000200088A (ja) * 1998-12-29 2000-07-18 Nagase & Co Ltd カラオケ装置
US6533420B1 (en) * 1999-01-22 2003-03-18 Dimension Technologies, Inc. Apparatus and method for generating and projecting autostereoscopic images
JP2000261752A (ja) 1999-03-05 2000-09-22 Hewlett Packard Co <Hp> 音声付き画像録画再生装置
US7139767B1 (en) * 1999-03-05 2006-11-21 Canon Kabushiki Kaisha Image processing apparatus and database
US6323942B1 (en) 1999-04-30 2001-11-27 Canesta, Inc. CMOS-compatible three-dimensional image sensor IC
JP2000315259A (ja) * 1999-05-06 2000-11-14 Sharp Corp データベース作成装置及びデータベース作成プログラムを記録した記録媒体
US7164413B2 (en) * 1999-05-19 2007-01-16 Digimarc Corporation Enhanced input peripheral
US6791531B1 (en) 1999-06-07 2004-09-14 Dot On, Inc. Device and method for cursor motion control calibration and object selection
JP2000350865A (ja) 1999-06-11 2000-12-19 Mr System Kenkyusho:Kk 複合現実空間のゲーム装置、その画像処理方法およびプログラム記憶媒体
US6952198B2 (en) 1999-07-06 2005-10-04 Hansen Karl C System and method for communication with enhanced optical pointer
US6819318B1 (en) 1999-07-23 2004-11-16 Z. Jason Geng Method and apparatus for modeling via a three-dimensional image mosaic system
US6545706B1 (en) * 1999-07-30 2003-04-08 Electric Planet, Inc. System, method and article of manufacture for tracking a head of a camera-generated image of a person
US6417836B1 (en) 1999-08-02 2002-07-09 Lucent Technologies Inc. Computer input device having six degrees of freedom for controlling movement of a three-dimensional object
JP2001056742A (ja) 1999-08-19 2001-02-27 Alps Electric Co Ltd 入力装置
US6556704B1 (en) * 1999-08-25 2003-04-29 Eastman Kodak Company Method for forming a depth image from digital image data
AU5646299A (en) 1999-09-08 2001-04-10 3Dv Systems Ltd. 3d imaging system
CA2385173A1 (en) 1999-09-17 2001-03-29 Nature Technology Co., Ltd. Image capturing system, image processing system, and camera
US6375572B1 (en) * 1999-10-04 2002-04-23 Nintendo Co., Ltd. Portable game apparatus with acceleration sensor and information storage medium storing a game progam
JP3847058B2 (ja) 1999-10-04 2006-11-15 任天堂株式会社 ゲームシステム及びそれに用いられるゲーム情報記憶媒体
US6753849B1 (en) 1999-10-27 2004-06-22 Ken Curran & Associates Universal remote TV mouse
US6519359B1 (en) * 1999-10-28 2003-02-11 General Electric Company Range camera controller for acquiring 3D models
US20050037844A1 (en) * 2002-10-30 2005-02-17 Nike, Inc. Sigils for use with apparel
JP4403474B2 (ja) 1999-12-09 2010-01-27 ソニー株式会社 触覚提示機構及びこれを用いた力触覚提示装置
US6785329B1 (en) 1999-12-21 2004-08-31 Microsoft Corporation Automatic video object extraction
KR100384406B1 (ko) * 2000-01-24 2003-05-22 (주) 베스트소프트 컴퓨터 프로그램 구동조작장치
US6587835B1 (en) 2000-02-09 2003-07-01 G. Victor Treyz Shopping assistance with handheld computing device
TW522732B (en) * 2000-02-15 2003-03-01 Sorceron Inc Method and system for distributing captured motion data over a network
WO2001064481A2 (en) 2000-03-02 2001-09-07 Donnelly Corporation Video mirror systems incorporating an accessory module
JP2001265275A (ja) * 2000-03-16 2001-09-28 Olympus Optical Co Ltd 画像表示装置
US6924787B2 (en) * 2000-04-17 2005-08-02 Immersion Corporation Interface for controlling a graphical image
JP3652961B2 (ja) * 2000-06-27 2005-05-25 シャープ株式会社 音声処理装置、音声・動画像処理装置及び音声・動画像処理プログラムを記録した記録媒体
TW527518B (en) * 2000-07-14 2003-04-11 Massachusetts Inst Technology Method and system for high resolution, ultra fast, 3-D imaging
US6795068B1 (en) 2000-07-21 2004-09-21 Sony Computer Entertainment Inc. Prop input device and method for mapping an object from a two-dimensional camera image to a three-dimensional space for controlling action in a game program
US7227526B2 (en) 2000-07-24 2007-06-05 Gesturetek, Inc. Video-based image control system
WO2002008849A2 (en) 2000-07-24 2002-01-31 Herman Ehrenburg Computer-compatible, color-coded manual input system
AUPQ896000A0 (en) 2000-07-24 2000-08-17 Seeing Machines Pty Ltd Facial image processing system
US6873747B2 (en) * 2000-07-25 2005-03-29 Farid Askary Method for measurement of pitch in metrology and imaging systems
JP4278288B2 (ja) * 2000-07-31 2009-06-10 株式会社タイトー 踊り映像強調合成装置
AUPQ952400A0 (en) * 2000-08-18 2000-09-14 Telefonaktiebolaget Lm Ericsson (Publ) Improved method and system of effecting a financial transaction
JP4815661B2 (ja) 2000-08-24 2011-11-16 ソニー株式会社 信号処理装置及び信号処理方法
AU2001294452A1 (en) 2000-09-29 2002-04-08 Senseboard Technologies Ab Wearable data input interface
FR2814965A1 (fr) 2000-10-11 2002-04-12 Janick Simeray Ordinateur et jouets cooperants
US7016532B2 (en) * 2000-11-06 2006-03-21 Evryx Technologies Image capture and identification system and process
US7061507B1 (en) 2000-11-12 2006-06-13 Bitboys, Inc. Antialiasing method and apparatus for video applications
KR100385563B1 (ko) * 2000-12-01 2003-05-27 한국과학기술원 구동수단을 갖는 분광 광도계와 광도 측정방법
US6751338B1 (en) 2000-12-15 2004-06-15 Cognex Corporation System and method of using range image data with machine vision tools
US20020085097A1 (en) 2000-12-22 2002-07-04 Colmenarez Antonio J. Computer vision-based wireless pointing system
EP1371019A2 (en) 2001-01-26 2003-12-17 Zaxel Systems, Inc. Real-time virtual viewpoint in simulated reality environment
DE10103922A1 (de) 2001-01-30 2002-08-01 Physoptics Opto Electronic Gmb Interaktives Datensicht- und Bediensystem
US6741741B2 (en) 2001-02-01 2004-05-25 Xerox Corporation System and method for automatically detecting edges of scanned documents
US6789967B1 (en) 2001-02-02 2004-09-14 George Forester Distal chording keyboard
US20020134151A1 (en) 2001-02-05 2002-09-26 Matsushita Electric Industrial Co., Ltd. Apparatus and method for measuring distances
US6746124B2 (en) 2001-02-06 2004-06-08 Robert E. Fischer Flashlight producing uniform high brightness
GB2376397A (en) 2001-06-04 2002-12-11 Hewlett Packard Co Virtual or augmented reality
JP2003018604A (ja) * 2001-07-04 2003-01-17 Matsushita Electric Ind Co Ltd 画像信号符号化方法、画像信号符号化装置および記録媒体
US6931125B2 (en) 2001-07-10 2005-08-16 Gregory N. Smallwood Telephone equipment compatible, twelve button alphanumeric keypad
KR20030009919A (ko) * 2001-07-24 2003-02-05 삼성전자주식회사 관성 센서를 구비하는 컴퓨터 게임용 입력 장치
US7039253B2 (en) 2001-07-24 2006-05-02 Casio Computer Co., Ltd. Image display device, image display method, program, and projection system
US7148922B2 (en) 2001-08-02 2006-12-12 Olympus Optical Co., Ltd. Electronic camera which detects flash influence on an image and controls white balance in accordance with the flash influence
JP3442754B2 (ja) * 2001-08-10 2003-09-02 株式会社コナミコンピュータエンタテインメント東京 ガンシューティングゲーム装置、コンピュータの制御方法及びプログラム
AU2002323134A1 (en) * 2001-08-16 2003-03-03 Trans World New York Llc User-personalized media sampling, recommendation and purchasing system using real-time inventory database
US6709108B2 (en) * 2001-08-31 2004-03-23 Adaptive Optics Associates, Inc. Ophthalmic instrument with adaptive optic subsystem that measures aberrations (including higher order aberrations) of a human eye and that provides a view of compensation of such aberrations to the human eye
US7555157B2 (en) 2001-09-07 2009-06-30 Geoff Davidson System and method for transforming graphical images
KR100846761B1 (ko) 2001-09-11 2008-07-16 삼성전자주식회사 포인터 표시방법, 그 포인팅 장치, 및 그 호스트 장치
JP2003114640A (ja) * 2001-10-04 2003-04-18 Nec Corp プラズマディスプレイパネル及びその駆動方法
US20030093591A1 (en) 2001-11-09 2003-05-15 David Hohl System and method for fast data transfer to display driver
FR2832892B1 (fr) 2001-11-27 2004-04-02 Thomson Licensing Sa Camera video d'effets speciaux
US20030100363A1 (en) 2001-11-28 2003-05-29 Ali Guiseppe C. Method and apparatus for inputting appearance of computer operator into a computer program
US20040070565A1 (en) * 2001-12-05 2004-04-15 Nayar Shree K Method and apparatus for displaying images
KR20030048570A (ko) 2001-12-12 2003-06-25 한국전자통신연구원 부가적 버튼을 장착한 키패드 및 그의 작동방법
US7106366B2 (en) 2001-12-19 2006-09-12 Eastman Kodak Company Image capture system incorporating metadata to facilitate transcoding
US7305114B2 (en) 2001-12-26 2007-12-04 Cognex Technology And Investment Corporation Human/machine interface for a machine vision sensor and method for installing and operating the same
US7436887B2 (en) 2002-02-06 2008-10-14 Playtex Products, Inc. Method and apparatus for video frame sequence-based object tracking
US6990639B2 (en) 2002-02-07 2006-01-24 Microsoft Corporation System and process for controlling electronic components in a ubiquitous computing environment using multimodal integration
US20030160862A1 (en) 2002-02-27 2003-08-28 Charlier Michael L. Apparatus having cooperating wide-angle digital camera system and microphone array
US20030167211A1 (en) 2002-03-04 2003-09-04 Marco Scibora Method and apparatus for digitally marking media content
US7301547B2 (en) 2002-03-22 2007-11-27 Intel Corporation Augmented reality system
FR2837597A1 (fr) 2002-03-25 2003-09-26 Thomson Licensing Sa Procede de modelisation d'une scene 3d
US6847311B2 (en) * 2002-03-28 2005-01-25 Motorola Inc. Method and apparatus for character entry in a wireless communication device
GB2388418A (en) 2002-03-28 2003-11-12 Marcus James Eales Input or pointing device with a camera
US20030199324A1 (en) 2002-04-23 2003-10-23 Xiaoling Wang Apparatus and a method for more realistic shooting video games on computers or similar devices using visible or invisible light
JP2004021345A (ja) 2002-06-12 2004-01-22 Toshiba Corp 画像処理装置及びその方法
US20040001082A1 (en) * 2002-06-26 2004-01-01 Amir Said System and method of interaction with a computer controlled image display system using a projected light source
CA2390072C (en) 2002-06-28 2018-02-27 Adrian Gh Podoleanu Optical mapping apparatus with adjustable depth resolution and multiple functionality
US7227976B1 (en) 2002-07-08 2007-06-05 Videomining Corporation Method and system for real-time facial image enhancement
CN1167996C (zh) * 2002-07-11 2004-09-22 庄胜雄 可形成虚拟追踪球的手套滑鼠
US7613310B2 (en) * 2003-08-27 2009-11-03 Sony Computer Entertainment Inc. Audio input system
US7809145B2 (en) 2006-05-04 2010-10-05 Sony Computer Entertainment Inc. Ultra small microphone array
US7102615B2 (en) 2002-07-27 2006-09-05 Sony Computer Entertainment Inc. Man-machine interface using a deformable device
US7697700B2 (en) * 2006-05-04 2010-04-13 Sony Computer Entertainment Inc. Noise removal for electronic device with far field microphone on console
US8073157B2 (en) 2003-08-27 2011-12-06 Sony Computer Entertainment Inc. Methods and apparatus for targeted sound detection and characterization
US8797260B2 (en) 2002-07-27 2014-08-05 Sony Computer Entertainment Inc. Inertially trackable hand-held controller
US7646372B2 (en) * 2003-09-15 2010-01-12 Sony Computer Entertainment Inc. Methods and systems for enabling direction detection when interfacing with a computer program
US7545926B2 (en) 2006-05-04 2009-06-09 Sony Computer Entertainment Inc. Echo and noise cancellation
US7970147B2 (en) 2004-04-07 2011-06-28 Sony Computer Entertainment Inc. Video game controller with noise canceling logic
US7783061B2 (en) * 2003-08-27 2010-08-24 Sony Computer Entertainment Inc. Methods and apparatus for the targeted sound detection
US7623115B2 (en) 2002-07-27 2009-11-24 Sony Computer Entertainment Inc. Method and apparatus for light input device
US20040017355A1 (en) * 2002-07-24 2004-01-29 Youngtack Shim Cursor control systems and methods
US8686939B2 (en) 2002-07-27 2014-04-01 Sony Computer Entertainment Inc. System, method, and apparatus for three-dimensional input control
US7850526B2 (en) 2002-07-27 2010-12-14 Sony Computer Entertainment America Inc. System for tracking user manipulations within an environment
US20070061413A1 (en) * 2005-09-15 2007-03-15 Larsen Eric J System and method for obtaining user information from voices
US8313380B2 (en) 2002-07-27 2012-11-20 Sony Computer Entertainment America Llc Scheme for translating movements of a hand-held controller into inputs for a system
US7391409B2 (en) 2002-07-27 2008-06-24 Sony Computer Entertainment America Inc. Method and system for applying gearing effects to multi-channel mixed input
US20070015559A1 (en) * 2002-07-27 2007-01-18 Sony Computer Entertainment America Inc. Method and apparatus for use in determining lack of user activity in relation to a system
US7760248B2 (en) 2002-07-27 2010-07-20 Sony Computer Entertainment Inc. Selective sound source listening in conjunction with computer interactive processing
US7803050B2 (en) 2002-07-27 2010-09-28 Sony Computer Entertainment Inc. Tracking device with sound emitter for use in obtaining information for controlling game program execution
US7627139B2 (en) 2002-07-27 2009-12-01 Sony Computer Entertainment Inc. Computer image and audio processing of intensity and input devices for interfacing with a computer program
US20060282873A1 (en) 2002-07-27 2006-12-14 Sony Computer Entertainment Inc. Hand-held controller having detectable elements for tracking purposes
US8233642B2 (en) 2003-08-27 2012-07-31 Sony Computer Entertainment Inc. Methods and apparatuses for capturing an audio signal based on a location of the signal
US20070261077A1 (en) 2006-05-08 2007-11-08 Gary Zalewski Using audio/visual environment to select ads on game platform
US10086282B2 (en) 2002-07-27 2018-10-02 Sony Interactive Entertainment Inc. Tracking device for use in obtaining information for controlling game program execution
US8160269B2 (en) 2003-08-27 2012-04-17 Sony Computer Entertainment Inc. Methods and apparatuses for adjusting a listening area for capturing sounds
US20070260517A1 (en) 2006-05-08 2007-11-08 Gary Zalewski Profile detection
US7854655B2 (en) * 2002-07-27 2010-12-21 Sony Computer Entertainment America Inc. Obtaining input for controlling execution of a game program
US9393487B2 (en) 2002-07-27 2016-07-19 Sony Interactive Entertainment Inc. Method for mapping movements of a hand-held controller to game commands
US9474968B2 (en) 2002-07-27 2016-10-25 Sony Interactive Entertainment America Llc Method and system for applying gearing effects to visual tracking
US20060256081A1 (en) 2002-07-27 2006-11-16 Sony Computer Entertainment America Inc. Scheme for detecting and tracking user manipulation of a game controller body
US20060264260A1 (en) 2002-07-27 2006-11-23 Sony Computer Entertainment Inc. Detectable and trackable hand-held controller
US8139793B2 (en) 2003-08-27 2012-03-20 Sony Computer Entertainment Inc. Methods and apparatus for capturing audio signals based on a visual image
US7352359B2 (en) * 2002-07-27 2008-04-01 Sony Computer Entertainment America Inc. Method and system for applying gearing effects to inertial tracking
US7918733B2 (en) 2002-07-27 2011-04-05 Sony Computer Entertainment America Inc. Multi-input game control mixer
GB2392286B (en) * 2002-08-19 2004-07-07 Chunghwa Telecom Co Ltd Personal identification system based on the reading of multiple one-dimensional barcodes scanned from scanned from PDA/cell phone screen
US7039199B2 (en) 2002-08-26 2006-05-02 Microsoft Corporation System and process for locating a speaker using 360 degree sound source localization
US6917688B2 (en) 2002-09-11 2005-07-12 Nanyang Technological University Adaptive noise cancelling microphone system
US20040063480A1 (en) * 2002-09-30 2004-04-01 Xiaoling Wang Apparatus and a method for more realistic interactive video games on computers or similar devices
EP1411461A1 (en) 2002-10-14 2004-04-21 STMicroelectronics S.r.l. User controlled device for sending control signals to an electric appliance, in particular user controlled pointing device such as mouse or joystick, with 3D-motion detection
US6995666B1 (en) 2002-10-16 2006-02-07 Luttrell Clyde K Cellemetry-operated railroad switch heater
JP2004145448A (ja) 2002-10-22 2004-05-20 Toshiba Corp 端末装置、サーバ装置および画像加工方法
DE60330885D1 (de) 2002-10-30 2010-02-25 Nike International Ltd Kleidungsstücke mit bewegungserfassungsmarker für videospiele
US8206219B2 (en) 2002-10-30 2012-06-26 Nike, Inc. Interactive gaming apparel for interactive gaming
US20040095327A1 (en) 2002-11-14 2004-05-20 Lo Fook Loong Alphanumeric data input system and method
US8012025B2 (en) 2002-12-13 2011-09-06 Applied Minds, Llc Video game controller hub with control input reduction and combination schemes
US7212308B2 (en) 2002-12-18 2007-05-01 Morgan Carol L Interactive photo kiosk
US20040140955A1 (en) 2003-01-21 2004-07-22 Metz Kristofer Erik Input device for a computer and method of operation
JP2004261236A (ja) * 2003-02-20 2004-09-24 Konami Co Ltd ゲームシステム
GB2398690B (en) * 2003-02-21 2006-05-10 Sony Comp Entertainment Europe Control of data processing
GB2398691B (en) * 2003-02-21 2006-05-31 Sony Comp Entertainment Europe Control of data processing
US7161634B2 (en) * 2003-03-06 2007-01-09 Huaya Microelectronics, Ltd. Encoding system for error diffusion dithering
US7665041B2 (en) * 2003-03-25 2010-02-16 Microsoft Corporation Architecture for controlling a computer using hand gestures
US20040212589A1 (en) 2003-04-24 2004-10-28 Hall Deirdre M. System and method for fusing and displaying multiple degree of freedom positional input data from multiple input sources
US7519186B2 (en) 2003-04-25 2009-04-14 Microsoft Corporation Noise reduction systems and methods for voice applications
US7379559B2 (en) 2003-05-28 2008-05-27 Trw Automotive U.S. Llc Method and apparatus for determining an occupant's head location in an actuatable occupant restraining system
US6881147B2 (en) * 2003-06-06 2005-04-19 Nyko Technologies, Inc. Video game controller with integrated microphone and speaker
TW571812U (en) 2003-06-11 2004-01-11 Vision Electronics Co Ltd Audio device for TV game machine
ATE339757T1 (de) * 2003-06-17 2006-10-15 Sony Ericsson Mobile Comm Ab Verfahren und vorrichtung zur sprachaktivitätsdetektion
US7156311B2 (en) * 2003-07-16 2007-01-02 Scanbuy, Inc. System and method for decoding and analyzing barcodes using a mobile device
US7489299B2 (en) * 2003-10-23 2009-02-10 Hillcrest Laboratories, Inc. User interface devices and methods employing accelerometers
EP1694195B1 (en) * 2003-10-28 2009-04-08 Welch Allyn, Inc. Digital documenting ophthalmoscope
US20050105777A1 (en) 2003-11-18 2005-05-19 Kozlowski William J.Jr. Personalized multimedia summary
US7277526B2 (en) 2004-04-09 2007-10-02 Lyncean Technologies, Inc. Apparatus, system, and method for high flux, compact compton x-ray source
TWI376520B (en) 2004-04-30 2012-11-11 Hillcrest Lab Inc Free space pointing devices and methods
ES2664472T3 (es) 2004-04-30 2018-04-19 Idhl Holdings, Inc. Dispositivos de señalización de espacio libre con compensación de inclinación y facilidad de uso mejorada
US7296007B1 (en) 2004-07-06 2007-11-13 Ailive, Inc. Real time context learning by software agents
US7263462B2 (en) 2004-07-30 2007-08-28 Ailive, Inc. Non-disruptive embedding of specialized elements
US7613610B1 (en) 2005-03-14 2009-11-03 Escription, Inc. Transcription data extraction
US7918732B2 (en) 2005-05-06 2011-04-05 Milton Charles Van Noland Manifold compatibility electronic omni axis human interface
US7927216B2 (en) * 2005-09-15 2011-04-19 Nintendo Co., Ltd. Video game system with wireless modular handheld controller
US7620316B2 (en) 2005-11-28 2009-11-17 Navisense Method and device for touchless control of a camera
US7834850B2 (en) 2005-11-29 2010-11-16 Navisense Method and system for object control
US7636645B1 (en) 2007-06-18 2009-12-22 Ailive Inc. Self-contained inertial navigation system for interactive control using movable controllers
JP4481280B2 (ja) * 2006-08-30 2010-06-16 富士フイルム株式会社 画像処理装置、及び画像処理方法
US8277316B2 (en) * 2006-09-14 2012-10-02 Nintendo Co., Ltd. Method and apparatus for using a common pointing input to control 3D viewpoint and object targeting
GB0622451D0 (en) 2006-11-10 2006-12-20 Intelligent Earth Ltd Object position and orientation detection device
US7636697B1 (en) 2007-01-29 2009-12-22 Ailive Inc. Method and system for rapid evaluation of logical expressions
US7937243B2 (en) 2007-08-03 2011-05-03 Ailive, Inc. Method and apparatus for non-disruptive embedding of specialized elements
US20090221368A1 (en) 2007-11-28 2009-09-03 Ailive Inc., Method and system for creating a shared game space for a networked game
US8419545B2 (en) 2007-11-28 2013-04-16 Ailive, Inc. Method and system for controlling movements of objects in a videogame
US8655622B2 (en) * 2008-07-05 2014-02-18 Ailive, Inc. Method and apparatus for interpreting orientation invariant motion

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5563988A (en) * 1994-08-01 1996-10-08 Massachusetts Institute Of Technology Method and system for facilitating wireless, full-body, real-time user interaction with a digitally represented visual environment
US6195104B1 (en) * 1997-12-23 2001-02-27 Philips Electronics North America Corp. System and method for permitting three-dimensional navigation through a virtual reality environment using camera-based gesture inputs
US6441825B1 (en) * 1999-10-04 2002-08-27 Intel Corporation Video token tracking system for animation
US20030020718A1 (en) * 2001-02-28 2003-01-30 Marshall Carl S. Approximating motion using a three-dimensional model

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
JOJIC N ET AL: "Tracking self-occluding articulated objects in dense disparity maps", COMPUTER VISION, 1999. THE PROCEEDINGS OF THE SEVENTH IEEE INTERNATIONAL CONFERENCE ON KERKYRA, GREECE 20-27 SEPT. 1999, LOS ALAMITOS, CA, USA,IEEE COMPUT. SOC, US, 20 September 1999 (1999-09-20), pages 123 - 130, XP010350445, ISBN: 0-7695-0164-8 *

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006108279A1 (en) * 2005-04-11 2006-10-19 Martin Lizee Method and apparatus for virtual presence
WO2006112308A1 (ja) * 2005-04-15 2006-10-26 The University Of Tokyo モーションキャプチャシステム、及びモーションキャプチャシステムにおける特徴点の三次元再構成法
JP5061350B2 (ja) * 2005-04-15 2012-10-31 国立大学法人 東京大学 モーションキャプチャシステム、及びモーションキャプチャシステムにおける特徴点の三次元再構成法
US9782660B2 (en) 2007-11-30 2017-10-10 Nike, Inc. Athletic training system and method
US8897495B2 (en) 2009-10-07 2014-11-25 Microsoft Corporation Systems and methods for tracking a model
US9522328B2 (en) 2009-10-07 2016-12-20 Microsoft Technology Licensing, Llc Human tracking system
US9582717B2 (en) 2009-10-07 2017-02-28 Microsoft Technology Licensing, Llc Systems and methods for tracking a model
US9821226B2 (en) 2009-10-07 2017-11-21 Microsoft Technology Licensing, Llc Human tracking system
US10845459B2 (en) 2013-06-13 2020-11-24 Basf Se Detector for optically detecting at least one object
US10823818B2 (en) 2013-06-13 2020-11-03 Basf Se Detector for optically detecting at least one object
US11041718B2 (en) 2014-07-08 2021-06-22 Basf Se Detector for determining a position of at least one object
US11125880B2 (en) 2014-12-09 2021-09-21 Basf Se Optical detector
US10775505B2 (en) 2015-01-30 2020-09-15 Trinamix Gmbh Detector for an optical detection of at least one object
US10955936B2 (en) 2015-07-17 2021-03-23 Trinamix Gmbh Detector for optically detecting at least one object
US10412283B2 (en) 2015-09-14 2019-09-10 Trinamix Gmbh Dual aperture 3D camera and method using differing aperture areas
US11211513B2 (en) 2016-07-29 2021-12-28 Trinamix Gmbh Optical sensor and detector for an optical detection
US10890491B2 (en) 2016-10-25 2021-01-12 Trinamix Gmbh Optical detector for an optical detection
US11428787B2 (en) 2016-10-25 2022-08-30 Trinamix Gmbh Detector for an optical detection of at least one object
US10948567B2 (en) 2016-11-17 2021-03-16 Trinamix Gmbh Detector for optically detecting at least one object
US11415661B2 (en) 2016-11-17 2022-08-16 Trinamix Gmbh Detector for optically detecting at least one object
US11635486B2 (en) 2016-11-17 2023-04-25 Trinamix Gmbh Detector for optically detecting at least one object
US11698435B2 (en) 2016-11-17 2023-07-11 Trinamix Gmbh Detector for optically detecting at least one object
US11860292B2 (en) 2016-11-17 2024-01-02 Trinamix Gmbh Detector and methods for authenticating at least one object
US11060922B2 (en) 2017-04-20 2021-07-13 Trinamix Gmbh Optical detector
US11067692B2 (en) 2017-06-26 2021-07-20 Trinamix Gmbh Detector for determining a position of at least one object

Also Published As

Publication number Publication date
EP1593096B1 (en) 2009-09-23
EP1593096A1 (en) 2005-11-09
US9881382B2 (en) 2018-01-30
KR100965348B1 (ko) 2010-06-22
JP4723863B2 (ja) 2011-07-13
DE60329424D1 (de) 2009-11-05
US9177387B2 (en) 2015-11-03
TW200415527A (en) 2004-08-16
US20180158196A1 (en) 2018-06-07
AU2003279810A1 (en) 2004-09-06
KR20050099547A (ko) 2005-10-13
CN1764931A (zh) 2006-04-26
CN100409261C (zh) 2008-08-06
US10410359B2 (en) 2019-09-10
TWI274295B (en) 2007-02-21
JP2006514366A (ja) 2006-04-27
US20160027188A1 (en) 2016-01-28
US20040155962A1 (en) 2004-08-12
AU2003279810B2 (en) 2008-06-19
ATE443903T1 (de) 2009-10-15

Similar Documents

Publication Publication Date Title
US10410359B2 (en) Methods for capturing images of markers of a person to control interfacing with an application
EP1636762B1 (en) System and method for providing a real-time three-dimensional interactive environment
EP3479202B1 (en) Augmenting virtual reality content with real world content
CN105955456B (zh) 虚拟现实与增强现实融合的方法、装置及智能穿戴设备
US7084887B1 (en) Marker layout method, mixed reality apparatus, and mixed reality space image generation method
CN102323855B (zh) 具有可变刚度的可再成形的连接器
US8758132B2 (en) Methods and systems for enabling depth and direction detection when interfacing with a computer program
JP5576932B2 (ja) キャラクターにアニメーションまたはモーションを加えるシステムおよび方法
US8696458B2 (en) Motion tracking system and method using camera and non-camera sensors
CN110310329A (zh) 操作显示设备的方法、信息处理系统及非暂时性存储介质
JP3530772B2 (ja) 複合現実感装置及び複合現実空間画像の生成方法
JP2014509758A (ja) リアルタイムの仮想反射
US11103771B1 (en) Motion capture compression suit
CN206819290U (zh) 一种虚拟现实多人互动的系统
KR100917100B1 (ko) 입체 영상 디스플레이 장치 및 그 입체 영상 디스플레이장치에서의 표시부 위치 조절 방법
EP3363509A1 (en) Motion tracking apparatus and system
US20230214006A1 (en) Peripheral Tracking System and Method
US20240005600A1 (en) Nformation processing apparatus, information processing method, and information processing program
JP2000258123A (ja) 画像処理装置および方法、並びに提供媒体

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2003279810

Country of ref document: AU

WWE Wipo information: entry into national phase

Ref document number: 2004568305

Country of ref document: JP

Ref document number: 1020057014813

Country of ref document: KR

WWE Wipo information: entry into national phase

Ref document number: 2003773138

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 20038B02426

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 1020057014813

Country of ref document: KR

WWP Wipo information: published in national office

Ref document number: 2003773138

Country of ref document: EP