US20040201857A1 - Self-referenced tracking - Google Patents
Self-referenced tracking Download PDFInfo
- Publication number
- US20040201857A1 US20040201857A1 US10/837,373 US83737304A US2004201857A1 US 20040201857 A1 US20040201857 A1 US 20040201857A1 US 83737304 A US83737304 A US 83737304A US 2004201857 A1 US2004201857 A1 US 2004201857A1
- Authority
- US
- United States
- Prior art keywords
- user
- head
- orientation
- localized feature
- tracker
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
- G06F3/0421—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means by interrupting or reflecting a light beam, e.g. optical touch-screen
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/033—Indexing scheme relating to G06F3/033
- G06F2203/0331—Finger worn pointing device
Definitions
- This invention relates to self-referenced tracking.
- VR virtual reality
- HMDs head mounted devices
- this tracking has been achieved with a variety of mechanical, acoustic, magnetic, and optical systems. These systems require propagation of a signal between a fixed ”source” and the tracked ”sensor” and therefore limit the range of operation. They also require a degree of care in setting up the source or preparing the site that reduces their utility for field use.
- the invention provides a new tracking technique that is essentially ”sourceless” in that it can be used anywhere with no set-up of a source, yet it enables a wider range of virtual environment-style navigation and interaction techniques than does a simple head-orientation tracker, including manual interaction with virtual objects.
- the tracker and can be used by novice end users without any knowledge of tracking technology, because there is nothing to set up or configure.
- the invention features mounting a tracker on a user's head and using the tracker to track a position of a localized feature associated with a limb of the user relative to the user's head.
- the localized feature associated with the limb may include a hand-held object or a hand-mounted object or a point on a hand.
- the invention features mounting a sourceless orientation tracker on a user's head and using a position tracker to track a position of a first localized feature associated with a limb of the user relative to the user's head.
- the invention features tracking a point on a hand-held object such as a pen or a point on a hand-mounted object such as a ring or a point on a hand relative to a user's head.
- the invention features using a position tracker to determine a distance between a first localized feature associated with a user's limb and a second localized feature associated with the user's head.
- the invention features a position tracker which includes an acoustic position tracker, an electro-optical system that tracks LEDs, optical sensors or reflective marks, a video machine-vision device, a magnetic tracker with a magnetic source held in the hand and sensors integrated in the headset or vice versa, or a radio frequency position locating device.
- the invention features a sourceless orientation tracker including an inertial sensor, a tilt-sensor, or a magnetic compass sensor.
- the invention features mounting a display device on the user's head and displaying a first object at a first position on the display device.
- the invention features changing the orientation of a display device, and, after changing the orientation of the display device, redisplaying the first object at a second position on the display device based on the change in orientation.
- the invention features determining the second position for displaying the first object so as to make the position of the first object appear to be fixed relative to a first coordinate reference frame, which frame does not rotate with the display device during said changing of the orientation of the display device.
- the invention features displaying the first object in response to a signal from a computer.
- the invention features mounting a wearable computer on the user's body, and displaying a first object in response to a signal from the wearable computer.
- the invention features displaying at least a portion of a virtual environment, such as a fly-through virtual environment, or a virtual treadmill, on the display device.
- a virtual environment such as a fly-through virtual environment, or a virtual treadmill
- the invention features displaying a graphical user interface for a computer on the display device.
- first object being a window, icon or menu in the graphical user interface.
- the invention features the first object being a pointer for the graphical user interface.
- the invention features changing the position of the first localized feature relative to the position tracker and, after changing the position of the first localized feature, redisplaying the first object at a second position on the display device determined based on the change in the position of the first localized feature.
- the invention features displaying a second object on the display device, so that after changing the position of the first localized feature, the displayed position of the second object on the display device does not change in response to the change in the position of the first localized feature.
- the invention features determining the second position so as to make the position of the first object appear to coincide with the position of the first localized feature as seen or felt by the user.
- the invention features changing the orientation of the first coordinate reference frame in response to a signal being received by the computer.
- the invention features changing the orientation of the first coordinate reference frame in response to a change in the position of the first localized feature.
- the invention features changing the orientation of the first coordinate reference frame in response to a signal representative of the location of the user.
- the invention features changing the orientation of the first coordinate reference frame in response to a signal representative of a destination.
- the invention features changing the orientation of the first coordinate reference frame in response to a signal representative of a change in the user's immediate surroundings.
- the invention features changing the orientation of the first coordinate reference frame is changed in response to a signal representative of a change in the physiological state or physical state of the user.
- the invention features redisplaying the first object further comprises changing the apparent size of the first object according to the change in position of the first localized feature.
- the invention features mounting a portable beacon, transponder or passive marker at a fixed point in the environment and determining the position vector of a second localized feature associated with the user's head relative to the fixed point.
- the invention features determining the position vector of the first localized feature relative to the fixed point.
- the invention features mounting a sourceless orientation tracker on a second user's head and determining the position of a localized feature associated with the body of the second user relative to the fixed point.
- the invention features determining the position vector of a second localized feature associated with the user's head relative to the fixed point without determining the distance between the second localized feature and more than one fixed point in the environment.
- the invention features displaying the first object at a third position after displaying the first object at the third position, changing the orientation of the display, and after changing the orientation of the display, continuing to display the first object at the third position.
- the invention features the first object being a window in a wraparound computer interface.
- the invention features redisplaying the changed position of the first localized feature not being within the field of view of the display when the first object is redisplayed.
- the invention features displaying the first object at a position coinciding with the position of the first localized object when the first localized object is within the field of view of the display.
- the invention features positioning the first localized feature at a first point positioning the first localized feature at a second point and calculating the distance between the first point and the second point.
- the invention features determining a position vector of the first localized feature relative to a second localized feature associated with the user's head and modifying the position vector based on an orientation of the user's head.
- the invention features setting an assumed position for the user's head in a coordinate system and setting a position for the first localized feature in the coordinate system based on the assumed position of the user's head and said position vector.
- the invention features measuring the orientation of the user's head relative to a fixed frame of reference.
- the invention features setting a virtual travel speed and direction for the user modifying the assumed position for the user's head based on the user's virtual travel speed and direction.
- the invention features mounting on the head of a user a three degree of freedom orientation tracker for tracking the orientation of the head, and a three degree of freedom position tracker for tracking the position of a first localized feature on the user's limb relative to a second localized feature on the user's head, computing a position vector for the first localized feature relative to the second localized feature, determining a rotation matrix based on information received from the rotation tracker, and transforming the position vector into a position vector for a fixed frame of reference based on the rotation matrix.
- the invention features using an acoustic or radio frequency position tracker to track a position of a first localized feature associated with a limb of the user relative to the user's head.
- the invention features mounting a video camera on the back of the user's head and displaying an image generated by the video camera in a portion of a display device mounted on the user's head.
- the invention features mounting a first inertial sensor on a user's head, mounting a second inertial sensor elsewhere on the user's body or in an object held by the user, and tracking the position of one inertial sensor relative to the other.
- Some embodiments of the invention include sensing data at the first and second inertial sensors and using the sensed data to track the position of one inertial sensor relative to the other, tracking the position of the inertial sensor is done without reference to any signal received from a source not mounted on or held by the user and correcting the drift of the relative position or orientation of the second inertial sensor relative to the first inertial sensor by measurements between devices on the user's head and devices elsewhere on the users body.
- the device is easy to don, can track both head and hand, adds no new cables to a wearable computer system, works anywhere indoors or outdoors with no preparation, and is simpler than alternatives such as vision-based self-tracking.
- FIG. 1 is a perspective view of a self-referenced tracking device mounted on a head.
- FIG. 2 is a block diagram.
- FIG. 3 is a graph of tracking coverage and relative resolution.
- FIG. 4 is a view of an information cockpit.
- FIG. 5 shows a user using a virtual reality game.
- implementations of the invention may combine a sourceless head orientation tracker 30 with a head-worn tracking device 12 that tracks a hand-mounted 3D beacon 14 relative to the head 16 .
- One implementation uses a wireless ultrasonic tracker 12 , which has the potential for low cost, lightweight, low power, good resolution, and high update rates when tracking at the relatively close ranges typical of head-hand displacements.
- this arrangement provides a simple and easy to don hardware system.
- a fully integrated wearable VR system using this tracker there are only three parts (a wearable computer 10 , a headset 15 with an integrated tracking system, and a hand-mounted beacon 14 ) and one cable connection 18 .
- the microprocessor and its power and communications link to the wearable the cost and complexity are reduced.
- the hand position measured in head space can be transformed into world space with good seen/felt position match using an assumed head pose, no matter how inaccurate.
- Implementations of the invention may exhibit:
- a simple proof-of-concept implementation combines an InterSense IS-300 sourceless inertial orientation tracker 40 (available from InterSense, Inc., in Burlington, Mass.) with a Pegasus FreeD ultrasonic position tracker 50 (available from Pegasus Technologies Ltd. in Holon, Israel).
- the IS-300 has an ”InertiaCube” inertial sensor assembly 42 , just over an inch on a side, cabled to a small computational unit 44 that outputs orientation data through a serial port 46 .
- the FreeD product consists of a finger-worn wireless ultrasonic emitter 50 A with two mouse buttons 54 , and an L-shaped receiver bar 50 B which normally mounts on the frame of a computer monitor, and outputs x,y,z data through a serial port.
- the FreeD therefore measures the ring position relative to the head-fixed coordinate frame whose orientation was measured by the IS-300.
- Data from both trackers is transmitted to a PC 62 (Pentium 300 MHz, Windows 98) running a program 63 that uses Windows DirectX and Direct3D capabilities to display graphics and effect interaction techniques.
- the graphics output window of Direct3D is maximized to take control over the entire screen, and VGA output 64 (640 ⁇ 480 at 60 Hz) is passed into the V-Cap HMD as well as a desktop monitor.
- the program 63 includes a tracker driver 71 and a fairly conventional VR rendering environment 72 that expects to receive 6 -DOF head and hand tracking data from the tracker driver as well as button states 65 for the hand tracking device.
- the interaction techniques to be described are implemented in the tracker driver.
- the basic functions of the tracker driver, when tracking a single 3-DOF point on the hand, are:
- [0070] Package the orientation data with the current head position in world-frame, and output the combined 6-DOF data record 73 for the head to the VR program.
- the current assumed world-frame head position is the same as the previous one unless the user is in the process of performing a navigation interaction such as flying. In this case the position is incremented based on the flying speed and direction.
- InterTrax 2 is an inertial head orientation module called InterSense 2 (available from InterSense and designed for use with consumer HMDs such as the Sony Glasstron and Olympus EyeTrek). Using tiny piezoelectric camcorder gyros, and solid-state accelerometers and magnetometers, InterTrax 2 is designed as a single long narrow circuit board 30 (FIG. 1) to lie across the top of the head mounted display unit along the brow line. It is 9 cm long, 2 cm wide, and 0.5 cm thick with all components, except for a vertical gyro in the center, which sticks up 1 cm higher.
- InterTrax 2 available from InterSense and designed for use with consumer HMDs such as the Sony Glasstron and Olympus EyeTrek.
- InterTrax 2 is designed as a single long narrow circuit board 30 (FIG. 1) to lie across the top of the head mounted display unit along the brow line. It is 9 cm long, 2 cm wide, and 0.5 cm thick with all components, except for a vertical gyro in
- It contains a low-power embedded 16-bit processor that runs a simplified fixed-point version of the GEOS drift-corrected orientation-tracking algorithm used in the IS-300. It communicates to the host through a single USB connector through which it draws its power, and can be manufactured for very low cost in volume. It is expected to achieve accuracy on the order of 2-3°, which is sufficient because the accuracy with which the hand avatar follows the physical hand is totally independent of orientation tracking accuracy.
- Another component is an embedded ultrasonic rangefinder (perhaps based on the Pegasus FreeD technology).
- an embedded ultrasonic rangefinder perhaps based on the Pegasus FreeD technology.
- three microphones 80 , 82 , 84 and their ultrasonic pulse detection circuits together with the InterTrax 2 board are embedded in a rigid plastic assembly designed to fit elegantly over the brow of an HMD. (In some embodiments, all components would be embedded inside the HMD display unit while sharing the HMD's cable 18 , but in others, the added components are clipped on)
- the InterTrax 2 processor has enough unused timer inputs and processing bandwidth to timestamp the signals from the three ultrasonic pulse detectors and relay this data down its USB link.
- the ultrasonic tracking technology can be modified to take advantage of the very short range requirements.
- ultrasonic frequency may be increased from 40 KHz to a higher frequency. This increases the attenuation in air, and virtually eliminates reverberation and interference between nearby users.
- the system can take advantage of the much reduced reverberation and the short time-of-flight to increase the update rate of tracking to, say, 240 Hz, thus allowing the system to average 4 position samples for each 60 Hz graphics update, or track up to 4 beacons at 60 Hz.
- GDOP Geometric Dilution of Precision
- the intended headset geometry, tracking range and optical field of view are illustrated superimposed on an isogram of a vertical slice through the GDOP data in FIG. 3.
- the plane of the microphones is angled downward 45° to insure that the system has tracking coverage for hands in the lap.
- the resolution at any point in space is the range measurement resolution (about 0.1 mm for short range ultrasonic measurements using 40 KHz) multiplied by the GDOP value, divided by 2 as a result of the 4 ⁇ oversampling and averaging.
- the expected resolution is approximately 0.5 mm at a distance of 400 mm away from the headset.
- a goal of a wearable computer is to keep the user's hands free to perform tasks. For this reason, the system uses a wireless 3-DOF ring pointer for interaction.
- the FreeD ring-mouse previously described is approximately the right size.
- the tracker will need to be triggered by a unique IR code from the headset, so that multiple beacons can be tracked.
- a pen-style input device may be more useful.
- An implementation could use a wireless 5-DOF pen using the same basic technology as the 3-DOF ring pointer, but employing two emitters that are activated in an alternating sequence.
- a compact omni-directional pen could be implemented using cylindrical radiating ultrasonic transducers that have been developed by Virtual Ink (Boston, Mass.), mounted at the ends of a cylindrical electronics unit approximately the size of a normal pen, with two mouse buttons.
- Ultrasonic beacons from InterSense are of suitable size and functionality.
- Implementations of the invention have advantages over conventional world-frame tracking systems for implementing these techniques effectively.
- any error in head orientation tracking will cause significant mismatch between the visual representation of the virtual hand and the felt position of the real hand, making it difficult to accurately activate hidden menus while the virtual hand is not in view.
- the head orientation accuracy is immaterial and visual-proprioceptive match will be good to the accuracy of the ultrasonic tracker—typically 1-2 mm.
- This section describes a few techniques to permit user locomotion and view control.
- the usual navigation interface device in fly-through virtual environments is a joystick. This is appropriate for a flight simulator, but reduces one's sense of presence in terrestrial environments, where turning one's body toward the destination is more instinctive than turning the world until the destination is in front. Implementations of the invention support this more immersive type of flying. No matter how one turns, if she raises a hand in front of her it will be trackable, and can be used to control flight speed and direction. Better yet, she can use two-handed flying, which can be performed with the arms in a relaxed position and allows backwards motion, or the scaled-world grab method to reach out to a distant object and pull oneself to it in one motion.
- head motion parallax is an important visual cue. It can be achieved with the tracking system of the invention on demand by using a trick.
- the system uses the 3-DOF position vector from the user's head to the hand-mounted beacon to track the position of the hand relative to the head, maintaining the head location fixed.
- the user may hold the hand still (say on a desk), and push a button to reverse this process, so that the tracker driver interprets the negative of the measured vector (in world frame) as a position update of the head relative to the stationary hand. He can then move his head back and forth to look around an object, and release the button when his viewpoint is repositioned for optimal viewing. After flying or walking to an area, this may be a convenient way of making finely controlled viewpoint adjustments using natural neck motion. Note that this operation is equivalent to grabbing the world and moving it around with one's hand, which may be a more convenient maneuver while standing.
- Implementations of the invention can perform full 6-DOF head tracking using only one fixed reference point in the environment, while most acoustic and optical trackers require at least three. This works in the invention because head orientation is completely constrained by the sourceless head-tracker. This observation suggests another interesting trick.
- Head-stabilized objects are displayed at a fixed location on the HMD screen, so they move with your head motion and require no tracking.
- World-stabilized objects are fixed to locations in the physical environment. To cause them to stay fixed despite user head-motion requires full 6-DOF head tracking.
- Body-stabilized objects are displayed at a fixed location on the information surround, a kind of cylindrical or spherical bubble of information that follows the user's body position around. Head orientation tracking allows the user to look at different parts of the surroundings by turning his head, but position tracking is not needed.
- the desk was thus slaved to the user's torso, and the user could easily locate windows on it using his innate knowledge of head turn relative to the torso.
- This is intuitive but has the drawback that an additional orientation sensor must be mounted on the user's torso.
- This adds cost makes the system more difficult to don, and causes the virtual desk to shift around in response to slight postural shifting of the user's torso, wobbling of the sensor mount, or metallic distortion of the relative magnetic heading between the two sensors.
- An implementation of the invention uses a variation on this theme, based on an ”information cockpit” metaphor instead of a body-stabilized desk.
- the information cockpit consists of a clear windshield, optionally drawn as a thin wireframe border, and a cluster of virtual instruments around it.
- the user's head is always in the center of the cockpit, but the heading direction of the cockpit stays fixed until the user changes it.
- the user first positions the windshield towards the objects he will be working on with his hands, and keeps the windshield area fairly clear of augmentations so that he can see what he is doing. Thereafter, the user can turn to look at the instruments, with or without turning his torso, and the instruments will not move.
- the implementation provides the user with steering techniques.
- FIG. 4 shows an example of an information cockpit for an outdoor navigation application.
- the active field-of-view of the see-through HMD is indicated by heavy black rectangle 400 .
- the head moves this active view port around the scene and reveals the other augmentations once they are inside of it.
- There are additional icons 402 in the dashboard that are stabilized to the information cockpit, and therefore can only be seen when the user looks down a little to check them.
- the full-size map application window 404 pops up in the middle of the active display area. The user may either quickly examine it then minimize it again, or save it for on-going reference by fixing it to a convenient spot on the information cockpit ”windshield” 410 as has been done in FIG. 4. The user can see a corner of the map in the current view, but can look at the whole map again by looking up and to the right.
- Virtual rear view mirrors 406 (fed by a video camera on the back of the head) have likewise been placed in three locations on the virtual cockpit, but the user can re-position or close any of these four information instruments at any time.
- the heading direction of the cockpit is controlled by the application in order to guide the user to a destination.
- the application Orients the cockpit along the direction from the user's current position to the destination, so he need only follow the dotted lines 408 to their vanishing point on the horizon to walk in the correct direction.
- This provides a virtual sidewalk in the forest, much as pilots are guided by virtual tunnel-in-the-sky displays.
- the computer would use map correlation to orient the cockpit along the current road in the suggested walking direction.
- the ring tracker can be used for several purposes in wearable computer applications: direct pointing to objects, virtual mouse pad cursor control, command gestures, and measuring or digitizing.
- HUD Heads-Up-Display
- the cursor is associated with a point on the spherical information cockpit surface, only a portion of which is visible in the HMD, so the cursor could be out of view and quite difficult to find.
- a wiggling gesture is then used to bring it back into the current center of display.
- the ring tracking must be calculated in the cockpit stabilized coordinate frame, which means that if the user turns to the right, an ”in-and-out” motion switches from cockpit x-axis to y-axis and has an unexpected effect.
- the ring position is transformed into cylindrical polar coordinates and the radial and tangential components are used to control cursor vertical and horizontal motion respectively.
- Ring tracker gestures may be used as a substitute for voice commands in situations where visual theatrics are more acceptable than audible ones, or where it is too noisy for reliable speech recognition.
- gestures should commence outside of the direct pointing and virtual mouse pad regions, in order to avoid accidentally selecting and moving objects. This leaves the sides and top of the viewing frustum, and the first few inches in front of the face (which are not used for direct pointing).
- the gestures are executed by depressing a mouse button, possibly making a certain movement, then releasing the button. They are always relative to the head in order to exploit proprioception, and the fact that the head is tracked, while the rest of the body is not.
- Many gestures may be defined, but the most commonly needed is a boresight command to reset the heading direction of the cockpit to the current forward direction of the person's head as he walks about.
- a more elaborate procedure may be employed.
- the user first looks at the first object, positions the pointer beacon on it and depresses a button.
- a world frame position vector (p 1 ) of the first object is stored and then the tracking mode is switched to 6-DOF tracking of the head relative to the stationary hand-held pointer, as previously described.
- the user While holding the pointer stationary on the object and keeping the button depressed, the user then repositions his head until the second object is in view, releases the button, and holds his head still while moving the pointer to the second object, then clicking it to capture the second position vector (p 2 ) in the same world coordinate frame as the first.
- This technique may be practiced either with a single pointing beacon operated by one hand, or using separate pointing beacons in each hand, to achieve approximately the same functionality as a conventional tape measure, but with the added benefit that the measurements are automatically stored on a digital computer.
- Relationships among remote objects may also be measured using standard triangulation surveying methods, exploiting the functional similarity of a see-through HMD optic with orientation tracker to a surveyor's theodolite (although a tripod mounted theodolite is likely to be more accurate).
- the previous section presented the information cockpit as a specific variation on Feiner's body-stabilized information surround.
- the cockpit metaphor also allows the user to make use of the head-stabilized and world-stabilized coordinate frames at the same time.
- the previous section gave one example of this in which the pilot drags information from the cockpit onto the HUD, which makes it head-stabilized. For example, one may wish to have an alerting device always visible in the HUD that pops up notifications whenever a phone call, page or email is received, or when a scheduled meeting is about to begin, etc.
- Some implementations of the invention use an inertial orientation sensor to track the rotation of the head, and an acoustic or optical position tracker to track the position of the hand relative to the head. For many applications, the performance of the acoustic or optical position tracker is sufficient. Furthermore, it has the great advantage that the item being tracked can be a small wireless transponder, or even a passive marker. For some applications, such as the ring-mounted pointing device for wearable computing, this is an overwhelming advantage.
- the virtual object controlled by the hand tracker e.g. a virtual sword or gun or racquet
- the hand tracker e.g. a virtual sword or gun or racquet
- Acoustic, magnetic, or videometric hand trackers may introduce noticeable latency or jitter in these applications.
- Inertial position and orientation trackers are well known to provide extremely low latency and low jitter, but they require drift correction, especially if tracking position and not just orientation is desired.
- the user's head and hand may both be tracked with 6 degrees of freedom relative to an external reference frame by using inertial sensors on the head and on the hand to measure their motion with a high update rate and low latency. The drift of these inertial sensors is corrected by making measurements with an ultrasonic, optical or magnetic tracking reference device mounted in the environment.
- the drift and latency issues can be addressed without the requirement of a reference device mounted in the environment.
- inertial sensors be attached to the moving body being used as the reference frame
- the moving reference frame may be the user's head and the tracked object may be the user's hand or hand-mounted or hand-held object
- the object being tracked here, e.g., the user's hand.
- the techniques utilize angular rate and linear acceleration signals from the sourceless orientation trackers on the reference frame and on the tracked object to derive a differential inertial signal representative of the motion of the object relative to the frame. In embodiments of the present invention, this technique may be used to derive a differential inertial signal representative of the motion of the hand relative to the head.
- FIG. 5 illustrates a user wearing a portable VR tennis game or training system.
- the computer and batteries are contained in backpack 502 , which is cabled to HMD 500 to which are mounted inertial sensors 506 and ultrasonic transducers 510 .
- He is holding a hand-held object 516 , in this case a tennis racquet, to which are attached inertial sensors 508 and ultrasonic transducers 512 .
- These hand-mounted devices may be powered by their own batteries and communicate by wireless means to the system on the users head and torso, or there may be an additional cable between the racquet and the backpack.
- the signals from inertial sensors 506 are processed by a first algorithm, preferably a drift-corrected inertial orientation tracking algorithm such as described in U.S. Pat. No. 5,645,077 to obtain a sourceless measurement of the head orientation.
- the signals from the hand-mounted inertial sensors 508 and the head-mounted inertial sensors 506 are jointly processed to track both the position and orientation of the hand relative to the head, preferably using an algorithm such as described in Foxlin ( 2000 ) and co-pending U.S. patent application Ser. No. 09/556,135.
- the drift of this relative inertial tracking is corrected by the relative range measurements 514 .
- earphones 504 to provide 3D spatialized audio
- a haptic feedback device 518 to provide tactile feedback to the user when the virtual ball has hit the virtual racquet.
- such a system may be used for other types of activities, such as a sword-fighting or gun-fighting game or trainer, a surgical trainer, an immersive design environment, a human-computer interface, or any other application known or not yet known which requires tracking of a user's head and one or more limbs or limb-mounted devices. While it is especially advantageous for mobile or portable applications in which the computer is wearable, this is not a requirement, and the user may be cabled to an off-body computer or communicate with an off-body computer through a wireless connection. In this case, it is still an advantage of the current invention that the tracking is accomplished without setting up an off-body reference device.
- the head mounted position tracker need not be acoustic. It may be an electro-optical system which tracks LEDs, optical sensors, or reflective markers, or a video machine-vision device that recognizes the hands or fingers or some special markers mounted on the hands or fingers or handheld object, or even a magnetic tracker with a magnetic source held in the hand and sensors integrated in the headset or vice versa, or an RF position locating device.
- the implementation described above use inertial sourceless orientation trackers.
- Other implementations may use other forms of head orientation trackers, including trackers based on tilt-sensing or magnetic compass sensors, or any other form of head orientation tracker.
- some implementations may use no head orientation tracker. In this case, the tracking system would not enable the user to look around in a virtual environment by turning his head, but it would still be useful for manual interaction with computers using head-worn displays.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Optics & Photonics (AREA)
- Position Input By Displaying (AREA)
- User Interface Of Digital Computer (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
Description
- This application is a continuation of and claims priority under 35 USC §120 to U.S. patent application Ser. No. 09/770,691, filed on Jan. 26, 2001, and is entitled under 35 USC § 119(e) to the benefit of the filing date of U.S. Provisional Patent Application Serial No. 60/178,797, filed on Jan. 28, 2000, the contents of both which are hereby incorporated by reference.
- This invention relates to self-referenced tracking.
- Virtual reality (VR) systems require tracking of the orientation and position of a user's head and hands with respect to a world coordinate frame in order to control view parameters for head mounted devices (HMDs) and allow manual interactions with the virtual world. In laboratory VR setups, this tracking has been achieved with a variety of mechanical, acoustic, magnetic, and optical systems. These systems require propagation of a signal between a fixed ”source” and the tracked ”sensor” and therefore limit the range of operation. They also require a degree of care in setting up the source or preparing the site that reduces their utility for field use.
- The emerging fields of wearable computing and augmented reality (AR) require tracking systems to be wearable and capable of operating essentially immediately in arbitrary environments. ”Sourceless” orientation trackers have been developed based on geomagnetic and/or inertial sensors. They allow enough control to look around the virtual environment and fly through it, but they don't enable the ”reach-out-and-grab” interactions that make virtual environments so intuitive and which are needed to facilitate computer interaction.
- In one aspect, in general, the invention provides a new tracking technique that is essentially ”sourceless” in that it can be used anywhere with no set-up of a source, yet it enables a wider range of virtual environment-style navigation and interaction techniques than does a simple head-orientation tracker, including manual interaction with virtual objects. The tracker and can be used by novice end users without any knowledge of tracking technology, because there is nothing to set up or configure.
- In another aspect, in general, the invention features mounting a tracker on a user's head and using the tracker to track a position of a localized feature associated with a limb of the user relative to the user's head. The localized feature associated with the limb may include a hand-held object or a hand-mounted object or a point on a hand.
- In another aspect, in general, the invention features mounting a sourceless orientation tracker on a user's head and using a position tracker to track a position of a first localized feature associated with a limb of the user relative to the user's head.
- In another aspect, in general, the invention features tracking a point on a hand-held object such as a pen or a point on a hand-mounted object such as a ring or a point on a hand relative to a user's head.
- In another aspect, in general, the invention features using a position tracker to determine a distance between a first localized feature associated with a user's limb and a second localized feature associated with the user's head.
- In another aspect, in general, the invention features a position tracker which includes an acoustic position tracker, an electro-optical system that tracks LEDs, optical sensors or reflective marks, a video machine-vision device, a magnetic tracker with a magnetic source held in the hand and sensors integrated in the headset or vice versa, or a radio frequency position locating device.
- In another aspect, in general, the invention features a sourceless orientation tracker including an inertial sensor, a tilt-sensor, or a magnetic compass sensor.
- In another aspect, in general, the invention features mounting a display device on the user's head and displaying a first object at a first position on the display device.
- In another aspect, in general, the invention features changing the orientation of a display device, and, after changing the orientation of the display device, redisplaying the first object at a second position on the display device based on the change in orientation.
- In another aspect, in general, the invention features determining the second position for displaying the first object so as to make the position of the first object appear to be fixed relative to a first coordinate reference frame, which frame does not rotate with the display device during said changing of the orientation of the display device.
- In another aspect, in general, the invention features displaying the first object in response to a signal from a computer.
- In another aspect, in general, the invention features mounting a wearable computer on the user's body, and displaying a first object in response to a signal from the wearable computer.
- In another aspect, in general, the invention features displaying at least a portion of a virtual environment, such as a fly-through virtual environment, or a virtual treadmill, on the display device.
- In another aspect, in general, the invention features displaying a graphical user interface for a computer on the display device.
- In another aspect, in general, the invention features first object being a window, icon or menu in the graphical user interface.
- In another aspect, in general, the invention features the first object being a pointer for the graphical user interface.
- In another aspect, in general, the invention features changing the position of the first localized feature relative to the position tracker and, after changing the position of the first localized feature, redisplaying the first object at a second position on the display device determined based on the change in the position of the first localized feature.
- In another aspect, in general, the invention features displaying a second object on the display device, so that after changing the position of the first localized feature, the displayed position of the second object on the display device does not change in response to the change in the position of the first localized feature.
- In another aspect, in general, the invention features determining the second position so as to make the position of the first object appear to coincide with the position of the first localized feature as seen or felt by the user.
- In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame in response to a signal being received by the computer.
- In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame in response to a change in the position of the first localized feature.
- In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame in response to a signal representative of the location of the user.
- In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame in response to a signal representative of a destination.
- In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame in response to a signal representative of a change in the user's immediate surroundings.
- In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame is changed in response to a signal representative of a change in the physiological state or physical state of the user.
- In another aspect, in general, the invention features redisplaying the first object further comprises changing the apparent size of the first object according to the change in position of the first localized feature.
- In another aspect, in general, the invention features mounting a portable beacon, transponder or passive marker at a fixed point in the environment and determining the position vector of a second localized feature associated with the user's head relative to the fixed point.
- In another aspect, in general, the invention features determining the position vector of the first localized feature relative to the fixed point.
- In another aspect, in general, the invention features mounting a sourceless orientation tracker on a second user's head and determining the position of a localized feature associated with the body of the second user relative to the fixed point.
- In another aspect, in general, the invention features determining the position vector of a second localized feature associated with the user's head relative to the fixed point without determining the distance between the second localized feature and more than one fixed point in the environment.
- In another aspect, in general, the invention features displaying the first object at a third position after displaying the first object at the third position, changing the orientation of the display, and after changing the orientation of the display, continuing to display the first object at the third position.
- In another aspect, in general, the invention features the first object being a window in a wraparound computer interface.
- In another aspect, in general, the invention features redisplaying the changed position of the first localized feature not being within the field of view of the display when the first object is redisplayed.
- In another aspect, in general, the invention features displaying the first object at a position coinciding with the position of the first localized object when the first localized object is within the field of view of the display.
- In another aspect, in general, the invention features positioning the first localized feature at a first point positioning the first localized feature at a second point and calculating the distance between the first point and the second point.
- In another aspect, in general, the invention features determining a position vector of the first localized feature relative to a second localized feature associated with the user's head and modifying the position vector based on an orientation of the user's head.
- In another aspect, in general, the invention features setting an assumed position for the user's head in a coordinate system and setting a position for the first localized feature in the coordinate system based on the assumed position of the user's head and said position vector.
- In another aspect, in general, the invention features measuring the orientation of the user's head relative to a fixed frame of reference.
- In another aspect, in general, the invention features setting a virtual travel speed and direction for the user modifying the assumed position for the user's head based on the user's virtual travel speed and direction.
- In another aspect, in general, the invention features mounting on the head of a user a three degree of freedom orientation tracker for tracking the orientation of the head, and a three degree of freedom position tracker for tracking the position of a first localized feature on the user's limb relative to a second localized feature on the user's head, computing a position vector for the first localized feature relative to the second localized feature, determining a rotation matrix based on information received from the rotation tracker, and transforming the position vector into a position vector for a fixed frame of reference based on the rotation matrix.
- In another aspect, in general, the invention features using an acoustic or radio frequency position tracker to track a position of a first localized feature associated with a limb of the user relative to the user's head.
- In another aspect, in general, the invention features mounting a video camera on the back of the user's head and displaying an image generated by the video camera in a portion of a display device mounted on the user's head.
- In another aspect, in general, the invention features mounting a first inertial sensor on a user's head, mounting a second inertial sensor elsewhere on the user's body or in an object held by the user, and tracking the position of one inertial sensor relative to the other.
- Some embodiments of the invention include sensing data at the first and second inertial sensors and using the sensed data to track the position of one inertial sensor relative to the other, tracking the position of the inertial sensor is done without reference to any signal received from a source not mounted on or held by the user and correcting the drift of the relative position or orientation of the second inertial sensor relative to the first inertial sensor by measurements between devices on the user's head and devices elsewhere on the users body.
- Among the advantages of the invention are one or more of the following. The device is easy to don, can track both head and hand, adds no new cables to a wearable computer system, works anywhere indoors or outdoors with no preparation, and is simpler than alternatives such as vision-based self-tracking.
- The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims.
- FIG. 1 is a perspective view of a self-referenced tracking device mounted on a head.
- FIG. 2 is a block diagram.
- FIG. 3 is a graph of tracking coverage and relative resolution.
- FIG. 4 is a view of an information cockpit.
- FIG. 5 shows a user using a virtual reality game.
- Like reference symbols in the various drawings indicate like elements.
- As seen in FIG. 1, implementations of the invention may combine a sourceless
head orientation tracker 30 with a head-worntracking device 12 that tracks a hand-mounted3D beacon 14 relative to thehead 16. One implementation uses a wirelessultrasonic tracker 12, which has the potential for low cost, lightweight, low power, good resolution, and high update rates when tracking at the relatively close ranges typical of head-hand displacements. - As FIG. 1 illustrates, this arrangement provides a simple and easy to don hardware system. In a fully integrated wearable VR system using this tracker there are only three parts (a
wearable computer 10, aheadset 15 with an integrated tracking system, and a hand-mounted beacon 14) and onecable connection 18. This is possible because the entireultrasonic receiver system 12 for tracking the beacon can be reduced to a few small signal-conditioning circuits and integrated with thesourceless orientation tracker 30 in the head-worndisplay 15. By sharing the microprocessor and its power and communications link to the wearable, the cost and complexity are reduced. - The benefits of this combination of elements stem from these realizations:
- 1. It is usually not important to track the hand unless it is in front of the head. Thus range and line-of-sight limitations are no problem if the tracker is mounted on the forehead.
- 2. The hand position measured in head space can be transformed into world space with good seen/felt position match using an assumed head pose, no matter how inaccurate.
- 3. Using one fixed beacon, the same tracking hardware can provide full 6-DOF tracking.
- Implementations of the invention may exhibit:
- 1. A new tracking concept that enables immersive visualization and intuitive manual interaction using a wearable system in arbitrary unprepared environments.
- 2. An information cockpit metaphor for a wearable computer user interface and a set of interaction techniques based on this metaphor.
- As shown in FIG. 2, a simple proof-of-concept implementation combines an InterSense IS-300 sourceless inertial orientation tracker40 (available from InterSense, Inc., in Burlington, Mass.) with a Pegasus FreeD ultrasonic position tracker 50 (available from Pegasus Technologies Ltd. in Holon, Israel). The IS-300 has an ”InertiaCube”
inertial sensor assembly 42, just over an inch on a side, cabled to a smallcomputational unit 44 that outputs orientation data through aserial port 46. The FreeD product consists of a finger-worn wirelessultrasonic emitter 50A with two mouse buttons 54, and an L-shapedreceiver bar 50B which normally mounts on the frame of a computer monitor, and outputs x,y,z data through a serial port. For our experiments we mounted the InertiaCube and the L-shaped receiver bar on thevisor 60 of a V-Cap 1000 see-through HMD (available from Virtual Vision of Seattle, Wash.). The FreeD therefore measures the ring position relative to the head-fixed coordinate frame whose orientation was measured by the IS-300. - Data from both trackers is transmitted to a PC62 (Pentium 300 MHz, Windows 98) running a
program 63 that uses Windows DirectX and Direct3D capabilities to display graphics and effect interaction techniques. The graphics output window of Direct3D is maximized to take control over the entire screen, and VGA output 64 (640×480 at 60 Hz) is passed into the V-Cap HMD as well as a desktop monitor. - The
program 63 includes atracker driver 71 and a fairly conventionalVR rendering environment 72 that expects to receive 6-DOF head and hand tracking data from the tracker driver as well as button states 65 for the hand tracking device. The interaction techniques to be described are implemented in the tracker driver. The basic functions of the tracker driver, when tracking a single 3-DOF point on the hand, are: - 1. Read in and parse the
orientation data 68 from the IS-300 and theposition triad 70 from the FreeD. - 2. Package the orientation data with the current head position in world-frame, and output the combined 6-
DOF data record 73 for the head to the VR program. The current assumed world-frame head position is the same as the previous one unless the user is in the process of performing a navigation interaction such as flying. In this case the position is incremented based on the flying speed and direction. - 3. Transform the hand position vector from head frame to world frame by first multiplying by the rotation matrix from head to world frame obtained from the orientation tracker, then adding the current assumed world-frame head position. Output the result to the VR program as a 3-
DOF position record 74 for the hand device. - The simple implementation just described is wearable, but cannot be integrated into an HMD elegantly, largely due to the size and power consumption of the IS-300 processing unit. A low-cost wearable version using available technologies could be implemented as follows:
- The core of this implementation is an inertial head orientation module called InterTrax 2 (available from InterSense and designed for use with consumer HMDs such as the Sony Glasstron and Olympus EyeTrek). Using tiny piezoelectric camcorder gyros, and solid-state accelerometers and magnetometers, InterTrax 2 is designed as a single long narrow circuit board30 (FIG. 1) to lie across the top of the head mounted display unit along the brow line. It is 9 cm long, 2 cm wide, and 0.5 cm thick with all components, except for a vertical gyro in the center, which sticks up 1 cm higher. It contains a low-power embedded 16-bit processor that runs a simplified fixed-point version of the GEOS drift-corrected orientation-tracking algorithm used in the IS-300. It communicates to the host through a single USB connector through which it draws its power, and can be manufactured for very low cost in volume. It is expected to achieve accuracy on the order of 2-3°, which is sufficient because the accuracy with which the hand avatar follows the physical hand is totally independent of orientation tracking accuracy.
- Another component is an embedded ultrasonic rangefinder (perhaps based on the Pegasus FreeD technology). As shown in FIG. 1, three
microphones cable 18, but in others, the added components are clipped on) The InterTrax 2 processor has enough unused timer inputs and processing bandwidth to timestamp the signals from the three ultrasonic pulse detectors and relay this data down its USB link. - The ultrasonic tracking technology can be modified to take advantage of the very short range requirements. First, ultrasonic frequency may be increased from 40 KHz to a higher frequency. This increases the attenuation in air, and virtually eliminates reverberation and interference between nearby users. Second, the system can take advantage of the much reduced reverberation and the short time-of-flight to increase the update rate of tracking to, say, 240 Hz, thus allowing the system to average 4 position samples for each 60 Hz graphics update, or track up to 4 beacons at 60 Hz. To calculate the resolution that this would yield in various parts of the tracking volume we calculated the Geometric Dilution of Precision (GDOP) throughout the tracking volume given the intended geometry of the microphone mounts on the headset. The intended headset geometry, tracking range and optical field of view are illustrated superimposed on an isogram of a vertical slice through the GDOP data in FIG. 3. The plane of the microphones is angled downward 45° to insure that the system has tracking coverage for hands in the lap. The resolution at any point in space is the range measurement resolution (about 0.1 mm for short range ultrasonic measurements using 40 KHz) multiplied by the GDOP value, divided by 2 as a result of the 4× oversampling and averaging. Thus the expected resolution is approximately 0.5 mm at a distance of 400 mm away from the headset.
- A goal of a wearable computer is to keep the user's hands free to perform tasks. For this reason, the system uses a wireless 3-DOF ring pointer for interaction. The FreeD ring-mouse previously described is approximately the right size. In some implementations of the system, the tracker will need to be triggered by a unique IR code from the headset, so that multiple beacons can be tracked.
- In interactive visualization and design (IVD) and many other VR applications, a pen-style input device may be more useful. An implementation could use a wireless 5-DOF pen using the same basic technology as the 3-DOF ring pointer, but employing two emitters that are activated in an alternating sequence. A compact omni-directional pen could be implemented using cylindrical radiating ultrasonic transducers that have been developed by Virtual Ink (Boston, Mass.), mounted at the ends of a cylindrical electronics unit approximately the size of a normal pen, with two mouse buttons.
- An additional device that could be included in the system and whose applications are discussed below is a small wireless anchor beacon that can be easily stuck to any surface. Ultrasonic beacons from InterSense are of suitable size and functionality.
- Portable VR Application
- Object Selection and Manipulation Exploiting Proprioception
- M. Mine, F. Brooks, and C. Sequin. (Moving Objects in Space: Exploiting Proprioception in Virtual Environment Interaction. In SIGGRAPH 97 Conference Proceedings, ACM Annual Conference Series, August, 1997), have discussed the benefits of designing virtual environment interaction techniques that exploit our proprioceptive sense of the relative pose of our head, hands and body. A variety of techniques were presented, such as direct manipulation of objects within arms reach, scaled-world grab, hiding tools and menus on the users body, and body-relative gestures.
- Implementations of the invention have advantages over conventional world-frame tracking systems for implementing these techniques effectively. With conventional trackers, any error in head orientation tracking will cause significant mismatch between the visual representation of the virtual hand and the felt position of the real hand, making it difficult to accurately activate hidden menus while the virtual hand is not in view. With implementations of the invention, the head orientation accuracy is immaterial and visual-proprioceptive match will be good to the accuracy of the ultrasonic tracker—typically 1-2 mm.
- Locomotion & View Control Tricks
- This section describes a few techniques to permit user locomotion and view control.
- Flying and Scaled-World Grab
- The usual navigation interface device in fly-through virtual environments is a joystick. This is appropriate for a flight simulator, but reduces one's sense of presence in terrestrial environments, where turning one's body toward the destination is more instinctive than turning the world until the destination is in front. Implementations of the invention support this more immersive type of flying. No matter how one turns, if she raises a hand in front of her it will be trackable, and can be used to control flight speed and direction. Better yet, she can use two-handed flying, which can be performed with the arms in a relaxed position and allows backwards motion, or the scaled-world grab method to reach out to a distant object and pull oneself to it in one motion.
- Walking Using Head Accelerometers as a Pedometer
- For exploratory walk-throughs, the sense of presence is greatest for walking, somewhat reduced for walking-in-place, and much further reduced for flying. M. Slater, A. Steed and M. Usoh (The Virtual Treadmill: A Naturalistic Metaphor for Navigation in Immersive Virtual Environments. InFirst Eurographics Workshop on Virtual Reality, M. Goebel Ed. 1993), and M. Slater, M. Usoh and A. Steed (Steps and Ladders in Virtual Reality. In Proc. Virtual Reality Software & Technology 94, G. Singh, S. K. Feiner, and D. Thalmann, Eds. Singapore: World Scientific, pages 45-54, August 1994) have described a ”virtual treadmill” technique in which a neural network is trained to recognize the bouncing pattern of a position tracker on an HMD, and thus control virtual motion. Inertial head-orientation trackers do not normally output the position obtained by double integrating the accelerometers, because it drifts too much to be useful, but it seems reasonable that pattern analysis of the acceleration signals would produce good results.
- Head-Motion Parallax Using Anchor Beacon
- When working with close objects, head motion parallax is an important visual cue. It can be achieved with the tracking system of the invention on demand by using a trick. Normally, the system uses the 3-DOF position vector from the user's head to the hand-mounted beacon to track the position of the hand relative to the head, maintaining the head location fixed. When desired, the user may hold the hand still (say on a desk), and push a button to reverse this process, so that the tracker driver interprets the negative of the measured vector (in world frame) as a position update of the head relative to the stationary hand. He can then move his head back and forth to look around an object, and release the button when his viewpoint is repositioned for optimal viewing. After flying or walking to an area, this may be a convenient way of making finely controlled viewpoint adjustments using natural neck motion. Note that this operation is equivalent to grabbing the world and moving it around with one's hand, which may be a more convenient maneuver while standing.
- Implementations of the invention can perform full 6-DOF head tracking using only one fixed reference point in the environment, while most acoustic and optical trackers require at least three. This works in the invention because head orientation is completely constrained by the sourceless head-tracker. This observation suggests another interesting trick. One may carry an extra wireless anchor beacon in a pocket and place it down on the table or stick it to a wall near a work area. Within range of this beacon, he can enjoy full 6-DOF tracking of both head and hand.
- Wearable Computing Information Cockpit Interface
- Information Cockpit Metaphor
- In the field of wearable computing, three modes of displaying objects in a head-mounted display have been discussed. Head-stabilized objects are displayed at a fixed location on the HMD screen, so they move with your head motion and require no tracking. World-stabilized objects are fixed to locations in the physical environment. To cause them to stay fixed despite user head-motion requires full 6-DOF head tracking. Body-stabilized objects are displayed at a fixed location on the information surround, a kind of cylindrical or spherical bubble of information that follows the user's body position around. Head orientation tracking allows the user to look at different parts of the surroundings by turning his head, but position tracking is not needed.
- Pure head-stabilized displays are usually used with small opaque monocular monitors mounted off to the side of the user's field of view. Without head tracking, this is better than having a display directly in front of the eye with information constantly blocking the frontal view. Use of this paradigm is widespread, and most of the wearable computer vendors provide this style of untracked sidecar display. This is roughly equivalent to wearing your desktop computer on your belt with the monitor mounted on a headband so that it is always available for hands-free viewing.
- At the other end of the spectrum are world-stabilized AR displays, which must be implemented using see-through optics placed directly in front of the eyes. For a variety of applications such as surgery, construction and maintenance, this is a highly valuable capability. However, it requires sophisticated tracking and calibration, and is likely to remain a high-end subset of the total wearable computing market for quite a few years.
- In the middle ground of complexity are the less common body-stabilized displays, which also tend to be implemented with see through HMDs. As implemented by S. Feiner, B. MacIntyre, M. Haupt, and E. Solomon (Windows on the World: 2D Windows for 3D Augmented Reality. In Proc. ACM UIST 93. ACM Press, November 1993) objects were drawn on a 170° horizontal by 90° vertical portion of a sphere. To prevent user disorientation, this hemispherical ”virtual desk” was kept in front of the user's body by mounting an additional orientation tracker on the user's torso, and using the difference between the head yaw and torso yaw to pan the viewport. The desk was thus slaved to the user's torso, and the user could easily locate windows on it using his innate knowledge of head turn relative to the torso. This is intuitive but has the drawback that an additional orientation sensor must be mounted on the user's torso. This adds cost, makes the system more difficult to don, and causes the virtual desk to shift around in response to slight postural shifting of the user's torso, wobbling of the sensor mount, or metallic distortion of the relative magnetic heading between the two sensors. An implementation of the invention uses a variation on this theme, based on an ”information cockpit” metaphor instead of a body-stabilized desk.
- The information cockpit consists of a clear windshield, optionally drawn as a thin wireframe border, and a cluster of virtual instruments around it. As with the body-stabilized technique, the user's head is always in the center of the cockpit, but the heading direction of the cockpit stays fixed until the user changes it. Generally, the user first positions the windshield towards the objects he will be working on with his hands, and keeps the windshield area fairly clear of augmentations so that he can see what he is doing. Thereafter, the user can turn to look at the instruments, with or without turning his torso, and the instruments will not move. To prevent the user from becoming disoriented or being forced to strain his neck as he moves around, the implementation provides the user with steering techniques.
- Outdoor Navigation Application
- FIG. 4 shows an example of an information cockpit for an outdoor navigation application. The active field-of-view of the see-through HMD is indicated by heavy
black rectangle 400. Thus only the augmentations within this rectangle are visible to the user, but rotating the head moves this active view port around the scene and reveals the other augmentations once they are inside of it. In this example there are a few frequently-usedicons 401 that are fixed (i.e. head-stabilized) in the upper right of the heads-up display that will always be visible. There areadditional icons 402 in the dashboard that are stabilized to the information cockpit, and therefore can only be seen when the user looks down a little to check them. Some of these are miniature information instruments, such as dials and gauges, while others are icons used to bring up larger information instruments such as a web browser or interactive map display. By clicking on the map icon on the dashboard, the full-sizemap application window 404 pops up in the middle of the active display area. The user may either quickly examine it then minimize it again, or save it for on-going reference by fixing it to a convenient spot on the information cockpit ”windshield” 410 as has been done in FIG. 4. The user can see a corner of the map in the current view, but can look at the whole map again by looking up and to the right. Virtual rear view mirrors 406 (fed by a video camera on the back of the head) have likewise been placed in three locations on the virtual cockpit, but the user can re-position or close any of these four information instruments at any time. In this example, the heading direction of the cockpit is controlled by the application in order to guide the user to a destination. Using a GPS receiver in the user's wearable computer, the application orients the cockpit along the direction from the user's current position to the destination, so he need only follow the dottedlines 408 to their vanishing point on the horizon to walk in the correct direction. This provides a virtual sidewalk in the forest, much as pilots are guided by virtual tunnel-in-the-sky displays. In an urban setting, the computer would use map correlation to orient the cockpit along the current road in the suggested walking direction. - Steering and Interaction
- The ring tracker can be used for several purposes in wearable computer applications: direct pointing to objects, virtual mouse pad cursor control, command gestures, and measuring or digitizing.
- Direct pointing to objects
- When the ring tracker enters the viewing frustum of the HMD, the cursor jumps to the location of the ring and follows it. This provides rapid direct selection of objects, taking full advantage of natural eye-hand coordination. In the virtual cockpit, one may glance up from the windshield to a side panel, see an instrument he wants to use, reach out to exactly where he sees it and click on it with one of the ring buttons to activate it or drag it into another view.
- Many useful operations can be accomplished most easily with direct selection and manipulation of objects. You can move and resize windows (i.e. instruments) the usual 2D way by dragging their borders. However, you can also exploit the 3D tracking of the ring to simultaneously move and resize an instrument. Simply grab the title bar and pull it toward you to make it larger or away from you to make it smaller, while simultaneously positioning it. If you pull it in towards your head far enough, as if to attach it to your HMD, it will change colors, indicating that if you let go of it, it will remain as a head-stabilized object. This is effectively like grabbing an instrument off your cockpit panel and attaching it to your Heads-Up-Display (HUD) so that it will always be visible in the foreground no matter where you look. By pushing it away far enough it will convert back to a cockpit panel instrument.
- One of the cockpit windows that can be manipulated in a similar manner is the windshield itself. Simply click on any clear area of the ”glass” where there aren't any graphical objects you might accidentally select, then drag it left/right or up/down to rotate the whole cockpit in space. This is one way of”steering” the cockpit, which is particularly useful for small course corrections or size adjustments or to refocus your attention on another area of the workbench nearby.
- Virtual mouse pad cursor control
- Though fast and intuitive, the direct pointing technique would become very tiring if used to work with an instrument that requires extended repetitive clicking, such as a web browser or hypertext manual. A virtual mouse pad technique can overcome this problem. As soon as the user's hand drops below the viewing frustum of the HMD, the cursor control automatically switches into this mode, in which left-and-right motion of the ring moves the cursor left-and-right, in-and-out motion moves it up and down, and vertical position has no effect. This allows the user to rest his hand comfortably in his lap or on a desk, and control the cursor by sliding his hand horizontally a few inches as if on an imaginary mouse pad.
- It is desirable that if the user positions the cursor on a particular object then moves his head without moving the ring, the cursor will remain on the object. This means that the cursor is drawn as an object in the cockpit-stabilized coordinates rather than the head-stabilized screen coordinates. This has several implications. First, the cursor is associated with a point on the spherical information cockpit surface, only a portion of which is visible in the HMD, so the cursor could be out of view and quite difficult to find. A wiggling gesture is then used to bring it back into the current center of display. Second, the ring tracking must be calculated in the cockpit stabilized coordinate frame, which means that if the user turns to the right, an ”in-and-out” motion switches from cockpit x-axis to y-axis and has an unexpected effect. To avoid this, the ring position is transformed into cylindrical polar coordinates and the radial and tangential components are used to control cursor vertical and horizontal motion respectively.
- Command Gestures
- Ring tracker gestures may be used as a substitute for voice commands in situations where visual theatrics are more acceptable than audible ones, or where it is too noisy for reliable speech recognition. In general, gestures should commence outside of the direct pointing and virtual mouse pad regions, in order to avoid accidentally selecting and moving objects. This leaves the sides and top of the viewing frustum, and the first few inches in front of the face (which are not used for direct pointing). The gestures are executed by depressing a mouse button, possibly making a certain movement, then releasing the button. They are always relative to the head in order to exploit proprioception, and the fact that the head is tracked, while the rest of the body is not. Many gestures may be defined, but the most commonly needed is a boresight command to reset the heading direction of the cockpit to the current forward direction of the person's head as he walks about.
- Measuring or Digitizing
- Most people can hold their head very still, which opens the possibility that the ring tracker can be used to make measurements between two points that are close enough that both can be seen without moving the head. This might be useful in an application such as taking inventory of how many pieces of each size are in a stockroom. Likewise, ail application might ask you to quickly digitize a few corners of a component so it can determine based on the dimensions what model of the component you are looking at and locate the appropriate manual pages.
- To measure the distance between two close objects that are both within the display FOV at the same time, the user clicks both objects while holding his head still. The distance is computed as the norm of the difference of the two vector positions thus stored.
- For two objects that are too far apart to be in the display FOV at once, a more elaborate procedure may be employed. The user first looks at the first object, positions the pointer beacon on it and depresses a button. At the moment the button is pressed, a world frame position vector (p1) of the first object is stored and then the tracking mode is switched to 6-DOF tracking of the head relative to the stationary hand-held pointer, as previously described. While holding the pointer stationary on the object and keeping the button depressed, the user then repositions his head until the second object is in view, releases the button, and holds his head still while moving the pointer to the second object, then clicking it to capture the second position vector (p2) in the same world coordinate frame as the first. This technique may be practiced either with a single pointing beacon operated by one hand, or using separate pointing beacons in each hand, to achieve approximately the same functionality as a conventional tape measure, but with the added benefit that the measurements are automatically stored on a digital computer.
- Relationships among remote objects may also be measured using standard triangulation surveying methods, exploiting the functional similarity of a see-through HMD optic with orientation tracker to a surveyor's theodolite (although a tripod mounted theodolite is likely to be more accurate).
- Mixed Display and AR-on-Demand Applications
- The previous section presented the information cockpit as a specific variation on Feiner's body-stabilized information surround. However, the cockpit metaphor also allows the user to make use of the head-stabilized and world-stabilized coordinate frames at the same time. The previous section gave one example of this in which the pilot drags information from the cockpit onto the HUD, which makes it head-stabilized. For example, one may wish to have an alerting device always visible in the HUD that pops up notifications whenever a phone call, page or email is received, or when a scheduled meeting is about to begin, etc.
- Likewise, one may wish to grab a certain instrument and paste it onto a physical object in worldspace. For example, while debugging a circuit board, you could overlay an interactive block diagram or schematic on the board, and attach a virtual scope trace to your hand that is holding the scope probe (which is possible because the hand is tracked by the ring pointer). To do this, you must first plant an anchor beacon, then click three corners of the circuit board to align the block diagram to it.
- One important reason to plant anchor beacons is to create a shared AR workspace for communication or collaboration with coworkers as described in M. Billinghurst, S. Weghorst and T. Furness (Shared Space: An Augmented Reality Approach for Computer Supported Cooperative Work. Virtual Reality Vol. 3(1) 1998) and D. Schmalstieg, A. Fuhrmann, Z. Szalavari, and M. Gervautz (Studierstube: An Environment for Collaboration in Augmented Reality. In CVE 96 Workshop Proceedings, September, 1996) incorporated by reference. Imagine a paperless construction site with numerous workers building a structure according to the plans they are viewing on their wearable computers. It is nice that they don't have to drag large rolls of blueprints around, but they have no way to stand around a blueprint and point to things. The solution is for someone to drop two anchor pins on a table, defining the top two corners of a virtual blueprint or model that each person can see in correct perspective from his own vantage point.
- A Variant Technique for Tracking the User's Hand
- Some implementations of the invention use an inertial orientation sensor to track the rotation of the head, and an acoustic or optical position tracker to track the position of the hand relative to the head. For many applications, the performance of the acoustic or optical position tracker is sufficient. Furthermore, it has the great advantage that the item being tracked can be a small wireless transponder, or even a passive marker. For some applications, such as the ring-mounted pointing device for wearable computing, this is an overwhelming advantage.
- However, for some applications, such as a virtual reality game, it may be desired to have the virtual object controlled by the hand tracker (e.g. a virtual sword or gun or racquet) respond to the hand motion with extremely fast smooth response. Acoustic, magnetic, or videometric hand trackers may introduce noticeable latency or jitter in these applications. Inertial position and orientation trackers are well known to provide extremely low latency and low jitter, but they require drift correction, especially if tracking position and not just orientation is desired. In a typical virtual reality application, the user's head and hand may both be tracked with6 degrees of freedom relative to an external reference frame by using inertial sensors on the head and on the hand to measure their motion with a high update rate and low latency. The drift of these inertial sensors is corrected by making measurements with an ultrasonic, optical or magnetic tracking reference device mounted in the environment.
- In some implementations of the present invention, the drift and latency issues can be addressed without the requirement of a reference device mounted in the environment. Foxlin, ”Head-tracking Relative to a Moving Vehicle or Simulator Platform Using Differential Inertial Sensors,” Proceedings of Helmet and Head-Mounted Displays V, SPIE vol. 4021 (2000) and co-pending U.S. patent application Ser. No. 09/556,135, which are incorporated herein by reference, describe techniques which enable the use of inertial sensors to track the motion of an object relative to a reference frame that is moving, even where the motion is not known completely. These techniques require that inertial sensors be attached to the moving body being used as the reference frame (in the cited references an example of the reference frame is given of a vehicle or motion-platform and an example of the tracked object is given as a head; in the present invention, the moving reference frame may be the user's head and the tracked object may be the user's hand or hand-mounted or hand-held object), as well as to the object being tracked (here, e.g., the user's hand). The techniques utilize angular rate and linear acceleration signals from the sourceless orientation trackers on the reference frame and on the tracked object to derive a differential inertial signal representative of the motion of the object relative to the frame. In embodiments of the present invention, this technique may be used to derive a differential inertial signal representative of the motion of the hand relative to the head.
- FIG. 5 illustrates a user wearing a portable VR tennis game or training system. The computer and batteries are contained in
backpack 502, which is cabled toHMD 500 to which are mountedinertial sensors 506 andultrasonic transducers 510. He is holding a hand-heldobject 516, in this case a tennis racquet, to which are attachedinertial sensors 508 andultrasonic transducers 512. These hand-mounted devices may be powered by their own batteries and communicate by wireless means to the system on the users head and torso, or there may be an additional cable between the racquet and the backpack. The signals frominertial sensors 506 are processed by a first algorithm, preferably a drift-corrected inertial orientation tracking algorithm such as described in U.S. Pat. No. 5,645,077 to obtain a sourceless measurement of the head orientation. In addition, the signals from the hand-mountedinertial sensors 508 and the head-mountedinertial sensors 506 are jointly processed to track both the position and orientation of the hand relative to the head, preferably using an algorithm such as described in Foxlin (2000) and co-pending U.S. patent application Ser. No. 09/556,135. The drift of this relative inertial tracking is corrected by therelative range measurements 514. In the illustrated system there are alsoearphones 504 to provide 3D spatialized audio, and ahaptic feedback device 518 to provide tactile feedback to the user when the virtual ball has hit the virtual racquet. - In general such a system may be used for other types of activities, such as a sword-fighting or gun-fighting game or trainer, a surgical trainer, an immersive design environment, a human-computer interface, or any other application known or not yet known which requires tracking of a user's head and one or more limbs or limb-mounted devices. While it is especially advantageous for mobile or portable applications in which the computer is wearable, this is not a requirement, and the user may be cabled to an off-body computer or communicate with an off-body computer through a wireless connection. In this case, it is still an advantage of the current invention that the tracking is accomplished without setting up an off-body reference device.
- Other embodiments are within the scope of the claims.
- The implementations described above track the hand with a head-mounted acoustic tracking system because this technology can be totally embedded in a lightweight headset and achieve high resolution tracking over a very wide FOV.
- However, the head mounted position tracker need not be acoustic. It may be an electro-optical system which tracks LEDs, optical sensors, or reflective markers, or a video machine-vision device that recognizes the hands or fingers or some special markers mounted on the hands or fingers or handheld object, or even a magnetic tracker with a magnetic source held in the hand and sensors integrated in the headset or vice versa, or an RF position locating device.
- The implementation described above use inertial sourceless orientation trackers. Other implementations may use other forms of head orientation trackers, including trackers based on tilt-sensing or magnetic compass sensors, or any other form of head orientation tracker. In fact, some implementations may use no head orientation tracker. In this case, the tracking system would not enable the user to look around in a virtual environment by turning his head, but it would still be useful for manual interaction with computers using head-worn displays.
- A number of embodiments of the invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention.
Claims (59)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/837,373 US20040201857A1 (en) | 2000-01-28 | 2004-04-30 | Self-referenced tracking |
US11/463,776 US7301648B2 (en) | 2000-01-28 | 2006-08-10 | Self-referenced tracking |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17879700P | 2000-01-28 | 2000-01-28 | |
US09/770,691 US6757068B2 (en) | 2000-01-28 | 2001-01-26 | Self-referenced tracking |
US10/837,373 US20040201857A1 (en) | 2000-01-28 | 2004-04-30 | Self-referenced tracking |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/770,691 Continuation US6757068B2 (en) | 2000-01-28 | 2001-01-26 | Self-referenced tracking |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/463,776 Continuation US7301648B2 (en) | 2000-01-28 | 2006-08-10 | Self-referenced tracking |
Publications (1)
Publication Number | Publication Date |
---|---|
US20040201857A1 true US20040201857A1 (en) | 2004-10-14 |
Family
ID=22653982
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/770,691 Expired - Lifetime US6757068B2 (en) | 2000-01-28 | 2001-01-26 | Self-referenced tracking |
US10/837,373 Abandoned US20040201857A1 (en) | 2000-01-28 | 2004-04-30 | Self-referenced tracking |
US11/463,776 Expired - Lifetime US7301648B2 (en) | 2000-01-28 | 2006-08-10 | Self-referenced tracking |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/770,691 Expired - Lifetime US6757068B2 (en) | 2000-01-28 | 2001-01-26 | Self-referenced tracking |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/463,776 Expired - Lifetime US7301648B2 (en) | 2000-01-28 | 2006-08-10 | Self-referenced tracking |
Country Status (3)
Country | Link |
---|---|
US (3) | US6757068B2 (en) |
AU (1) | AU2001233019A1 (en) |
WO (1) | WO2001056007A1 (en) |
Cited By (102)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020154070A1 (en) * | 2001-03-13 | 2002-10-24 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and control program |
US20030156144A1 (en) * | 2002-02-18 | 2003-08-21 | Canon Kabushiki Kaisha | Information processing apparatus and method |
US20050162409A1 (en) * | 2000-08-18 | 2005-07-28 | International Business Machines Corporation | Projector and camera arrangement with shared optics and optical marker for use with whiteboard systems |
US7063256B2 (en) * | 2003-03-04 | 2006-06-20 | United Parcel Service Of America | Item tracking and processing systems and methods |
US20060167649A1 (en) * | 2005-01-25 | 2006-07-27 | Alexander Jeffrey M | Enhanced hang-timer for console simulation |
US20070081695A1 (en) * | 2005-10-04 | 2007-04-12 | Eric Foxlin | Tracking objects with markers |
US20070166669A1 (en) * | 2005-12-19 | 2007-07-19 | Raydon Corporation | Perspective tracking system |
US20070209586A1 (en) * | 2006-03-10 | 2007-09-13 | Ebensberger Jason M | Virtual coatings application system |
US20070209585A1 (en) * | 2006-03-10 | 2007-09-13 | Ebensberger Jason M | Virtual coatings application system |
US20070273610A1 (en) * | 2006-05-26 | 2007-11-29 | Itt Manufacturing Enterprises, Inc. | System and method to display maintenance and operational instructions of an apparatus using augmented reality |
US20070273557A1 (en) * | 2006-05-26 | 2007-11-29 | Itt Manufacturing Enterprises,Inc. | Augmented reality-based system and method providing status and control of unmanned vehicles |
US20080039868A1 (en) * | 2006-07-05 | 2008-02-14 | Aesculap Ag & Co. Kg | Calibration method and calibration device for a surgical referencing unit |
US20080124698A1 (en) * | 2006-11-28 | 2008-05-29 | Ebensberger Jason M | Virtual coatings application system with structured training and remote instructor capabilities |
US20080218331A1 (en) * | 2007-03-08 | 2008-09-11 | Itt Manufacturing Enterprises, Inc. | Augmented reality-based system and method to show the location of personnel and sensors inside occluded structures and provide increased situation awareness |
US20080266323A1 (en) * | 2007-04-25 | 2008-10-30 | Board Of Trustees Of Michigan State University | Augmented reality user interaction system |
US20080275670A1 (en) * | 2005-01-25 | 2008-11-06 | Drop Zone Corporation | Hang timer for determining time of flight of an object |
US20080280676A1 (en) * | 2007-05-07 | 2008-11-13 | Samsung Electronics Co. Ltd. | Wireless gaming method and wireless gaming-enabled mobile terminal |
US20090202975A1 (en) * | 2008-02-11 | 2009-08-13 | Michael Bolick | Virtual blasting system for removal of coating and/or rust from a virtual surface |
US20090209216A1 (en) * | 2008-02-20 | 2009-08-20 | Sony Corporation | Reflector for wireless television transmissions |
US7602301B1 (en) | 2006-01-09 | 2009-10-13 | Applied Technology Holdings, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US20090273542A1 (en) * | 2005-12-20 | 2009-11-05 | Kakuya Yamamoto | Content presentation apparatus, and content presentation method |
US20120200703A1 (en) * | 2009-10-22 | 2012-08-09 | Bluebird Aero Systems Ltd. | Imaging system for uav |
US20120218186A1 (en) * | 2011-02-27 | 2012-08-30 | Mr. David Brock, SR. | 3D Configuration Management System (CMS) Visualization and Management System |
US20130007668A1 (en) * | 2011-07-01 | 2013-01-03 | James Chia-Ming Liu | Multi-visor: managing applications in head mounted displays |
JP2013012024A (en) * | 2011-06-29 | 2013-01-17 | Olympus Corp | Information processing system, portable electronic apparatus, program and information storage medium |
WO2014022239A1 (en) * | 2012-07-29 | 2014-02-06 | Qualcomm Incorporated | Anatomical gestures detection system using radio signals |
US20140160170A1 (en) * | 2012-12-06 | 2014-06-12 | Nokia Corporation | Provision of an Image Element on a Display Worn by a User |
US8888786B2 (en) | 2003-06-09 | 2014-11-18 | OrthAlign, Inc. | Surgical orientation device and method |
US8911447B2 (en) | 2008-07-24 | 2014-12-16 | OrthAlign, Inc. | Systems and methods for joint replacement |
US8974467B2 (en) | 2003-06-09 | 2015-03-10 | OrthAlign, Inc. | Surgical orientation system and method |
US8974468B2 (en) | 2008-09-10 | 2015-03-10 | OrthAlign, Inc. | Hip surgery systems and methods |
US20150126281A1 (en) * | 2005-10-07 | 2015-05-07 | Percept Technologies Inc. | Enhanced optical and perceptual digital eyewear |
US9235064B2 (en) | 2005-10-07 | 2016-01-12 | Percept Technologies Inc. | Digital eyewear |
US9271756B2 (en) | 2009-07-24 | 2016-03-01 | OrthAlign, Inc. | Systems and methods for joint replacement |
WO2016072785A1 (en) * | 2014-11-07 | 2016-05-12 | Samsung Electronics Co., Ltd. | Direction based electronic device for displaying object and method thereof |
US9339226B2 (en) | 2010-01-21 | 2016-05-17 | OrthAlign, Inc. | Systems and methods for joint replacement |
US20160259404A1 (en) * | 2015-03-05 | 2016-09-08 | Magic Leap, Inc. | Systems and methods for augmented reality |
US9498231B2 (en) | 2011-06-27 | 2016-11-22 | Board Of Regents Of The University Of Nebraska | On-board tool tracking system and methods of computer assisted surgery |
US9504909B2 (en) | 2011-05-05 | 2016-11-29 | Qualcomm Incorporated | Method and apparatus of proximity and stunt recording for outdoor gaming |
US20160378294A1 (en) * | 2015-06-24 | 2016-12-29 | Shawn Crispin Wright | Contextual cursor display based on hand tracking |
US9549742B2 (en) | 2012-05-18 | 2017-01-24 | OrthAlign, Inc. | Devices and methods for knee arthroplasty |
US9649160B2 (en) | 2012-08-14 | 2017-05-16 | OrthAlign, Inc. | Hip replacement navigation system and method |
US9892563B2 (en) * | 2008-10-27 | 2018-02-13 | Sri International | System and method for generating a mixed reality environment |
US10030931B1 (en) * | 2011-12-14 | 2018-07-24 | Lockheed Martin Corporation | Head mounted display-based training tool |
US10105149B2 (en) | 2013-03-15 | 2018-10-23 | Board Of Regents Of The University Of Nebraska | On-board tool tracking system and methods of computer assisted surgery |
WO2018199979A1 (en) * | 2017-04-28 | 2018-11-01 | Hewlett-Packard Development Company, L.P. | Determining position and orientation of a user's torso for a display system |
WO2018231819A1 (en) * | 2017-06-17 | 2018-12-20 | Tactual Labs Co. | Six degrees of freedom tracking of objects using sensors |
US10180734B2 (en) | 2015-03-05 | 2019-01-15 | Magic Leap, Inc. | Systems and methods for augmented reality |
US10219811B2 (en) | 2011-06-27 | 2019-03-05 | Board Of Regents Of The University Of Nebraska | On-board tool tracking system and methods of computer assisted surgery |
CN109689173A (en) * | 2016-04-26 | 2019-04-26 | 奇跃公司 | It is tracked using the electromagnetism of augmented reality system |
US10363149B2 (en) | 2015-02-20 | 2019-07-30 | OrthAlign, Inc. | Hip replacement navigation system and method |
US10471478B2 (en) | 2017-04-28 | 2019-11-12 | United Parcel Service Of America, Inc. | Conveyor belt assembly for identifying an asset sort location and methods of utilizing the same |
WO2020028191A1 (en) | 2018-08-03 | 2020-02-06 | Magic Leap, Inc. | Unfused pose-based drift correction of a fused pose of a totem in a user interaction system |
WO2020077389A1 (en) * | 2018-10-15 | 2020-04-23 | Idearlabs Pty Ltd | "a method and system for determining an orientation of a user" |
US10649211B2 (en) | 2016-08-02 | 2020-05-12 | Magic Leap, Inc. | Fixed-distance virtual and augmented reality systems and methods |
US10762598B2 (en) | 2017-03-17 | 2020-09-01 | Magic Leap, Inc. | Mixed reality system with color virtual content warping and method of generating virtual content using same |
US10769752B2 (en) | 2017-03-17 | 2020-09-08 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
WO2020182309A1 (en) * | 2019-03-14 | 2020-09-17 | Huawei Technologies Co., Ltd. | Ultrasonic hand tracking system |
US10812936B2 (en) | 2017-01-23 | 2020-10-20 | Magic Leap, Inc. | Localization determination for mixed reality systems |
US10838207B2 (en) * | 2015-03-05 | 2020-11-17 | Magic Leap, Inc. | Systems and methods for augmented reality |
US10846864B2 (en) * | 2015-06-10 | 2020-11-24 | VTouch Co., Ltd. | Method and apparatus for detecting gesture in user-based spatial coordinate system |
US10861237B2 (en) | 2017-03-17 | 2020-12-08 | Magic Leap, Inc. | Mixed reality system with multi-source virtual content compositing and method of generating virtual content using same |
US10863995B2 (en) | 2017-03-14 | 2020-12-15 | OrthAlign, Inc. | Soft tissue measurement and balancing systems and methods |
US10869771B2 (en) | 2009-07-24 | 2020-12-22 | OrthAlign, Inc. | Systems and methods for joint replacement |
US10909711B2 (en) | 2015-12-04 | 2021-02-02 | Magic Leap, Inc. | Relocalization systems and methods |
US10915165B2 (en) * | 2016-07-29 | 2021-02-09 | Emmanuel Lusinchi | Methods and systems for controlling a displacement of a virtual point of view in a virtual reality environment |
US10918499B2 (en) | 2017-03-14 | 2021-02-16 | OrthAlign, Inc. | Hip replacement navigation systems and methods |
US10943521B2 (en) | 2018-07-23 | 2021-03-09 | Magic Leap, Inc. | Intra-field sub code timing in field sequential displays |
US10948994B2 (en) | 2016-02-29 | 2021-03-16 | Huawei Technologies Co., Ltd. | Gesture control method for wearable system and wearable system |
EP3738655A3 (en) * | 2006-05-04 | 2021-03-17 | Sony Interactive Entertainment LLC | Method and apparatus for use in determining lack of user activity, determining an activity level of a user, and/or adding a new player in relation to a system |
US11116574B2 (en) | 2006-06-16 | 2021-09-14 | Board Of Regents Of The University Of Nebraska | Method and apparatus for computer aided surgery |
US11200870B2 (en) | 2018-06-05 | 2021-12-14 | Magic Leap, Inc. | Homography transformation matrices based temperature calibration of a viewing system |
US11244485B2 (en) | 2016-01-19 | 2022-02-08 | Magic Leap, Inc. | Augmented reality systems and methods utilizing reflections |
US11280937B2 (en) | 2017-12-10 | 2022-03-22 | Magic Leap, Inc. | Anti-reflective coatings on optical waveguides |
US11347960B2 (en) | 2015-02-26 | 2022-05-31 | Magic Leap, Inc. | Apparatus for a near-eye display |
US11379948B2 (en) | 2018-07-23 | 2022-07-05 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
US11425189B2 (en) | 2019-02-06 | 2022-08-23 | Magic Leap, Inc. | Target intent-based clock speed determination and adjustment to limit total heat generated by multiple processors |
US11428937B2 (en) | 2005-10-07 | 2022-08-30 | Percept Technologies | Enhanced optical and perceptual digital eyewear |
US11445232B2 (en) | 2019-05-01 | 2022-09-13 | Magic Leap, Inc. | Content provisioning system and method |
US11510027B2 (en) | 2018-07-03 | 2022-11-22 | Magic Leap, Inc. | Systems and methods for virtual and augmented reality |
US11514673B2 (en) | 2019-07-26 | 2022-11-29 | Magic Leap, Inc. | Systems and methods for augmented reality |
US11521296B2 (en) | 2018-11-16 | 2022-12-06 | Magic Leap, Inc. | Image size triggered clarification to maintain image sharpness |
US11567324B2 (en) | 2017-07-26 | 2023-01-31 | Magic Leap, Inc. | Exit pupil expander |
US11579441B2 (en) | 2018-07-02 | 2023-02-14 | Magic Leap, Inc. | Pixel intensity modulation using modifying gain values |
WO2023028190A1 (en) * | 2021-08-26 | 2023-03-02 | Street Smarts VR | Mount for adapting weapons to a virtual tracker |
US11598651B2 (en) | 2018-07-24 | 2023-03-07 | Magic Leap, Inc. | Temperature dependent calibration of movement detection devices |
US11624929B2 (en) | 2018-07-24 | 2023-04-11 | Magic Leap, Inc. | Viewing device with dust seal integration |
US11630507B2 (en) | 2018-08-02 | 2023-04-18 | Magic Leap, Inc. | Viewing system with interpupillary distance compensation based on head motion |
US11737832B2 (en) | 2019-11-15 | 2023-08-29 | Magic Leap, Inc. | Viewing system for use in a surgical environment |
US11762222B2 (en) | 2017-12-20 | 2023-09-19 | Magic Leap, Inc. | Insert for augmented reality viewing device |
US11762623B2 (en) | 2019-03-12 | 2023-09-19 | Magic Leap, Inc. | Registration of local content between first and second augmented reality viewers |
US11776509B2 (en) | 2018-03-15 | 2023-10-03 | Magic Leap, Inc. | Image correction due to deformation of components of a viewing device |
US11790554B2 (en) | 2016-12-29 | 2023-10-17 | Magic Leap, Inc. | Systems and methods for augmented reality |
US11856479B2 (en) | 2018-07-03 | 2023-12-26 | Magic Leap, Inc. | Systems and methods for virtual and augmented reality along a route with markers |
US11874468B2 (en) | 2016-12-30 | 2024-01-16 | Magic Leap, Inc. | Polychromatic light out-coupling apparatus, near-eye displays comprising the same, and method of out-coupling polychromatic light |
US11885871B2 (en) | 2018-05-31 | 2024-01-30 | Magic Leap, Inc. | Radar head pose localization |
US11911117B2 (en) | 2011-06-27 | 2024-02-27 | Board Of Regents Of The University Of Nebraska | On-board tool tracking system and methods of computer assisted surgery |
US11944428B2 (en) | 2015-11-30 | 2024-04-02 | Nike, Inc. | Apparel with ultrasonic position sensing and haptic feedback for activities |
EP4240008A4 (en) * | 2020-10-30 | 2024-04-17 | Panasonic Intellectual Property Management Co., Ltd. | Display device |
US12016719B2 (en) | 2018-08-22 | 2024-06-25 | Magic Leap, Inc. | Patient viewing system |
US12033081B2 (en) | 2019-11-14 | 2024-07-09 | Magic Leap, Inc. | Systems and methods for virtual and augmented reality |
US12044851B2 (en) | 2018-12-21 | 2024-07-23 | Magic Leap, Inc. | Air pocket structures for promoting total internal reflection in a waveguide |
Families Citing this family (474)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7749089B1 (en) | 1999-02-26 | 2010-07-06 | Creative Kingdoms, Llc | Multi-media interactive play system |
JP3363837B2 (en) * | 1999-06-11 | 2003-01-08 | キヤノン株式会社 | User interface device and information processing method |
AU2001233019A1 (en) * | 2000-01-28 | 2001-08-07 | Intersense, Inc. | Self-referenced tracking |
US6761637B2 (en) | 2000-02-22 | 2004-07-13 | Creative Kingdoms, Llc | Method of game play using RFID tracking device |
US7878905B2 (en) | 2000-02-22 | 2011-02-01 | Creative Kingdoms, Llc | Multi-layered interactive play experience |
US7445550B2 (en) | 2000-02-22 | 2008-11-04 | Creative Kingdoms, Llc | Magical wand and interactive play experience |
US7110013B2 (en) * | 2000-03-15 | 2006-09-19 | Information Decision Technology | Augmented reality display integrated with self-contained breathing apparatus |
US7066781B2 (en) | 2000-10-20 | 2006-06-27 | Denise Chapman Weston | Children's toy with wireless tag/transponder |
SE517765C2 (en) * | 2000-11-16 | 2002-07-16 | Ericsson Telefon Ab L M | Registration of moving images by means of a portable communication device and an accessory device co-located with the object |
GB2374502B (en) * | 2001-01-29 | 2004-12-29 | Hewlett Packard Co | Distinguishing real-world sounds from audio user interface sounds |
GB2374503B (en) * | 2001-01-29 | 2005-04-13 | Hewlett Packard Co | Audio user interface with audio field orientation indication |
US20030227476A1 (en) * | 2001-01-29 | 2003-12-11 | Lawrence Wilcock | Distinguishing real-world sounds from audio user interface sounds |
GB2374506B (en) * | 2001-01-29 | 2004-11-17 | Hewlett Packard Co | Audio user interface with cylindrical audio field organisation |
GB2376283B (en) * | 2001-06-04 | 2005-03-16 | Hewlett Packard Co | Foot activated user input |
WO2003000367A1 (en) * | 2001-06-19 | 2003-01-03 | Faeger Jan G | A device and a method for creating an environment for a creature |
US7714880B2 (en) * | 2001-11-16 | 2010-05-11 | Honeywell International Inc. | Method and apparatus for displaying images on a display |
EP1315120A1 (en) * | 2001-11-26 | 2003-05-28 | Siemens Aktiengesellschaft | Pen input system |
US20070066396A1 (en) | 2002-04-05 | 2007-03-22 | Denise Chapman Weston | Retail methods for providing an interactive product to a consumer |
US6967566B2 (en) | 2002-04-05 | 2005-11-22 | Creative Kingdoms, Llc | Live-action interactive adventure game |
US20040002843A1 (en) * | 2002-05-13 | 2004-01-01 | Consolidated Global Fun Unlimited, Llc | Method and system for interacting with simulated phenomena |
US20050009608A1 (en) * | 2002-05-13 | 2005-01-13 | Consolidated Global Fun Unlimited | Commerce-enabled environment for interacting with simulated phenomena |
US20070265089A1 (en) * | 2002-05-13 | 2007-11-15 | Consolidated Global Fun Unlimited | Simulated phenomena interaction game |
JP4029675B2 (en) * | 2002-06-19 | 2008-01-09 | セイコーエプソン株式会社 | Image / tactile information input device |
US8947347B2 (en) * | 2003-08-27 | 2015-02-03 | Sony Computer Entertainment Inc. | Controlling actions in a video game unit |
US8797260B2 (en) | 2002-07-27 | 2014-08-05 | Sony Computer Entertainment Inc. | Inertially trackable hand-held controller |
US9393487B2 (en) * | 2002-07-27 | 2016-07-19 | Sony Interactive Entertainment Inc. | Method for mapping movements of a hand-held controller to game commands |
US10086282B2 (en) | 2002-07-27 | 2018-10-02 | Sony Interactive Entertainment Inc. | Tracking device for use in obtaining information for controlling game program execution |
US8570378B2 (en) | 2002-07-27 | 2013-10-29 | Sony Computer Entertainment Inc. | Method and apparatus for tracking three-dimensional movements of an object using a depth sensing camera |
US7674184B2 (en) | 2002-08-01 | 2010-03-09 | Creative Kingdoms, Llc | Interactive water attraction and quest game |
US7034779B2 (en) * | 2002-08-06 | 2006-04-25 | Information Decision Technologeis, Llc | Advanced ruggedized augmented reality instrumented self contained breathing apparatus |
US20050121031A1 (en) * | 2002-08-06 | 2005-06-09 | Ebersole John F.Jr. | Impact-protected advanced ruggedized augmented reality instrumented self contained breathing apparatus |
AU2003268119A1 (en) * | 2002-08-21 | 2004-03-11 | Neuroptics, Inc. | Intelligent patient interface for ophthalmic instruments |
JP2004151085A (en) * | 2002-09-27 | 2004-05-27 | Canon Inc | Method and apparatus for processing information |
ATE521953T1 (en) * | 2002-10-22 | 2011-09-15 | Artoolworks | TRACKING A SURFACE IN A THREE-DIMENSIONAL SCENE USING NATURAL VISUAL FEATURES OF THE SURFACE |
US6987257B2 (en) * | 2003-01-28 | 2006-01-17 | Honeywell International Inc. | Attitude determination system and method |
US8542717B2 (en) | 2003-03-03 | 2013-09-24 | Veroscan, Inc. | Interrogator and interrogation system employing the same |
US7019650B2 (en) * | 2003-03-03 | 2006-03-28 | Caducys, L.L.C. | Interrogator and interrogation system employing the same |
US7764178B2 (en) | 2003-03-03 | 2010-07-27 | Veroscan, Inc. | Interrogator and interrogation system employing the same |
US8063760B2 (en) | 2003-03-03 | 2011-11-22 | Veroscan, Inc. | Interrogator and interrogation system employing the same |
US8174366B2 (en) | 2003-03-03 | 2012-05-08 | Veroscan, Inc. | Interrogator and interrogation system employing the same |
US7893840B2 (en) | 2003-03-03 | 2011-02-22 | Veroscan, Inc. | Interrogator and interrogation system employing the same |
US9446319B2 (en) | 2003-03-25 | 2016-09-20 | Mq Gaming, Llc | Interactive gaming toy |
US7355561B1 (en) | 2003-09-15 | 2008-04-08 | United States Of America As Represented By The Secretary Of The Army | Systems and methods for providing images |
JP4218952B2 (en) * | 2003-09-30 | 2009-02-04 | キヤノン株式会社 | Data conversion method and apparatus |
US20080048931A1 (en) * | 2003-11-26 | 2008-02-28 | Rafael - Armament Development Authority Ltd. | Helmet System for Information or Weapon Systems |
US20050140696A1 (en) * | 2003-12-31 | 2005-06-30 | Buxton William A.S. | Split user interface |
EP1709519B1 (en) * | 2003-12-31 | 2014-03-05 | ABB Research Ltd. | A virtual control panel |
CA2558312A1 (en) | 2004-03-03 | 2005-09-15 | Caducys, L.L.C. | Interrogator and interrogation system employing the same |
WO2005119356A2 (en) * | 2004-05-28 | 2005-12-15 | Erik Jan Banning | Interactive direct-pointing system and calibration method |
WO2006006686A1 (en) * | 2004-07-15 | 2006-01-19 | Nippon Telegraph And Telephone Corporation | Inner force sense presentation device, inner force sense presentation method, and inner force sense presentation program |
US20060017654A1 (en) * | 2004-07-23 | 2006-01-26 | Romo Justin R | Virtual reality interactivity system and method |
EP1779344A4 (en) * | 2004-07-29 | 2009-08-19 | Motiva Llc | A human movement measurement system |
MX2007002011A (en) * | 2004-08-23 | 2007-08-07 | Gamecaster Inc | Apparatus, methods and systems for viewing and manipulating a virtual environment. |
US20060047414A1 (en) * | 2004-09-02 | 2006-03-02 | Matsushita Electric Industrial Co., Ltd. | Probe-car system using beacon and apparatus therefore |
US7501948B2 (en) | 2004-09-29 | 2009-03-10 | Lone Star Ip Holdings, Lp | Interrogation system employing prior knowledge about an object to discern an identity thereof |
US6934633B1 (en) * | 2004-10-15 | 2005-08-23 | The United States Of America As Represented By The Secretary Of The Navy | Helmet-mounted parachutist navigation system |
ITGE20050008A1 (en) | 2005-02-08 | 2006-08-09 | Scienza Ind Tecnologia S R L | METHOD AND APPARATUS FOR MONITORING HUMAN BODY MOVEMENTS IN SPORTS AND MEDICAL AREA. |
EP1712981A1 (en) * | 2005-04-15 | 2006-10-18 | Herman Bailey | Interactive augmented reality system |
US7672781B2 (en) * | 2005-06-04 | 2010-03-02 | Microstrain, Inc. | Miniaturized wireless inertial sensing system |
US7773074B2 (en) * | 2005-06-28 | 2010-08-10 | Siemens Medical Solutions Usa, Inc. | Medical diagnostic imaging three dimensional navigation device and methods |
US9285897B2 (en) | 2005-07-13 | 2016-03-15 | Ultimate Pointer, L.L.C. | Easily deployable interactive direct-pointing system and calibration method therefor |
US20070016372A1 (en) * | 2005-07-14 | 2007-01-18 | Gm Global Technology Operations, Inc. | Remote Perspective Vehicle Environment Observation System |
US8313379B2 (en) | 2005-08-22 | 2012-11-20 | Nintendo Co., Ltd. | Video game system with wireless modular handheld controller |
JP4805633B2 (en) | 2005-08-22 | 2011-11-02 | 任天堂株式会社 | Game operation device |
US7942745B2 (en) | 2005-08-22 | 2011-05-17 | Nintendo Co., Ltd. | Game operating device |
US7927216B2 (en) | 2005-09-15 | 2011-04-19 | Nintendo Co., Ltd. | Video game system with wireless modular handheld controller |
JP4262726B2 (en) | 2005-08-24 | 2009-05-13 | 任天堂株式会社 | Game controller and game system |
US8870655B2 (en) | 2005-08-24 | 2014-10-28 | Nintendo Co., Ltd. | Wireless game controllers |
US20070047517A1 (en) * | 2005-08-29 | 2007-03-01 | Hua Xu | Method and apparatus for altering a media activity |
US8308563B2 (en) | 2005-08-30 | 2012-11-13 | Nintendo Co., Ltd. | Game system and storage medium having game program stored thereon |
US8157651B2 (en) | 2005-09-12 | 2012-04-17 | Nintendo Co., Ltd. | Information processing program |
US20070136026A1 (en) * | 2005-11-28 | 2007-06-14 | Farheap Solutions Inc. | GPS-Guided Visual Display |
US8184367B2 (en) * | 2006-02-15 | 2012-05-22 | University Of Central Florida Research Foundation | Dynamically focused optical instrument |
US10653497B2 (en) | 2006-02-16 | 2020-05-19 | Globus Medical, Inc. | Surgical tool systems and methods |
US10357184B2 (en) | 2012-06-21 | 2019-07-23 | Globus Medical, Inc. | Surgical tool systems and method |
US10893912B2 (en) | 2006-02-16 | 2021-01-19 | Globus Medical Inc. | Surgical tool systems and methods |
JP4530419B2 (en) * | 2006-03-09 | 2010-08-25 | 任天堂株式会社 | Coordinate calculation apparatus and coordinate calculation program |
JP4151982B2 (en) | 2006-03-10 | 2008-09-17 | 任天堂株式会社 | Motion discrimination device and motion discrimination program |
JP4684147B2 (en) | 2006-03-28 | 2011-05-18 | 任天堂株式会社 | Inclination calculation device, inclination calculation program, game device, and game program |
US8688375B2 (en) | 2006-05-31 | 2014-04-01 | Trx Systems, Inc. | Method and system for locating and monitoring first responders |
SE0601216L (en) * | 2006-05-31 | 2007-12-01 | Abb Technology Ltd | Virtual workplace |
ATE529087T1 (en) * | 2006-08-15 | 2011-11-15 | Koninkl Philips Electronics Nv | ASSISTANCE SYSTEM FOR VISUALLY IMPAIRED PERSONS |
JP5294442B2 (en) * | 2006-09-13 | 2013-09-18 | 任天堂株式会社 | GAME DEVICE AND GAME PROGRAM |
US7950003B1 (en) * | 2006-12-07 | 2011-05-24 | Sony Computer Entertainment Inc. | Heads-up-display software development tool for analyzing and optimizing computer software |
US20080211768A1 (en) * | 2006-12-07 | 2008-09-04 | Randy Breen | Inertial Sensor Input Device |
US8248462B2 (en) * | 2006-12-15 | 2012-08-21 | The Board Of Trustees Of The University Of Illinois | Dynamic parallax barrier autosteroscopic display system and method |
US20080168402A1 (en) * | 2007-01-07 | 2008-07-10 | Christopher Blumenberg | Application Programming Interfaces for Gesture Operations |
US7844915B2 (en) | 2007-01-07 | 2010-11-30 | Apple Inc. | Application programming interfaces for scrolling operations |
US20080168478A1 (en) | 2007-01-07 | 2008-07-10 | Andrew Platzer | Application Programming Interfaces for Scrolling |
JP5127242B2 (en) | 2007-01-19 | 2013-01-23 | 任天堂株式会社 | Acceleration data processing program and game program |
CN101542587B (en) * | 2007-05-08 | 2012-09-05 | 松下电器产业株式会社 | Display device and image display method |
US9448072B2 (en) | 2007-05-31 | 2016-09-20 | Trx Systems, Inc. | System and method for locating, tracking, and/or monitoring the status of personnel and/or assets both indoors and outdoors |
US9395190B1 (en) | 2007-05-31 | 2016-07-19 | Trx Systems, Inc. | Crowd sourced mapping with robust structural features |
US9733091B2 (en) | 2007-05-31 | 2017-08-15 | Trx Systems, Inc. | Collaborative creation of indoor maps |
US9003304B2 (en) * | 2007-08-16 | 2015-04-07 | International Business Machines Corporation | Method and apparatus for moving an avatar in a virtual universe |
US7990387B2 (en) * | 2007-08-16 | 2011-08-02 | International Business Machines Corporation | Method and apparatus for spawning projected avatars in a virtual universe |
US8345049B2 (en) * | 2007-08-16 | 2013-01-01 | International Business Machine Corporation | Method and apparatus for predicting avatar movement in a virtual universe |
EP2587345A3 (en) | 2007-08-19 | 2013-06-26 | Ringbow Ltd. | Finger-worn devices and related methods of use |
US20090058850A1 (en) * | 2007-09-04 | 2009-03-05 | Wey Fun | System and method for intuitive interactive navigational control in virtual environments |
US20090079745A1 (en) * | 2007-09-24 | 2009-03-26 | Wey Fun | System and method for intuitive interactive navigational control in virtual environments |
US20090094188A1 (en) * | 2007-10-03 | 2009-04-09 | Edward Covannon | Facilitating identification of an object recorded in digital content records |
US8094090B2 (en) * | 2007-10-19 | 2012-01-10 | Southwest Research Institute | Real-time self-visualization system |
US8127235B2 (en) | 2007-11-30 | 2012-02-28 | International Business Machines Corporation | Automatic increasing of capacity of a virtual space in a virtual world |
US9001036B2 (en) | 2007-12-20 | 2015-04-07 | University Of Central Florida Research Foundation, Inc. | Systems and methods of camera-based fingertip tracking |
US20090164919A1 (en) | 2007-12-24 | 2009-06-25 | Cary Lee Bates | Generating data for managing encounters in a virtual world environment |
US8933876B2 (en) | 2010-12-13 | 2015-01-13 | Apple Inc. | Three dimensional user interface session control |
US9035876B2 (en) | 2008-01-14 | 2015-05-19 | Apple Inc. | Three-dimensional user interface session control |
US8217995B2 (en) * | 2008-01-18 | 2012-07-10 | Lockheed Martin Corporation | Providing a collaborative immersive environment using a spherical camera and motion capture |
US8645827B2 (en) | 2008-03-04 | 2014-02-04 | Apple Inc. | Touch event model |
US8717305B2 (en) | 2008-03-04 | 2014-05-06 | Apple Inc. | Touch event model for web pages |
US8416196B2 (en) | 2008-03-04 | 2013-04-09 | Apple Inc. | Touch event model programming interface |
US8875026B2 (en) * | 2008-05-01 | 2014-10-28 | International Business Machines Corporation | Directed communication in a virtual environment |
US8671349B2 (en) * | 2008-05-15 | 2014-03-11 | International Business Machines Corporation | Virtual universe teleportation suggestion service |
US8820939B2 (en) | 2008-06-17 | 2014-09-02 | The Invention Science Fund I, Llc | Projection associated methods and systems |
US8384005B2 (en) | 2008-06-17 | 2013-02-26 | The Invention Science Fund I, Llc | Systems and methods for selectively projecting information in response to at least one specified motion associated with pressure applied to at least one projection surface |
US20090309826A1 (en) | 2008-06-17 | 2009-12-17 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Systems and devices |
US8430515B2 (en) | 2008-06-17 | 2013-04-30 | The Invention Science Fund I, Llc | Systems and methods for projecting |
US8733952B2 (en) | 2008-06-17 | 2014-05-27 | The Invention Science Fund I, Llc | Methods and systems for coordinated use of two or more user responsive projectors |
US8608321B2 (en) | 2008-06-17 | 2013-12-17 | The Invention Science Fund I, Llc | Systems and methods for projecting in response to conformation |
US8936367B2 (en) | 2008-06-17 | 2015-01-20 | The Invention Science Fund I, Llc | Systems and methods associated with projecting in response to conformation |
US8540381B2 (en) | 2008-06-17 | 2013-09-24 | The Invention Science Fund I, Llc | Systems and methods for receiving information associated with projecting |
US20090310038A1 (en) | 2008-06-17 | 2009-12-17 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Projection in response to position |
US8944608B2 (en) | 2008-06-17 | 2015-02-03 | The Invention Science Fund I, Llc | Systems and methods associated with projecting in response to conformation |
US8641203B2 (en) | 2008-06-17 | 2014-02-04 | The Invention Science Fund I, Llc | Methods and systems for receiving and transmitting signals between server and projector apparatuses |
US8308304B2 (en) | 2008-06-17 | 2012-11-13 | The Invention Science Fund I, Llc | Systems associated with receiving and transmitting information related to projection |
US8723787B2 (en) | 2008-06-17 | 2014-05-13 | The Invention Science Fund I, Llc | Methods and systems related to an image capture projection surface |
US8267526B2 (en) | 2008-06-17 | 2012-09-18 | The Invention Science Fund I, Llc | Methods associated with receiving and transmitting information related to projection |
US9480919B2 (en) * | 2008-10-24 | 2016-11-01 | Excalibur Ip, Llc | Reconfiguring reality using a reality overlay device |
US8963804B2 (en) * | 2008-10-30 | 2015-02-24 | Honeywell International Inc. | Method and system for operating a near-to-eye display |
WO2010057304A1 (en) * | 2008-11-21 | 2010-05-27 | London Health Sciences Centre Research Inc. | Hands-free pointer system |
US8584026B2 (en) * | 2008-12-29 | 2013-11-12 | Avaya Inc. | User interface for orienting new users to a three dimensional computer-generated virtual environment |
US20120202569A1 (en) * | 2009-01-13 | 2012-08-09 | Primesense Ltd. | Three-Dimensional User Interface for Game Applications |
US8285499B2 (en) | 2009-03-16 | 2012-10-09 | Apple Inc. | Event recognition |
US9684521B2 (en) | 2010-01-26 | 2017-06-20 | Apple Inc. | Systems having discrete and continuous gesture recognizers |
US8566045B2 (en) | 2009-03-16 | 2013-10-22 | Apple Inc. | Event recognition |
US9311112B2 (en) | 2009-03-16 | 2016-04-12 | Apple Inc. | Event recognition |
US9067097B2 (en) * | 2009-04-10 | 2015-06-30 | Sovoz, Inc. | Virtual locomotion controller apparatus and methods |
JP5522349B2 (en) * | 2009-04-14 | 2014-06-18 | 任天堂株式会社 | INPUT SYSTEM, INFORMATION PROCESSING SYSTEM, PERIPHERAL DEVICE CONTROL METHOD, AND OPERATION DEVICE CONTROL PROGRAM |
JP5247590B2 (en) * | 2009-05-21 | 2013-07-24 | キヤノン株式会社 | Information processing apparatus and calibration processing method |
US20100311512A1 (en) * | 2009-06-04 | 2010-12-09 | Timothy James Lock | Simulator with enhanced depth perception |
US20100309197A1 (en) * | 2009-06-08 | 2010-12-09 | Nvidia Corporation | Interaction of stereoscopic objects with physical objects in viewing area |
US9801778B2 (en) * | 2009-06-19 | 2017-10-31 | Andrew Mahoney | System and method for alerting visually impaired users of nearby objects |
US8773330B2 (en) * | 2009-06-25 | 2014-07-08 | The Boeing Company | Method and apparatus for a virtual mission control station |
US8531526B1 (en) * | 2009-08-25 | 2013-09-10 | Clinton A. Spence | Wearable video recorder and monitor system and associated method |
US8736616B2 (en) * | 2009-09-15 | 2014-05-27 | Sony Corporation | Combining multi-sensory inputs for digital animation |
FR2950189B1 (en) * | 2009-09-17 | 2011-10-28 | Univ Paris 6 Pierre Et Marie Curie | DEVICE FOR RECONSTITUTING SPEECH BY ULTRASONIC SURVEY OF THE PHONATORY DEVICE |
US8532962B2 (en) | 2009-12-23 | 2013-09-10 | Honeywell International Inc. | Approach for planning, designing and observing building systems |
US20110181497A1 (en) * | 2010-01-26 | 2011-07-28 | Roni Raviv | Object related augmented reality play system |
US8467133B2 (en) | 2010-02-28 | 2013-06-18 | Osterhout Group, Inc. | See-through display with an optical assembly including a wedge-shaped illumination system |
US9129295B2 (en) | 2010-02-28 | 2015-09-08 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses with a fast response photochromic film system for quick transition from dark to clear |
US8488246B2 (en) | 2010-02-28 | 2013-07-16 | Osterhout Group, Inc. | See-through near-eye display glasses including a curved polarizing film in the image source, a partially reflective, partially transmitting optical element and an optically flat film |
US9097890B2 (en) | 2010-02-28 | 2015-08-04 | Microsoft Technology Licensing, Llc | Grating in a light transmissive illumination system for see-through near-eye display glasses |
US9182596B2 (en) | 2010-02-28 | 2015-11-10 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses with the optical assembly including absorptive polarizers or anti-reflective coatings to reduce stray light |
US8472120B2 (en) | 2010-02-28 | 2013-06-25 | Osterhout Group, Inc. | See-through near-eye display glasses with a small scale image source |
US20120249797A1 (en) | 2010-02-28 | 2012-10-04 | Osterhout Group, Inc. | Head-worn adaptive display |
US9366862B2 (en) | 2010-02-28 | 2016-06-14 | Microsoft Technology Licensing, Llc | System and method for delivering content to a group of see-through near eye display eyepieces |
US8482859B2 (en) | 2010-02-28 | 2013-07-09 | Osterhout Group, Inc. | See-through near-eye display glasses wherein image light is transmitted to and reflected from an optically flat film |
US9341843B2 (en) | 2010-02-28 | 2016-05-17 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses with a small scale image source |
US9759917B2 (en) | 2010-02-28 | 2017-09-12 | Microsoft Technology Licensing, Llc | AR glasses with event and sensor triggered AR eyepiece interface to external devices |
US9128281B2 (en) | 2010-09-14 | 2015-09-08 | Microsoft Technology Licensing, Llc | Eyepiece with uniformly illuminated reflective display |
US9223134B2 (en) | 2010-02-28 | 2015-12-29 | Microsoft Technology Licensing, Llc | Optical imperfections in a light transmissive illumination system for see-through near-eye display glasses |
US9097891B2 (en) | 2010-02-28 | 2015-08-04 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses including an auto-brightness control for the display brightness based on the brightness in the environment |
KR20130000401A (en) * | 2010-02-28 | 2013-01-02 | 오스터하우트 그룹 인코포레이티드 | Local advertising content on an interactive head-mounted eyepiece |
US9134534B2 (en) | 2010-02-28 | 2015-09-15 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses including a modular image source |
US9285589B2 (en) | 2010-02-28 | 2016-03-15 | Microsoft Technology Licensing, Llc | AR glasses with event and sensor triggered control of AR eyepiece applications |
US20150309316A1 (en) | 2011-04-06 | 2015-10-29 | Microsoft Technology Licensing, Llc | Ar glasses with predictive control of external device based on event input |
US9229227B2 (en) | 2010-02-28 | 2016-01-05 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses with a light transmissive wedge shaped illumination system |
US9091851B2 (en) | 2010-02-28 | 2015-07-28 | Microsoft Technology Licensing, Llc | Light control in head mounted displays |
US10180572B2 (en) | 2010-02-28 | 2019-01-15 | Microsoft Technology Licensing, Llc | AR glasses with event and user action control of external applications |
US8477425B2 (en) | 2010-02-28 | 2013-07-02 | Osterhout Group, Inc. | See-through near-eye display glasses including a partially reflective, partially transmitting optical element |
JP5530234B2 (en) * | 2010-03-29 | 2014-06-25 | オリンパス株式会社 | Operation input device and manipulator system |
SE536160C2 (en) * | 2010-04-16 | 2013-06-04 | Bae Systems Bofors Ab | Method and apparatus for display |
US8995678B2 (en) | 2010-04-30 | 2015-03-31 | Honeywell International Inc. | Tactile-based guidance system |
US8990049B2 (en) | 2010-05-03 | 2015-03-24 | Honeywell International Inc. | Building structure discovery and display from various data artifacts at scene |
US8538687B2 (en) | 2010-05-04 | 2013-09-17 | Honeywell International Inc. | System for guidance and navigation in a building |
JP5704833B2 (en) * | 2010-05-10 | 2015-04-22 | オリンパス株式会社 | Operation input device and manipulator system |
US8552999B2 (en) | 2010-06-14 | 2013-10-08 | Apple Inc. | Control selection approximation |
WO2011162753A1 (en) | 2010-06-23 | 2011-12-29 | Mako Sugical Corp. | Inertially tracked objects |
KR20120000161A (en) * | 2010-06-25 | 2012-01-02 | 삼성전자주식회사 | Pointing device, controlling method of the same, glass for 3d image and display apparatus |
US9201501B2 (en) | 2010-07-20 | 2015-12-01 | Apple Inc. | Adaptive projector |
CN102959616B (en) | 2010-07-20 | 2015-06-10 | 苹果公司 | Interactive reality augmentation for natural interaction |
US8959013B2 (en) | 2010-09-27 | 2015-02-17 | Apple Inc. | Virtual keyboard for a non-tactile three dimensional user interface |
CN103238339B (en) * | 2010-12-02 | 2015-12-09 | 尤特瑞登特生产公司 | Check and follow the tracks of the system and method for stereoscopic video images |
US8872762B2 (en) | 2010-12-08 | 2014-10-28 | Primesense Ltd. | Three dimensional user interface cursor control |
US20120157204A1 (en) * | 2010-12-20 | 2012-06-21 | Lai Games Australia Pty Ltd. | User-controlled projector-based games |
US8773946B2 (en) | 2010-12-30 | 2014-07-08 | Honeywell International Inc. | Portable housings for generation of building maps |
US20120188148A1 (en) * | 2011-01-24 | 2012-07-26 | Microvision, Inc. | Head Mounted Meta-Display System |
US8509483B2 (en) * | 2011-01-31 | 2013-08-13 | Qualcomm Incorporated | Context aware augmentation interactions |
US20120196684A1 (en) * | 2011-02-01 | 2012-08-02 | David Richardson | Combining motion capture and timing to create a virtual gaming experience |
EP3527121B1 (en) | 2011-02-09 | 2023-08-23 | Apple Inc. | Gesture detection in a 3d mapping environment |
US20120229509A1 (en) * | 2011-03-07 | 2012-09-13 | Liu Guangsong | System and method for user interaction |
WO2012125596A2 (en) | 2011-03-12 | 2012-09-20 | Parshionikar Uday | Multipurpose controller for electronic devices, facial expressions management and drowsiness detection |
US20120254809A1 (en) * | 2011-03-31 | 2012-10-04 | Nokia Corporation | Method and apparatus for motion gesture recognition |
WO2012131660A1 (en) | 2011-04-01 | 2012-10-04 | Ecole Polytechnique Federale De Lausanne (Epfl) | Robotic system for spinal and other surgeries |
US8810598B2 (en) | 2011-04-08 | 2014-08-19 | Nant Holdings Ip, Llc | Interference based augmented reality hosting platforms |
US9298363B2 (en) | 2011-04-11 | 2016-03-29 | Apple Inc. | Region activation for touch sensitive surface |
US9035774B2 (en) | 2011-04-11 | 2015-05-19 | Lone Star Ip Holdings, Lp | Interrogator and system employing the same |
US9342928B2 (en) | 2011-06-29 | 2016-05-17 | Honeywell International Inc. | Systems and methods for presenting building information |
US9459758B2 (en) | 2011-07-05 | 2016-10-04 | Apple Inc. | Gesture-based interface with enhanced features |
US8881051B2 (en) | 2011-07-05 | 2014-11-04 | Primesense Ltd | Zoom-based gesture user interface |
US9377865B2 (en) | 2011-07-05 | 2016-06-28 | Apple Inc. | Zoom-based gesture user interface |
US8179604B1 (en) | 2011-07-13 | 2012-05-15 | Google Inc. | Wearable marker for passive interaction |
US9243902B2 (en) | 2011-07-26 | 2016-01-26 | Thales Visionix, Inc. | System for light source location detection |
US9030498B2 (en) | 2011-08-15 | 2015-05-12 | Apple Inc. | Combining explicit select gestures and timeclick in a non-tactile three dimensional user interface |
US9122311B2 (en) | 2011-08-24 | 2015-09-01 | Apple Inc. | Visual feedback for tactile and non-tactile user interfaces |
US9218063B2 (en) | 2011-08-24 | 2015-12-22 | Apple Inc. | Sessionless pointing user interface |
JP5821464B2 (en) * | 2011-09-22 | 2015-11-24 | セイコーエプソン株式会社 | Head-mounted display device |
US9690100B1 (en) * | 2011-09-22 | 2017-06-27 | Sprint Communications Company L.P. | Wireless communication system with a liquid crystal display embedded in an optical lens |
CN103018903A (en) * | 2011-09-23 | 2013-04-03 | 奇想创造事业股份有限公司 | Head mounted display with displaying azimuth locking device and display method thereof |
US20150199081A1 (en) * | 2011-11-08 | 2015-07-16 | Google Inc. | Re-centering a user interface |
CN103108197A (en) | 2011-11-14 | 2013-05-15 | 辉达公司 | Priority level compression method and priority level compression system for three-dimensional (3D) video wireless display |
US20130131897A1 (en) * | 2011-11-23 | 2013-05-23 | Honeywell International Inc. | Three dimensional auditory reporting of unusual aircraft attitude |
US20130137076A1 (en) * | 2011-11-30 | 2013-05-30 | Kathryn Stone Perez | Head-mounted display based education and instruction |
US20130139082A1 (en) * | 2011-11-30 | 2013-05-30 | Google Inc. | Graphical Interface Having Adjustable Borders |
US9141194B1 (en) | 2012-01-04 | 2015-09-22 | Google Inc. | Magnetometer-based gesture sensing with a wearable device |
US20160011724A1 (en) * | 2012-01-06 | 2016-01-14 | Google Inc. | Hands-Free Selection Using a Ring-Based User-Interface |
US9230171B2 (en) | 2012-01-06 | 2016-01-05 | Google Inc. | Object outlining to initiate a visual search |
US9069382B1 (en) | 2012-01-06 | 2015-06-30 | Google Inc. | Using visual layers to aid in initiating a visual search |
US9829715B2 (en) | 2012-01-23 | 2017-11-28 | Nvidia Corporation | Eyewear device for transmitting signal and communication method thereof |
PL2734977T3 (en) * | 2012-02-22 | 2016-02-29 | Aselsan Elektronik Sanayi Ve Ticaret Anonim Sirketi | System and method for optimizing tracker system |
JP2013178639A (en) * | 2012-02-28 | 2013-09-09 | Seiko Epson Corp | Head mounted display device and image display system |
US9229534B2 (en) | 2012-02-28 | 2016-01-05 | Apple Inc. | Asymmetric mapping for tactile and non-tactile user interfaces |
CN104246682B (en) | 2012-03-26 | 2017-08-25 | 苹果公司 | Enhanced virtual touchpad and touch-screen |
US10021351B2 (en) | 2012-06-01 | 2018-07-10 | Ultradent Products, Inc. | Stereoscopic video imaging |
US9116666B2 (en) | 2012-06-01 | 2015-08-25 | Microsoft Technology Licensing, Llc | Gesture based region identification for holograms |
WO2014007955A1 (en) * | 2012-06-12 | 2014-01-09 | University Of Central Florida Research Foundation, Inc. | Systems and methods of camera-based body-motion tracking |
JP2015528713A (en) | 2012-06-21 | 2015-10-01 | グローバス メディカル インコーポレイティッド | Surgical robot platform |
US12004905B2 (en) | 2012-06-21 | 2024-06-11 | Globus Medical, Inc. | Medical imaging systems using robotic actuators and related methods |
US10350013B2 (en) | 2012-06-21 | 2019-07-16 | Globus Medical, Inc. | Surgical tool systems and methods |
US11786324B2 (en) | 2012-06-21 | 2023-10-17 | Globus Medical, Inc. | Surgical robotic automation with tracking markers |
US11298196B2 (en) | 2012-06-21 | 2022-04-12 | Globus Medical Inc. | Surgical robotic automation with tracking markers and controlled tool advancement |
US10231791B2 (en) | 2012-06-21 | 2019-03-19 | Globus Medical, Inc. | Infrared signal based position recognition system for use with a robot-assisted surgery |
US10646280B2 (en) | 2012-06-21 | 2020-05-12 | Globus Medical, Inc. | System and method for surgical tool insertion using multiaxis force and moment feedback |
US11116576B2 (en) | 2012-06-21 | 2021-09-14 | Globus Medical Inc. | Dynamic reference arrays and methods of use |
US10842461B2 (en) | 2012-06-21 | 2020-11-24 | Globus Medical, Inc. | Systems and methods of checking registrations for surgical systems |
US10758315B2 (en) | 2012-06-21 | 2020-09-01 | Globus Medical Inc. | Method and system for improving 2D-3D registration convergence |
US11395706B2 (en) | 2012-06-21 | 2022-07-26 | Globus Medical Inc. | Surgical robot platform |
US11317971B2 (en) | 2012-06-21 | 2022-05-03 | Globus Medical, Inc. | Systems and methods related to robotic guidance in surgery |
US11974822B2 (en) | 2012-06-21 | 2024-05-07 | Globus Medical Inc. | Method for a surveillance marker in robotic-assisted surgery |
US11857149B2 (en) | 2012-06-21 | 2024-01-02 | Globus Medical, Inc. | Surgical robotic systems with target trajectory deviation monitoring and related methods |
US11857266B2 (en) | 2012-06-21 | 2024-01-02 | Globus Medical, Inc. | System for a surveillance marker in robotic-assisted surgery |
US11045267B2 (en) | 2012-06-21 | 2021-06-29 | Globus Medical, Inc. | Surgical robotic automation with tracking markers |
US11793570B2 (en) | 2012-06-21 | 2023-10-24 | Globus Medical Inc. | Surgical robotic automation with tracking markers |
US11589771B2 (en) | 2012-06-21 | 2023-02-28 | Globus Medical Inc. | Method for recording probe movement and determining an extent of matter removed |
US10799298B2 (en) | 2012-06-21 | 2020-10-13 | Globus Medical Inc. | Robotic fluoroscopic navigation |
US10624710B2 (en) | 2012-06-21 | 2020-04-21 | Globus Medical, Inc. | System and method for measuring depth of instrumentation |
US11896446B2 (en) | 2012-06-21 | 2024-02-13 | Globus Medical, Inc | Surgical robotic automation with tracking markers |
US10874466B2 (en) | 2012-06-21 | 2020-12-29 | Globus Medical, Inc. | System and method for surgical tool insertion using multiaxis force and moment feedback |
US11399900B2 (en) | 2012-06-21 | 2022-08-02 | Globus Medical, Inc. | Robotic systems providing co-registration using natural fiducials and related methods |
US11864745B2 (en) | 2012-06-21 | 2024-01-09 | Globus Medical, Inc. | Surgical robotic system with retractor |
US11607149B2 (en) | 2012-06-21 | 2023-03-21 | Globus Medical Inc. | Surgical tool systems and method |
US11864839B2 (en) | 2012-06-21 | 2024-01-09 | Globus Medical Inc. | Methods of adjusting a virtual implant and related surgical navigation systems |
US10136954B2 (en) | 2012-06-21 | 2018-11-27 | Globus Medical, Inc. | Surgical tool systems and method |
US11963755B2 (en) | 2012-06-21 | 2024-04-23 | Globus Medical Inc. | Apparatus for recording probe movement |
US11253327B2 (en) | 2012-06-21 | 2022-02-22 | Globus Medical, Inc. | Systems and methods for automatically changing an end-effector on a surgical robot |
US8953154B2 (en) * | 2012-07-26 | 2015-02-10 | Vivonics, Inc. | Orientation tracking system and method |
US9578224B2 (en) | 2012-09-10 | 2017-02-21 | Nvidia Corporation | System and method for enhanced monoimaging |
US9268136B1 (en) | 2012-09-28 | 2016-02-23 | Google Inc. | Use of comparative sensor data to determine orientation of head relative to body |
DE102012219814A1 (en) | 2012-10-30 | 2014-04-30 | Bayerische Motoren Werke Aktiengesellschaft | Providing an operator input using a head-mounted display |
JP6066676B2 (en) * | 2012-11-06 | 2017-01-25 | 株式会社ソニー・インタラクティブエンタテインメント | Head mounted display and video presentation system |
US20140152558A1 (en) * | 2012-11-30 | 2014-06-05 | Tom Salter | Direct hologram manipulation using imu |
US9704350B1 (en) | 2013-03-14 | 2017-07-11 | Harmonix Music Systems, Inc. | Musical combat game |
US11156464B2 (en) | 2013-03-14 | 2021-10-26 | Trx Systems, Inc. | Crowd sourced mapping with robust structural features |
US11268818B2 (en) | 2013-03-14 | 2022-03-08 | Trx Systems, Inc. | Crowd sourced mapping with robust structural features |
WO2014143776A2 (en) | 2013-03-15 | 2014-09-18 | Bodhi Technology Ventures Llc | Providing remote interactions with host device using a wireless device |
JP6108926B2 (en) * | 2013-04-15 | 2017-04-05 | オリンパス株式会社 | Wearable device, program, and display control method for wearable device |
JP6308214B2 (en) * | 2013-05-15 | 2018-04-11 | ソニー株式会社 | Display control device, display control method, and recording medium |
US9733716B2 (en) | 2013-06-09 | 2017-08-15 | Apple Inc. | Proxy gesture recognizer |
US10194860B2 (en) | 2013-09-11 | 2019-02-05 | Industrial Technology Research Institute | Virtual image display system |
TWI518368B (en) * | 2013-09-11 | 2016-01-21 | 財團法人工業技術研究院 | Virtual image display apparatus |
JP6337433B2 (en) * | 2013-09-13 | 2018-06-06 | セイコーエプソン株式会社 | Head-mounted display device and method for controlling head-mounted display device |
US9283048B2 (en) | 2013-10-04 | 2016-03-15 | KB Medical SA | Apparatus and systems for precise guidance of surgical tools |
US9582516B2 (en) | 2013-10-17 | 2017-02-28 | Nant Holdings Ip, Llc | Wide area augmented reality location-based services |
DE102013019574A1 (en) | 2013-11-22 | 2015-05-28 | Audi Ag | Method for operating electronic data glasses and electronic data glasses |
US9530057B2 (en) * | 2013-11-26 | 2016-12-27 | Honeywell International Inc. | Maintenance assistant system |
US20150199106A1 (en) * | 2014-01-14 | 2015-07-16 | Caterpillar Inc. | Augmented Reality Display System |
US9241771B2 (en) | 2014-01-15 | 2016-01-26 | KB Medical SA | Notched apparatus for guidance of an insertable instrument along an axis during spinal surgery |
US9939934B2 (en) | 2014-01-17 | 2018-04-10 | Osterhout Group, Inc. | External user interface for head worn computing |
US10254856B2 (en) | 2014-01-17 | 2019-04-09 | Osterhout Group, Inc. | External user interface for head worn computing |
US9810906B2 (en) | 2014-06-17 | 2017-11-07 | Osterhout Group, Inc. | External user interface for head worn computing |
US10935788B2 (en) | 2014-01-24 | 2021-03-02 | Nvidia Corporation | Hybrid virtual 3D rendering approach to stereovision |
US10293205B2 (en) * | 2014-01-27 | 2019-05-21 | The Regents Of The University Of Michigan | IMU system for assessing head and torso orientation during physical motion |
US10039605B2 (en) | 2014-02-11 | 2018-08-07 | Globus Medical, Inc. | Sterile handle for controlling a robotic surgical system from a sterile field |
US10007329B1 (en) * | 2014-02-11 | 2018-06-26 | Leap Motion, Inc. | Drift cancelation for portable object detection and tracking |
CN104915979A (en) * | 2014-03-10 | 2015-09-16 | 苏州天魂网络科技有限公司 | System capable of realizing immersive virtual reality across mobile platforms |
CN106233227B (en) | 2014-03-14 | 2020-04-28 | 索尼互动娱乐股份有限公司 | Game device with volume sensing |
US9684369B2 (en) | 2014-04-08 | 2017-06-20 | Eon Reality, Inc. | Interactive virtual reality systems and methods |
US9542011B2 (en) | 2014-04-08 | 2017-01-10 | Eon Reality, Inc. | Interactive virtual reality systems and methods |
EP3134022B1 (en) | 2014-04-24 | 2018-01-10 | KB Medical SA | Surgical instrument holder for use with a robotic surgical system |
EP3157446B1 (en) | 2014-06-19 | 2018-08-15 | KB Medical SA | Systems for performing minimally invasive surgery |
FR3023020A1 (en) * | 2014-06-30 | 2016-01-01 | Orange | METHOD FOR MANAGING AN INTERFACE |
US10357257B2 (en) | 2014-07-14 | 2019-07-23 | KB Medical SA | Anti-skid surgical instrument for use in preparing holes in bone tissue |
US10765438B2 (en) | 2014-07-14 | 2020-09-08 | KB Medical SA | Anti-skid surgical instrument for use in preparing holes in bone tissue |
US10101793B2 (en) | 2014-07-18 | 2018-10-16 | Apple Inc. | Raise gesture detection in a device |
US10311638B2 (en) | 2014-07-25 | 2019-06-04 | Microsoft Technology Licensing, Llc | Anti-trip when immersed in a virtual reality environment |
US9766460B2 (en) | 2014-07-25 | 2017-09-19 | Microsoft Technology Licensing, Llc | Ground plane adjustment in a virtual reality environment |
US9858720B2 (en) * | 2014-07-25 | 2018-01-02 | Microsoft Technology Licensing, Llc | Three-dimensional mixed-reality viewport |
US10416760B2 (en) | 2014-07-25 | 2019-09-17 | Microsoft Technology Licensing, Llc | Gaze-based object placement within a virtual reality environment |
US10451875B2 (en) | 2014-07-25 | 2019-10-22 | Microsoft Technology Licensing, Llc | Smart transparency for virtual objects |
US20160061581A1 (en) * | 2014-08-26 | 2016-03-03 | Lusee, Llc | Scale estimating method using smart device |
US9508195B2 (en) * | 2014-09-03 | 2016-11-29 | Microsoft Technology Licensing, Llc | Management of content in a 3D holographic environment |
WO2016087539A2 (en) | 2014-12-02 | 2016-06-09 | KB Medical SA | Robot assisted volume removal during surgery |
US20160170482A1 (en) * | 2014-12-15 | 2016-06-16 | Seiko Epson Corporation | Display apparatus, and control method for display apparatus |
US9632657B2 (en) * | 2014-12-28 | 2017-04-25 | Sap Se | Auxiliary input device |
US10073516B2 (en) * | 2014-12-29 | 2018-09-11 | Sony Interactive Entertainment Inc. | Methods and systems for user interaction within virtual reality scene using head mounted display |
US10013808B2 (en) | 2015-02-03 | 2018-07-03 | Globus Medical, Inc. | Surgeon head-mounted display apparatuses |
US10555782B2 (en) | 2015-02-18 | 2020-02-11 | Globus Medical, Inc. | Systems and methods for performing minimally invasive spinal surgery with a robotic surgical system using a percutaneous technique |
US10111620B2 (en) * | 2015-02-27 | 2018-10-30 | Microsoft Technology Licensing, Llc | Enhanced motion tracking using transportable inertial sensors to determine that a frame of reference is established |
US10444018B2 (en) | 2015-02-27 | 2019-10-15 | Microsoft Technology Licensing, Llc | Computer-implemented method to test the sensitivity of a sensor for detecting movement of a tracking device within an established frame of reference of a moving platform |
KR102335910B1 (en) | 2015-03-01 | 2021-12-06 | 삼성전자주식회사 | Apparatus and method for controlling power |
GB2536650A (en) | 2015-03-24 | 2016-09-28 | Augmedics Ltd | Method and system for combining video-based and optic-based augmented reality in a near eye display |
US10114127B2 (en) * | 2015-05-11 | 2018-10-30 | The United States Of America, As Represented By The Secretary Of The Navy | Augmented reality visualization system |
US20170003738A1 (en) * | 2015-06-15 | 2017-01-05 | Survios, Inc. | Systems and methods for immersive physical interaction with a virtual environment |
US10139966B2 (en) | 2015-07-22 | 2018-11-27 | Osterhout Group, Inc. | External user interface for head worn computing |
US11003246B2 (en) | 2015-07-22 | 2021-05-11 | Mentor Acquisition One, Llc | External user interface for head worn computing |
WO2017014671A1 (en) * | 2015-07-20 | 2017-01-26 | Андрей Юрьевич ЗЕЛИНСКИЙ | Virtual reality driving simulator with added real objects |
US10058394B2 (en) | 2015-07-31 | 2018-08-28 | Globus Medical, Inc. | Robot arm and methods of use |
US10646298B2 (en) | 2015-07-31 | 2020-05-12 | Globus Medical, Inc. | Robot arm and methods of use |
TWI736542B (en) * | 2015-08-06 | 2021-08-21 | 日商新力股份有限公司 | Information processing device, data distribution server, information processing method, and non-temporary computer-readable recording medium |
US10080615B2 (en) | 2015-08-12 | 2018-09-25 | Globus Medical, Inc. | Devices and methods for temporary mounting of parts to bone |
EP3344179B1 (en) | 2015-08-31 | 2021-06-30 | KB Medical SA | Robotic surgical systems |
JP6576177B2 (en) * | 2015-09-09 | 2019-09-18 | キヤノン株式会社 | Information processing apparatus, information processing apparatus control method, and program |
US10034716B2 (en) | 2015-09-14 | 2018-07-31 | Globus Medical, Inc. | Surgical robotic systems and methods thereof |
CN105183170B (en) * | 2015-09-22 | 2018-09-25 | 京东方科技集团股份有限公司 | Wear-type wearable device and its information processing method, device |
WO2017053780A1 (en) | 2015-09-23 | 2017-03-30 | Iwear Holdings Corp. | Sending messages wirelessly from a garment |
GB2542609A (en) * | 2015-09-25 | 2017-03-29 | Nokia Technologies Oy | Differential headtracking apparatus |
US9771092B2 (en) | 2015-10-13 | 2017-09-26 | Globus Medical, Inc. | Stabilizer wheel assembly and methods of use |
US10338688B2 (en) | 2015-12-24 | 2019-07-02 | Samsung Electronics Co., Ltd. | Electronic device and method of controlling the same |
US11351472B2 (en) * | 2016-01-19 | 2022-06-07 | Disney Enterprises, Inc. | Systems and methods for using a gyroscope to change the resistance of moving a virtual weapon |
US10117632B2 (en) | 2016-02-03 | 2018-11-06 | Globus Medical, Inc. | Portable medical imaging system with beam scanning collimator |
US10448910B2 (en) | 2016-02-03 | 2019-10-22 | Globus Medical, Inc. | Portable medical imaging system |
US10842453B2 (en) | 2016-02-03 | 2020-11-24 | Globus Medical, Inc. | Portable medical imaging system |
US11883217B2 (en) | 2016-02-03 | 2024-01-30 | Globus Medical, Inc. | Portable medical imaging system and method |
US11058378B2 (en) | 2016-02-03 | 2021-07-13 | Globus Medical, Inc. | Portable medical imaging system |
US11663783B2 (en) | 2016-02-10 | 2023-05-30 | Disney Enterprises, Inc. | Systems and methods for using augmented reality with the internet of things |
US9874931B1 (en) | 2016-02-22 | 2018-01-23 | Rockwell Collins, Inc. | Head-tracking system and method |
US9906981B2 (en) | 2016-02-25 | 2018-02-27 | Nvidia Corporation | Method and system for dynamic regulation and control of Wi-Fi scans |
US10866119B2 (en) | 2016-03-14 | 2020-12-15 | Globus Medical, Inc. | Metal detector for detecting insertion of a surgical device into a hollow tube |
US10976809B2 (en) * | 2016-03-14 | 2021-04-13 | Htc Corporation | Interaction method for virtual reality |
US10824253B2 (en) | 2016-05-09 | 2020-11-03 | Mentor Acquisition One, Llc | User interface systems for head-worn computers |
US10684478B2 (en) | 2016-05-09 | 2020-06-16 | Mentor Acquisition One, Llc | User interface systems for head-worn computers |
US10466491B2 (en) | 2016-06-01 | 2019-11-05 | Mentor Acquisition One, Llc | Modular systems for head-worn computers |
EP3241518A3 (en) | 2016-04-11 | 2018-01-24 | Globus Medical, Inc | Surgical tool systems and methods |
KR20170126295A (en) * | 2016-05-09 | 2017-11-17 | 엘지전자 주식회사 | Head mounted display device and method for controlling the same |
US10198874B2 (en) | 2016-05-13 | 2019-02-05 | Google Llc | Methods and apparatus to align components in virtual reality environments |
US10146335B2 (en) * | 2016-06-09 | 2018-12-04 | Microsoft Technology Licensing, Llc | Modular extension of inertial controller for six DOF mixed reality input |
US10146334B2 (en) | 2016-06-09 | 2018-12-04 | Microsoft Technology Licensing, Llc | Passive optical and inertial tracking in slim form-factor |
EP3430490B1 (en) | 2016-07-22 | 2021-06-23 | Hewlett-Packard Development Company, L.P. | Outer cases for computing devices |
CN109478339A (en) * | 2016-07-29 | 2019-03-15 | 三菱电机株式会社 | Display device, display control unit and display control method |
US10345925B2 (en) | 2016-08-03 | 2019-07-09 | Google Llc | Methods and systems for determining positional data for three-dimensional interactions inside virtual reality environments |
US9891705B1 (en) | 2016-08-25 | 2018-02-13 | Rockwell Collins, Inc. | Automatic boresighting of head-worn display |
US11039893B2 (en) | 2016-10-21 | 2021-06-22 | Globus Medical, Inc. | Robotic surgical systems |
GB2555838A (en) | 2016-11-11 | 2018-05-16 | Sony Corp | An apparatus, computer program and method |
US10545219B2 (en) * | 2016-11-23 | 2020-01-28 | Chirp Microsystems | Three dimensional object-localization and tracking using ultrasonic pulses |
WO2018095804A1 (en) * | 2016-11-25 | 2018-05-31 | Sensoryx AG | Wearable motion tracking system |
EP3552077B1 (en) * | 2016-12-06 | 2021-04-28 | Vuelosophy Inc. | Systems and methods for tracking motion and gesture of heads and eyes |
EP3333590A1 (en) * | 2016-12-12 | 2018-06-13 | Nxp B.V. | Apparatus and associated methods |
US10484623B2 (en) | 2016-12-20 | 2019-11-19 | Microsoft Technology Licensing, Llc | Sensor with alternating visible and infrared sensitive pixels |
US10942252B2 (en) * | 2016-12-26 | 2021-03-09 | Htc Corporation | Tracking system and tracking method |
US10895628B2 (en) * | 2016-12-29 | 2021-01-19 | Htc Corporation | Tracking system, tracking device and tracking method |
CN106775258A (en) * | 2017-01-04 | 2017-05-31 | 虹软(杭州)多媒体信息技术有限公司 | The method and apparatus that virtual reality is interacted are realized using gesture control |
EP3351202B1 (en) | 2017-01-18 | 2021-09-08 | KB Medical SA | Universal instrument guide for robotic surgical systems |
JP7233841B2 (en) | 2017-01-18 | 2023-03-07 | ケービー メディカル エスアー | Robotic Navigation for Robotic Surgical Systems |
JP2018114280A (en) | 2017-01-18 | 2018-07-26 | ケービー メディカル エスアー | Universal instrument guide for robotic surgical system, surgical instrument system, and method of using them |
US20180253159A1 (en) * | 2017-03-01 | 2018-09-06 | Osterhout Group, Inc. | User interface systems for head-worn computers |
US11071594B2 (en) | 2017-03-16 | 2021-07-27 | KB Medical SA | Robotic navigation of robotic surgical systems |
US10747301B2 (en) * | 2017-03-28 | 2020-08-18 | Magic Leap, Inc. | Augmented reality system with spatialized audio tied to user manipulated virtual object |
US10736409B2 (en) * | 2017-04-06 | 2020-08-11 | Affl Associates, Llc | Flag football system |
AU2018258679B2 (en) | 2017-04-27 | 2022-03-31 | Magic Leap, Inc. | Light-emitting user input device |
US10444865B2 (en) * | 2017-05-01 | 2019-10-15 | Google Llc | Tracking of position and orientation of objects in virtual reality systems |
TWI646449B (en) * | 2017-05-12 | 2019-01-01 | 華碩電腦股份有限公司 | Three-dimensional positioning system and method thereof |
CN109247945A (en) * | 2017-07-12 | 2019-01-22 | 松下知识产权经营株式会社 | measuring device |
US11135015B2 (en) | 2017-07-21 | 2021-10-05 | Globus Medical, Inc. | Robot surgical platform |
US10216265B1 (en) | 2017-08-07 | 2019-02-26 | Rockwell Collins, Inc. | System and method for hybrid optical/inertial headtracking via numerically stable Kalman filter |
US10152141B1 (en) * | 2017-08-18 | 2018-12-11 | Osterhout Group, Inc. | Controller movement tracking with light emitters |
DE102017120741A1 (en) * | 2017-09-08 | 2019-03-14 | Tim Millhoff | Device, system and method for decoupling a VR system from infrastructure and localized hardware |
US10871570B2 (en) * | 2017-09-14 | 2020-12-22 | Everysight Ltd. | System and method for position and orientation tracking |
US11794338B2 (en) | 2017-11-09 | 2023-10-24 | Globus Medical Inc. | Robotic rod benders and related mechanical and motor housings |
JP6778242B2 (en) | 2017-11-09 | 2020-10-28 | グローバス メディカル インコーポレイティッド | Surgical robot systems for bending surgical rods, and related methods and equipment |
US11357548B2 (en) | 2017-11-09 | 2022-06-14 | Globus Medical, Inc. | Robotic rod benders and related mechanical and motor housings |
US11134862B2 (en) | 2017-11-10 | 2021-10-05 | Globus Medical, Inc. | Methods of selecting surgical implants and related devices |
EP3639523A4 (en) | 2017-11-14 | 2020-04-22 | Samsung Electronics Co., Ltd. | Method and apparatus for managing a wide view content in a virtual reality environment |
US10514545B2 (en) * | 2017-12-08 | 2019-12-24 | Facebook Technologies, Llc | Selective tracking of a head-mounted display |
US20190254753A1 (en) | 2018-02-19 | 2019-08-22 | Globus Medical, Inc. | Augmented reality navigation systems for use with robotic surgical systems and methods of their use |
AU2019231697B2 (en) | 2018-03-07 | 2020-01-30 | Magic Leap, Inc. | Visual tracking of peripheral devices |
US10528133B2 (en) * | 2018-03-13 | 2020-01-07 | Facebook Technologies, Llc | Bracelet in a distributed artificial reality system |
US10572002B2 (en) | 2018-03-13 | 2020-02-25 | Facebook Technologies, Llc | Distributed artificial reality system with contextualized hand tracking |
JP6952868B2 (en) * | 2018-03-14 | 2021-10-27 | 三菱電機株式会社 | Information processing equipment, information processing systems, and information processing programs |
US11266530B2 (en) * | 2018-03-22 | 2022-03-08 | Jennifer Hendrix | Route guidance and obstacle avoidance system |
US10573023B2 (en) | 2018-04-09 | 2020-02-25 | Globus Medical, Inc. | Predictive visualization of medical imaging scanner component movement |
US11226683B2 (en) * | 2018-04-20 | 2022-01-18 | Hewlett-Packard Development Company, L.P. | Tracking stylus in a virtual reality system |
EP3787543A4 (en) | 2018-05-02 | 2022-01-19 | Augmedics Ltd. | Registration of a fiducial marker for an augmented reality system |
CN108762488A (en) * | 2018-05-04 | 2018-11-06 | 梦卓科技(深圳)有限公司 | A kind of single base station portable V R system based on wireless human body motion capture and optical alignment |
US11204491B2 (en) | 2018-05-30 | 2021-12-21 | Magic Leap, Inc. | Compact variable focus configurations |
WO2019229698A1 (en) * | 2018-05-31 | 2019-12-05 | Purple Tambourine Limited | Interacting with a virtual environment using a pointing controller |
CN112513785A (en) | 2018-06-08 | 2021-03-16 | 奇跃公司 | Augmented reality viewer with automatic surface selection placement and content orientation placement |
US11150746B2 (en) * | 2018-06-28 | 2021-10-19 | Google Llc | Wearable electronic devices having user interface mirroring based on device position |
US20210357017A1 (en) * | 2018-07-10 | 2021-11-18 | Hewlett-Packard Development Company, L.P. | Motion matching in virtual environments |
USD930614S1 (en) | 2018-07-24 | 2021-09-14 | Magic Leap, Inc. | Totem controller having an illumination region |
US10671163B2 (en) * | 2018-07-24 | 2020-06-02 | Microsoft Technology Licensing, Llc | Refining virtual mesh models through physical contacts |
USD918176S1 (en) | 2018-07-24 | 2021-05-04 | Magic Leap, Inc. | Totem controller having an illumination region |
US10817047B2 (en) | 2018-09-19 | 2020-10-27 | XRSpace CO., LTD. | Tracking system and tacking method using the same |
US10974132B2 (en) | 2018-10-02 | 2021-04-13 | Disney Enterprises, Inc. | Systems and methods to provide a shared interactive experience across multiple presentation devices based on detection of one or more extraterrestrial bodies |
WO2020084625A1 (en) * | 2018-10-25 | 2020-04-30 | Beyeonics Surgical Ltd. | Ui for head mounted display system |
US11337742B2 (en) | 2018-11-05 | 2022-05-24 | Globus Medical Inc | Compliant orthopedic driver |
US11278360B2 (en) | 2018-11-16 | 2022-03-22 | Globus Medical, Inc. | End-effectors for surgical robotic systems having sealed optical components |
US10939977B2 (en) | 2018-11-26 | 2021-03-09 | Augmedics Ltd. | Positioning marker |
US11766296B2 (en) | 2018-11-26 | 2023-09-26 | Augmedics Ltd. | Tracking system for image-guided surgery |
US10445899B1 (en) | 2018-11-26 | 2019-10-15 | Capital One Services, Llc | System and method for recalibrating an augmented reality experience using physical markers |
US11602402B2 (en) | 2018-12-04 | 2023-03-14 | Globus Medical, Inc. | Drill guide fixtures, cranial insertion fixtures, and related methods and robotic systems |
US11744655B2 (en) | 2018-12-04 | 2023-09-05 | Globus Medical, Inc. | Drill guide fixtures, cranial insertion fixtures, and related methods and robotic systems |
US10390581B1 (en) * | 2019-01-29 | 2019-08-27 | Rockwell Collins, Inc. | Radio frequency head tracker |
WO2020176779A1 (en) * | 2019-02-28 | 2020-09-03 | Magic Leap, Inc. | Method and system utilizing phased array beamforming for six degree of freedom tracking for an emitter in augmented reality systems |
US11918313B2 (en) | 2019-03-15 | 2024-03-05 | Globus Medical Inc. | Active end effectors for surgical robots |
US11382549B2 (en) | 2019-03-22 | 2022-07-12 | Globus Medical, Inc. | System for neuronavigation registration and robotic trajectory guidance, and related methods and devices |
US20200297357A1 (en) | 2019-03-22 | 2020-09-24 | Globus Medical, Inc. | System for neuronavigation registration and robotic trajectory guidance, robotic surgery, and related methods and devices |
US11571265B2 (en) | 2019-03-22 | 2023-02-07 | Globus Medical Inc. | System for neuronavigation registration and robotic trajectory guidance, robotic surgery, and related methods and devices |
US11806084B2 (en) | 2019-03-22 | 2023-11-07 | Globus Medical, Inc. | System for neuronavigation registration and robotic trajectory guidance, and related methods and devices |
US11419616B2 (en) | 2019-03-22 | 2022-08-23 | Globus Medical, Inc. | System for neuronavigation registration and robotic trajectory guidance, robotic surgery, and related methods and devices |
US11317978B2 (en) | 2019-03-22 | 2022-05-03 | Globus Medical, Inc. | System for neuronavigation registration and robotic trajectory guidance, robotic surgery, and related methods and devices |
US11014008B2 (en) | 2019-03-27 | 2021-05-25 | Disney Enterprises, Inc. | Systems and methods for game profile development based on virtual and/or real activities |
US10816813B1 (en) * | 2019-04-05 | 2020-10-27 | Disney Enterprises, Inc. | Systems and methods for enhancing accuracy of spatial location and rotational orientation determination of wearable head-mounted display device |
US10916061B2 (en) | 2019-04-24 | 2021-02-09 | Disney Enterprises, Inc. | Systems and methods to synchronize real-world motion of physical objects with presentation of virtual content |
US11287505B2 (en) | 2019-05-13 | 2022-03-29 | Cast Group Of Companies Inc. | Electronic tracking device and related system |
US11045179B2 (en) | 2019-05-20 | 2021-06-29 | Global Medical Inc | Robot-mounted retractor system |
US11486961B2 (en) * | 2019-06-14 | 2022-11-01 | Chirp Microsystems | Object-localization and tracking using ultrasonic pulses with reflection rejection |
JP7429231B2 (en) * | 2019-06-21 | 2024-02-07 | マクセル株式会社 | head mounted display device |
US11628023B2 (en) | 2019-07-10 | 2023-04-18 | Globus Medical, Inc. | Robotic navigational system for interbody implants |
US11980506B2 (en) | 2019-07-29 | 2024-05-14 | Augmedics Ltd. | Fiducial marker |
US11571171B2 (en) | 2019-09-24 | 2023-02-07 | Globus Medical, Inc. | Compound curve cable chain |
US11864857B2 (en) | 2019-09-27 | 2024-01-09 | Globus Medical, Inc. | Surgical robot with passive end effector |
US11426178B2 (en) | 2019-09-27 | 2022-08-30 | Globus Medical Inc. | Systems and methods for navigating a pin guide driver |
US11890066B2 (en) | 2019-09-30 | 2024-02-06 | Globus Medical, Inc | Surgical robot with passive end effector |
US11510684B2 (en) | 2019-10-14 | 2022-11-29 | Globus Medical, Inc. | Rotary motion passive end effector for surgical robots in orthopedic surgeries |
WO2021086304A1 (en) * | 2019-10-28 | 2021-05-06 | Hewlett-Packard Development Company, L.P. | Provision of feedback to an actuating object |
US11416065B1 (en) * | 2019-11-08 | 2022-08-16 | Meta Platforms Technologies, Llc | Synthesizing haptic and sonic feedback for textured materials in interactive virtual environments |
US11821996B1 (en) * | 2019-11-12 | 2023-11-21 | Lockheed Martin Corporation | Outdoor entity and weapon tracking and orientation |
US11599257B2 (en) * | 2019-11-12 | 2023-03-07 | Cast Group Of Companies Inc. | Electronic tracking device and charging apparatus |
US11992373B2 (en) | 2019-12-10 | 2024-05-28 | Globus Medical, Inc | Augmented reality headset with varied opacity for navigated robotic surgery |
US12064189B2 (en) | 2019-12-13 | 2024-08-20 | Globus Medical, Inc. | Navigated instrument for use in robotic guided surgery |
US11382712B2 (en) | 2019-12-22 | 2022-07-12 | Augmedics Ltd. | Mirroring in image guided surgery |
US11464581B2 (en) | 2020-01-28 | 2022-10-11 | Globus Medical, Inc. | Pose measurement chaining for extended reality surgical navigation in visible and near infrared spectrums |
US11382699B2 (en) | 2020-02-10 | 2022-07-12 | Globus Medical Inc. | Extended reality visualization of optical tool tracking volume for computer assisted navigation in surgery |
US11207150B2 (en) | 2020-02-19 | 2021-12-28 | Globus Medical, Inc. | Displaying a virtual model of a planned instrument attachment to ensure correct selection of physical instrument attachment |
US11253216B2 (en) | 2020-04-28 | 2022-02-22 | Globus Medical Inc. | Fixtures for fluoroscopic imaging systems and related navigation systems and methods |
US11153555B1 (en) | 2020-05-08 | 2021-10-19 | Globus Medical Inc. | Extended reality headset camera system for computer assisted navigation in surgery |
US11510750B2 (en) | 2020-05-08 | 2022-11-29 | Globus Medical, Inc. | Leveraging two-dimensional digital imaging and communication in medicine imagery in three-dimensional extended reality applications |
US11382700B2 (en) | 2020-05-08 | 2022-07-12 | Globus Medical Inc. | Extended reality headset tool tracking and control |
US11360552B1 (en) | 2020-06-02 | 2022-06-14 | Rockwell Collins, Inc. | High assurance headtracking via structured light projection for head worn display (HWD) |
US11320650B1 (en) | 2020-06-02 | 2022-05-03 | Rockwell Collins, Inc. | High assurance head tracking system incorporating ground truth fiducials |
US11317973B2 (en) | 2020-06-09 | 2022-05-03 | Globus Medical, Inc. | Camera tracking bar for computer assisted navigation during surgery |
US12070276B2 (en) | 2020-06-09 | 2024-08-27 | Globus Medical Inc. | Surgical object tracking in visible light via fiducial seeding and synthetic image registration |
US11389252B2 (en) | 2020-06-15 | 2022-07-19 | Augmedics Ltd. | Rotating marker for image guided surgery |
US11382713B2 (en) | 2020-06-16 | 2022-07-12 | Globus Medical, Inc. | Navigated surgical system with eye to XR headset display calibration |
US11877807B2 (en) | 2020-07-10 | 2024-01-23 | Globus Medical, Inc | Instruments for navigated orthopedic surgeries |
US11793588B2 (en) | 2020-07-23 | 2023-10-24 | Globus Medical, Inc. | Sterile draping of robotic arms |
US11737831B2 (en) | 2020-09-02 | 2023-08-29 | Globus Medical Inc. | Surgical object tracking template generation for computer assisted navigation during surgical procedure |
US11523785B2 (en) | 2020-09-24 | 2022-12-13 | Globus Medical, Inc. | Increased cone beam computed tomography volume length without requiring stitching or longitudinal C-arm movement |
US11911112B2 (en) | 2020-10-27 | 2024-02-27 | Globus Medical, Inc. | Robotic navigational system |
US11941814B2 (en) | 2020-11-04 | 2024-03-26 | Globus Medical Inc. | Auto segmentation using 2-D images taken during 3-D imaging spin |
US11717350B2 (en) | 2020-11-24 | 2023-08-08 | Globus Medical Inc. | Methods for robotic assistance and navigation in spinal surgery and related systems |
US11914762B2 (en) | 2020-12-28 | 2024-02-27 | Meta Platforms Technologies, Llc | Controller position tracking using inertial measurement units and machine learning |
US20220218431A1 (en) | 2021-01-08 | 2022-07-14 | Globus Medical, Inc. | System and method for ligament balancing with robotic assistance |
US11857273B2 (en) | 2021-07-06 | 2024-01-02 | Globus Medical, Inc. | Ultrasonic robotic surgical navigation |
US11896445B2 (en) | 2021-07-07 | 2024-02-13 | Augmedics Ltd. | Iliac pin and adapter |
US11439444B1 (en) | 2021-07-22 | 2022-09-13 | Globus Medical, Inc. | Screw tower and rod reduction tool |
US11918304B2 (en) | 2021-12-20 | 2024-03-05 | Globus Medical, Inc | Flat panel registration fixture and method of using same |
US11995228B2 (en) | 2022-01-11 | 2024-05-28 | Rockwell Collins, Inc. | Head tracking system with extended kalman filter combining absolute and relative navigation |
US12048493B2 (en) | 2022-03-31 | 2024-07-30 | Globus Medical, Inc. | Camera tracking system identifying phantom markers during computer assisted surgery navigation |
US20230360323A1 (en) * | 2022-05-09 | 2023-11-09 | SB22, Inc. | Systems and methods for the generating content overlays for virtual reality systems |
US11586286B1 (en) | 2022-05-18 | 2023-02-21 | Bank Of America Corporation | System and method for navigating on an augmented reality display |
US11720380B1 (en) | 2022-05-18 | 2023-08-08 | Bank Of America Corporation | System and method for updating augmented reality navigation instructions based on a detected error |
US11822736B1 (en) * | 2022-05-18 | 2023-11-21 | Google Llc | Passive-accessory mediated gesture interaction with a head-mounted device |
WO2024057210A1 (en) | 2022-09-13 | 2024-03-21 | Augmedics Ltd. | Augmented reality eyewear for image-guided medical intervention |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4988981A (en) * | 1987-03-17 | 1991-01-29 | Vpl Research, Inc. | Computer data entry and manipulation apparatus and method |
US5526022A (en) * | 1993-01-06 | 1996-06-11 | Virtual I/O, Inc. | Sourceless orientation sensor |
US5645077A (en) * | 1994-06-16 | 1997-07-08 | Massachusetts Institute Of Technology | Inertial orientation tracker apparatus having automatic drift compensation for tracking human head and other similarly sized body |
US5812257A (en) * | 1990-11-29 | 1998-09-22 | Sun Microsystems, Inc. | Absolute position tracker |
US5854843A (en) * | 1995-06-07 | 1998-12-29 | The United States Of America As Represented By The Secretary Of The Air Force | Virtual navigator, and inertial angular measurement system |
US5991085A (en) * | 1995-04-21 | 1999-11-23 | I-O Display Systems Llc | Head-mounted personal visual display apparatus with image generator and holder |
US6176837B1 (en) * | 1998-04-17 | 2001-01-23 | Massachusetts Institute Of Technology | Motion tracking system |
US6474159B1 (en) * | 2000-04-21 | 2002-11-05 | Intersense, Inc. | Motion-tracking |
US20030158699A1 (en) * | 1998-12-09 | 2003-08-21 | Christopher P. Townsend | Orientation sensor |
US6757068B2 (en) * | 2000-01-28 | 2004-06-29 | Intersense, Inc. | Self-referenced tracking |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0788394B2 (en) * | 1987-04-22 | 1995-09-27 | 東燃株式会社 | Method for producing epoxy group-containing silane compound |
ATE214508T1 (en) * | 1990-11-30 | 2002-03-15 | Sun Microsystems Inc | COMPACT HEAD TRACKING SYSTEM FOR CHEAP VIRTUAL REALITY SYSTEM |
US5646077A (en) * | 1993-01-07 | 1997-07-08 | Unitika Ltd | Binder fiber and nonwoven fabrics using the fiber |
US5615132A (en) | 1994-01-21 | 1997-03-25 | Crossbow Technology, Inc. | Method and apparatus for determining position and orientation of a moveable object using accelerometers |
EP0852732A1 (en) | 1995-09-21 | 1998-07-15 | Omniplanar, Inc. | Method and apparatus for determining position and orientation |
TW395121B (en) * | 1996-02-26 | 2000-06-21 | Seiko Epson Corp | Personal wearing information display device and the display method using such device |
JP2001504605A (en) * | 1996-08-14 | 2001-04-03 | ラティポフ,ヌラフメド,ヌリスラモビチ | Method for tracking and displaying a user's location and orientation in space, method for presenting a virtual environment to a user, and systems for implementing these methods |
DE19830359A1 (en) | 1998-07-07 | 2000-01-20 | Helge Zwosta | Spatial position and movement determination of body and body parts for remote control of machine and instruments |
US6757066B2 (en) * | 2002-01-28 | 2004-06-29 | Zygo Corporation | Multiple degree of freedom interferometer |
-
2001
- 2001-01-26 AU AU2001233019A patent/AU2001233019A1/en not_active Abandoned
- 2001-01-26 WO PCT/US2001/002632 patent/WO2001056007A1/en active Search and Examination
- 2001-01-26 US US09/770,691 patent/US6757068B2/en not_active Expired - Lifetime
-
2004
- 2004-04-30 US US10/837,373 patent/US20040201857A1/en not_active Abandoned
-
2006
- 2006-08-10 US US11/463,776 patent/US7301648B2/en not_active Expired - Lifetime
Patent Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4988981A (en) * | 1987-03-17 | 1991-01-29 | Vpl Research, Inc. | Computer data entry and manipulation apparatus and method |
US4988981B1 (en) * | 1987-03-17 | 1999-05-18 | Vpl Newco Inc | Computer data entry and manipulation apparatus and method |
US5812257A (en) * | 1990-11-29 | 1998-09-22 | Sun Microsystems, Inc. | Absolute position tracker |
US5526022A (en) * | 1993-01-06 | 1996-06-11 | Virtual I/O, Inc. | Sourceless orientation sensor |
US5645077A (en) * | 1994-06-16 | 1997-07-08 | Massachusetts Institute Of Technology | Inertial orientation tracker apparatus having automatic drift compensation for tracking human head and other similarly sized body |
US5991085A (en) * | 1995-04-21 | 1999-11-23 | I-O Display Systems Llc | Head-mounted personal visual display apparatus with image generator and holder |
US5854843A (en) * | 1995-06-07 | 1998-12-29 | The United States Of America As Represented By The Secretary Of The Air Force | Virtual navigator, and inertial angular measurement system |
US6176837B1 (en) * | 1998-04-17 | 2001-01-23 | Massachusetts Institute Of Technology | Motion tracking system |
US20030158699A1 (en) * | 1998-12-09 | 2003-08-21 | Christopher P. Townsend | Orientation sensor |
US6757068B2 (en) * | 2000-01-28 | 2004-06-29 | Intersense, Inc. | Self-referenced tracking |
US6474159B1 (en) * | 2000-04-21 | 2002-11-05 | Intersense, Inc. | Motion-tracking |
Cited By (212)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050162409A1 (en) * | 2000-08-18 | 2005-07-28 | International Business Machines Corporation | Projector and camera arrangement with shared optics and optical marker for use with whiteboard systems |
US7355584B2 (en) * | 2000-08-18 | 2008-04-08 | International Business Machines Corporation | Projector and camera arrangement with shared optics and optical marker for use with whiteboard systems |
US20020154070A1 (en) * | 2001-03-13 | 2002-10-24 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and control program |
US7038699B2 (en) * | 2001-03-13 | 2006-05-02 | Canon Kabushiki Kaisha | Image processing apparatus and method with setting of prohibited region and generation of computer graphics data based on prohibited region and first or second position/orientation |
US20030156144A1 (en) * | 2002-02-18 | 2003-08-21 | Canon Kabushiki Kaisha | Information processing apparatus and method |
US7610558B2 (en) * | 2002-02-18 | 2009-10-27 | Canon Kabushiki Kaisha | Information processing apparatus and method |
US7063256B2 (en) * | 2003-03-04 | 2006-06-20 | United Parcel Service Of America | Item tracking and processing systems and methods |
US8888786B2 (en) | 2003-06-09 | 2014-11-18 | OrthAlign, Inc. | Surgical orientation device and method |
US11903597B2 (en) | 2003-06-09 | 2024-02-20 | OrthAlign, Inc. | Surgical orientation system and method |
US11179167B2 (en) | 2003-06-09 | 2021-11-23 | OrthAlign, Inc. | Surgical orientation system and method |
US8974467B2 (en) | 2003-06-09 | 2015-03-10 | OrthAlign, Inc. | Surgical orientation system and method |
US7650257B2 (en) | 2005-01-25 | 2010-01-19 | Drop Zone Inc. | Enhanced hang-timer for console simulation |
US20080275670A1 (en) * | 2005-01-25 | 2008-11-06 | Drop Zone Corporation | Hang timer for determining time of flight of an object |
US20060167649A1 (en) * | 2005-01-25 | 2006-07-27 | Alexander Jeffrey M | Enhanced hang-timer for console simulation |
US8108177B2 (en) * | 2005-01-25 | 2012-01-31 | Drop Zone Corp. | Hang timer for determining time of flight of an object |
US8224024B2 (en) * | 2005-10-04 | 2012-07-17 | InterSense, LLC | Tracking objects with markers |
US20070081695A1 (en) * | 2005-10-04 | 2007-04-12 | Eric Foxlin | Tracking objects with markers |
US9235064B2 (en) | 2005-10-07 | 2016-01-12 | Percept Technologies Inc. | Digital eyewear |
US9244293B2 (en) | 2005-10-07 | 2016-01-26 | Percept Technologies Inc. | Digital eyewear |
US9239473B2 (en) | 2005-10-07 | 2016-01-19 | Percept Technologies Inc. | Digital eyewear |
US11675216B2 (en) | 2005-10-07 | 2023-06-13 | Percept Technologies | Enhanced optical and perceptual digital eyewear |
US9658473B2 (en) * | 2005-10-07 | 2017-05-23 | Percept Technologies Inc | Enhanced optical and perceptual digital eyewear |
US20150126281A1 (en) * | 2005-10-07 | 2015-05-07 | Percept Technologies Inc. | Enhanced optical and perceptual digital eyewear |
US11428937B2 (en) | 2005-10-07 | 2022-08-30 | Percept Technologies | Enhanced optical and perceptual digital eyewear |
US11294203B2 (en) | 2005-10-07 | 2022-04-05 | Percept Technologies | Enhanced optical and perceptual digital eyewear |
US10795183B1 (en) * | 2005-10-07 | 2020-10-06 | Percept Technologies Inc | Enhanced optical and perceptual digital eyewear |
WO2007062377A3 (en) * | 2005-11-23 | 2008-01-03 | Drop Zone Corp | Enhanced hang-timer for console simulation |
WO2007062377A2 (en) * | 2005-11-23 | 2007-05-31 | Drop Zone Corp. | Enhanced hang-timer for console simulation |
US9052161B2 (en) | 2005-12-19 | 2015-06-09 | Raydon Corporation | Perspective tracking system |
US20070166669A1 (en) * | 2005-12-19 | 2007-07-19 | Raydon Corporation | Perspective tracking system |
US20090273542A1 (en) * | 2005-12-20 | 2009-11-05 | Kakuya Yamamoto | Content presentation apparatus, and content presentation method |
US11452914B2 (en) | 2006-01-09 | 2022-09-27 | Nike, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US7978081B2 (en) | 2006-01-09 | 2011-07-12 | Applied Technology Holdings, Inc. | Apparatus, systems, and methods for communicating biometric and biomechanical information |
US20100201500A1 (en) * | 2006-01-09 | 2010-08-12 | Harold Dan Stirling | Apparatus, systems, and methods for communicating biometric and biomechanical information |
US11653856B2 (en) | 2006-01-09 | 2023-05-23 | Nike, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US7821407B2 (en) | 2006-01-09 | 2010-10-26 | Applied Technology Holdings, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US7825815B2 (en) | 2006-01-09 | 2010-11-02 | Applied Technology Holdings, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US20100204616A1 (en) * | 2006-01-09 | 2010-08-12 | Applied Technology Holdings, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US11717185B2 (en) | 2006-01-09 | 2023-08-08 | Nike, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US9907997B2 (en) | 2006-01-09 | 2018-03-06 | Nike, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US7602301B1 (en) | 2006-01-09 | 2009-10-13 | Applied Technology Holdings, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US20100121228A1 (en) * | 2006-01-09 | 2010-05-13 | Applied Technology Holdings, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US20100121227A1 (en) * | 2006-01-09 | 2010-05-13 | Applied Technology Holdings, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US11399758B2 (en) | 2006-01-09 | 2022-08-02 | Nike, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US10675507B2 (en) | 2006-01-09 | 2020-06-09 | Nike, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US11819324B2 (en) | 2006-01-09 | 2023-11-21 | Nike, Inc. | Apparatus, systems, and methods for gathering and processing biometric and biomechanical data |
US20070209586A1 (en) * | 2006-03-10 | 2007-09-13 | Ebensberger Jason M | Virtual coatings application system |
US20070209585A1 (en) * | 2006-03-10 | 2007-09-13 | Ebensberger Jason M | Virtual coatings application system |
US7839417B2 (en) | 2006-03-10 | 2010-11-23 | University Of Northern Iowa Research Foundation | Virtual coatings application system |
US7839416B2 (en) | 2006-03-10 | 2010-11-23 | University Of Northern Iowa Research Foundation | Virtual coatings application system |
EP3738655A3 (en) * | 2006-05-04 | 2021-03-17 | Sony Interactive Entertainment LLC | Method and apparatus for use in determining lack of user activity, determining an activity level of a user, and/or adding a new player in relation to a system |
US7920071B2 (en) | 2006-05-26 | 2011-04-05 | Itt Manufacturing Enterprises, Inc. | Augmented reality-based system and method providing status and control of unmanned vehicles |
US20070273557A1 (en) * | 2006-05-26 | 2007-11-29 | Itt Manufacturing Enterprises,Inc. | Augmented reality-based system and method providing status and control of unmanned vehicles |
US20070273610A1 (en) * | 2006-05-26 | 2007-11-29 | Itt Manufacturing Enterprises, Inc. | System and method to display maintenance and operational instructions of an apparatus using augmented reality |
JP2009538487A (en) * | 2006-05-26 | 2009-11-05 | アイティーティー マニュファクチャリング エンタープライジーズ, インコーポレイテッド | System and method for displaying device maintenance and operation instructions using augmented reality |
US9323055B2 (en) * | 2006-05-26 | 2016-04-26 | Exelis, Inc. | System and method to display maintenance and operational instructions of an apparatus using augmented reality |
WO2007139676A3 (en) * | 2006-05-26 | 2008-10-30 | Itt Mfg Enterprises Inc | System and method to display maintenance and operational instructions of an apparatus using augmented reality |
US11116574B2 (en) | 2006-06-16 | 2021-09-14 | Board Of Regents Of The University Of Nebraska | Method and apparatus for computer aided surgery |
US11857265B2 (en) | 2006-06-16 | 2024-01-02 | Board Of Regents Of The University Of Nebraska | Method and apparatus for computer aided surgery |
US20080039868A1 (en) * | 2006-07-05 | 2008-02-14 | Aesculap Ag & Co. Kg | Calibration method and calibration device for a surgical referencing unit |
US7702477B2 (en) | 2006-07-05 | 2010-04-20 | Aesculap Ag | Calibration method and calibration device for a surgical referencing unit |
US20080124698A1 (en) * | 2006-11-28 | 2008-05-29 | Ebensberger Jason M | Virtual coatings application system with structured training and remote instructor capabilities |
US9324229B2 (en) | 2007-03-08 | 2016-04-26 | Exelis, Inc. | System and method to display maintenance and operational instructions of an apparatus using augmented reality |
US20080218331A1 (en) * | 2007-03-08 | 2008-09-11 | Itt Manufacturing Enterprises, Inc. | Augmented reality-based system and method to show the location of personnel and sensors inside occluded structures and provide increased situation awareness |
US20080266323A1 (en) * | 2007-04-25 | 2008-10-30 | Board Of Trustees Of Michigan State University | Augmented reality user interaction system |
US20080280676A1 (en) * | 2007-05-07 | 2008-11-13 | Samsung Electronics Co. Ltd. | Wireless gaming method and wireless gaming-enabled mobile terminal |
US8506404B2 (en) * | 2007-05-07 | 2013-08-13 | Samsung Electronics Co., Ltd. | Wireless gaming method and wireless gaming-enabled mobile terminal |
US20090202975A1 (en) * | 2008-02-11 | 2009-08-13 | Michael Bolick | Virtual blasting system for removal of coating and/or rust from a virtual surface |
US7817162B2 (en) | 2008-02-11 | 2010-10-19 | University Of Northern Iowa Research Foundation | Virtual blasting system for removal of coating and/or rust from a virtual surface |
US20090209216A1 (en) * | 2008-02-20 | 2009-08-20 | Sony Corporation | Reflector for wireless television transmissions |
US11871965B2 (en) | 2008-07-24 | 2024-01-16 | OrthAlign, Inc. | Systems and methods for joint replacement |
US9192392B2 (en) | 2008-07-24 | 2015-11-24 | OrthAlign, Inc. | Systems and methods for joint replacement |
US10864019B2 (en) | 2008-07-24 | 2020-12-15 | OrthAlign, Inc. | Systems and methods for joint replacement |
US9572586B2 (en) | 2008-07-24 | 2017-02-21 | OrthAlign, Inc. | Systems and methods for joint replacement |
US8911447B2 (en) | 2008-07-24 | 2014-12-16 | OrthAlign, Inc. | Systems and methods for joint replacement |
US10206714B2 (en) | 2008-07-24 | 2019-02-19 | OrthAlign, Inc. | Systems and methods for joint replacement |
US8998910B2 (en) | 2008-07-24 | 2015-04-07 | OrthAlign, Inc. | Systems and methods for joint replacement |
US11684392B2 (en) | 2008-07-24 | 2023-06-27 | OrthAlign, Inc. | Systems and methods for joint replacement |
US11547451B2 (en) | 2008-07-24 | 2023-01-10 | OrthAlign, Inc. | Systems and methods for joint replacement |
US9855075B2 (en) | 2008-07-24 | 2018-01-02 | OrthAlign, Inc. | Systems and methods for joint replacement |
US11540746B2 (en) | 2008-09-10 | 2023-01-03 | OrthAlign, Inc. | Hip surgery systems and methods |
US11179062B2 (en) | 2008-09-10 | 2021-11-23 | OrthAlign, Inc. | Hip surgery systems and methods |
US9931059B2 (en) | 2008-09-10 | 2018-04-03 | OrthAlign, Inc. | Hip surgery systems and methods |
US10321852B2 (en) | 2008-09-10 | 2019-06-18 | OrthAlign, Inc. | Hip surgery systems and methods |
US8974468B2 (en) | 2008-09-10 | 2015-03-10 | OrthAlign, Inc. | Hip surgery systems and methods |
US9892563B2 (en) * | 2008-10-27 | 2018-02-13 | Sri International | System and method for generating a mixed reality environment |
US11633293B2 (en) | 2009-07-24 | 2023-04-25 | OrthAlign, Inc. | Systems and methods for joint replacement |
US9271756B2 (en) | 2009-07-24 | 2016-03-01 | OrthAlign, Inc. | Systems and methods for joint replacement |
US10869771B2 (en) | 2009-07-24 | 2020-12-22 | OrthAlign, Inc. | Systems and methods for joint replacement |
US9775725B2 (en) | 2009-07-24 | 2017-10-03 | OrthAlign, Inc. | Systems and methods for joint replacement |
US10238510B2 (en) | 2009-07-24 | 2019-03-26 | OrthAlign, Inc. | Systems and methods for joint replacement |
US20120200703A1 (en) * | 2009-10-22 | 2012-08-09 | Bluebird Aero Systems Ltd. | Imaging system for uav |
US9339226B2 (en) | 2010-01-21 | 2016-05-17 | OrthAlign, Inc. | Systems and methods for joint replacement |
US20120218186A1 (en) * | 2011-02-27 | 2012-08-30 | Mr. David Brock, SR. | 3D Configuration Management System (CMS) Visualization and Management System |
US9504909B2 (en) | 2011-05-05 | 2016-11-29 | Qualcomm Incorporated | Method and apparatus of proximity and stunt recording for outdoor gaming |
US10219811B2 (en) | 2011-06-27 | 2019-03-05 | Board Of Regents Of The University Of Nebraska | On-board tool tracking system and methods of computer assisted surgery |
US10080617B2 (en) | 2011-06-27 | 2018-09-25 | Board Of Regents Of The University Of Nebraska | On-board tool tracking system and methods of computer assisted surgery |
US9498231B2 (en) | 2011-06-27 | 2016-11-22 | Board Of Regents Of The University Of Nebraska | On-board tool tracking system and methods of computer assisted surgery |
US11911117B2 (en) | 2011-06-27 | 2024-02-27 | Board Of Regents Of The University Of Nebraska | On-board tool tracking system and methods of computer assisted surgery |
JP2013012024A (en) * | 2011-06-29 | 2013-01-17 | Olympus Corp | Information processing system, portable electronic apparatus, program and information storage medium |
US20130007668A1 (en) * | 2011-07-01 | 2013-01-03 | James Chia-Ming Liu | Multi-visor: managing applications in head mounted displays |
US9727132B2 (en) * | 2011-07-01 | 2017-08-08 | Microsoft Technology Licensing, Llc | Multi-visor: managing applications in augmented reality environments |
US10030931B1 (en) * | 2011-12-14 | 2018-07-24 | Lockheed Martin Corporation | Head mounted display-based training tool |
US10716580B2 (en) | 2012-05-18 | 2020-07-21 | OrthAlign, Inc. | Devices and methods for knee arthroplasty |
US9549742B2 (en) | 2012-05-18 | 2017-01-24 | OrthAlign, Inc. | Devices and methods for knee arthroplasty |
US9235241B2 (en) | 2012-07-29 | 2016-01-12 | Qualcomm Incorporated | Anatomical gestures detection system using radio signals |
WO2014022239A1 (en) * | 2012-07-29 | 2014-02-06 | Qualcomm Incorporated | Anatomical gestures detection system using radio signals |
US11911119B2 (en) | 2012-08-14 | 2024-02-27 | OrthAlign, Inc. | Hip replacement navigation system and method |
US9649160B2 (en) | 2012-08-14 | 2017-05-16 | OrthAlign, Inc. | Hip replacement navigation system and method |
US10603115B2 (en) | 2012-08-14 | 2020-03-31 | OrthAlign, Inc. | Hip replacement navigation system and method |
US11653981B2 (en) | 2012-08-14 | 2023-05-23 | OrthAlign, Inc. | Hip replacement navigation system and method |
WO2014087044A1 (en) * | 2012-12-06 | 2014-06-12 | Nokia Corporation | Provision of an image element on a display worn by a user |
US20140160170A1 (en) * | 2012-12-06 | 2014-06-12 | Nokia Corporation | Provision of an Image Element on a Display Worn by a User |
US10105149B2 (en) | 2013-03-15 | 2018-10-23 | Board Of Regents Of The University Of Nebraska | On-board tool tracking system and methods of computer assisted surgery |
WO2016072785A1 (en) * | 2014-11-07 | 2016-05-12 | Samsung Electronics Co., Ltd. | Direction based electronic device for displaying object and method thereof |
US10642041B2 (en) | 2014-11-07 | 2020-05-05 | Samsung Electronics Co., Ltd | Direction based electronic device for displaying object and method thereof |
US10363149B2 (en) | 2015-02-20 | 2019-07-30 | OrthAlign, Inc. | Hip replacement navigation system and method |
US11020245B2 (en) | 2015-02-20 | 2021-06-01 | OrthAlign, Inc. | Hip replacement navigation system and method |
US11756335B2 (en) | 2015-02-26 | 2023-09-12 | Magic Leap, Inc. | Apparatus for a near-eye display |
US11347960B2 (en) | 2015-02-26 | 2022-05-31 | Magic Leap, Inc. | Apparatus for a near-eye display |
US11619988B2 (en) | 2015-03-05 | 2023-04-04 | Magic Leap, Inc. | Systems and methods for augmented reality |
US20160259404A1 (en) * | 2015-03-05 | 2016-09-08 | Magic Leap, Inc. | Systems and methods for augmented reality |
US10838207B2 (en) * | 2015-03-05 | 2020-11-17 | Magic Leap, Inc. | Systems and methods for augmented reality |
AU2016225963B2 (en) * | 2015-03-05 | 2021-05-13 | Magic Leap, Inc. | Systems and methods for augmented reality |
US10180734B2 (en) | 2015-03-05 | 2019-01-15 | Magic Leap, Inc. | Systems and methods for augmented reality |
US11256090B2 (en) | 2015-03-05 | 2022-02-22 | Magic Leap, Inc. | Systems and methods for augmented reality |
US11429183B2 (en) | 2015-03-05 | 2022-08-30 | Magic Leap, Inc. | Systems and methods for augmented reality |
CN107533233A (en) * | 2015-03-05 | 2018-01-02 | 奇跃公司 | System and method for augmented reality |
US10678324B2 (en) | 2015-03-05 | 2020-06-09 | Magic Leap, Inc. | Systems and methods for augmented reality |
US10846864B2 (en) * | 2015-06-10 | 2020-11-24 | VTouch Co., Ltd. | Method and apparatus for detecting gesture in user-based spatial coordinate system |
US10409443B2 (en) * | 2015-06-24 | 2019-09-10 | Microsoft Technology Licensing, Llc | Contextual cursor display based on hand tracking |
US20160378294A1 (en) * | 2015-06-24 | 2016-12-29 | Shawn Crispin Wright | Contextual cursor display based on hand tracking |
US11944428B2 (en) | 2015-11-30 | 2024-04-02 | Nike, Inc. | Apparel with ultrasonic position sensing and haptic feedback for activities |
US11288832B2 (en) | 2015-12-04 | 2022-03-29 | Magic Leap, Inc. | Relocalization systems and methods |
US10909711B2 (en) | 2015-12-04 | 2021-02-02 | Magic Leap, Inc. | Relocalization systems and methods |
US11244485B2 (en) | 2016-01-19 | 2022-02-08 | Magic Leap, Inc. | Augmented reality systems and methods utilizing reflections |
US10948994B2 (en) | 2016-02-29 | 2021-03-16 | Huawei Technologies Co., Ltd. | Gesture control method for wearable system and wearable system |
US11460698B2 (en) | 2016-04-26 | 2022-10-04 | Magic Leap, Inc. | Electromagnetic tracking with augmented reality systems |
US10948721B2 (en) | 2016-04-26 | 2021-03-16 | Magic Leap, Inc. | Electromagnetic tracking with augmented reality systems |
CN109689173A (en) * | 2016-04-26 | 2019-04-26 | 奇跃公司 | It is tracked using the electromagnetism of augmented reality system |
US10915165B2 (en) * | 2016-07-29 | 2021-02-09 | Emmanuel Lusinchi | Methods and systems for controlling a displacement of a virtual point of view in a virtual reality environment |
US11073699B2 (en) | 2016-08-02 | 2021-07-27 | Magic Leap, Inc. | Fixed-distance virtual and augmented reality systems and methods |
US10649211B2 (en) | 2016-08-02 | 2020-05-12 | Magic Leap, Inc. | Fixed-distance virtual and augmented reality systems and methods |
US11536973B2 (en) | 2016-08-02 | 2022-12-27 | Magic Leap, Inc. | Fixed-distance virtual and augmented reality systems and methods |
US11790554B2 (en) | 2016-12-29 | 2023-10-17 | Magic Leap, Inc. | Systems and methods for augmented reality |
US11874468B2 (en) | 2016-12-30 | 2024-01-16 | Magic Leap, Inc. | Polychromatic light out-coupling apparatus, near-eye displays comprising the same, and method of out-coupling polychromatic light |
US11206507B2 (en) | 2017-01-23 | 2021-12-21 | Magic Leap, Inc. | Localization determination for mixed reality systems |
US11711668B2 (en) | 2017-01-23 | 2023-07-25 | Magic Leap, Inc. | Localization determination for mixed reality systems |
US10812936B2 (en) | 2017-01-23 | 2020-10-20 | Magic Leap, Inc. | Localization determination for mixed reality systems |
US11786261B2 (en) | 2017-03-14 | 2023-10-17 | OrthAlign, Inc. | Soft tissue measurement and balancing systems and methods |
US11547580B2 (en) | 2017-03-14 | 2023-01-10 | OrthAlign, Inc. | Hip replacement navigation systems and methods |
US10918499B2 (en) | 2017-03-14 | 2021-02-16 | OrthAlign, Inc. | Hip replacement navigation systems and methods |
US10863995B2 (en) | 2017-03-14 | 2020-12-15 | OrthAlign, Inc. | Soft tissue measurement and balancing systems and methods |
US11410269B2 (en) | 2017-03-17 | 2022-08-09 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
US10861130B2 (en) | 2017-03-17 | 2020-12-08 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
US11315214B2 (en) | 2017-03-17 | 2022-04-26 | Magic Leap, Inc. | Mixed reality system with color virtual content warping and method of generating virtual con tent using same |
US11978175B2 (en) | 2017-03-17 | 2024-05-07 | Magic Leap, Inc. | Mixed reality system with color virtual content warping and method of generating virtual content using same |
US11423626B2 (en) | 2017-03-17 | 2022-08-23 | Magic Leap, Inc. | Mixed reality system with multi-source virtual content compositing and method of generating virtual content using same |
US10762598B2 (en) | 2017-03-17 | 2020-09-01 | Magic Leap, Inc. | Mixed reality system with color virtual content warping and method of generating virtual content using same |
US10769752B2 (en) | 2017-03-17 | 2020-09-08 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
US10861237B2 (en) | 2017-03-17 | 2020-12-08 | Magic Leap, Inc. | Mixed reality system with multi-source virtual content compositing and method of generating virtual content using same |
US10964119B2 (en) | 2017-03-17 | 2021-03-30 | Magic Leap, Inc. | Mixed reality system with multi-source virtual content compositing and method of generating virtual content using same |
CN110431468A (en) * | 2017-04-28 | 2019-11-08 | 惠普发展公司,有限责任合伙企业 | Determine position and the orientation of user's trunk for display system |
US20200057476A1 (en) * | 2017-04-28 | 2020-02-20 | Hewlett-Packard Development Company, L.P. | Determining position and orientation of a user's torso for a display system |
WO2018199979A1 (en) * | 2017-04-28 | 2018-11-01 | Hewlett-Packard Development Company, L.P. | Determining position and orientation of a user's torso for a display system |
US11090689B2 (en) | 2017-04-28 | 2021-08-17 | United Parcel Service Of America, Inc. | Conveyor belt assembly for identifying an asset sort location and methods of utilizing the same |
US10471478B2 (en) | 2017-04-28 | 2019-11-12 | United Parcel Service Of America, Inc. | Conveyor belt assembly for identifying an asset sort location and methods of utilizing the same |
US11858010B2 (en) | 2017-04-28 | 2024-01-02 | United Parcel Service Of America, Inc. | Conveyor belt assembly for identifying an asset sort location and methods of utilizing the same |
US11216045B2 (en) * | 2017-04-28 | 2022-01-04 | Hewlett-Packard Development Company, L.P. | Determining position and orientation of a user's torso for a display system |
WO2018231819A1 (en) * | 2017-06-17 | 2018-12-20 | Tactual Labs Co. | Six degrees of freedom tracking of objects using sensors |
GB2588468A (en) * | 2017-06-17 | 2021-04-28 | Tactual Labs Co | Six degrees of freedom tracking of objects using sensors |
US10838516B2 (en) | 2017-06-17 | 2020-11-17 | Tactual Labs Co. | Six degrees of freedom tracking of objects using sensors |
CN110753851A (en) * | 2017-06-17 | 2020-02-04 | 触觉实验室股份有限公司 | Six degree of freedom tracking of an object using sensors |
US11567324B2 (en) | 2017-07-26 | 2023-01-31 | Magic Leap, Inc. | Exit pupil expander |
US11927759B2 (en) | 2017-07-26 | 2024-03-12 | Magic Leap, Inc. | Exit pupil expander |
US11953653B2 (en) | 2017-12-10 | 2024-04-09 | Magic Leap, Inc. | Anti-reflective coatings on optical waveguides |
US11280937B2 (en) | 2017-12-10 | 2022-03-22 | Magic Leap, Inc. | Anti-reflective coatings on optical waveguides |
US11762222B2 (en) | 2017-12-20 | 2023-09-19 | Magic Leap, Inc. | Insert for augmented reality viewing device |
US11908434B2 (en) | 2018-03-15 | 2024-02-20 | Magic Leap, Inc. | Image correction due to deformation of components of a viewing device |
US11776509B2 (en) | 2018-03-15 | 2023-10-03 | Magic Leap, Inc. | Image correction due to deformation of components of a viewing device |
US11885871B2 (en) | 2018-05-31 | 2024-01-30 | Magic Leap, Inc. | Radar head pose localization |
US11200870B2 (en) | 2018-06-05 | 2021-12-14 | Magic Leap, Inc. | Homography transformation matrices based temperature calibration of a viewing system |
US11579441B2 (en) | 2018-07-02 | 2023-02-14 | Magic Leap, Inc. | Pixel intensity modulation using modifying gain values |
US12001013B2 (en) | 2018-07-02 | 2024-06-04 | Magic Leap, Inc. | Pixel intensity modulation using modifying gain values |
US11510027B2 (en) | 2018-07-03 | 2022-11-22 | Magic Leap, Inc. | Systems and methods for virtual and augmented reality |
US11856479B2 (en) | 2018-07-03 | 2023-12-26 | Magic Leap, Inc. | Systems and methods for virtual and augmented reality along a route with markers |
US10943521B2 (en) | 2018-07-23 | 2021-03-09 | Magic Leap, Inc. | Intra-field sub code timing in field sequential displays |
US11501680B2 (en) | 2018-07-23 | 2022-11-15 | Magic Leap, Inc. | Intra-field sub code timing in field sequential displays |
US11379948B2 (en) | 2018-07-23 | 2022-07-05 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
US11790482B2 (en) | 2018-07-23 | 2023-10-17 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
US11624929B2 (en) | 2018-07-24 | 2023-04-11 | Magic Leap, Inc. | Viewing device with dust seal integration |
US11598651B2 (en) | 2018-07-24 | 2023-03-07 | Magic Leap, Inc. | Temperature dependent calibration of movement detection devices |
US11630507B2 (en) | 2018-08-02 | 2023-04-18 | Magic Leap, Inc. | Viewing system with interpupillary distance compensation based on head motion |
US11216086B2 (en) | 2018-08-03 | 2022-01-04 | Magic Leap, Inc. | Unfused pose-based drift correction of a fused pose of a totem in a user interaction system |
US11609645B2 (en) | 2018-08-03 | 2023-03-21 | Magic Leap, Inc. | Unfused pose-based drift correction of a fused pose of a totem in a user interaction system |
EP3830631A4 (en) * | 2018-08-03 | 2021-10-27 | Magic Leap, Inc. | Unfused pose-based drift correction of a fused pose of a totem in a user interaction system |
WO2020028191A1 (en) | 2018-08-03 | 2020-02-06 | Magic Leap, Inc. | Unfused pose-based drift correction of a fused pose of a totem in a user interaction system |
US11960661B2 (en) | 2018-08-03 | 2024-04-16 | Magic Leap, Inc. | Unfused pose-based drift correction of a fused pose of a totem in a user interaction system |
US12016719B2 (en) | 2018-08-22 | 2024-06-25 | Magic Leap, Inc. | Patient viewing system |
WO2020077389A1 (en) * | 2018-10-15 | 2020-04-23 | Idearlabs Pty Ltd | "a method and system for determining an orientation of a user" |
US11521296B2 (en) | 2018-11-16 | 2022-12-06 | Magic Leap, Inc. | Image size triggered clarification to maintain image sharpness |
US12044851B2 (en) | 2018-12-21 | 2024-07-23 | Magic Leap, Inc. | Air pocket structures for promoting total internal reflection in a waveguide |
US11425189B2 (en) | 2019-02-06 | 2022-08-23 | Magic Leap, Inc. | Target intent-based clock speed determination and adjustment to limit total heat generated by multiple processors |
US11762623B2 (en) | 2019-03-12 | 2023-09-19 | Magic Leap, Inc. | Registration of local content between first and second augmented reality viewers |
WO2020182309A1 (en) * | 2019-03-14 | 2020-09-17 | Huawei Technologies Co., Ltd. | Ultrasonic hand tracking system |
US11445232B2 (en) | 2019-05-01 | 2022-09-13 | Magic Leap, Inc. | Content provisioning system and method |
US11514673B2 (en) | 2019-07-26 | 2022-11-29 | Magic Leap, Inc. | Systems and methods for augmented reality |
US12033081B2 (en) | 2019-11-14 | 2024-07-09 | Magic Leap, Inc. | Systems and methods for virtual and augmented reality |
US11737832B2 (en) | 2019-11-15 | 2023-08-29 | Magic Leap, Inc. | Viewing system for use in a surgical environment |
EP4240008A4 (en) * | 2020-10-30 | 2024-04-17 | Panasonic Intellectual Property Management Co., Ltd. | Display device |
WO2023028190A1 (en) * | 2021-08-26 | 2023-03-02 | Street Smarts VR | Mount for adapting weapons to a virtual tracker |
US11852436B2 (en) | 2021-08-26 | 2023-12-26 | Street Smarts VR, Inc. | Mount for adapting weapons to a virtual tracker |
Also Published As
Publication number | Publication date |
---|---|
US6757068B2 (en) | 2004-06-29 |
AU2001233019A1 (en) | 2001-08-07 |
US20060284792A1 (en) | 2006-12-21 |
US7301648B2 (en) | 2007-11-27 |
WO2001056007A8 (en) | 2002-01-24 |
WO2001056007A1 (en) | 2001-08-02 |
US20020024675A1 (en) | 2002-02-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6757068B2 (en) | Self-referenced tracking | |
Foxlin et al. | Weartrack: A self-referenced head and hand tracker for wearable computers and portable vr | |
US11513605B2 (en) | Object motion tracking with remote device | |
US11816296B2 (en) | External user interface for head worn computing | |
US11886638B2 (en) | External user interface for head worn computing | |
US20220163799A1 (en) | External user interface for head worn computing | |
WO2019153824A1 (en) | Virtual object control method, device, computer apparatus, and storage medium | |
KR101546654B1 (en) | Method and apparatus for providing augmented reality service in wearable computing environment | |
EP3997552A1 (en) | Virtual user interface using a peripheral device in artificial reality environments | |
CN103119628B (en) | Utilize three-dimensional user interface effect on the display of kinetic characteristic | |
US20150205351A1 (en) | External user interface for head worn computing | |
US20170017323A1 (en) | External user interface for head worn computing | |
US20160025977A1 (en) | External user interface for head worn computing | |
CN107646098A (en) | System for tracking portable equipment in virtual reality | |
CN116027894A (en) | Passive optical and inertial tracking for slim form factors | |
CN116360604A (en) | Method and apparatus for aligning components in a virtual reality environment | |
US11886650B2 (en) | Stylus-based input system for a head-mounted device | |
WO2017015093A1 (en) | External user interface for head worn computing | |
JP2017091433A (en) | Head-mounted type display device, method of controlling head-mounted type display device, and computer program | |
WO2019026713A1 (en) | Information processing device, information processing method, and program | |
US20230349693A1 (en) | System and method for generating input data from pose estimates of a manipulated object by using light data and relative motion data | |
JP2018200415A (en) | Head-mounted display device, program, and method for controlling head-mounted display device | |
US10592010B1 (en) | Electronic device system with input tracking and visual output | |
CN106681488A (en) | Digital eyewear operated by head | |
WO2022228056A1 (en) | Human-computer interaction method and device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: GC NORTH ACQUISITION, LLC, PENNSYLVANIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERSENSE INCORPORATED;REEL/FRAME:027347/0300 Effective date: 20111118 |
|
AS | Assignment |
Owner name: INTERSENSE, LLC, PENNSYLVANIA Free format text: CHANGE OF NAME;ASSIGNOR:GC NORTH ACQUISITION, LLC;REEL/FRAME:028469/0864 Effective date: 20111122 Owner name: INDIGO TECHNOLOGIES, LLC, PENNSYLVANIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERSENSE, LLC;REEL/FRAME:028469/0444 Effective date: 20120628 |
|
AS | Assignment |
Owner name: THALES VISIONIX, INC., MARYLAND Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INDIGO TECHNOLOGIES, LLC;REEL/FRAME:030566/0711 Effective date: 20121228 |