WO2011089538A1 - A stereo-calibration-less multiple-camera human-tracking system for human-computer 3d interaction - Google Patents
A stereo-calibration-less multiple-camera human-tracking system for human-computer 3d interaction Download PDFInfo
- Publication number
- WO2011089538A1 WO2011089538A1 PCT/IB2011/050138 IB2011050138W WO2011089538A1 WO 2011089538 A1 WO2011089538 A1 WO 2011089538A1 IB 2011050138 W IB2011050138 W IB 2011050138W WO 2011089538 A1 WO2011089538 A1 WO 2011089538A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- user
- camera
- screen
- fixture
- backed
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/0304—Detection arrangements using opto-electronic means
- G06F3/0325—Detection arrangements using opto-electronic means using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
Definitions
- One of the problems of multiple-camera 3D tracking is stereo calibration. What is proposed is a system that does not require a stereo calibration step by the user or installer of the system.
- 3D human-computer interaction e.g. with virtual object 18 in Fig.1
- the cameras do not need be exposed as shown by 8 in Fig. 1. That is an under-the-hood drawing.
- the screen-corner-brackets could typically provide casing for the cameras as denoted by 16 in Fig. 1 for discreetness and extra protection and security.
- the casing in front of the cameras would need to be transparent to whatever wavelengths of electromagnetic radiation the cameras need to detect, or could be transparent to those wavelengths exclusively.
- brackets are significantly better than using, for example, a single bar at the top of the screen because the user does not have to perform any kind of measurements to place them in a precise way. Also, the units allow for more compact packing of the system than a long bar would, allowing for more convenient transit, storage and stocking of the system.
- the cameras used for 3D tracking would be pre-mounted on the brackets securely and in a precise way, so that their position and orientation in relation to their bracket is already known. This would then be used in conjunction with the calculation of the screen size as described above to deduce the position and orientation of the cameras in relation to the screen.
- the position and orientation of the cameras relative to the screen is sufficient information to deduce the 3D location, relative to the screen, of any tracked object in view of two or more of the cameras.
- Markers mounted on the user's stereoscopic glasses such as LEDs 11 and 12 in Fig. 1 provide one way of estimating those positions. Another way would be tracking the user's head and estimating the left and right eye positions that way. This requires a little more computation than tracking LEDs but would be far more desirable.
- the system would also typically need to track another part of the user, or a device that the user is in control of, to enable interaction with the virtual world.
- FIG. 1 An example of such a user-controlled device is denoted by the proposed handsets 13 and 14 in Fig. 1.
- Markers such as LEDs 15 in Fig. 1 mounted on each device can provide a simpler way of tracking the device's position and orientation than tracking the device or user by itself.
- the system is significantly beneficial in comparison to systems that require stereo calibration because it would be incredibly easy and quick for the user to set up.
- a good way of increasing the range of view of the cameras is to use fisheye lenses on them.
Abstract
A human-computer 3D interaction system that does not require a stereo calibration step by the user or installer of the system is proposed.
Description
One of the problems of multiple-camera 3D
tracking is stereo calibration. What is proposed is a
system that does not require a stereo calibration step by
the user or installer of the system.
In 3D human-computer interaction (e.g. with
virtual object 18 in Fig.1), it is important to establish
a 3D coordinate space relative to the screen (17 in Fig.
1). Normally one would perform a stereo calibration step
to establish the position and orientation of the cameras in
order to enable 3D tracking of objects located within
viewing range of two or more of the cameras.
However for home use this step is
cumbersome, and requires re-doing every time any camera is
moved even slightly, for example by accident when someone
brushes past.
What is therefore proposed first is the use
of right-angled brackets such as 1 and 2 in Fig.1, which
would be mounted on corners of the user's screen by
the user, in line with the edges of the screen, via a
self-attaching method, such as magnets, a magnet and a
ferrous metal (one of which is attached to the screen by
adhesive), adhesive or suction pumps. The size of the
user's screen would then either then be input into the
system by the user, or calculated automatically by the
system by using at least one ultrasonic receiver such as 3
in Fig. 1 and transmitter such as 4 in Fig.1 mounted on
different brackets, and using a time-of-flight calculation
to calculate the distance between the two, and hence
deduce the size of the screen, or using at least one
camera such as 5 in Fig. 2, to pick up two or more objects
or features in known locations on the other bracket such as
LEDs 6 and 7 in Fig. 2, and using their pixel locations on
the camera (or cameras) to calculate, given the known
information, the actual distance to those objects from
that camera.
The cameras do not need be exposed as shown
by 8 in Fig. 1. That is an under-the-hood drawing. The
screen-corner-brackets could typically provide casing for
the cameras as denoted by 16 in Fig. 1 for discreetness
and extra protection and security. The casing in front of
the cameras would need to be transparent to whatever
wavelengths of electromagnetic radiation the cameras need
to detect, or could be transparent to those wavelengths exclusively.
Using screen-corner-mounted brackets is
significantly better than using, for example, a single bar
at the top of the screen because the user does not have to
perform any kind of measurements to place them in a
precise way. Also, the units allow for more compact packing
of the system than a long bar would, allowing for more
convenient transit, storage and stocking of the system.
The cameras used for 3D tracking, denoted
by 8 in Fig. 1, would be pre-mounted on the brackets
securely and in a precise way, so that their position and
orientation in relation to their bracket is already known.
This would then be used in conjunction with the calculation
of the screen size as described above to deduce the
position and orientation of the cameras in relation to the
screen. The position and orientation of the cameras relative
to the screen is sufficient information to deduce the 3D
location, relative to the screen, of any tracked object in
view of two or more of the cameras.
Typically, in order for realistic 3D
interaction, all versions of the system would need to
track, or estimate using tracking, either the 3D locations
of the user's left and right eyes or a single
compromise between the two, in order to place at least one
virtual camera position (two if stereoscopic presentation
is being used) in the virtual 3D world to match that
location, and set its viewing frustum to the asymmetric
pyramid shape that the user's viewing position (or
each eye position respectively) would make with the
corners of the screen if you were to draw a straight line
from that position to each of the four corners of the
screen at any given time. Example virtual camera frustums
for a stereoscopic system with respect to the left and right
eyes is denoted by 9 and 10 in Fig. 1. Markers mounted on
the user's stereoscopic glasses such as LEDs 11 and
12 in Fig. 1 provide one way of estimating those
positions. Another way would be tracking the user's
head and estimating the left and right eye positions that
way. This requires a little more computation than tracking
LEDs but would be far more desirable.
The system would also typically need to
track another part of the user, or a device that the user
is in control of, to enable interaction with the virtual world.
An example of such a user-controlled
device is denoted by the proposed handsets 13 and 14 in
Fig. 1. Markers such as LEDs 15 in Fig. 1 mounted on each
device can provide a simpler way of tracking the
device's position and orientation than tracking the
device or user by itself.
The system is significantly beneficial in
comparison to systems that require stereo calibration
because it would be incredibly easy and quick for the user
to set up.
A good way of increasing the range of view
of the cameras is to use fisheye lenses on them.
Claims (18)
- A system which comprises one or more fixtures, each of which has two straight edges at right angles to one another, and which, by its shape, leaves empty the internal region of the conceptual right angle considered from the imaginary point of intersection of the two straight edges and along the two straight edges up to infinity, where each fixture has at least one camera attached to it in a known position and orientation in relation to the fixture, and where each camera is able to send its data to a computer system
- A system as claimed in Claim 1 in which the fixture is backed by a magnet
- A system as claimed in Claim 1 in which the fixture is backed by a ferrous metal
- A system as claimed in Claim 2 which is accompanied by a piece backed by adhesive, which forms an empty inner right angle shape, that includes a ferrous metal
- A system as claimed in Claim 2, Claim 3 or Claim 4 which is accompanied by a piece backed by adhesive, which forms an empty inner right angle shape, that includes a magnet
- A system as claimed in Claim 1 in which the fixture is backed by adhesive
- A system as claimed in Claim 1 in which the fixture is backed by suction pumps
- A system as claimed in Claim 1 which is accompanied by suction pumps
- A system as claimed in any of Claims 1 to 8 in which the cameras use fisheye lenses
- A system as claimed in any of Claims 1 to 9 which automatically calculates the size of a screen on which the fixtures are mounted by using at least one camera and at least two distinguishable feature points within its view, both mounted in known locations on another of the fixtures, and the pixel locations of the features as detected by each camera
- A system as claimed in any of Claims 1 to 9 which automatically calculates the size of a screen on which the fixtures are mounted by using at least one ultrasonic transmitter and one ultrasonic sensor on different fixtures within the system, and a the time of flight calculation of at least one ultrasonic signal from the transmitter to the sensor
- A system as claimed in any of Claims 1 to 11 which calculates an estimate of a user’s viewing location in 3D relative to the screen using images from the cameras and the hitherto known locations and orientations of the cameras relative to the screen
- A system as claimed in Claim 12 which sets the location of a virtual camera relative to a virtual 3D environment to match the estimated viewing location of the user, and which sets the virtual viewing frustum of the virtual camera in the virtual 3D environment to match the shape conceivable if straight lines are drawn between the estimated viewing location of the user and the corners of the screen at a given moment
- A system as claimed in any of Claims 1 to 13 which additionally estimates the 3D locations of one or more other parts of the user
- A system as claimed in any of Claims 1 to 14 which additionally estimates the 3D locations of one or more devices held in the user’s hands
- A system as claimed in any of Claims 1 to 15 which additionally estimates the 3D locations of one or more devices mounted on the user’s body
- A system which includes in a single package, an indication of how and all of the components required to assemble the system as claimed in any of the preceding claims
- A system as herein described and illustrated by the accompanying drawings
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB1016811.0 | 2010-01-25 | ||
GB1016811A GB2477174A (en) | 2010-01-25 | 2010-10-06 | Right angled camera housing |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011089538A1 true WO2011089538A1 (en) | 2011-07-28 |
Family
ID=43743557
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2011/050138 WO2011089538A1 (en) | 2010-01-25 | 2011-01-12 | A stereo-calibration-less multiple-camera human-tracking system for human-computer 3d interaction |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2011089538A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102650746A (en) * | 2012-03-31 | 2012-08-29 | 京东方科技集团股份有限公司 | Active shutter type 3D (Three-dimensional) eyeglasses |
CN103294278A (en) * | 2013-06-20 | 2013-09-11 | 苏州速腾电子科技有限公司 | Triangular fixing sheet of touch screen |
CN107145822A (en) * | 2017-03-24 | 2017-09-08 | 深圳奥比中光科技有限公司 | Deviate the method and system of user's body feeling interaction demarcation of depth camera |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2001063550A2 (en) * | 2000-02-21 | 2001-08-30 | Tool-Tribe International A/S | Position detection device |
US20050248539A1 (en) * | 2004-05-05 | 2005-11-10 | Morrison Gerald D | Apparatus and method for detecting a pointer relative to a touch surface |
GB2451461A (en) * | 2007-07-28 | 2009-02-04 | Naveen Chawla | Camera based 3D user and wand tracking human-computer interaction system |
-
2011
- 2011-01-12 WO PCT/IB2011/050138 patent/WO2011089538A1/en active Application Filing
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2001063550A2 (en) * | 2000-02-21 | 2001-08-30 | Tool-Tribe International A/S | Position detection device |
US20050248539A1 (en) * | 2004-05-05 | 2005-11-10 | Morrison Gerald D | Apparatus and method for detecting a pointer relative to a touch surface |
GB2451461A (en) * | 2007-07-28 | 2009-02-04 | Naveen Chawla | Camera based 3D user and wand tracking human-computer interaction system |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102650746A (en) * | 2012-03-31 | 2012-08-29 | 京东方科技集团股份有限公司 | Active shutter type 3D (Three-dimensional) eyeglasses |
US9494804B2 (en) | 2012-03-31 | 2016-11-15 | Boe Technology Group Co., Ltd. | Active-shutter 3D glasses and operating method thereof |
CN103294278A (en) * | 2013-06-20 | 2013-09-11 | 苏州速腾电子科技有限公司 | Triangular fixing sheet of touch screen |
CN107145822A (en) * | 2017-03-24 | 2017-09-08 | 深圳奥比中光科技有限公司 | Deviate the method and system of user's body feeling interaction demarcation of depth camera |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20200278449A1 (en) | System and method for concurrent odometry and mapping | |
US11017610B2 (en) | System and method for fault detection and recovery for concurrent odometry and mapping | |
US10262466B2 (en) | Systems and methods for adjusting a combined image visualization based on depth information | |
US10242454B2 (en) | System for depth data filtering based on amplitude energy values | |
WO2016095057A1 (en) | Peripheral tracking for an augmented reality head mounted device | |
US20180276863A1 (en) | System and method for merging maps | |
US8155387B2 (en) | Method and system for position determination using image deformation | |
WO2017218213A3 (en) | Miniature vision-inertial navigation system with extended dynamic range | |
US20130101206A1 (en) | Method, System and Computer Program Product for Segmenting an Image | |
WO2011089538A1 (en) | A stereo-calibration-less multiple-camera human-tracking system for human-computer 3d interaction | |
CN108259827A (en) | A kind of method, apparatus for realizing security protection, AR equipment and system | |
CN104504868A (en) | Device and method for monitoring bad habits of terminal device user | |
US20240054780A1 (en) | Object detections for virtual reality | |
CN101887330B (en) | Electronic equipment as well as single-camera object-positioning device and method thereof | |
JP2017055446A (en) | Estimation of control feature from remote controller having camera | |
JP5213883B2 (en) | Composite display device | |
GB2477174A (en) | Right angled camera housing | |
US11935286B2 (en) | Method and device for detecting a vertical planar surface | |
Rydell et al. | Chameleon v2: Improved imaging-inertial indoor navigation | |
KR101729923B1 (en) | Method for implementing 3d image using screen image and augmented reality image and server and system implementing the same | |
TW201935032A (en) | Electronic device and positioning method | |
CN106131541A (en) | Intelligent display device based on augmented reality and method | |
Amemiya et al. | Camera pose estimation with a two-dimensional marker Grid for haptic navigation | |
Gomez et al. | Real-time image registration of RGB webcams and colorless 3d time-of-flight cameras | |
GB2477145A (en) | 3D display with ultrasonic head tracking |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11703262 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 11703262 Country of ref document: EP Kind code of ref document: A1 |