US7572011B2 - Contact lenses for use in motion capture - Google Patents
Contact lenses for use in motion capture Download PDFInfo
- Publication number
- US7572011B2 US7572011B2 US11/460,611 US46061106A US7572011B2 US 7572011 B2 US7572011 B2 US 7572011B2 US 46061106 A US46061106 A US 46061106A US 7572011 B2 US7572011 B2 US 7572011B2
- Authority
- US
- United States
- Prior art keywords
- tracking feature
- motion capture
- contact lens
- tracking
- eye
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
Definitions
- the present invention relates generally to computer animation, and more specifically to motion capture techniques.
- Motion capture is a technique of digitally recording the movements of real things, such as humans, as a source of motion data for computer animation.
- the technique involves a performer wearing a set of markers that are tracked by sensors as the performer moves.
- a computer records the movements to provide a digital representation of the motion.
- the digital representation of the motion can then be used in animation so that an animator does not have to draw each frame.
- One embodiment provides an apparatus for use in motion capture, comprising: a contact lens; and a tracking feature incorporated into the contact lens.
- Another embodiment provides a method for use in motion capture, comprising the steps of: inserting a contact lens having a tracking feature into a performer's eye; and tracking movements of the performer's eye using the tracking feature.
- FIG. 1 illustrates an apparatus made in accordance with an embodiment of the present invention
- FIG. 2 is an exploded view of the apparatus shown in FIG. 1 illustrating an assembly thereof in accordance with an embodiment of the present invention
- FIG. 3 illustrates the apparatus shown in FIG. 1 inserted in a person's eye in accordance with an embodiment of the present invention
- FIG. 4 illustrates an apparatus made in accordance with another embodiment of the present invention
- FIG. 5 illustrates an apparatus made in accordance with another embodiment of the present invention.
- FIG. 6 is a flow diagram illustrating a method for use in motion capture in accordance with an embodiment of the present invention.
- motion capture or tracking has expanded into education, training, sports and recently computer animation for cinema and video games.
- a performer wears reflective markers near each joint to identify the motion by the positions or angles between the markers. Reflective markers are tracked, optimally at least two times the rate of the desired motion, to submillimeter positions.
- the motion capture computer software records the positions, angles, velocities, accelerations and impulses, providing an accurate digital representation of the motion.
- Optical systems triangulate the three dimensional (3D) position of a marker between a number of cameras calibrated to provide overlapping projections. Tracking a large number of markers or multiple performers or expanding the capture area is accomplished by the addition of more cameras. These systems produce data with three degrees of freedom for each marker, and rotational information must be inferred from the relative orientation of three or more markers. For example, shoulder, elbow and wrist markers may be used to provide the angle of the elbow.
- the look of the character's face can be greatly improved by creating realistic eye movement.
- creating realistic eye movement is difficult because eye and eyelid animation is complex. Namely, such animation seeks to combine several different movements, such as voluntary movements (e.g. the point of focus of the character's eyes), involuntary movements (e.g. the character's facial expression), and automatic movements (e.g. blinking of the eyes).
- voluntary movements e.g. the point of focus of the character's eyes
- involuntary movements e.g. the character's facial expression
- automatic movements e.g. blinking of the eyes.
- Embodiments of the present invention provide contact lenses that include a tracking feature for use in motion capture of eye movements. Namely, such contact lenses may be worn by the performer during the motion capture process to track the movements of the performer's eyes.
- the optical contact lens 20 which may be used in motion capture, includes a tracking feature incorporated into the contact lens.
- the optical contact lens 20 includes a reflector or reflective dot 22 embedded therein.
- the reflective dot 22 comprises a tracking feature and may be used as a marker in the motion capture process.
- the reflective dot 22 may comprise a reflective material. In other embodiments the reflective dot 22 may comprise many different shapes.
- FIG. 2 shows an exploded view of the optical contact lens 20 and illustrates one manner in which it may be assembled in accordance with an embodiment of the present invention.
- the reflective dot 22 may be embedded between two layers of plastic or contact lens material 24 , 26 .
- a typical construction may comprise 45% methafilcon B, and 55% water in a buffered 0.9% saline solution.
- FIG. 3 illustrates the optical contact lens 20 being worn in a user's eye.
- the weight of the reflective dot 22 will naturally and automatically center itself at the bottom of the iris, allowing the user to see clearly out into space. Offsets may then be calculated at the time of, or after, the motion capture process so that the reflective dot 22 appears in the actual center of the eye.
- a pair of the optical contact lenses 20 may be ordered to fit a specific user, and in some embodiments they may even contain corrective prescriptions.
- some embodiments of the present invention provide for a reflective material that consistently rests near the bottom of iris. If needed, bright lights may be used with the camera set-up during the motion capture process to reduce dilation of the pupils. Such bright light can cause the pupils to tighten up nicely.
- the reflective dot 22 comprises the tracking feature of the contact lens 20 and may be used as a marker in motion capture. It should be well understood, however, that in other embodiments the tracking feature may comprise many different shapes or patterns, or the tracking feature may comprise any other machine recognizable feature.
- the tracking feature may occupy any portion or all of the lens.
- the location of the tracking feature may be at the bottom of the iris as shown, or outside the iris, or across a larger portion of the lens, or some combination thereof.
- the tracking feature may be visually blocking/opaque, dithered to enable partial transparency to allow the user to see, or a combination thereof.
- a combination opaque and transparent lens may be optimized so that the portion over the iris is less opaque but the region around the iris gets increasingly opaque.
- the tracking feature may comprise multiple colors.
- the tracking feature may comprise a color different than the rest of the contact lens.
- the colors may be disposed in sectors and wedges. Each such wedge may have a different color than its neighboring wedge, each sector may have a different color than its neighboring sector, etc.
- the colors may be chosen so that each neighbor has a high contrast color from its other neighbors (e.g. neon orange vs. neon green) to yield maximum contrast between color regions.
- the tracking feature comprises four dots 42 , 44 , 46 , 48 , which are arranged in a north, south, east, west pattern. Such a pattern allows for tracking of the eye at its extreme left, right, up or down, or at any radian of rotation, where one or more of the dots 42 , 44 , 46 , 48 might become covered up by the eye lid or surrounding skin. While four dots are illustrated it should be well understood that any number of dots may be used and that instead of dots the tracking feature may comprise any other shape or pattern. Furthermore, the dots 42 , 44 , 46 , 48 may all be the same color, different colors, or some the same color and some a different color.
- FIG. 5 illustrates a contact lens 60 made in accordance with yet another embodiment of the present invention.
- the tracking feature comprises eight regions 62 , 64 , 66 , 68 , 70 , 72 , 74 , 76 which are arranged in a north, south, east, west, northeast, southeast, southwest, northwest pattern. Similar to above, such a pattern allows for tracking of the eye at its extreme left, right, up or down, or at any radian of rotation, where one or more of the regions 62 , 64 , 66 , 68 , 70 , 72 , 74 , 76 might become covered up by the eye lid or surrounding skin.
- regions 62 , 64 , 66 , 68 , 70 , 72 , 74 , 76 may all be the same color, different colors, or some the same color and some a different color.
- transparent, semi-transparent, or even opaque motion capture contact lenses include a tracking feature.
- the tracking feature may comprise one or more specific colors or colors in combination with a shape or pattern target marking.
- the tracking feature may include any shape, pattern or arrangement that is trackable or detectable by a motion capture system so that eye movement may be tracked.
- the use of such lenses to capture eye movements in a motion capture process can be used to reduce the problem of “dead eye” in video games and other animations.
- a method 90 in accordance with an embodiment of the present invention.
- the method 90 may be used in motion capture processes, such as for use in making video games and other animations.
- step 92 a contact lens having a tracking feature is inserted into a performer's eye.
- the performer then moves in some manner.
- step 94 the movements of the performer's eye are tracked using the tracking feature.
- step 96 which is an optional step as indicated by the dashed lines, offsets may be calculated to account for the positioning of the tracking feature on the contact lens. For example, if the tracking feature comprises an off center reflective dot as shown in FIGS. 1-3 , an offset may be calculated to account for the dot not being centered. Such calculation may be performed by any type of computer system or the like.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Ophthalmology & Optometry (AREA)
- Biomedical Technology (AREA)
- Human Computer Interaction (AREA)
- Medical Informatics (AREA)
- Physics & Mathematics (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Eyeglasses (AREA)
- Lenses (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
Claims (5)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/460,611 US7572011B2 (en) | 2006-07-27 | 2006-07-27 | Contact lenses for use in motion capture |
PCT/US2007/072584 WO2008014085A2 (en) | 2006-07-27 | 2007-06-29 | Contact lenses for use in motion capture |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/460,611 US7572011B2 (en) | 2006-07-27 | 2006-07-27 | Contact lenses for use in motion capture |
Publications (2)
Publication Number | Publication Date |
---|---|
US20080024722A1 US20080024722A1 (en) | 2008-01-31 |
US7572011B2 true US7572011B2 (en) | 2009-08-11 |
Family
ID=38982180
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/460,611 Active 2026-10-03 US7572011B2 (en) | 2006-07-27 | 2006-07-27 | Contact lenses for use in motion capture |
Country Status (2)
Country | Link |
---|---|
US (1) | US7572011B2 (en) |
WO (1) | WO2008014085A2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11559391B2 (en) * | 2013-11-26 | 2023-01-24 | Strathspey Crown LLC | Intraocular lens having input and output electronics |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8996409B2 (en) | 2007-06-06 | 2015-03-31 | Sony Computer Entertainment Inc. | Management of online trading services using mediated communications |
US8290604B2 (en) * | 2008-08-19 | 2012-10-16 | Sony Computer Entertainment America Llc | Audience-condition based media selection |
US8447421B2 (en) | 2008-08-19 | 2013-05-21 | Sony Computer Entertainment Inc. | Traffic-based media selection |
US8484219B2 (en) | 2010-09-21 | 2013-07-09 | Sony Computer Entertainment America Llc | Developing a knowledge base associated with a user that facilitates evolution of an intelligent user interface |
US8504487B2 (en) | 2010-09-21 | 2013-08-06 | Sony Computer Entertainment America Llc | Evolution of a user interface based on learned idiosyncrasies and collected data of a user |
US8632182B2 (en) * | 2011-05-05 | 2014-01-21 | Sony Computer Entertainment Inc. | Interface using eye tracking contact lenses |
US9641297B2 (en) * | 2012-04-09 | 2017-05-02 | Intel Corporation | Enhancements to wireless networks to support short message service (SMS) communication in the packet switched domain |
US9105178B2 (en) | 2012-12-03 | 2015-08-11 | Sony Computer Entertainment Inc. | Remote dynamic configuration of telemetry reporting through regular expressions |
GB2526380A (en) * | 2014-05-24 | 2015-11-25 | Iriss Medical Technologies Ltd | Contact lens assessment |
EP3243429A1 (en) | 2016-05-13 | 2017-11-15 | Erasmus University Medical Center Rotterdam | Method for measuring a subject's eye movement and scleral contact lens |
CN108664118B (en) * | 2017-03-27 | 2021-06-15 | 腾讯科技(深圳)有限公司 | Eyeball tracking method and device, contact lenses and virtual reality system |
WO2023159526A1 (en) * | 2022-02-25 | 2023-08-31 | 曹庆恒 | Eyeball tracking system and use method therefor |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3804496A (en) * | 1972-12-11 | 1974-04-16 | Stanford Research Inst | Two dimensional eye tracker and method for tracking an eye |
US6152564A (en) * | 1999-12-06 | 2000-11-28 | Bertec Corporation | Infrared eye movement measurement device |
US7021761B2 (en) * | 2002-06-28 | 2006-04-04 | Bausch & Lomb Incorporated | Lens with colored portion and coated surface |
US20070121065A1 (en) * | 2005-03-24 | 2007-05-31 | Cox David D | Device and method for tracking eye gaze direction |
-
2006
- 2006-07-27 US US11/460,611 patent/US7572011B2/en active Active
-
2007
- 2007-06-29 WO PCT/US2007/072584 patent/WO2008014085A2/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3804496A (en) * | 1972-12-11 | 1974-04-16 | Stanford Research Inst | Two dimensional eye tracker and method for tracking an eye |
US6152564A (en) * | 1999-12-06 | 2000-11-28 | Bertec Corporation | Infrared eye movement measurement device |
US7021761B2 (en) * | 2002-06-28 | 2006-04-04 | Bausch & Lomb Incorporated | Lens with colored portion and coated surface |
US20070121065A1 (en) * | 2005-03-24 | 2007-05-31 | Cox David D | Device and method for tracking eye gaze direction |
Non-Patent Citations (4)
Title |
---|
Patent Cooperation Treaty; "International Search Report" issued in corresponding PCT/US07/72584; mailed Aug. 27, 2008; 2 pages (86355PC). |
Patent Cooperation Treaty; "Notification of Transmittal of the International Search Report and the Written Opinion of the International Searching Authority or the Declaration" issued in corresponding PCT/US07/72584; mailed Aug. 27, 2008; 2 pages (86355PC). |
Patent Cooperation Treaty; "Written Opinion of the International Searching Authority" issued in corresponding PCT/US07/72584; mailed Aug. 27, 2008; 3 pages (86355PC). |
Wikipedia, Motion Capture, online article: http://en.wikipedia.org/wiki/Motion-capture. first printed Jul. 13, 2006, 6 pages. |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11559391B2 (en) * | 2013-11-26 | 2023-01-24 | Strathspey Crown LLC | Intraocular lens having input and output electronics |
US11998444B2 (en) | 2013-11-26 | 2024-06-04 | Crown Holdings, Llc | Intraocular lens having input and output electronics |
Also Published As
Publication number | Publication date |
---|---|
US20080024722A1 (en) | 2008-01-31 |
WO2008014085A3 (en) | 2009-04-09 |
WO2008014085A2 (en) | 2008-01-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7572011B2 (en) | Contact lenses for use in motion capture | |
JP7454544B2 (en) | Systems and methods for generating augmented reality and virtual reality images | |
US10410359B2 (en) | Methods for capturing images of markers of a person to control interfacing with an application | |
US20200184700A1 (en) | Apparatus and method for performing motion capture using a random pattern on capture surfaces | |
JP6095763B2 (en) | Gesture registration device, gesture registration program, and gesture registration method | |
CN101681423B (en) | Method of capturing, processing, and rendering images | |
US20190200003A1 (en) | System and method for 3d space-dimension based image processing | |
US10380758B2 (en) | Method for tracking subject head position from monocular-source image sequence | |
CN109801379A (en) | General augmented reality glasses and its scaling method | |
JP2000350859A (en) | Marker arranging method and composite reality really feeling device | |
JP6965277B2 (en) | Methods and scleral contact lenses for measuring subject eye movements | |
CN110269586A (en) | For capturing the device and method in the visual field of the people with dim spot | |
JP5362357B2 (en) | Capture and process facial movement data | |
Wei et al. | Avatar-mediated face tracking and lip reading for human computer interaction | |
US5706421A (en) | Method and system for reproducing an animated image sequence using wide-angle images | |
Zhuang et al. | A modern approach to intelligent animation: theory and practice | |
US20230390622A1 (en) | Sports training aid | |
Jain | Attention-guided algorithms to retarget and augment animations, stills, and videos | |
EP3690703A1 (en) | Postures recognition of objects in augmented reality applications | |
CN115461774A (en) | Method for recommending customized contact lenses and device for implementing method | |
Neumann et al. | Talking heads: Introducing the tool of 3d motion fields in the study of action | |
Knappmeyer et al. | Interactions between facial form and facial motion during the processing of identity |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SONY COMPUTER ENTERTAINMENT AMERICA INC., CALIFORN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:POLLARD, BRAD;REEL/FRAME:018041/0526 Effective date: 20060727 |
|
AS | Assignment |
Owner name: SONY COMPUTER ENTERTAINMENT AMERICA INC., CALIFORN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ZALEWSKI, GARY M.;REEL/FRAME:022889/0562 Effective date: 20090625 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: SONY INTERACTIVE ENTERTAINMENT AMERICA LLC, CALIFO Free format text: CHANGE OF NAME;ASSIGNOR:SONY COMPUTER ENTERTAINMENT AMERICA LLC;REEL/FRAME:038611/0846 Effective date: 20160331 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
AS | Assignment |
Owner name: SONY INTERACTIVE ENTERTAINMENT LLC, CALIFORNIA Free format text: MERGER;ASSIGNOR:SONY INTERACTIVE ENTERTAINMENT AMERICA LLC;REEL/FRAME:053323/0567 Effective date: 20180315 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 12 |