CA2668776A1 - Head-mounted display apparatus for profiling system - Google Patents

Head-mounted display apparatus for profiling system Download PDF

Info

Publication number
CA2668776A1
CA2668776A1 CA002668776A CA2668776A CA2668776A1 CA 2668776 A1 CA2668776 A1 CA 2668776A1 CA 002668776 A CA002668776 A CA 002668776A CA 2668776 A CA2668776 A CA 2668776A CA 2668776 A1 CA2668776 A1 CA 2668776A1
Authority
CA
Canada
Prior art keywords
eye
subsurface medium
image
characterizing
reference system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CA002668776A
Other languages
French (fr)
Other versions
CA2668776C (en
Inventor
Daniel Rioux
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
MICROMENTIS Inc
Original Assignee
Micromentis Inc.
Daniel Rioux
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Micromentis Inc., Daniel Rioux filed Critical Micromentis Inc.
Publication of CA2668776A1 publication Critical patent/CA2668776A1/en
Application granted granted Critical
Publication of CA2668776C publication Critical patent/CA2668776C/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • G02B27/0172Head mounted characterised by optical features
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/239Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B23/00Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
    • G02B23/02Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices involving prisms or mirrors
    • G02B23/10Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices involving prisms or mirrors reflecting into the field of view additional indications, e.g. from collimator
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106Processing image signals
    • H04N13/111Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H04N13/117Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/275Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/332Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/344Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/366Image reproducers using viewer tracking
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/0132Head-up displays characterised by optical features comprising binocular systems
    • G02B2027/0134Head-up displays characterised by optical features comprising binocular systems of stereoscopic type
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/0138Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • G02B2027/0178Eyeglass type
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0179Display position adjusting means not related to the information to be displayed
    • G02B2027/0187Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Astronomy & Astrophysics (AREA)
  • Processing Or Creating Images (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Image Processing (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

The invention provides a head-mounted display to visualize a medium through a surface by displaying an image characterizing the medium under the surface provided by a profiling system and referenced in the real environment of the user. An image of the medium under the surface is projected in front of one or both eyes of a person wearing the head-mounted display, in superimposition with the real environment of the user. The head-mounted display comprises a positioning sensor, such as an inertial positioning sensor, for determining its position and orientation in the real environment. As the user moves around the medium, the image of the medium is updated to display the medium as if it could be seen through the surface. In one embodiment of the invention, the image of the medium under surface is displayed in stereoscopy, the user thereby visualizing the medium in three dimensions.

Claims (24)

1. A head-mounted display apparatus for use by a user to visualize a characterization of a subsurface medium, said display apparatus comprising:

an input for receiving a model characterizing the subsurface medium in a three-dimensional representation, in a reference system, the model being provided using a profiling system;

a positioning sensor for sensing a position and orientation of a first eye of the user in said reference system;

a processing unit for perspectively projecting said model on a first surface located in front of the first eye with said first position and orientation, to provide a first image characterizing the subsurface medium; and a first display system for displaying, on said first surface, said first image characterizing the subsurface medium in superimposition with a first image of a real environment in front of the first eye.
2. The head-mounted display apparatus as claimed in claim 1, further comprising a second display system for displaying, on a second surface located in front of a second eye of the user, a second image characterizing the subsurface medium in superimposition with an image of a real environment in front of the second eye, said processing unit being further for perspectively projecting said model on said second surface to provide said second image characterizing the subsurface medium, the characterization being thereby visualized in stereoscopy.
3. The head-mounted display apparatus as claimed in claim 2, further comprising a first and second camera, one disposed in front of each of the first and the second surface for providing said images of the real environment in front of the first and the second eye, said processing unit being further for superimposing said images characterizing the subsurface medium with said images of the real environment in front of the eyes.
4. The head-mounted display apparatus as claimed in claim 1, wherein said first display system comprises a see-through screen transmitting said image of a real environment, said first image characterizing the subsurface medium being displayed onto said see-through screen.
5. The head-mounted display apparatus as claimed in any of claims 1 to 4, wherein the characterization of the subsurface medium comprises a tomography.
6. The head-mounted display apparatus as claimed in any of claims 1 to 5, wherein said positioning sensor comprises a three-axis accelerometer translation sensor for referencing a position of said first eye in said reference system, and a three-axis accelerometer rotation sensor for referencing an orientation of said first eye in said reference system.
7. A system for use by a user to visualize a characterization of a subsurface medium, the system comprising:

a profiling system for providing the characterization of the subsurface medium;

a three-dimensional model processor for processing said characterization of the subsurface medium to provide a model characterizing the subsurface medium in a three-dimensional graphical representation, in a reference system; and a head-mounted display device having:
an input for receiving said model;

a positioning sensor for sensing a position and orientation of a first eye of the user in said reference system;

a processing unit for perspectively projecting said model on a first surface located in front of the first eye with said position and orientation, to provide a first image characterizing the subsurface medium; and a first display system for displaying, on said first surface, said first image characterizing the subsurface medium in superimposition with an image of a real environment in front of the first eye.
8. The system as claimed in claim 7, wherein said head-mounted display device further has a second display system for displaying, on a second surface located in front of a second eye of the user, a second image characterizing the subsurface medium in superimposition with an image of a real environment in front of the second eye, said processing unit being further for perspectively projecting said model on said second surface to provide said second image characterizing the subsurface medium, the characterization being thereby visualized in stereoscopy.
9. The system as claimed in claim 8, further comprising a first and second camera, one disposed in front of each of the first and the second surfaces for providing said images of a real environment in front of the first and the second eye, said processing unit being further for superimposing said images characterizing the subsurface medium under the surface with said images of the real environment in front of the eyes.
10. The system as claimed in claim 7, wherein said first display system comprises a see-through screen transmitting said image of a real environment, said first image characterizing the subsurface medium being displayed onto said see-through screen.
11. The system as claimed in any of claims 7 to 10, wherein said characterization of the subsurface medium comprises a tomography.
12. The system as claimed in any of claims 7 to 11, wherein said three-dimensional modeling processor comprises a geotechnical-based three-dimensional modeling software.
13. The system as claimed in any of claims 7 to 12, wherein said positioning sensor comprises a three-axis accelerometer translation sensor for referencing a position of said first eye in said reference system, and a three-axis accelerometer rotation sensor for referencing an orientation of said first eye in said reference system.
14. A method for a user to visualize a characterization of a subsurface medium, the method comprising:

providing the characterization of the subsurface medium;

processing said characterization of the subsurface medium to provide a model characterizing the subsurface medium in a three dimensional graphical representation, in a reference system;

sensing a first position and orientation of a first eye of the user in said reference system;

defining a first surface located in front of said first eye;

perspectively projecting said model on a first surface located in front of the first eye to provide a first image characterizing the subsurface medium;

providing an image of a real environment in front of the first eye; and displaying on said first surface said first image characterizing the subsurface medium in superimposition with said image of a real environment in front of the first eye.
15. The method as claimed in claim 14, further comprising:

determining a second position and orientation of the second eye of the user in said reference system with the first sensed position and orientation;

defining a second surface located in front of the second eye with said second position and orientation;

perspectively projecting said model on said second surface to provide a second image characterizing the subsurface medium;

providing an image of a real environment in front of the second eye; and displaying on said second surface said second image characterizing the subsurface medium in superimposition with said image of a real environment in front of the second eye, the characterization being thereby visualized in stereoscopy.
16. The method as claimed in claim 15, further comprising :
acquiring said image of a real environment in front of the first eye; and acquiring said image of a real environment in front of the second eye;
17. The method as claimed in claim 14, further comprising transmitting said image of a real environment through a see-through screen, said displaying comprising displaying said first image characterizing the subsurface medium on said see-through screen.
18. The method as claimed in any of claims 14 to 17, wherein said characterization of the subsurface medium comprises a tomography.
19. The method as claimed in any of claims 14 to 18, wherein said processing comprises using geotechnical-based modeling algorithm.
20. The method as claimed in any of claims 14 to 19, wherein said perspectively projecting comprises:

selecting regions of said subsurface medium having a given characteristic, graphically representing said region to provide a three-dimensional graphical representation, and perspectively projecting said graphical representation on said first surface to provide said first image characterizing the subsurface medium.
21. A head-mounted display apparatus for use by a user to visualize a characterization of a subsurface medium, said display apparatus comprising:

an input for receiving a model characterizing the subsurface medium in a three-dimensional representation, in a reference system;

a positioning sensor for sensing a position and orientation of a first eye of the user in said reference system;

a processing unit for perspectively projecting said model on a first surface located in front of the first eye with said first position and orientation, to provide a first image characterizing the subsurface medium; and a first display system for displaying, on said first surface, said first image characterizing the subsurface medium in superimposition with a first image of a real environment in front of the first eye.
22. A method for referencing a head-mounted display device in a global reference system, the method comprising:

providing three target points disposed in the global reference system and defining a target plane;

displaying a first reticle to a first eye and a second reticle to a second eye of the head mounted display device;

aligning the first and second reticles from one another;

aligning the reticles to a first target point and reading a first position and orientation of the head-mounted display device in a device reference system;

aligning the reticles to a second target point and reading a second position and orientation of the head-mounted display device in a device reference system;

aligning the reticles to a third target point and reading a third position and orientation of the head-mounted display device in a device reference system;
calculating a translation matrix between the global reference system and the device reference system using the first, second and third positions and orientations; and saving the calculated translation matrix in memory.
23. The method as claimed in claim 22, further comprising:

displaying a virtual plane corresponding to the target plane, according to the translation matrix;

aligning the virtual plane with the target plane and reading a forth orientation of the head-mounted display device in a device reference system;

calculating a translation matrix between the global reference system and the device reference system using the forth orientation and the translation matrix;
and saving the calculated rotation matrix in memory.
24. A portable head-mounted display apparatus for use by a user to visualize a characterization of a subsurface medium, said display apparatus comprising:

an input for receiving, from a model processor, a model characterizing the subsurface medium in a three-dimensional graphical representation, in a reference system;

a memory for saving said model, said input to be disconnected from said model processor after saving said model;

a positioning sensor for sensing a position and orientation of the head-mounted display apparatus in said reference system;

a processing unit for determining a pair of stereoscopic images characterizing the subsurface medium, using said model and said position and orientation; and a stereoscopic display systems for displaying, in front of the eyes of the user, said pair of stereoscopic images characterizing the subsurface medium in superimposition with a pair of images of a real environment.
CA2668776A 2007-01-31 2007-01-31 Head-mounted display apparatus for profiling system Expired - Fee Related CA2668776C (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CA2007/000138 WO2008092230A1 (en) 2007-01-31 2007-01-31 Head-mounted display apparatus for profiling system

Publications (2)

Publication Number Publication Date
CA2668776A1 true CA2668776A1 (en) 2008-08-07
CA2668776C CA2668776C (en) 2010-05-04

Family

ID=39673589

Family Applications (1)

Application Number Title Priority Date Filing Date
CA2668776A Expired - Fee Related CA2668776C (en) 2007-01-31 2007-01-31 Head-mounted display apparatus for profiling system

Country Status (7)

Country Link
EP (1) EP2108135A4 (en)
JP (1) JP5118152B2 (en)
CN (1) CN101595417B (en)
AU (1) AU2007345525B2 (en)
CA (1) CA2668776C (en)
MX (1) MX2009007256A (en)
WO (1) WO2008092230A1 (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8417325B2 (en) * 2007-10-12 2013-04-09 Cardiac Pacemakers, Inc. Differentiating decompensation detection based on co-morbidities in heart failure
JP4852062B2 (en) * 2008-03-28 2012-01-11 株式会社東芝 Monocular image display device and monocular image display method
EP2478492B1 (en) * 2009-09-16 2019-03-20 Sydac Pty Ltd Visual presentation system
JP5977922B2 (en) * 2011-02-24 2016-08-24 セイコーエプソン株式会社 Information processing apparatus, information processing apparatus control method, and transmissive head-mounted display apparatus
DE102011115739A1 (en) * 2011-10-11 2013-04-11 Daimler Ag Method for integrating virtual objects in vehicle displays
JP5884576B2 (en) * 2012-03-16 2016-03-15 セイコーエプソン株式会社 Head-mounted display device and method for controlling head-mounted display device
CN105247861B (en) 2013-03-22 2017-11-10 精工爱普生株式会社 Infrared video shows glasses
CN103605209A (en) * 2013-11-05 2014-02-26 中国科学技术大学 Transmission type stereoscopic display glasses device
CN104484033B (en) * 2014-11-21 2017-10-03 上海同筑信息科技有限公司 Virtual reality display method and system based on BIM
CN104581128A (en) * 2014-12-29 2015-04-29 青岛歌尔声学科技有限公司 Head-mounted display device and method for displaying external image information therein
CN104795017B (en) * 2015-04-24 2019-07-19 深圳市虚拟现实科技有限公司 Display control method and head-mounted display apparatus
US9442575B1 (en) * 2015-05-15 2016-09-13 Atheer, Inc. Method and apparatus for applying free space input for surface constrained control

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6094625A (en) * 1997-07-03 2000-07-25 Trimble Navigation Limited Augmented vision for survey work and machine control
JP3653196B2 (en) * 1998-06-30 2005-05-25 飛島建設株式会社 Construction support information system using virtual reality.
US6222675B1 (en) * 1998-12-01 2001-04-24 Kaiser Electro-Optics, Inc. Area of interest head-mounted display using low resolution, wide angle; high resolution, narrow angle; and see-through views
US6536553B1 (en) * 2000-04-25 2003-03-25 The United States Of America As Represented By The Secretary Of The Army Method and apparatus using acoustic sensor for sub-surface object detection and visualization
US6735888B2 (en) * 2001-05-18 2004-05-18 Witten Technologies Inc. Virtual camera on the bucket of an excavator displaying 3D images of buried pipes
US6522474B2 (en) * 2001-06-11 2003-02-18 Eastman Kodak Company Head-mounted optical apparatus for stereoscopic display
CA2366030A1 (en) * 2001-12-20 2003-06-20 Global E Bang Inc. Profiling system
US7292269B2 (en) * 2003-04-11 2007-11-06 Mitsubishi Electric Research Laboratories Context aware projector

Also Published As

Publication number Publication date
EP2108135A4 (en) 2013-02-20
AU2007345525A1 (en) 2008-08-07
JP5118152B2 (en) 2013-01-16
JP2010517187A (en) 2010-05-20
CN101595417A (en) 2009-12-02
WO2008092230A1 (en) 2008-08-07
EP2108135A1 (en) 2009-10-14
MX2009007256A (en) 2009-09-09
CN101595417B (en) 2012-11-28
AU2007345525B2 (en) 2012-03-08
CA2668776C (en) 2010-05-04

Similar Documents

Publication Publication Date Title
CA2668776A1 (en) Head-mounted display apparatus for profiling system
US9201568B2 (en) Three-dimensional tracking of a user control device in a volume
US7830334B2 (en) Image displaying method and apparatus
US10269139B2 (en) Computer program, head-mounted display device, and calibration method
CN105144249B (en) Part installation exercise supports system and part mounting method
US20200363867A1 (en) Blink-based calibration of an optical see-through head-mounted display
SE527257C2 (en) Device and method for presenting an external image
US20120120113A1 (en) Method and apparatus for visualizing 2D product images integrated in a real-world environment
JP2006302034A (en) Image processing method and image processor
JPWO2009153975A1 (en) Electronic mirror device
CN104520905A (en) Three-dimensional environment sharing system, and three-dimensional environment sharing method
JP2022122876A (en) image display system
US20210082176A1 (en) Passthrough visualization
JP2006252468A (en) Image processing method and image processing system
JP4747296B2 (en) How to visualize occluded space
CN110431468A (en) Determine position and the orientation of user's trunk for display system
CN107884930B (en) Head-mounted device and control method
JP4689344B2 (en) Information processing method and information processing apparatus
JP6563802B2 (en) Head mounted display for transportation inspection and head mounted display program for transportation inspection
JP6975001B2 (en) Tilt measuring device and tilt measuring method
WO2018017125A1 (en) Display of supplemental information
JP6890524B2 (en) Attitude control system
JP7465133B2 (en) Information processing device and information processing method
JP2017111724A (en) Head-mounted display for piping
JP2017111721A (en) Head-mounted display for clean room, control method of head-mounted display for clean room, and control program for head-mounted display for clean room

Legal Events

Date Code Title Description
EEER Examination request
MKLA Lapsed

Effective date: 20190131