WO2016182504A1 - A virtual reality headset - Google Patents

A virtual reality headset Download PDF

Info

Publication number
WO2016182504A1
WO2016182504A1 PCT/SG2015/050101 SG2015050101W WO2016182504A1 WO 2016182504 A1 WO2016182504 A1 WO 2016182504A1 SG 2015050101 W SG2015050101 W SG 2015050101W WO 2016182504 A1 WO2016182504 A1 WO 2016182504A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
virtual reality
reality headset
camera
image
Prior art date
Application number
PCT/SG2015/050101
Other languages
French (fr)
Inventor
Bryan Shwo-kang CHOW
Original Assignee
Chow Bryan Shwo-Kang
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chow Bryan Shwo-Kang filed Critical Chow Bryan Shwo-Kang
Priority to PCT/SG2015/050101 priority Critical patent/WO2016182504A1/en
Publication of WO2016182504A1 publication Critical patent/WO2016182504A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/163Wearable computers, e.g. on a belt
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/0138Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/014Head-up displays characterised by optical features comprising information/image processing systems
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0179Display position adjusting means not related to the information to be displayed
    • G02B2027/0187Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye

Definitions

  • the present invention relates to a Virtual Reality headset, in particular, one capable of capturing facial expression.
  • An existing Virtual Reality headset may include a camera but the camera only captures images of surroundings of a user or images of the eyes of a user. Furthermore, such images are insufficient for good representation of the physical state or emotion of the user in a virtual environment accessible to one or more Virtual Reality headsets.
  • Figure 1 illustrates a Virtual Reality headset according to an example of the present disclosure.
  • Figure 2 illustrates another Virtual Reality headset according to an example of the present disclosure.
  • Figure 3 illustrates architecture of a processor according to an example of the present disclosure.
  • a Virtual Reality headset comprising: a frame wearable by a user; one or more displays disposed in the frame and facing eyes of the user; a camera for capturing at least one image of a facial expression including at least part of a mouth of the user, the camera being disposed at a distance away from the user for capturing the at least one image of the facial expression of the user; and a communication interface for transmitting the captured at least one image to a processor configured for generating an image displayable on the one or more displays or another one or more displays based on the at least one image captured by the camera.
  • FIG 1 shows an example Virtual Reality (VR) headset 100 of the present disclosure.
  • the Virtual Reality headset 100 has a frame 102 wearable by a user (the user is not shown in Figure 1 ), one or more displays 128 (not visible and covered by the frame 102 in Figure 1 ) disposed in the frame 102 and facing eyes of the user, and a camera 104 for capturing at least one image of a facial expression including at least part of a mouth of the user.
  • the camera 104 is disposed at a distance 1 12 away from the user for capturing the at least one image of the facial expression of the user.
  • the Virtual Reality headset 100 has a communication interface 1 16, in this case, in the form of a data transfer wire, for transmitting the captured at least one image to a processor 130 configured for generating an image displayable on the one or more displays 128 or another one or more displays (not shown in Figure 1 ; also known herein as "the other one or more displays") based on the at least one image captured by the camera 104.
  • a processor 130 configured for generating an image displayable on the one or more displays 128 or another one or more displays (not shown in Figure 1 ; also known herein as "the other one or more displays" based on the at least one image captured by the camera 104.
  • the distance 1 12 is from the camera 104 to an axis 120 orthogonal to an edge 122 of the frame 102.
  • the camera 104 may be an optical camera typically used on a smartphone device, mobile device or a digital camera capable of capturing at least one image of the facial expression including at least part of the mouth of the user at a relatively close distance away from the user.
  • the at least one image may be one photograph, a series of photographs or a real time video feed.
  • a video or video feed is understood to be a plurality of images.
  • the processor 130 may be a central processing unit of a computer or a gaming device.
  • the camera 104 may include an image sensor based on semiconductor charge-coupled devices (CCD), active pixel sensors in complementary metal-oxide-semiconductor (CMOS), N-type metal-oxide-semiconductor (NMOS, Live MOS) technologies and the like.
  • CCD semiconductor charge-coupled devices
  • CMOS complementary metal-oxide-semiconductor
  • NMOS N-type metal-oxide-semiconductor
  • Live MOS Live MOS
  • the frame 102 includes a head strap 1 14 for strapping the Virtual Reality headset 100 to a head of the user and an eye goggle portion 1 18 for covering both eyes of the user.
  • the one or more displays 128 in the frame 102 refer specifically to two displays 128 for enabling stereoscopic view for 3- dimensional images.
  • the two displays 128 are facing eyes of the user looking into the eye goggle portion 1 18 of the frame 102. More specifically, in this case, each of the two displays 128 faces each eye of the user.
  • the other one or more displays may be one or more displays located in another Virtual Reality headset or one or more displays of or connected to a network connected device such as a laptop, a desktop computer, a smartphone, a tablet, and the like.
  • the processor 130 is configured for processing, in real-time, data of the at least one image captured to map the at least one image captured to a model of the user in a virtual environment displayable in the image displayable on the one or more displays 128 or the other one or more displays. For instance, one way to realise this is to carry out texture mapping of the at least one image captured to the model of the user in the virtual environment. It is appreciated that other methods to map and display the at least one captured image in the virtual environment can also be employed.
  • the model of the user in the virtual environment can be a two dimensional (2D) model or a three dimensional (3D) model.
  • the video or image of the user can be overlaid on a 3D model of the head and face of the user or representative avatar in real time.
  • the at least one image of the captured facial expression including at least part of a mouth of the user can be mapped to a face portion at a location of an avatar representative of the user in the virtual environment, wherein the location of the face portion is corresponding to location of the captured facial expression.
  • the image generated by the processor 130 based on the at least one image captured by the camera 104 will include the avatar representative of the user, or a 2D or 3D model of the user in the virtual environment.
  • the image generated by the processor 130 will be displayed on the other one or more displays instead of the one or more displays 128 to show the avatar representative of the user, or a 2D or 3D model of the user in the virtual environment to another user viewing the other one or more displays.
  • the avatar representative of the user, or a 2D or 3D model of the user in the virtual environment may be displayed in the one or more displays 128 to allow the user to view his/her avatar representation, or 2D or 3D model representation. It is appreciated that both the one or more displays 128 and the other one or more displays can concurrently show the avatar representative of the user, or a 2D or 3D model of the user in the virtual environment.
  • animation may be applied to the generated image including the avatar, or the 2D or 3D model of the user in some instances.
  • the at least one image captured can be mapped without processing or with minimal processing (e.g. only correction or compensation of optical camera distortion or defects) to the face portion of the avatar, or the 2D or 3D model of the user so as to show actual image or footage of the facial expression of the user.
  • Including at least part of the mouth of the user provides clearer indication of a physical state or an emotional state of the user as opposed to capturing the facial expression without the mouth of the user. Capturing at least part of the mouth also provides mouth movement information in the case the user is speaking and this can also be mapped into the virtual environment, in an avatar representative of the user, or in a 2D or 3D model of the user in the virtual environment.
  • the at least part of the mouth of the user can include 20%- 100% of the mouth and may or may not include at least an edge of the mouth.
  • more parts of the mouth should be captured and/or at least an edge of the mouth is captured.
  • the camera 104 is at a location directed to capture the mouth entirely.
  • the virtual environment including the avatar, or 2D or 3D model of the user therein, may be generated by the processor 130 for user interaction in a social media application.
  • the social media application may be a computer-mediated tool that allow people to create, share or exchange information, ideas, and pictures/videos in virtual communities and networks.
  • the social media application may include a group of Internet- or network- based applications that allow creation and exchange of user-generated content.
  • the social media application may depend on mobile and web-based technologies to create highly interactive platforms through which individuals and communities share, co-create, discuss, and modify user-generated content.
  • the virtual environment including the avatar, or 2D or 3D model of the user therein, may also be generated by the processor 130 for user interaction in a game application.
  • the game application may support multi-player modes during game play for displaying an avatar, or a 2D or 3D model of other users logged in through a server to play the game provided by the game application.
  • the virtual environment, including the avatar, or a 2D or 3D model of the user is displayed on the one or more displays 128 viewable by the user.
  • the processor 130 may be configured for compensating distortion of the at least one image captured for generating the image displayable on the one or more displays 128 or the other one or more displays. For instance, perspective distortion in the form of a warping or transformation of an object (in this case, the captured facial expression of the user) and its surrounding area that differs significantly from what the object would look like with a normal focal length may result from the use of wide angle lens. Such distortion correction can be accomplished via available image processing techniques.
  • the camera 104 is mounted to the frame 102.
  • the frame 102 includes an elongate member 1 10 extending away from a face of the user and the camera 104 is mounted on the elongate member 1 10.
  • the elongate member 1 10 includes a connection 108 at one end mounted to the frame 102, and another end holding the camera 104 and directing the camera 104 in a direction for capturing the facial expression of the user.
  • the connection 108 may be a fixed connection on the frame 102 or made detachable from the frame 202.
  • the elongate member 1 10 is moveable to adjust position of the camera 104 with respect to the face of the user.
  • the elongate member 1 10 can be made of a bendable material that maintains shape after bending to enable the position adjustment of the camera 104. This enables the distance 1 12 of the camera 104 from the face of the user to be adjusted.
  • the Virtual Reality headset 100 comprises a microphone 124 (an optional component).
  • the microphone is located adjacent to the camera 104 for capturing sound made by the user.
  • the processor 130 can be configured to synchronize the sound with mouth movements captured by the camera 104 and displayed in the virtual environment.
  • the distance 1 12 may be set at a distance of 1 cm to 10 cm (e.g. 5 cm) away from the face of the user. This is a range of distance observed to be desirable for capturing the facial expression of the user without having the camera 104 located too close or too far away from the user, which may compromise camera image capture, the aesthetics of the Virtual Reality headset 100 or make the Virtual Reality headset 100 inconvenient for handling as the camera 104 can be an obstruction.
  • the user can advantageously adjust the elongate member 1 10 to set the distance 1 12 of the camera 104 away from the face of the user between 0 cm (come into contact with the user) to as far as the elongate member 1 10 can bring the camera 104 away from the face of the user.
  • the minimum of the distance 1 12 that can still allow the camera 104 to function properly to capture facial expression of the user is a function of the specification of the camera, including wideness of wide angle capture of the camera used. It is appreciated that this function can be derived through experimentation. With regard to the distance range of 1 cm to 10 cm, 1 cm is a probable minimum distance that will likely not affect camera image capture (sufficient details of the facial expression of the user should be captured) performed using optical lens typically used in smartphones. mobile devices, and digital cameras. It is noted that 0 cm (i.e. camera contacting user's face) will impede mouth movements and cover an image sensor in the camera that is directed to capture images of the mouth of the user.
  • 10 cm is a comfortable distance to avoid the camera being too far out from the user's face, which may affect aesthetics of the Virtual Reality headset and cause obstruction. It is appreciated that beyond 10 cm is possible as well as long as the facial expression including at least a part of the mouth is captured with sufficient details for processing and displaying in the virtual environment.
  • the communication interface 1 16 may include a power supply line to draw electrical power from the processor 130 to operate the Virtual Reality headset 100.
  • the processor 130 can in turn obtain electrical power from mains electricity or from a portable power supply.
  • the Virtual Reality headset 100 requires electrical power to run the one or more displays 128, the camera 104 and/or the microphone 124. It is appreciated that in another example, the Virtual Reality headset 100 can be configured to draw electrical power from a portable power supply or from both the portable power supply and through the power supply line.
  • the portable power supply may be incorporated in, for instance, the frame 102 of the Virtual Reality headset 100. In this case, the processor 130 may draw power from the portable power supply.
  • FIG 2 shows another example of a Virtual Reality headset 200 similar (i.e. containing the same features) to the Virtual Reality headset 100 in Figure 1 except for one key difference.
  • the key difference is that instead of having the camera 104 and the elongate member 1 10 of the Virtual Reality headset 100, there is in the Virtual Reality headset 200 a camera 204 mounted to a frame 202 at a portion along a line of symmetry 206 of a face of a user (not shown in Figure 2) and the camera 204 is directed in a direction for capturing the facial expression of the user (including at least part of the mouth of the user).
  • the camera 204 can be permanently mounted to the frame 202 or made to be detachable from the frame 202.
  • a distance 212 the same as the distance 1 12 in Figure 1 may be set at a fixed distance of about 1 cm to 10 cm (e.g. 5 cm) away from the face of the user.
  • the minimum of the distance 212 is similarly governed by the function described with reference to the Virtual Reality headset 100 in Figure 1 .
  • the Virtual Reality headset 200 has a communication interface (not visible in Figure 2; operates like the communication interface 1 16 in Figure 1 ) for transmitting the captured at least one image of the facial expression of the user to a processor 208 (operates like the processor 130 in Figure 1 ) configured for processing an image displayable on one or more displays 228 (not visible in Figure 2 and covered by the frame 202 of the Virtual Reality headset 200) or another one or more displays (not shown in Figure 2; similar to the other one or more displays discussed with reference to Figure 1 ) based on the at least one image captured by the camera 204.
  • the one or more displays 228 is similar to the one or more displays 128 in Figure 1 and is located in an eye goggle portion 218 of the frame 202.
  • the camera 204 is directed to capture the mouth entirely. It is appreciated that in another example, the camera 204 may be directed to capture only part of the mouth.
  • the Virtual Reality headset 200 may also comprise a microphone 224 (similar to microphone 124 in Figure 1 ).
  • the microphone 224 is located adjacent to the camera 204 for capturing sound made by the user.
  • the processor 208 can be configured to synchronize the sound with mouth movements captured by the camera 204 and displayed in a virtual environment.
  • the Virtual Reality headset 200 has a processor 208 (not visible in Figure 2; indicated by broken lines for illustration) mounted in or integrated with the electronics of the Virtual Reality headset 200.
  • the processor 208 resides in the eye goggle portion 218 of the frame 202. This is a different location from the processor 130 in Figure 1 , which is residing outside the Virtual Reality headset 200.
  • the communication interface (not shown in Figure 2) of the Virtual Reality headset 200 can similarly be a data transfer wire connecting the processor 208 to the camera 204.
  • the Virtual Reality headset 100 in Figure 1 can also relocate the processor 130 to the same location as the processor 208 and the Virtual Reality headset 200 can have the processor 208 located in the same location as the processor 130 in Figure 1 .
  • the processing performed by the processor 208 are the same as the processing performed by the processor 130.
  • a further difference between the Virtual Reality headset of 100 in Figure 1 and the Virtual Reality headset 200 is that the Virtual Reality headset 200 has a portable power supply (not shown in Figure 2; can be located anywhere in, for instance, the frame 202 of the Virtual Reality headset 200 and may be connected to the processor 208) incorporated into the Virtual Reality headset 200 for operating the Virtual Reality headset 200.
  • the Virtual Reality headset 200 requires electrical power to run the one or more displays 228, the camera 204, the processor 208, and/or the microphone 224.
  • the Virtual Reality headset 200 can be configured to draw electrical power from the processor 208 through a power supply line like the Virtual Reality headset 100 in Figure.
  • the Virtual Reality headset 200 can be configured to draw electrical power from both the portable power supply and through the power supply line.
  • the processor 208 can in turn obtain electrical power from mains electricity or from a portable power supply.
  • FIG. 1 there can be a Virtual Reality headset having both the cameras 104 in Figure 1 and 204 in Figure 2 as described to ensure better image capture.
  • the Virtual Reality headset may also have both the processor 130 in Figure 1 and the processor 208 in Figure 2 as described.
  • the processor 130 in Figure 1 and the processor 208 in Figure 2 may process different things (i.e. software) or share processing load.
  • the respective communication interfaces of the Virtual Reality headsets 100 in Figure 1 and 200 in Figure 2 may be a wireless data transmitter instead of a data transfer wire for transferring data wirelessly between the camera 104 and the processor 130 or the camera 204 and the processor 208 respectively.
  • the wireless data communication may be based on proprietary wireless data communication standards, Bluetooth, WiFi, Near Field Communication (NFC), Digital Living Network Alliance (DLNA) wireless technology, and the like.
  • the stereoscopic views can provide input to the respective processors 130 and 208 of the Virtual Reality headsets 100 and 200 for generating an image or video containing 3D content displayable on one or more displays (this can be the one or more displays 128 or 228 of the respective Virtual Reality headsets 100 or 200 or the respective other one or more displays).
  • a user may view the 3D content displayed on the one or more displays 128 or 228 of the respective Virtual Reality headsets 100 or 200, or on the respective other one or more displays through 3D lens.
  • Such 3D lens may be provided in the eye goggle portion 1 18 in Figure 1 or 218 in Figure 2 of the respective Virtual Reality Headset 100 or 200 to enable the user to view 3D content displayed on the one or more displays 128 or 228 of the respective Virtual Reality Headset 100 or 200.
  • one or more cameras that include any type of image sensor for capturing a facial expression of a user.
  • the one or more cameras is/are mounted on or integrated to, and form part of a Virtual Reality (VR) headset for capturing facial expressions including mouth movements of the user.
  • a video, including audio captured through a microphone mounted to the VR headset are transmitted, preferably in real-time, to a processor for processing, for instance, to render the facial expressions including mouth movements of the user onto an avatar, 2D or 3D model of the user in a virtual environment.
  • the virtual environment including the avatar, 2D or 3D model of the user is generated by the processor as a video that is transmitted across a network (including the Internet or Intranet) for displaying in one or more displays of the remote VR headset of another user.
  • a network including the Internet or Intranet
  • Examples of the present disclosure may have one or more of the following advantages.
  • Existing Virtual Reality (VR) headsets are targeted to provide "singular" experiences for a single participant.
  • an example VR headset of the present disclosure can advantageously convey facial expressions of each of the multiple participants by having a camera to capture facial expressions of each of the multiple participants when needed.
  • an example VR headset of the present disclosure can capture images of the facial expression of the user and then process the images for reconstruction and/or overlaying in a virtual environment. This can provide a better representation of the facial expressions.
  • An example VR headset of the present disclosure can advantageously provide a camera mounted to an elongate member adjustable to move the position of the camera or by positioning the camera at a fixed location along a line of symmetry of the face of the user suitable for capturing at least an image of a facial expression of the user.
  • a camera is mounted on the headset in a manner directed in the direction of the mouth area of the user to capture at least one image that may be a video feed of the facial expression of the user.
  • the at least one image or video feed can then be processed by a processor and overlaid on a 3D model of the user in a virtual environment that is displayed remotely to the user via one or more displays of the VR headset and/or one or more displays accessible to one or more participants in the virtual environment.
  • the Virtual Reality headset can capture at least one images of a facial expression including at least a part of the mouth of a user remotely and direct data of the at least one images to a processor for processing to enable the at least one images to be overlaid in a virtual environment for purposes of virtual reality telepresence.
  • FIG 3 shows in more detail an example of the processor 130 in Figure 1 and the processor 208 in Figure 2.
  • the processor 130 and the processor 208 may each comprise a processing unit 302 for processing software including one or more computer programs for running one or more computer applications to enable processing of, in realtime, data of at least one image captured to map the at least one image captured to a model of the user in a virtual environment displayable in the image displayable on the one or more displays (128 in Figure 1 or 228 in Figure 2) or the other one or more displays as described earlier.
  • Other processing enabled are compensating distortion of the at least one image captured for generating the image displayable on the one or more displays (128 in Figure 1 or 228 in Figure 2) or the other one or more displays as described earlier, generating the virtual environment including the avatar, the 2D or 3D model of the user as described earlier, generating the virtual environment for user interaction in a game application and/or generating the virtual environment for user interaction in a social media application.
  • the processor 130 in Figure 1 and the processor 208 in Figure 2 may reside in a gaming machine, a mobile device, a tablet device, a desktop or portable computer, and the like.
  • the processing unit 302 may include user input modules such as a computer mouse 336, keyboard/keypad 304, and/or a plurality of output devices such as a display 308.
  • the display 308 may be a touch screen capable of receiving user input as well. It is appreciated that in another example, another display (not shown) like the display 308 may be included.
  • the processing unit 302 may be connected to a computer network 312 via a suitable transceiver device 314 (i.e. a network interface), to enable access to e.g. the Internet or other network systems such as a wired Local Area Network (LAN) or Wide Area Network (WAN).
  • LAN Local Area Network
  • WAN Wide Area Network
  • the processing unit 302 may be connected to one or more external wireless communication enabled devices 334 via a suitable wireless transceiver device 332 e.g. a WiFi transceiver, Bluetooth module, Mobile telecommunication transceiver suitable for Global System for Mobile Communication (GSM), 3G, 3.5G, 4G telecommunication systems, and the like.
  • a suitable wireless transceiver device 332 e.g. a WiFi transceiver, Bluetooth module, Mobile telecommunication transceiver suitable for Global System for Mobile Communication (GSM), 3G, 3.5G, 4G telecommunication systems, and the like.
  • GSM Global System for Mobile Communication
  • the processing unit 302 may include a processor integrated circuit 318, a Random Access Memory (RAM) 320 and a Read Only Memory (ROM) 322.
  • RAM Random Access Memory
  • ROM Read Only Memory
  • the processing unit 302 may also include a number of Input/Output (I/O) interfaces, for example I/O interface 338 to the computer mouse 336, a memory card slot 316, I/O interface 324 to the display 308, and I/O interface 326 to the keyboard/keypad 304.
  • I/O Input/Output
  • the components of the processing unit 302 typically communicate via an interconnected bus 528 and in a manner known to the person skilled in the relevant art.
  • the computer programs may further include one or more software applications for e.g. image processing, instant messaging platform, audio/video playback, internet accessibility, operating the processor 130 and the processor 208 (i.e. operating system), network security, file accessibility, database management, which are applications typically equipped on a desktop, electronic apparatus, mobile device or portable computer.
  • software applications for e.g. image processing, instant messaging platform, audio/video playback, internet accessibility, operating the processor 130 and the processor 208 (i.e. operating system), network security, file accessibility, database management, which are applications typically equipped on a desktop, electronic apparatus, mobile device or portable computer.
  • the computer programs may be supplied to the user of the processor 130 and the processor 208 encoded on a data storage medium such as a CD-ROM, on a flash memory carrier or a Hard Disk Drive, and are to be read using a corresponding data storage medium drive of a data storage device 330.
  • a data storage medium such as a CD-ROM, on a flash memory carrier or a Hard Disk Drive
  • Such computer or application programs may also be downloaded from the computer network 312.
  • the application programs are read and controlled in its execution by the processor integrated circuit 318.
  • Intermediate storage of program data may be accomplished using RAM 320.
  • one or more of the computer or application programs may be stored on any non-transitory machine- or computer- readable medium.
  • the machine- or computer- readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a general purpose computer.
  • the machine- or computer- readable medium may also include a hard-wired medium such as that exemplified in the Internet system, or wireless medium such as that exemplified in the Wireless LAN (WLAN) system and the like.
  • WLAN Wireless LAN

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Hardware Design (AREA)
  • Optics & Photonics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

A Virtual Reality headset, the Virtual Reality headset comprising: a frame wearable by a user; one or more displays disposed in the frame and facing eyes of the user; a camera for capturing at least one image of a facial expression including at least part of a mouth of the user, the camera being disposed at a distance away from the user for capturing the at least one image of the facial expression of the user; a communication interface for transmitting the captured at least one image to a processor configured for generating an image displayable on the one or more displays or another one or more displays based on the at least one image captured by the camera.

Description

A Virtual Reality Headset
Field The present invention relates to a Virtual Reality headset, in particular, one capable of capturing facial expression.
Background An existing Virtual Reality headset may include a camera but the camera only captures images of surroundings of a user or images of the eyes of a user. Furthermore, such images are insufficient for good representation of the physical state or emotion of the user in a virtual environment accessible to one or more Virtual Reality headsets. Brief Description of the Drawings
Various embodiments of the present disclosure will now be described, by way of example only, and with reference to the accompanying drawings in which: Figure 1 illustrates a Virtual Reality headset according to an example of the present disclosure.
Figure 2 illustrates another Virtual Reality headset according to an example of the present disclosure.
Figure 3 illustrates architecture of a processor according to an example of the present disclosure.
Summary
According to one aspect of an example of the present invention, there is provided a Virtual Reality headset, the Virtual Reality headset comprising: a frame wearable by a user; one or more displays disposed in the frame and facing eyes of the user; a camera for capturing at least one image of a facial expression including at least part of a mouth of the user, the camera being disposed at a distance away from the user for capturing the at least one image of the facial expression of the user; and a communication interface for transmitting the captured at least one image to a processor configured for generating an image displayable on the one or more displays or another one or more displays based on the at least one image captured by the camera.
Detailed Description
Figure 1 shows an example Virtual Reality (VR) headset 100 of the present disclosure. The Virtual Reality headset 100 has a frame 102 wearable by a user (the user is not shown in Figure 1 ), one or more displays 128 (not visible and covered by the frame 102 in Figure 1 ) disposed in the frame 102 and facing eyes of the user, and a camera 104 for capturing at least one image of a facial expression including at least part of a mouth of the user. The camera 104 is disposed at a distance 1 12 away from the user for capturing the at least one image of the facial expression of the user. The Virtual Reality headset 100 has a communication interface 1 16, in this case, in the form of a data transfer wire, for transmitting the captured at least one image to a processor 130 configured for generating an image displayable on the one or more displays 128 or another one or more displays (not shown in Figure 1 ; also known herein as "the other one or more displays") based on the at least one image captured by the camera 104. As a head of the user will contact the frame 102, in this example, the distance 1 12 is from the camera 104 to an axis 120 orthogonal to an edge 122 of the frame 102.
For example, the camera 104 may be an optical camera typically used on a smartphone device, mobile device or a digital camera capable of capturing at least one image of the facial expression including at least part of the mouth of the user at a relatively close distance away from the user. The at least one image may be one photograph, a series of photographs or a real time video feed. In the present disclosure, a video or video feed is understood to be a plurality of images. The processor 130 may be a central processing unit of a computer or a gaming device. As an example, the camera 104 may include an image sensor based on semiconductor charge-coupled devices (CCD), active pixel sensors in complementary metal-oxide-semiconductor (CMOS), N-type metal-oxide-semiconductor (NMOS, Live MOS) technologies and the like.
In the present example, the frame 102 includes a head strap 1 14 for strapping the Virtual Reality headset 100 to a head of the user and an eye goggle portion 1 18 for covering both eyes of the user. Furthermore, in the present example, the one or more displays 128 in the frame 102 refer specifically to two displays 128 for enabling stereoscopic view for 3- dimensional images. The two displays 128 are facing eyes of the user looking into the eye goggle portion 1 18 of the frame 102. More specifically, in this case, each of the two displays 128 faces each eye of the user. The other one or more displays (not shown in Figure 1 ) mentioned earlier may be one or more displays located in another Virtual Reality headset or one or more displays of or connected to a network connected device such as a laptop, a desktop computer, a smartphone, a tablet, and the like.
In the present example, the processor 130 is configured for processing, in real-time, data of the at least one image captured to map the at least one image captured to a model of the user in a virtual environment displayable in the image displayable on the one or more displays 128 or the other one or more displays. For instance, one way to realise this is to carry out texture mapping of the at least one image captured to the model of the user in the virtual environment. It is appreciated that other methods to map and display the at least one captured image in the virtual environment can also be employed. The model of the user in the virtual environment can be a two dimensional (2D) model or a three dimensional (3D) model. For instance, the video or image of the user can be overlaid on a 3D model of the head and face of the user or representative avatar in real time. More specifically, the at least one image of the captured facial expression including at least part of a mouth of the user can be mapped to a face portion at a location of an avatar representative of the user in the virtual environment, wherein the location of the face portion is corresponding to location of the captured facial expression.
The image generated by the processor 130 based on the at least one image captured by the camera 104 will include the avatar representative of the user, or a 2D or 3D model of the user in the virtual environment. In some instances, the image generated by the processor 130 will be displayed on the other one or more displays instead of the one or more displays 128 to show the avatar representative of the user, or a 2D or 3D model of the user in the virtual environment to another user viewing the other one or more displays. However, in other instances, the avatar representative of the user, or a 2D or 3D model of the user in the virtual environment may be displayed in the one or more displays 128 to allow the user to view his/her avatar representation, or 2D or 3D model representation. It is appreciated that both the one or more displays 128 and the other one or more displays can concurrently show the avatar representative of the user, or a 2D or 3D model of the user in the virtual environment.
It is appreciated that animation may be applied to the generated image including the avatar, or the 2D or 3D model of the user in some instances. In other instances, the at least one image captured can be mapped without processing or with minimal processing (e.g. only correction or compensation of optical camera distortion or defects) to the face portion of the avatar, or the 2D or 3D model of the user so as to show actual image or footage of the facial expression of the user.
Including at least part of the mouth of the user provides clearer indication of a physical state or an emotional state of the user as opposed to capturing the facial expression without the mouth of the user. Capturing at least part of the mouth also provides mouth movement information in the case the user is speaking and this can also be mapped into the virtual environment, in an avatar representative of the user, or in a 2D or 3D model of the user in the virtual environment. For example, the at least part of the mouth of the user can include 20%- 100% of the mouth and may or may not include at least an edge of the mouth. For better indication of a physical state or an emotional state of the user, more parts of the mouth should be captured and/or at least an edge of the mouth is captured. In the example of Figure 1 , the camera 104 is at a location directed to capture the mouth entirely.
The virtual environment, including the avatar, or 2D or 3D model of the user therein, may be generated by the processor 130 for user interaction in a social media application. The social media application may be a computer-mediated tool that allow people to create, share or exchange information, ideas, and pictures/videos in virtual communities and networks. The social media application may include a group of Internet- or network- based applications that allow creation and exchange of user-generated content. Furthermore, the social media application may depend on mobile and web-based technologies to create highly interactive platforms through which individuals and communities share, co-create, discuss, and modify user-generated content.
The virtual environment, including the avatar, or 2D or 3D model of the user therein, may also be generated by the processor 130 for user interaction in a game application. The game application may support multi-player modes during game play for displaying an avatar, or a 2D or 3D model of other users logged in through a server to play the game provided by the game application. Depending on the game type or in the case of a single player mode, it is possible that the virtual environment, including the avatar, or a 2D or 3D model of the user is displayed on the one or more displays 128 viewable by the user.
Furthermore, the processor 130 may be configured for compensating distortion of the at least one image captured for generating the image displayable on the one or more displays 128 or the other one or more displays. For instance, perspective distortion in the form of a warping or transformation of an object (in this case, the captured facial expression of the user) and its surrounding area that differs significantly from what the object would look like with a normal focal length may result from the use of wide angle lens. Such distortion correction can be accomplished via available image processing techniques.
In the present example, the camera 104 is mounted to the frame 102. The frame 102 includes an elongate member 1 10 extending away from a face of the user and the camera 104 is mounted on the elongate member 1 10. The elongate member 1 10 includes a connection 108 at one end mounted to the frame 102, and another end holding the camera 104 and directing the camera 104 in a direction for capturing the facial expression of the user. The connection 108 may be a fixed connection on the frame 102 or made detachable from the frame 202. The elongate member 1 10 is moveable to adjust position of the camera 104 with respect to the face of the user. The elongate member 1 10 can be made of a bendable material that maintains shape after bending to enable the position adjustment of the camera 104. This enables the distance 1 12 of the camera 104 from the face of the user to be adjusted.
The Virtual Reality headset 100 comprises a microphone 124 (an optional component). In the present example, the microphone is located adjacent to the camera 104 for capturing sound made by the user. The processor 130 can be configured to synchronize the sound with mouth movements captured by the camera 104 and displayed in the virtual environment.
The distance 1 12 may be set at a distance of 1 cm to 10 cm (e.g. 5 cm) away from the face of the user. This is a range of distance observed to be desirable for capturing the facial expression of the user without having the camera 104 located too close or too far away from the user, which may compromise camera image capture, the aesthetics of the Virtual Reality headset 100 or make the Virtual Reality headset 100 inconvenient for handling as the camera 104 can be an obstruction. In the present example, the user can advantageously adjust the elongate member 1 10 to set the distance 1 12 of the camera 104 away from the face of the user between 0 cm (come into contact with the user) to as far as the elongate member 1 10 can bring the camera 104 away from the face of the user. However, it is noted that the minimum of the distance 1 12 that can still allow the camera 104 to function properly to capture facial expression of the user is a function of the specification of the camera, including wideness of wide angle capture of the camera used. It is appreciated that this function can be derived through experimentation. With regard to the distance range of 1 cm to 10 cm, 1 cm is a probable minimum distance that will likely not affect camera image capture (sufficient details of the facial expression of the user should be captured) performed using optical lens typically used in smartphones. mobile devices, and digital cameras. It is noted that 0 cm (i.e. camera contacting user's face) will impede mouth movements and cover an image sensor in the camera that is directed to capture images of the mouth of the user. 10 cm is a comfortable distance to avoid the camera being too far out from the user's face, which may affect aesthetics of the Virtual Reality headset and cause obstruction. It is appreciated that beyond 10 cm is possible as well as long as the facial expression including at least a part of the mouth is captured with sufficient details for processing and displaying in the virtual environment.
The communication interface 1 16 may include a power supply line to draw electrical power from the processor 130 to operate the Virtual Reality headset 100. The processor 130 can in turn obtain electrical power from mains electricity or from a portable power supply. The Virtual Reality headset 100 requires electrical power to run the one or more displays 128, the camera 104 and/or the microphone 124. It is appreciated that in another example, the Virtual Reality headset 100 can be configured to draw electrical power from a portable power supply or from both the portable power supply and through the power supply line. The portable power supply may be incorporated in, for instance, the frame 102 of the Virtual Reality headset 100. In this case, the processor 130 may draw power from the portable power supply.
Figure 2 shows another example of a Virtual Reality headset 200 similar (i.e. containing the same features) to the Virtual Reality headset 100 in Figure 1 except for one key difference. The key difference is that instead of having the camera 104 and the elongate member 1 10 of the Virtual Reality headset 100, there is in the Virtual Reality headset 200 a camera 204 mounted to a frame 202 at a portion along a line of symmetry 206 of a face of a user (not shown in Figure 2) and the camera 204 is directed in a direction for capturing the facial expression of the user (including at least part of the mouth of the user). The camera 204 can be permanently mounted to the frame 202 or made to be detachable from the frame 202.
In this case, a distance 212 the same as the distance 1 12 in Figure 1 may be set at a fixed distance of about 1 cm to 10 cm (e.g. 5 cm) away from the face of the user. The minimum of the distance 212 is similarly governed by the function described with reference to the Virtual Reality headset 100 in Figure 1 .
Furthermore, similarly, the Virtual Reality headset 200 has a communication interface (not visible in Figure 2; operates like the communication interface 1 16 in Figure 1 ) for transmitting the captured at least one image of the facial expression of the user to a processor 208 (operates like the processor 130 in Figure 1 ) configured for processing an image displayable on one or more displays 228 (not visible in Figure 2 and covered by the frame 202 of the Virtual Reality headset 200) or another one or more displays (not shown in Figure 2; similar to the other one or more displays discussed with reference to Figure 1 ) based on the at least one image captured by the camera 204. The one or more displays 228 is similar to the one or more displays 128 in Figure 1 and is located in an eye goggle portion 218 of the frame 202. In the example of Figure 2, the camera 204 is directed to capture the mouth entirely. It is appreciated that in another example, the camera 204 may be directed to capture only part of the mouth.
The Virtual Reality headset 200 may also comprise a microphone 224 (similar to microphone 124 in Figure 1 ). In the present example, the microphone 224 is located adjacent to the camera 204 for capturing sound made by the user. Similarly, the processor 208 can be configured to synchronize the sound with mouth movements captured by the camera 204 and displayed in a virtual environment.
Another difference between the Virtual Reality headset 100 in Figure 1 and the Virtual Reality headset 200 is that the Virtual Reality headset 200 has a processor 208 (not visible in Figure 2; indicated by broken lines for illustration) mounted in or integrated with the electronics of the Virtual Reality headset 200. In this case, more specifically, the processor 208 resides in the eye goggle portion 218 of the frame 202. This is a different location from the processor 130 in Figure 1 , which is residing outside the Virtual Reality headset 200. The communication interface (not shown in Figure 2) of the Virtual Reality headset 200 can similarly be a data transfer wire connecting the processor 208 to the camera 204. It is appreciated that the Virtual Reality headset 100 in Figure 1 can also relocate the processor 130 to the same location as the processor 208 and the Virtual Reality headset 200 can have the processor 208 located in the same location as the processor 130 in Figure 1 . The processing performed by the processor 208 are the same as the processing performed by the processor 130. A further difference between the Virtual Reality headset of 100 in Figure 1 and the Virtual Reality headset 200 is that the Virtual Reality headset 200 has a portable power supply (not shown in Figure 2; can be located anywhere in, for instance, the frame 202 of the Virtual Reality headset 200 and may be connected to the processor 208) incorporated into the Virtual Reality headset 200 for operating the Virtual Reality headset 200. The Virtual Reality headset 200 requires electrical power to run the one or more displays 228, the camera 204, the processor 208, and/or the microphone 224. It is appreciated that in another example, where the processor 208 is at the same location as the processor 130 in Figure 1 , the Virtual Reality headset 200 can be configured to draw electrical power from the processor 208 through a power supply line like the Virtual Reality headset 100 in Figure. In yet another example, the Virtual Reality headset 200 can be configured to draw electrical power from both the portable power supply and through the power supply line. In a further example, the processor 208 can in turn obtain electrical power from mains electricity or from a portable power supply.
In another example, there can be a Virtual Reality headset having both the cameras 104 in Figure 1 and 204 in Figure 2 as described to ensure better image capture. The Virtual Reality headset may also have both the processor 130 in Figure 1 and the processor 208 in Figure 2 as described. In this case, the processor 130 in Figure 1 and the processor 208 in Figure 2 may process different things (i.e. software) or share processing load.
It is appreciated that the respective communication interfaces of the Virtual Reality headsets 100 in Figure 1 and 200 in Figure 2 may be a wireless data transmitter instead of a data transfer wire for transferring data wirelessly between the camera 104 and the processor 130 or the camera 204 and the processor 208 respectively. The wireless data communication may be based on proprietary wireless data communication standards, Bluetooth, WiFi, Near Field Communication (NFC), Digital Living Network Alliance (DLNA) wireless technology, and the like.
It is appreciated that in another example incorporating all or some of the features of the Virtual Reality headsets 100 and 200 described with reference to Figures 1 and 2 respectively, there can be a modification to have one camera for capturing the at least one image from a first angle and one or more camera (not shown in the Figures) for capturing at least one other image of the facial expression of the user from a second angle different from the first angle. This is for capturing stereoscopic views of the facial expressions of the user. The stereoscopic views can provide input to the respective processors 130 and 208 of the Virtual Reality headsets 100 and 200 for generating an image or video containing 3D content displayable on one or more displays (this can be the one or more displays 128 or 228 of the respective Virtual Reality headsets 100 or 200 or the respective other one or more displays). A user may view the 3D content displayed on the one or more displays 128 or 228 of the respective Virtual Reality headsets 100 or 200, or on the respective other one or more displays through 3D lens. Such 3D lens may be provided in the eye goggle portion 1 18 in Figure 1 or 218 in Figure 2 of the respective Virtual Reality Headset 100 or 200 to enable the user to view 3D content displayed on the one or more displays 128 or 228 of the respective Virtual Reality Headset 100 or 200. In other examples of the present disclosure, there may be included one or more cameras that include any type of image sensor for capturing a facial expression of a user. The one or more cameras is/are mounted on or integrated to, and form part of a Virtual Reality (VR) headset for capturing facial expressions including mouth movements of the user. A video, including audio captured through a microphone mounted to the VR headset are transmitted, preferably in real-time, to a processor for processing, for instance, to render the facial expressions including mouth movements of the user onto an avatar, 2D or 3D model of the user in a virtual environment. The virtual environment including the avatar, 2D or 3D model of the user is generated by the processor as a video that is transmitted across a network (including the Internet or Intranet) for displaying in one or more displays of the remote VR headset of another user. Examples of the present disclosure may have one or more of the following advantages. Existing Virtual Reality (VR) headsets are targeted to provide "singular" experiences for a single participant. For a virtual environment (or VR environment) with multiple participants, an example VR headset of the present disclosure can advantageously convey facial expressions of each of the multiple participants by having a camera to capture facial expressions of each of the multiple participants when needed.
Although mouth movements can be simulated based on audio data, more complicated expressions such as a smile or frown, and actions such as a kiss, cannot be detected and reconstructed based on existing head tracking and audio data provided by an existing VR headset. However, an example VR headset of the present disclosure can capture images of the facial expression of the user and then process the images for reconstruction and/or overlaying in a virtual environment. This can provide a better representation of the facial expressions.
Furthermore, a fixed camera (such as one mounted on a desk, a monitor or display of a device) cannot reliably capture facial expressions and can also be obstructed by the headset itself. An example VR headset of the present disclosure can advantageously provide a camera mounted to an elongate member adjustable to move the position of the camera or by positioning the camera at a fixed location along a line of symmetry of the face of the user suitable for capturing at least an image of a facial expression of the user.
It has been discovered that most facial expressions are conveyed in a lower half of a face of a user below the eyes. In an example VR headset of the present disclosure, advantageously, a camera is mounted on the headset in a manner directed in the direction of the mouth area of the user to capture at least one image that may be a video feed of the facial expression of the user. The at least one image or video feed can then be processed by a processor and overlaid on a 3D model of the user in a virtual environment that is displayed remotely to the user via one or more displays of the VR headset and/or one or more displays accessible to one or more participants in the virtual environment.
Advantageously, the Virtual Reality headset can capture at least one images of a facial expression including at least a part of the mouth of a user remotely and direct data of the at least one images to a processor for processing to enable the at least one images to be overlaid in a virtual environment for purposes of virtual reality telepresence.
Figure 3 shows in more detail an example of the processor 130 in Figure 1 and the processor 208 in Figure 2. The processor 130 and the processor 208 may each comprise a processing unit 302 for processing software including one or more computer programs for running one or more computer applications to enable processing of, in realtime, data of at least one image captured to map the at least one image captured to a model of the user in a virtual environment displayable in the image displayable on the one or more displays (128 in Figure 1 or 228 in Figure 2) or the other one or more displays as described earlier. Other processing enabled are compensating distortion of the at least one image captured for generating the image displayable on the one or more displays (128 in Figure 1 or 228 in Figure 2) or the other one or more displays as described earlier, generating the virtual environment including the avatar, the 2D or 3D model of the user as described earlier, generating the virtual environment for user interaction in a game application and/or generating the virtual environment for user interaction in a social media application. The processor 130 in Figure 1 and the processor 208 in Figure 2 may reside in a gaming machine, a mobile device, a tablet device, a desktop or portable computer, and the like.
Furthermore, the processing unit 302 may include user input modules such as a computer mouse 336, keyboard/keypad 304, and/or a plurality of output devices such as a display 308. The display 308 may be a touch screen capable of receiving user input as well. It is appreciated that in another example, another display (not shown) like the display 308 may be included. The processing unit 302 may be connected to a computer network 312 via a suitable transceiver device 314 (i.e. a network interface), to enable access to e.g. the Internet or other network systems such as a wired Local Area Network (LAN) or Wide Area Network (WAN). Optionally, the processing unit 302 may be connected to one or more external wireless communication enabled devices 334 via a suitable wireless transceiver device 332 e.g. a WiFi transceiver, Bluetooth module, Mobile telecommunication transceiver suitable for Global System for Mobile Communication (GSM), 3G, 3.5G, 4G telecommunication systems, and the like. Through the computer network 312, the processing unit 302 can gain access to one or more storages i.e. data storages, databases, data servers and the like connectable to the computer network 312 to retrieve and/or store data in the one or more storages. The processing unit 302 may include a processor integrated circuit 318, a Random Access Memory (RAM) 320 and a Read Only Memory (ROM) 322. The processing unit 302 may also include a number of Input/Output (I/O) interfaces, for example I/O interface 338 to the computer mouse 336, a memory card slot 316, I/O interface 324 to the display 308, and I/O interface 326 to the keyboard/keypad 304.
The components of the processing unit 302 typically communicate via an interconnected bus 528 and in a manner known to the person skilled in the relevant art.
The computer programs may further include one or more software applications for e.g. image processing, instant messaging platform, audio/video playback, internet accessibility, operating the processor 130 and the processor 208 (i.e. operating system), network security, file accessibility, database management, which are applications typically equipped on a desktop, electronic apparatus, mobile device or portable computer.
The computer programs may be supplied to the user of the processor 130 and the processor 208 encoded on a data storage medium such as a CD-ROM, on a flash memory carrier or a Hard Disk Drive, and are to be read using a corresponding data storage medium drive of a data storage device 330. Such computer or application programs may also be downloaded from the computer network 312. The application programs are read and controlled in its execution by the processor integrated circuit 318. Intermediate storage of program data may be accomplished using RAM 320.
In more detail, one or more of the computer or application programs may be stored on any non-transitory machine- or computer- readable medium. The machine- or computer- readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a general purpose computer. The machine- or computer- readable medium may also include a hard-wired medium such as that exemplified in the Internet system, or wireless medium such as that exemplified in the Wireless LAN (WLAN) system and the like. The computer program when loaded and executed on such a general-purpose computer effectively results in an apparatus that implements the steps of the computing methods in examples herein described.
Many modifications and other examples can be made to the Virtual Reality headset by those skilled in the art having the understanding of the above described disclosure together with the drawings. Therefore, it is to be understood that the Virtual Reality headset is not to be limited to the above description contained herein only, and that possible modifications are to be included in the claims of the disclosure.

Claims

Claims
1 . A Virtual Reality headset, the Virtual Reality headset comprising:
a frame wearable by a user;
one or more displays disposed in the frame and facing eyes of the user;
a camera for capturing at least one image of a facial expression including at least part of a mouth of the user, the camera being disposed at a distance away from the user for capturing the at least one image of the facial expression of the user; and
a communication interface for transmitting the captured at least one image to a processor configured for generating an image displayable on the one or more displays or another one or more displays based on the at least one image captured by the camera.
2. The Virtual Reality headset as claimed in claim 1 , wherein the processor is configured for processing, in real-time, data of the at least one image captured to map the at least one image captured to a model of the user in a virtual environment displayable in the image displayable on the one or more displays or the other one or more displays.
3. The Virtual Reality headset as claimed in claim 2, wherein the model of the user in the virtual environment is a three dimensional model.
4. The Virtual Reality headset as claimed in any one of the preceding claims, wherein the camera is mounted to the frame.
5. The Virtual Reality headset as claimed in any one of the preceding claims, wherein the frame comprises an elongate member extending away from a face of the user and the camera is mounted on the elongate member.
6. The Virtual Reality headset as claimed in claim 5, wherein the elongate member is moveable to adjust position of the camera with respect to the face of the user.
7. The Virtual Reality headset as claimed in any one of the preceding claims, wherein the camera is mounted to the frame at a portion along a line of symmetry of a face of the user and the camera is directed in a direction for capturing the facial expression of the user.
8. The Virtual Reality headset as claimed in any one of the preceding claims, wherein the frame comprises the processor.
9. The Virtual Reality headset as claimed in any one of the preceding claims, wherein the communication interface is a wireless data transmitter.
10. The Virtual Reality headset as claimed in any one of the preceding claims, wherein the Virtual Reality headset comprises a microphone.
1 1 . The Virtual Reality headset as claimed in any one of the preceding claims, wherein the distance of the camera away from the face of the user is 1 cm to 10 cm.
12. The Virtual Reality headset as claimed in any one of the preceding claims, wherein the camera captures the at least one image from a first angle and the Virtual Reality headset includes one or more camera for capturing at least one other image of the facial expression of the user from a second angle different from the first angle.
13. The Virtual Reality headset as claimed in any one of the preceding claims, wherein the processor is configured for compensating distortion of the at least one image captured for generating the image displayable on the one or more displays or the other one or more displays.
14. The Virtual Reality headset as claimed in any one of the preceding claims, wherein the virtual environment is generated for user interaction in a social media application.
15. The Virtual Reality headset as claimed in any one of the preceding claims, wherein the virtual environment is generated for user interaction in a game application.
PCT/SG2015/050101 2015-05-08 2015-05-08 A virtual reality headset WO2016182504A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/SG2015/050101 WO2016182504A1 (en) 2015-05-08 2015-05-08 A virtual reality headset

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/SG2015/050101 WO2016182504A1 (en) 2015-05-08 2015-05-08 A virtual reality headset

Publications (1)

Publication Number Publication Date
WO2016182504A1 true WO2016182504A1 (en) 2016-11-17

Family

ID=57249293

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2015/050101 WO2016182504A1 (en) 2015-05-08 2015-05-08 A virtual reality headset

Country Status (1)

Country Link
WO (1) WO2016182504A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10325405B1 (en) 2016-04-08 2019-06-18 Relay Cars LLC Social media sharing in a virtual reality application
CN111107912A (en) * 2017-07-14 2020-05-05 惠普发展公司,有限责任合伙企业 Virtual reality head-mounted device support

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010064361A1 (en) * 2008-12-02 2010-06-10 ブラザー工業株式会社 Head-mounted display
JP2013175929A (en) * 2012-02-24 2013-09-05 Nikon Corp Information output device and information output method
WO2014080678A1 (en) * 2012-11-22 2014-05-30 ソニー株式会社 Information processing device, system, information processing method and program
US20140254939A1 (en) * 2011-11-24 2014-09-11 Ntt Docomo, Inc. Apparatus and method for outputting information on facial expression

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010064361A1 (en) * 2008-12-02 2010-06-10 ブラザー工業株式会社 Head-mounted display
US20140254939A1 (en) * 2011-11-24 2014-09-11 Ntt Docomo, Inc. Apparatus and method for outputting information on facial expression
JP2013175929A (en) * 2012-02-24 2013-09-05 Nikon Corp Information output device and information output method
WO2014080678A1 (en) * 2012-11-22 2014-05-30 ソニー株式会社 Information processing device, system, information processing method and program

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10325405B1 (en) 2016-04-08 2019-06-18 Relay Cars LLC Social media sharing in a virtual reality application
CN111107912A (en) * 2017-07-14 2020-05-05 惠普发展公司,有限责任合伙企业 Virtual reality head-mounted device support
CN111107912B (en) * 2017-07-14 2023-08-25 惠普发展公司,有限责任合伙企业 Virtual reality head-mounted equipment support

Similar Documents

Publication Publication Date Title
US10691202B2 (en) Virtual reality system including social graph
US11889229B2 (en) Modifying video streams with supplemental content for video conferencing
Kasahara et al. JackIn head: immersive visual telepresence system with omnidirectional wearable camera for remote collaboration
EP3358835B1 (en) Improved method and system for video conferences with hmds
US9479736B1 (en) Rendered audiovisual communication
WO2020210213A1 (en) Multiuser asymmetric immersive teleconferencing
US9030486B2 (en) System and method for low bandwidth image transmission
US10701426B1 (en) Virtual reality system including social graph
EP2661077A1 (en) System and method for eye alignment in video
US11218669B1 (en) System and method for extracting and transplanting live video avatar images
CN109997175B (en) Determining the size of a virtual object
Müller et al. PanoVC: Pervasive telepresence using mobile phones
US10694145B1 (en) Presenting a portion of a first display on a second display positioned relative to the first display
CN113282163A (en) Head-mounted device with adjustable image sensing module and system thereof
KR101784095B1 (en) Head-mounted display apparatus using a plurality of data and system for transmitting and receiving the plurality of data
US20210097740A1 (en) Multi-user and multi-surrogate virtual encounters
CN109478343A (en) Capture and rendering are related to the information of virtual environment
US20230231983A1 (en) System and method for determining directionality of imagery using head tracking
JP2018033107A (en) Video distribution device and distribution method
WO2016182504A1 (en) A virtual reality headset
US9699123B2 (en) Methods, systems, and non-transitory machine-readable medium for incorporating a series of images resident on a user device into an existing web browser session
JPWO2020090128A1 (en) Image processing equipment, methods, computer programs
WO2023075810A1 (en) System and method for extracting, transplanting live images for streaming blended, hyper-realistic reality
WO2020090128A1 (en) Image processing device, method, and computer program
TW202414341A (en) Systems and methods of automated imaging domain transfer

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15891995

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15891995

Country of ref document: EP

Kind code of ref document: A1