WO2017156622A1 - Dispositif de capture audiovisuel monté sur la tête - Google Patents

Dispositif de capture audiovisuel monté sur la tête Download PDF

Info

Publication number
WO2017156622A1
WO2017156622A1 PCT/CA2017/050326 CA2017050326W WO2017156622A1 WO 2017156622 A1 WO2017156622 A1 WO 2017156622A1 CA 2017050326 W CA2017050326 W CA 2017050326W WO 2017156622 A1 WO2017156622 A1 WO 2017156622A1
Authority
WO
WIPO (PCT)
Prior art keywords
head
mounted device
microphone
visual
camera lens
Prior art date
Application number
PCT/CA2017/050326
Other languages
English (en)
Inventor
Michael Godfrey
Original Assignee
Rising Sun Productions Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rising Sun Productions Limited filed Critical Rising Sun Productions Limited
Publication of WO2017156622A1 publication Critical patent/WO2017156622A1/fr

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/243—Image signal generators using stereoscopic image cameras using three or more 2D image sensors
    • A—HUMAN NECESSITIES
    • A42—HEADWEAR
    • A42B—HATS; HEAD COVERINGS
    • A42B1/00—Hats; Caps; Hoods
    • A42B1/24—Hats; Caps; Hoods with means for attaching articles thereto, e.g. memorandum tablets or mirrors
    • A—HUMAN NECESSITIES
    • A42—HEADWEAR
    • A42B—HATS; HEAD COVERINGS
    • A42B1/00—Hats; Caps; Hoods
    • A42B1/24—Hats; Caps; Hoods with means for attaching articles thereto, e.g. memorandum tablets or mirrors
    • A42B1/245—Means for mounting audio or communication systems
    • A—HUMAN NECESSITIES
    • A42—HEADWEAR
    • A42B—HATS; HEAD COVERINGS
    • A42B3/00—Helmets; Helmet covers ; Other protective head coverings
    • A42B3/04—Parts, details or accessories of helmets
    • A42B3/0406—Accessories for helmets
    • A42B3/042—Optical devices
    • G—PHYSICS
    • G02—OPTICS
    • G02C—SPECTACLES; SUNGLASSES OR GOGGLES INSOFAR AS THEY HAVE THE SAME FEATURES AS SPECTACLES; CONTACT LENSES
    • G02C11/00—Non-optical adjuncts; Attachment thereof
    • G02C11/10—Electronic devices other than hearing aids
    • G—PHYSICS
    • G02—OPTICS
    • G02C—SPECTACLES; SUNGLASSES OR GOGGLES INSOFAR AS THEY HAVE THE SAME FEATURES AS SPECTACLES; CONTACT LENSES
    • G02C3/00—Special supporting arrangements for lens assemblies or monocles
    • G02C3/003—Arrangements for fitting and securing to the head in the position of use
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B37/00—Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe

Definitions

  • the present invention relates to a head-mounted audiovisual capture device and related methods and systems for capturing audiovisual content about the device. More particularly, the present invention relates to a head-mounted audiovisual capture device for capturing 360 degrees of immersive audiovisual content for use with virtual and augmented reality applications.
  • Virtual reality and its inevitable successor, augmented reality, provide humans with a highly immersive sensory experience.
  • Virtual reality is a highly interactive form of tactile audiovisual entertainment, broadcast and communication that involves the three-dimensional capture, creation and delivery of an environment, either live or in a simulated format.
  • virtual reality video is experienced by a user via electronic visual devices such as a head-mounted virtual reality viewer or virtual reality glasses.
  • Virtual reality audio is generally experienced by a user simultaneously via head- mounted headphones or multichannel loudspeaker systems.
  • a virtual reality video viewer essentially blocks out the typical way humans experience naturally occurring stimuli and substitutes real-life sight and sound with electronic content information.
  • a user now exists in a naturally sensory- deprived state where they are relying on electronics for sensory stimulation and communication.
  • the virtual reality video and sound artificially re-create reality and stimulate brain sensors to perceive the audiovisual material as reality.
  • Virtual reality allows a user to feel fully immersed in a created three- dimensional world where the sense of realism experienced by the user is immense.
  • Virtual reality cameras may contain camera lenses that point outwardly from a central point in space on a single device, otherwise known as a single-point source.
  • a single-point source captures visuals in a first-person perspective of approximately 180 to 360 degrees.
  • virtual reality microphones capture 360 degrees of audio through microphones that point outwardly from a central point in space on a single device.
  • this multi-lens, multi- microphone camera array will continue to be designed to be smaller and smaller in size, due to the way it is designed to capture in an outwardly direction, it will nevertheless continue to capture the actual camera-operator or the person holding the camera from a secondary perspective.
  • the intention of this camera/microphone array is for the content to be shot in a first-person perspective. This, however, will not be possible due to the angle of the camera. Instead, it will be shot from a secondary perspective. Further, the shadow of the camera-operator or other production equipment may be visible or there may be a blocked view in at least one direction, or there may be an acoustic shadow of the total 360 degrees sound. Overall, the sound and visual captured will be compromised and imperfect.
  • first-person perspective capture, content creation and final viewer perspective are the most effective methods for humans to electronically sense the total three-dimensional experience of virtual reality.
  • first-person perspective capture, content creation and final viewer perspective are the most effective methods for humans to electronically sense the total three-dimensional experience of virtual reality.
  • the benefits of live broadcast or recorded 360-degree audiovisual communication in a first-person perspective would be numerous.
  • a method for capturing audiovisual content in a three-dimensional space from a head-mounted device comprising: receiving visual input from at least one camera lens positioned on the head-mounted device, the at least one camera lens configured to capture at least 180 degrees of visual content away from the head-mounted device; simultaneously receiving audio input from at least two microphone elements positioned on the head-mounted device, wherein of the at least two microphone elements, a first microphone element is proximate to a right ear of a user wearing the head-mounted device and wherein a second microphone element is proximate to a left ear of the user wearing the head-mounted device, wherein the at least two microphone elements capture audio content about the head-mounted device; and electrically receiving the visual content received and the audio content received at at least one electronic processor for storage, processing or transmission.
  • the present invention provides a system for capturing audiovisual content in a three-dimensional space from a head-mounted device, the system comprising: at least one camera lens positioned on the head-mounted device, wherein the at least one camera lens receives visual input and is configured to capture at least 180 degrees of visual content away from the head-mounted device; at least two microphone elements positioned on the head-mounted device, wherein of the at least two microphone elements, a first microphone element is proximate to a left ear of a user wearing the head-mounted device and a second microphone element is proximate to a right ear of the user wearing the head-mounted device, wherein the at least two microphone elements capture audio content about the head-mounted device; and at least one electronic processor for executing stored computer- readable instructions which when executed, cause the at least one electronic processor to electrically receive the visual content received and electrically receive the audio content received from any of the at least one camera lens and the at least two microphone elements and thereafter store, process or transmit the visual and audio content
  • the present invention provides a head-mounted device for capturing audiovisual content in a three-dimensional space, the device comprising: a head-worn item; at least one camera lens for receiving visual input, the at least one camera lens positioned on the head-worn item to capture at least 180 degrees of visual content away from the head-mounted device; at least two microphone elements positioned on the head-worn item, wherein of the at least two microphone elements, a first microphone element is proximate to a left ear of a user wearing the head-mounted device and a second microphone element is proximate to a right ear of the user wearing the head-mounted device, wherein the at least two microphone elements capture audio content about the head-mounted device; and at least one electronic processor for executing stored computer-readable instructions which when executed, cause the at least one electronic processor to electrically receive the visual content received and electrically receive the audio content received from any of the at least one camera lens and the at least two microphone elements and thereafter store, process or transmit the visual and audio content.
  • a head-mounted device for capturing audiovisual content in a three-dimensional space, the device comprising: a head-worn item; at least one camera lens for receiving visual input, the at least one camera lens positioned on the head-worn item to capture at least 180 degrees of visual content away from the head-mounted device; at least one microphone element group positioned on the head-worn item, the at least one microphone element group comprising four microphone elements arranged in a tetrahedral array, wherein the microphone elements capture audio content about the head-mounted device; and at least one electronic processor for executing stored computer-readable instructions which when executed, cause the at least one electronic processor to electrically receive the visual content received and electrically receive the audio content received from any of the at least one camera lens and the microphone elements and thereafter store, process or transmit the visual and audio content.
  • Figure 1A is a flow diagram of a method for simultaneously capturing 360 degrees of visual and audio content in a three-dimensional space from a head-mounted device according to an embodiment of the present invention
  • Figure IB is a flow diagram of a method for simultaneously capturing 360 degrees of visual and audio content in a three-dimensional space from a head-mounted device and storing the audiovisual content according to an embodiment of the present invention
  • Figure 2 is a diagram of a system for simultaneously capturing 360 degrees of visual and audio content in a three-dimensional space from a head-mounted device in accordance with an embodiment of the present invention
  • Figure 3A is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention
  • Figure 3B is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention
  • Figure 3C is a top view of three configurations of camera lenses and microphone elements on a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention
  • Figure 3D is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention
  • Figure 4 is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention
  • Figure 5 is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention
  • Figure 6 is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention
  • Figure 7 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention
  • Figure 8 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention
  • Figure 9 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention.
  • Figure 10 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention
  • Figure 11 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention
  • Figure 12A is a back perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention
  • Figure 12B is a side perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention
  • Figure 12C is a top view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention
  • Figure 13 is a side view of a head-mounted device for capturing audiovisual content in a three- dimensional space in accordance with an embodiment of the present invention
  • Figure 14 is a perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention.
  • Figure 15 is a front perspective view of a head-mounted device for capturing audiovisual content in a three-dimensional space in accordance with an embodiment of the present invention.
  • FIG. 1 to 15 a method, system and head-mounted device for simultaneously capturing as much as 360 degrees of visual and audio content in a three-dimensional space defined by an X-axis, a Y-axis, and a Z-axis for full three-dimensional virtual reality production is disclosed.
  • a method 100 may simultaneously capture as much as 360 degrees of visual and audio content in a three-dimensional space from a head-mounted device 201.
  • the method 100 may comprise receiving visual input from at least one camera lens 110 positioned on the head-mounted device 201, wherein the at least one camera lens 110 may be configured to capture at least 180 degrees of visual content.
  • the method 100 may further comprise receiving audio input from at least two microphone elements 105 positioned on the head- mounted device 201.
  • At least one microphone element 105 is proximate to a right ear of a user wearing the head-mounted device and at least one microphone element 105 is proximate to a left ear of the user wearing the head-mounted device.
  • the at least two microphone elements 105 may capture 360 degrees of audio content when the device is in use.
  • the method 100 may also comprise electrically receiving the visual content and the audio content at at least one electronic processor 135 electrically connected to the at least one camera lens 110 and the at least two microphone elements 105. According to an embodiment, there may be separate electronic processors 135 for visual content received and audio content received.
  • the visual content and the audio content received at the at least one electronic processor 135 may be stored integrally or transmitted with a wired or wireless connection to a storage medium, such as a computer- readable and writeable memory 150, and may then be retrieved from the memory 150 for further processing and playback by another device, such as remote computer 160.
  • a storage medium such as a computer- readable and writeable memory 150
  • the head-mounted device 201 may comprise a head-worn item such as a cap, a headband, a helmet, a hat or other head-worn item such as head-worn straps contained or made part of a virtual reality content viewer.
  • Camera lens 110 may capture immersive video according to techniques known in the art.
  • One or more camera lenses 110 may be used according to embodiments of the invention. These camera lenses 110 may be of a chosen size appropriate for the head-worn item and the number of camera lenses on the head-worn item.
  • One or more microphone elements 105 which may also of a chosen size appropriate for the head-worn item and the number of microphone elements or sensors on the head-worn item, may capture immersive audio according to techniques known in the art.
  • the microphone elements 105 or microphone sensors may be omnidirectional, cardioid, hypercardioid, or bidirectional microphone elements or sensors.
  • the visual and audio content received may then transmitted and processed as desired for viewing in a virtual reality or augmented reality environment.
  • the head-mounted device 201 may include a head-worn item appropriate for use by military, security or law enforcement personnel such as a protective or reinforced helmet.
  • the at least one camera lens 110 and the at least two microphone elements 105 may be positioned on the head-mounted device 201 facing outwards and the positions of the at least one camera lens 110 and the at least two microphone elements 105 may be positioned in various locations as shown in Figures 1A to 15. According to an embodiment, at least one microphone element 105 will be proximate to the left ear of the user and at least one microphone element will be proximate to the right ear of the user.
  • Additional microphone elements 105 may be placed on the head-mounted device 201 which may help better determine the points of origination of sounds in capturing 360 degrees of audio content and may supply more complete coverage of all points in space in order to create a more complete soundscape for post- production workflows and processes that may utilize the extra information for immersive content delivery.
  • the additional microphone elements 105 are spaced away from each of the first and second microphone elements, such that the first, second, and any additional microphone elements are at different coordinates in a three-dimensional Cartesian coordinate system.
  • the visual and audio content that may be captured by the at least one camera lens 110 and the at least two microphone elements 105, respectively, may create a 360-degree audio and at least a 180-degree visual image or moving image when combined.
  • the visual content captured from the at least one camera lens 110 and the audio content captured from the at least two microphone elements 105 may be directly or discretely output for external processing or storage.
  • the audio content received and the visual content received may be connected to outboard electronic equipment such as amplifiers and processors.
  • the electronic processor 135 may store the visual content and the content output separately for future processing.
  • the electronic processor 135 may merge the visual content and the audio content received into a single data type such as through the use of a live video stitching application that may combine the video and audio cameras signals together in a VR viewable format such as an MP4 or VRML file encoded with MPEG4 or H.264 encoding.
  • a live video stitching application may combine the video and audio cameras signals together in a VR viewable format such as an MP4 or VRML file encoded with MPEG4 or H.264 encoding.
  • This may be used for live retransmission through a network connection or another wired or wireless connection, or for storage on the head-mounted device 201 such as in memory included as part of the device or in remote memory such as on a linked phone, or on a network-accessible device having its own memory.
  • a system 200 for simultaneously capturing 360 degrees of visual and audio content in a three-dimensional space from a head-mounted device 201 may comprise at least one camera lens 110 positioned on the head-mounted device 201, wherein the at least one camera lens 110 receives visual input.
  • the system 200 may further comprise at least two microphone elements 105 positioned on the head-mounted device 201, wherein one microphone element 105 is proximate to the left ear of the user wearing the head-mounted device 201 and one microphone element 105 is proximate to the right ear of the user wearing the head-mounted device 201.
  • Each microphone element 105 may receive audio input.
  • the system 200 may also comprise a processor 135 for executing stored computer-readable instructions which when executed, may cause the processor 135 to receive and process and/or store received audio content and visual content from any of the at least one camera lens 110 and the at least two microphone elements 110.
  • the at least one camera lens 110 and the at least two microphone elements 105 may be positioned on the head-mounted device 201 facing outwards. According to an embodiment, the positions of the at least one camera lens 110 and the at least two microphone elements 105 may be varied and reconfigured by removing and reattaching the elements at a different location, for example, by attachment using adhesives or hook and loop fasteners, on the head-mounted device 201. Additional microphone elements 105 may also be placed on the head-mounted device 201 which may help better determine the points of origination of sounds in capturing 360 degrees of audio content.
  • the at least one camera lens 110 may capture at least 180 degrees of visual content in a three-dimensional space and the at least two microphone elements 105 may capture at least 360 degrees of audio content in a three-dimensional space.
  • the visual and audio content captured by the at least one camera lens 110 and the at least two microphone elements 110, respectively, may create a 360-degree audio and at least a 180-degree visual image or moving image when combined.
  • At least two camera lenses 110 may capture at least 270 degrees of visual content in a three-dimensional space and the at least two microphone elements 105 may capture at least 360 degrees of audio content in a three-dimensional space.
  • the visual and audio content captured by the at least two camera lenses 110 and the at least two microphone elements 105, respectively, may create a 360-degree audio and at least a 270-degree visual image or moving image when combined.
  • At least two camera lenses 110 may capture at least 360 degrees of visual content in a three-dimensional space and the at least two microphone elements 105 may capture at least 360 degrees of audio content in a three-dimensional space.
  • the visual and audio content captured by the at least two camera lenses 110 and the at least two microphone elements 105, respectively, may create a 360-degree audio and at least a 360-degree visual image or moving image when combined.
  • the visual input and the audio input may be combined to output a 360-degree sphere of audiovisual content.
  • Individually derived components of the audiovisual content may be stitched, merged, processed or broadcast simultaneously and may be received and played, with or without further processing, by an electronic viewfinder designed for virtual reality or augmented reality.
  • the electronic viewfinder may be used in conjunction with stereo headphones or another loudspeaker arrangement comprising of at least two speakers.
  • the audiovisual content received may be broadcast live over 360-degree one-way communication or two-way communication.
  • the audiovisual content may be recorded by a recording device (which may be part of electronic processor 135 or remote computer-readable and writeable memory) on the head-mounted device 201.
  • the various electronics used in the system may be contained in the head-mounted device 201, or in one or more separate units on the user, such as a smartphone, or remotely such as in an electronic cloud.
  • the head-mounted device 201 may communicate with a second head-mounted device via a network or linked telephone or other communication device.
  • the link between the head-mounted device 201 and linked telephone, such as a smartphone, may be by short-range wireless communication such as BluetoothTM, ZigBeeTM, infrared or wi-fi.
  • merged visual and audio output may contain spatial information and orientation from where original audiovisual signals physically originated and from where original audiovisual signals were physically captured in the three-dimensional space. Further processing or program applications may stitch a total of the individually derived components of the audiovisual content together to create a viable and believable 360-degree audiovisual sphere for playback to a viewer.
  • Processors may include stereoscopic visual processors and other means to enhance the visual signal, as well as "de-fish" or fisheye to rectilinear image software to straighten visual artifacts of ultra-wide angle lenses. Processing may also include low light processing and color and image enhancement processing. Spatialization or HRTF (head related transfer function) processors, additional audio delays, and reverberation processors may also be used to achieve a desired three-dimensional effect for sound.
  • a storage medium may be used in the signal chain before the processors, wherein the storage medium may be used live or subsequent to initially capturing the audiovisual content.
  • Audio signals and visual signals that may be output at the end of the signal chain to the viewer may be 360 degrees or 360x360 video, defined by an X-axis, a Y-axis, and a Z-axis, or stereoscopic three-dimensional video or a combination of both.
  • the audio signals may be stereophonic, stereo binaural, multichannel loudspeaker or other immersive audio signals.
  • the audio signals and the visual signals may be discrete or combined for delivery.
  • the audio content may be captured by the methods disclosed in U.S Patent No. 5,778,083A, entitled “GLOBAL SOUND MICROPHONE SYSTEM,” published on July 7, 1998.
  • the audio content may be captured by the methods disclosed in U.S. Patent No. 6,980,661B2, entitled “METHOD OF AND APPARATUS FOR PRODUCING APPARENT MULTIDIMENSIONAL SOUND,” filed on November 14, 2001 as U.S. Patent Application No. 09/987,217 with priority claim to U.S. Provisional Application No. 60/248,225 filed on November 15, 2000.
  • a head-mounted device 201 for simultaneously capturing 360 degrees of visual and audio content in a three-dimensional space may comprise a head-worn item, at least one camera lens 110 mounted or positioned on the head-worn item, wherein the at least one camera lens 110 may receive visual input.
  • the head-mounted device 201 may further comprise at least two microphone elements 105 mounted or positioned on the head-mounted device 201, wherein one microphone element 105 is proximate to the left ear of the user wearing the head-mounted device 201 and one microphone element 105 is proximate to the right ear of the user wearing the head-mounted device 201.
  • Each microphone element 105 may receive audio input.
  • the at least one camera lens 110 and the at least two microphone elements 105 may be positioned on the head- mounted device 201 facing outwards and the positions of the at least one camera lens 110 and the at least two microphone elements 105 may be mounted or positioned according to various configurations as suggested throughout the figures. Additional microphone elements 105 may also be mounted or positioned on the head-mounted device 201 to better determine points of origination of sound to capture 360 degrees of audio content.
  • the head-mounted device 201 may comprise a head-worn item such as a virtual reality viewer.
  • the virtual reality viewer may comprise camera lenses 110 and microphone elements 105 around a perimeter of the virtual reality viewer in alternating positions, on a strap across the head of the user in alternating positions and on a strap extending from the right ear of the user to the left ear of the user in alternating positions.
  • the head-mounted device 201 may comprise a head-worn item such as a helmet.
  • the helmet may comprise camera lenses 110 and microphone elements 105 positioned linearly in alternating positions across a top of the helmet.
  • head-mounted device 201 may, for example, comprise three camera lenses 110 and three microphone elements 105, or four camera lenses 110 and four microphone elements 105, or six camera lenses 110 and six microphone elements 105.
  • the camera lenses 110 and the microphone elements 105 may be positioned around the perimeter of the head-mounted device 201 in alternating positions.
  • the head-mounted device 201 may be a head-worn item such as a cap.
  • the cap may comprise camera lenses 110 and microphone elements 105 positioned around the perimeter of the cap in a tiered fashion in alternating positions.
  • the head-mounted device 201 may comprise a head-worn item such as a cap, for example cap 300.
  • cap 300 may comprise a first camera lens 110 mounted or positioned on the cap 300 proximate to the center of a forehead of the user wearing the head-mounted device 201.
  • the cap 300 may further comprise two microphone elements 105 mounted thereon or therein, wherein a first microphone element 105 may be mounted or positioned on the cap 300 proximate to the right ear of the user and a second microphone element 105 may be mounted or positioned on the cap 300 proximate to the left ear of the user.
  • the user may also be wearing an electronic viewer 115.
  • the camera lenses 110 may be mounted in holes made in a head- worn item for the particular shape and size of the camera lens 110 such that the camera lenses may be aligned and mounted firmly in position.
  • microphone elements 105 may be either be embedded in pockets designed in the manufacturing process for a head-worn item or can be mounted in rubber grommets to reduce vibration transmitting noise and then mounted in holes similar to camera lens holes described above.
  • sensor mounts may be sewn or clipped onto fabric that will hold camera lenses 110 and microphone elements 105 or they may be sewn in during manufacturing with wiring and electronics hidden in channels of fabric and foam on the garment itself, or may be placed in channels designed directly into the shell of a helmet.
  • a cap 300 may comprise three camera lenses 110 and three microphone elements 105 mounted thereon or therein, wherein a first camera lens 110 and a first microphone element 105 may be mounted or positioned on the cap 300 proximate to the right ear of the user, a second camera lens 110 and a second microphone element 105 may be mounted or positioned on the cap 300 proximate to the left ear of the user, and a third camera lens 110 and a third microphone element 105 may be mounted or positioned on the cap 300 proximate to the center of the forehead of the user.
  • the user may also be wearing one microphone element 105, such as a wireless microphone, in one or both ears.
  • a cap 300 may comprise three camera lenses 110 mounted thereon or therein, wherein a first camera lens 110 may be mounted or positioned on the cap 300 proximate to the right ear of the user, a second camera lens 110 may be mounted or positioned on the cap 300 proximate to the left ear of the user, and a third camera lens 110 may be mounted or positioned on the cap 300 proximate to the center of the forehead of the user.
  • the cap 300 may further comprise a tetrahedral microphone array 130 composed of a microphone element group of four microphone elements 105 mounted or positioned on top of the cap 300.
  • the user may also be wearing one microphone element 105, such as a wireless microphone, in one or both ears.
  • the head-mounted device 201 may be a head-worn item such as a helmet, for example helmet 700.
  • a helmet 700 may comprise two camera lenses 110 mounted thereon or therein, wherein a first camera lens 110 may be mounted or positioned on the helmet 700 proximate to the center of the forehead of the user and a second camera lens 110 may be mounted or positioned on the helmet 700 proximate to the back of a head of the user.
  • the helmet 700 may further comprise two microphone elements 105 mounted thereon or therein, wherein a first microphone element 150 may be mounted or positioned on the helmet 700 proximate to the back of the head of the user and a second microphone element 105 may be mounted or positioned on the helmet 700 proximate to the center of the forehead of the user.
  • the helmet 700 may further comprise a microphone 120 positioned proximate to the mouth of the user for receiving voice communication.
  • a helmet 700 may comprise three camera lenses 110 and three microphone elements 105 mounted around the perimeter of the helmet 700 in alternating positions, wherein a first camera lens 110 and a first microphone element 105 may be mounted or positioned on the helmet 700 proximate to the center of the forehead of the user, a second camera lens 110 and a second microphone element 105 may be mounted or positioned on the helmet 700 proximate to the left ear of the user, and a third camera lens 110 and a third microphone element 105 may be mounted or positioned on the helmet 700 proximate to the back of the head of the user.
  • the helmet 700 may further comprise a microphone 120 positioned proximate to the mouth of the user for receiving voice communication.
  • the user may also be wearing an electronic viewer 115.
  • a helmet 700 may comprise three camera lenses 110 and three microphone elements 105 mounted around the perimeter of the helmet 700 in alternating positions, wherein a first camera lens 110 and a first microphone element 105 may be mounted or positioned on the helmet 700 proximate to the center of the forehead of the user, a second camera lens 110 and a second microphone element 105 may be mounted or positioned on the helmet 700 proximate to the left ear of the user, and a third camera lens 110 and a third microphone element 105 may be mounted or positioned on the helmet 700 proximate to the back of the head of the user.
  • the helmet 700 may further comprise three camera lenses 110 and three microphone elements 105 mounted around the perimeter of a top half of the helmet 700 in alternating positions, wherein a fourth camera lens 105 and a fourth microphone element 105 may be mounted or positioned on the helmet 700 proximate to the center of the forehead of the user, a fifth camera lens 110 and a fifth microphone element 105 may be mounted or positioned on the helmet 700 proximate to a left side of the head of the user, and a sixth camera lens 110 and a sixth microphone element 105 may be mounted or positioned on the helmet 700 proximate to the back of the head of the user.
  • the helmet 700 may further comprise a seventh camera lens 110 and a seventh microphone element 105 on top of the helmet 700.
  • the helmet 700 may also comprise a microphone 120 proximate to the mouth of the user for voice communication and the user may be wearing an electronic viewer 115.
  • the helmet 700 may comprise three camera lenses 110 mounted thereon or therein, wherein a first camera lens 110 may be mounted or positioned on top of the helmet 700, a second camera lens 110 may be mounted or positioned on the helmet 700 proximate to a front of the head of the user, and a third camera lens 110 may be mounted or positioned on the helmet 700 proximate to the back of the head of the user.
  • the helmet 700 may further comprise three microphone elements 105 mounted thereon or therein, wherein a first microphone element 105 may be mounted or positioned proximate to the center of the forehead of the user, a second microphone element 105 may be mounted or positioned on the helmet 700 proximate to a front of the left ear of the user, and a third microphone element 105 may be mounted or positioned on the helmet 700 proximate to a back of the left ear of the user.
  • the helmet 700 may also comprise a microphone 120 positioned proximate to the mouth of the user for voice communication.
  • the head-mounted device 201 may be a head-worn item such as a virtual reality viewer, for example virtual reality viewer 1200.
  • a virtual reality viewer 1200 may comprise three camera lenses 110 mounted thereon, wherein a first camera lens 110 may be mounted or positioned on top of a strap over the head of the user, a second camera lens 110 may be mounted or positioned on a back-left side of a strap across the back of the head of the user, and a third camera lens 110 may be mounted or positioned on a back-right side of the strap across the back of the head of the user.
  • the virtual reality viewer 1200 may further comprise three microphone elements 105 mounted thereon or therein, wherein a first microphone element 105 may be mounted or positioned on the strap across the back of the head of the user proximate to the right ear of the user, a second microphone element 105 may be mounted or positioned on the strap across the back of the head of the user proximate to the left ear of the user, and a third microphone element 105 may be mounted or positioned on a middle of the strap across the back of the head of the user.
  • a virtual reality viewer 1200 may comprise three camera lenses 110 and three microphone elements 105 mounted or positioned on a front side of the viewfinder of virtual reality viewer 1200.
  • the virtual reality viewer 1200 may further comprise a further camera lens 110 and further microphone element 105 mounted or positioned on top of the strap over the head of the user.
  • the virtual reality viewer may also comprise two microphone elements 105 mounted or positioned on the strap across the back of the head of user, wherein one microphone element 105 may be mounted or positioned proximate to the right ear of the user and the other microphone element 105 may be mounted or positioned proximate to the left ear of the user.
  • a virtual reality viewer 1200 may comprise two camera lenses 110 and three microphone elements 105 mounted or positioned in alternating positions across a top of a viewfinder of the virtual reality viewer 1200.
  • the virtual reality viewer 1200 may further comprise two microphone elements 105 mounted or positioned on the strap across the back of the head of user, wherein one microphone element 105 may be mounted or positioned proximate to the right ear of the user and one microphone element 105 may be mounted or positioned proximate to the left ear of the user.
  • the head-mounted device 201 may be a head-worn item such as a headset, for example headset 1300.
  • the headset 1300 may comprise a camera lens 110 mounted or positioned on the headset 1300 proximate to the left ear of the user.
  • the headset 1300 may further comprise microphone elements 105, such as microphone sensors.
  • a microphone element 105 may be mounted or positioned on the headset 1300 proximate to the left ear of the user and a microphone element 105 may be mounted or positioned on the headset 1300 proximate to the right ear of the user.
  • microphone elements 105 may also be mounted or positioned along a strap across the back of the head of the user.
  • the head-mounted device 201 may be a head-worn item such as glasses, for example glasses 1400 or glasses 1500.
  • glasses 1400 may comprise a camera lens 110 mounted or positioned proximate to a left-side lens of the glasses 1400.
  • Glasses 1400 may further comprise two microphone elements 105, or microphone sensors, wherein a first microphone element 105 or sensor may be mounted or positioned on a left arm of the glasses 1400 and a second microphone element 105 or sensor may be mounted or positioned on a right arm of the glasses 1400 such that the first microphone element or sensor and the second microphone element or sensor may be positioned at least two inches apart.
  • glasses 1500 may comprise a camera lens 110 between the left-side lens and the right-side lens of the glasses 1500.
  • the glasses 1500 may further comprise four microphone elements 105, for example microphone sensors, wherein a first microphone element 105 or sensor may be mounted or positioned proximate to the left-side lens of the glasses 1500, a second microphone element 105 or sensor may be mounted or positioned proximate to the right-side lens of the glasses 1500 such that the first microphone element or sensor and the second microphone element or sensor may be positioned at least two inches apart.
  • a third microphone element 105 or sensor may be mounted or positioned on the right arm of the glasses 1500, and a fourth microphone element 105 or sensor may be mounted or positioned on the left arm of the glasses 1500.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Ophthalmology & Optometry (AREA)
  • Acoustics & Sound (AREA)
  • General Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Studio Devices (AREA)

Abstract

L'invention concerne un dispositif monté sur la tête permettant de capturer un contenu audiovisuel dans un espace tridimensionnel. Le dispositif est un article porté sur la tête ayant au moins un objectif et au moins deux éléments formant microphones. Les objectifs sont positionnés sur l'article porté sur la tête pour capturer au moins 180 degrés de contenu visuel depuis le dispositif monté sur la tête. Les éléments formant microphones capturent 360 degrés de contenu audio autour du dispositif monté sur la tête. Un premier élément formant microphone est positionné à proximité de l'oreille gauche d'un utilisateur lorsque le dispositif monté sur la tête est porté, et un second élément formant microphone est positionné à proximité de l'oreille droite d'un utilisateur lorsque le dispositif monté sur la tête est porté. Des objectifs et éléments formant microphones supplémentaires peuvent être ajoutés à l'article porté sur la tête. Un processeur électronique reçoit électriquement le contenu visuel et le contenu audio reçus en provenance des objectifs et des éléments formant microphones, puis stocke, traite ou transmet le contenu visuel et le contenu audio.
PCT/CA2017/050326 2016-03-13 2017-03-13 Dispositif de capture audiovisuel monté sur la tête WO2017156622A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201662307507P 2016-03-13 2016-03-13
US62/307,507 2016-03-13

Publications (1)

Publication Number Publication Date
WO2017156622A1 true WO2017156622A1 (fr) 2017-09-21

Family

ID=59850994

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2017/050326 WO2017156622A1 (fr) 2016-03-13 2017-03-13 Dispositif de capture audiovisuel monté sur la tête

Country Status (1)

Country Link
WO (1) WO2017156622A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109619744A (zh) * 2018-11-08 2019-04-16 厦门科路德科技有限公司 智能识别闯红灯的安全头盔的控制方法
CN110139016A (zh) * 2019-07-05 2019-08-16 昆山玖趣智能科技有限公司 一种头戴式摄像头多麦克风系统
CN114338619A (zh) * 2021-12-29 2022-04-12 鲁春宝 一种售后维修用远程视频支持装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5778083A (en) * 1994-10-31 1998-07-07 Godfrey; Mike Global sound microphone system
US6980661B2 (en) * 2000-11-15 2005-12-27 Mike Godfrey Method of and apparatus for producing apparent multidimensional sound
US20110157327A1 (en) * 2009-12-31 2011-06-30 Broadcom Corporation 3d audio delivery accompanying 3d display supported by viewer/listener position and orientation tracking
US20140125579A1 (en) * 2012-11-06 2014-05-08 Sony Computer Entertainment Inc. Head mounted display, motion detector, motion detection method, image presentation system and program
US20140180684A1 (en) * 2012-12-20 2014-06-26 Strubwerks, LLC Systems, Methods, and Apparatus for Assigning Three-Dimensional Spatial Data to Sounds and Audio Files
GB2516758A (en) * 2013-06-11 2015-02-04 Sony Comp Entertainment Europe Head-mountable apparatus and systems
CA2953335A1 (fr) * 2014-06-14 2015-12-17 Magic Leap, Inc. Procedes et systemes de creation d'une realite virtuelle et d'une realite augmentee

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5778083A (en) * 1994-10-31 1998-07-07 Godfrey; Mike Global sound microphone system
US6980661B2 (en) * 2000-11-15 2005-12-27 Mike Godfrey Method of and apparatus for producing apparent multidimensional sound
US20110157327A1 (en) * 2009-12-31 2011-06-30 Broadcom Corporation 3d audio delivery accompanying 3d display supported by viewer/listener position and orientation tracking
US20140125579A1 (en) * 2012-11-06 2014-05-08 Sony Computer Entertainment Inc. Head mounted display, motion detector, motion detection method, image presentation system and program
US20140180684A1 (en) * 2012-12-20 2014-06-26 Strubwerks, LLC Systems, Methods, and Apparatus for Assigning Three-Dimensional Spatial Data to Sounds and Audio Files
GB2516758A (en) * 2013-06-11 2015-02-04 Sony Comp Entertainment Europe Head-mountable apparatus and systems
CA2953335A1 (fr) * 2014-06-14 2015-12-17 Magic Leap, Inc. Procedes et systemes de creation d'une realite virtuelle et d'une realite augmentee

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109619744A (zh) * 2018-11-08 2019-04-16 厦门科路德科技有限公司 智能识别闯红灯的安全头盔的控制方法
CN110139016A (zh) * 2019-07-05 2019-08-16 昆山玖趣智能科技有限公司 一种头戴式摄像头多麦克风系统
CN114338619A (zh) * 2021-12-29 2022-04-12 鲁春宝 一种售后维修用远程视频支持装置

Similar Documents

Publication Publication Date Title
US10959037B1 (en) Gaze-directed audio enhancement
JP6747538B2 (ja) 情報処理装置
KR102197544B1 (ko) 공간화 오디오를 가진 혼합 현실 시스템
US20240105156A1 (en) Adaptive anc based on environmental triggers
JP6565903B2 (ja) 情報再生装置及び情報再生方法
US20180249276A1 (en) System and method for reproducing three-dimensional audio with a selectable perspective
WO2017183319A1 (fr) Robot et boîtier
US11719944B2 (en) Swappable strap for a head-mounted display system
JP6580516B2 (ja) 処理装置および画像決定方法
WO2017156622A1 (fr) Dispositif de capture audiovisuel monté sur la tête
CN204681518U (zh) 一种全景图像信息采集设备
JP6538003B2 (ja) アクチュエータ装置
WO2017183294A1 (fr) Dispositif actionneur
US10536666B1 (en) Systems and methods for transmitting aggregated video data
US10674259B2 (en) Virtual microphone
US11132834B2 (en) Privacy-aware artificial reality mapping
WO2017183292A1 (fr) Dispositif de traitement et procédé de détermination d'image
JP6615716B2 (ja) ロボットおよび筐体
KR101813103B1 (ko) 넥밴드형 캠코더
US11495004B1 (en) Systems and methods for lighting subjects for artificial reality scenes
JP6518620B2 (ja) 位相差増幅装置
US20210283827A1 (en) Method of covering a housing with a textile and related systems and devices
US20220408049A1 (en) Multi-layer stacked camera-image-sensor circuit
US20230388598A1 (en) Systems and methods for enabling artificial reality commentary
WO2022240319A1 (fr) Dispositif pour obtenir une vue panoramique

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17765607

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17765607

Country of ref document: EP

Kind code of ref document: A1