WO2018149266A1 - 一种基于增强现实的信息处理方法及设备 - Google Patents

一种基于增强现实的信息处理方法及设备 Download PDF

Info

Publication number
WO2018149266A1
WO2018149266A1 PCT/CN2018/073463 CN2018073463W WO2018149266A1 WO 2018149266 A1 WO2018149266 A1 WO 2018149266A1 CN 2018073463 W CN2018073463 W CN 2018073463W WO 2018149266 A1 WO2018149266 A1 WO 2018149266A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
augmented reality
display
virtual
information processing
Prior art date
Application number
PCT/CN2018/073463
Other languages
English (en)
French (fr)
Inventor
毛颖
钟张翼
Original Assignee
深圳梦境视觉智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳梦境视觉智能科技有限公司 filed Critical 深圳梦境视觉智能科技有限公司
Publication of WO2018149266A1 publication Critical patent/WO2018149266A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0172—Head mounted characterised by optical features
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0132—Head-up displays characterised by optical features comprising binocular systems
    • G02B2027/0134—Head-up displays characterised by optical features comprising binocular systems of stereoscopic type
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B2027/0178—Eyeglass type

Definitions

  • the embodiments of the present application relate to the field of augmented reality technologies, and in particular, to an information processing method and device based on augmented reality.
  • the main form of the meeting is that participants are concentrated in the conference room, and various types of digital information are transmitted through projectors, televisions, and audio equipment. Such as documents, slides, videos, images, etc., to all participants. If there are remote participants, they usually use web conferencing services such as Webex and GoToMeetings to share the content displayed on the local screen through the Internet and synchronize the content seen by both parties. And make a video or audio call over the network or by phone, so that both parties can see each other and zoom in.
  • web conferencing services such as Webex and GoToMeetings
  • Digital information presentation is not intuitive enough. Since the presentation of digital information mainly depends on the two-dimensional display screen or projection, in many cases where a three-dimensional model needs to be displayed, such as product display, engineering component demonstration, etc., the two-dimensional plane can only be observed from one or several angles. Not intuitive enough.
  • the sense of presence is limited, and the discussion is mixed with feelings and cannot be communicated face to face. Because the remote participants and local participants mainly let the other party see themselves through the camera shooting scene, and only display the other party's video on the computer screen, the presence is poor and it is difficult to integrate into the meeting for discussion. If the meeting is only communicated via network audio or telephone, the situation will be even worse. There will even be a heated discussion among the more people, and the participants at the other end of the phone will completely ignore it.
  • the technical problem to be solved by the embodiment of the present application is to provide an information processing method and device based on augmented reality, which has a sense of presence and intuitive display of content.
  • the embodiment of the present application provides an information processing method based on augmented reality, including: acquiring a first light including a real image of a conference room; emitting a second light including a virtual image; and including the conference room The first ray composition of the real image is combined with the second ray containing the virtual image.
  • the information processing method based on the augmented reality provided by the embodiment combines the real scene image of the conference room with the virtual image, and the virtual image can provide information prompts for the participants, and assist the participants to perform the conference.
  • the user experience is good and easy to use.
  • FIG. 1a is a schematic structural diagram of an information processing device based on augmented reality according to Embodiment 1 of the present application;
  • Figure 1b is a schematic view of the see-through light guiding element shown in Figure 1a when it is placed on the head frame;
  • Figure 1c is a first relationship diagram between a side view angle and a display brightness of the display module shown in Figure 1a;
  • Figure 1d is a second relationship diagram between the side view angle and the display brightness of the display module shown in Figure 1a;
  • Figure 1e is a third relationship diagram between a side view angle and a display brightness of the display module shown in Figure 1a;
  • FIG. 2a is a schematic diagram showing a positional relationship between a display module and a user's face when the augmented reality-based information processing device shown in FIG. 1a is worn;
  • Figure 2b is a schematic view showing the rotation of the display module shown in Figure 1a;
  • FIG. 3 is a schematic diagram of an imaging principle of the augmented reality based information processing apparatus shown in FIG. 1a;
  • FIG. 3 is a schematic diagram of an imaging principle of the augmented reality based information processing apparatus shown in FIG. 1a;
  • FIG. 4 is a schematic view of the augmented reality-based information processing apparatus shown in FIG. 1a when a diopter correction lens is provided;
  • FIG. 5 is a schematic diagram showing the distance relationship between the diagonal field of view area and the farthest end of the head frame to the foremost end of the user's head of the information processing device based on the augmented reality shown in FIG. 1a;
  • FIG. 6 is a schematic diagram of the augmented reality-based information processing device shown in FIG. 1a connected to an external device;
  • FIG. 7 is a schematic structural diagram of an information processing device based on augmented reality according to Embodiment 2 of the present application.
  • FIG. 8 is a schematic diagram of the augmented reality-based information processing device shown in FIG. 7 connected to an external device;
  • FIG. 9 is another schematic diagram of the operation of the augmented reality based information processing device shown in FIG. 7 when the external device is connected;
  • FIG. 10 is a schematic diagram of the operation of the information processing apparatus based on the augmented reality shown in FIG. 7; FIG.
  • FIG. 11 is a diagram showing an application example of an information processing method based on augmented reality provided by a third embodiment of the present application.
  • FIG. 12 is a schematic diagram of a first display mode in an information processing method based on augmented reality provided by a third embodiment of the present application;
  • FIG. 13 is a schematic diagram of a second display mode in an information processing method based on augmented reality provided by a third embodiment of the present application;
  • FIG. 14 is a schematic diagram of synchronous display in an augmented reality based information processing method according to a third embodiment of the present application.
  • FIG. 15 is a schematic diagram of a face recognition method based on an augmented reality information processing method provided by a third embodiment of the present application.
  • FIG. 16 is a schematic diagram of a remote reality display based on an augmented reality based information processing method provided by a third embodiment of the present application.
  • an augmented reality-based information processing device provided by the embodiment of the present application has a total weight of the information processing device based on the augmented reality of less than 350 grams, and includes: a head frame 11, two display modules 12, Two see-through light guiding elements 13. Wherein, the see-through light guiding element 13 is a partially transmissive, partially reflective optical synthesizing device.
  • the display module 12 and the see-through light guiding elements 13 are all disposed on the head frame 11.
  • the bracket 11 fixes the display module 12 and the see-through light guiding element 13.
  • the display module 12 is disposed on the upper side of the see-through light guiding element 13, and the light emitted by the display module 12 can be reflected after passing through the see-through light guiding element 13.
  • the display module 13 may also be located on the side of the see-through light guiding element 13.
  • the information processing device based on the augmented reality further includes a main board 17 disposed on the head frame 11 and located between the two display modules 12.
  • the main board 17 is provided with a processor for processing a virtual image signal and displaying the virtual image information on the display module 12.
  • the head frame 11 is used for wearing on the head of the user, and each of the see-through light guiding elements 13 has a concave surface which is disposed toward the eyes of the user.
  • the first light reflected through the concave surface of the see-through light guiding element 13 enters the left eye of the user, and the other first light reflected through the concave surface of the other see-through light guiding element 13 enters the right eye of the user to be at the user's
  • the vision of a 3D virtual scene is formed in the mind.
  • the first light is emitted by the display module 12, and the first light includes virtual image information of the left eye and the right eye.
  • two see-through light guiding elements 13 are disposed on the head frame 11 and are independently embedded in the head frame 11, respectively.
  • two regions corresponding to the left and right eyes of the user may be disposed on the raw material for fabricating the fluoroscopic light guiding element, and the shape and size of the region are different from the shape and size of each of the fluoroscopic light guiding members 13 when independently disposed as described above.
  • the same effect; the final effect is that a large see-through light guiding element is provided with two areas corresponding to the left and right eyes of the user.
  • the two fluoroscopic light guiding elements 13 are integrally formed.
  • the see-through light guiding elements provided to correspond to the left and right eye regions of the user are embedded in the head frame 11.
  • the display module 12 is detachably mounted on the head frame 11, for example, the display module is an intelligent display terminal such as a mobile phone or a tablet computer; or the display module is fixedly mounted on the head frame, for example, the display module and the head. Wear a frame integrated design.
  • Two display modules 12 can be mounted on the headgear frame 11.
  • the left eye and the right eye of the user are respectively provided with a display module 12, for example, one display module 12 is configured to emit a first light containing virtual image information of the left eye, and A display module 12 is configured to emit another first light that includes virtual image information of the right eye.
  • the two display modules 12 can be respectively located in one-to-one correspondence between the two perspective-type light guiding elements 13 .
  • the two display modules 12 are respectively located one by one.
  • the display module 12 can also be located on the side of the see-through light guiding element, that is, two see-through light guiding elements are located between the two display modules, when worn by an information processing device based on augmented reality At the user's head, the two display modules are located one-to-one in the side of the user's left and right eyes, respectively.
  • a single display module 12 can also be mounted on the headgear frame 11.
  • the single display module 12 has two display areas, one for emitting a first ray containing left eye virtual image information and the other for transmitting Another first ray of virtual image information for the right eye.
  • the display module includes, but is not limited to, an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), an LCOS (Liquid Crystal On Silicon), or the like.
  • LCD Liquid Crystal Display
  • OLED Organic Light-Emitting Diode
  • LCOS Liquid Crystal On Silicon
  • the horizontal axis represents the side view angle and the vertical axis represents the display brightness.
  • the display module 12 is an LCD
  • the brightness of the display module 12 varies with the angle of the viewer.
  • the side observation angle ⁇ at a display luminance of 50% is generally large.
  • the LCD When the LCD is applied to an augmented reality display system, it is more suitable for a small side viewing angle, and the brightness of such a display module 12 is concentrated in an angular area near the center. Because the augmented reality display system mainly uses an angled area near the center, the brightness of the first light and the second light projected into the user's eyes will be relatively high. Referring to FIG. 1d, the brightness of the first light and the second light emitted by the LCD applied to the augmented reality display system is generally small when the display brightness is 50%. Moreover, the distribution of the brightness of the first light and the second light emitted by the LCD applied in the augmented reality display system is symmetric about the 0 degree side view, and the side view angle is less than 60 degrees.
  • the display brightness of the brightness of the first light and the second light emitted by the display module 12 is maximum, and when the user's angle of view is shifted to both sides, the display brightness is gradually reduced, and the side view is gradually reduced. When it is less than 60 degrees, the display brightness is 0.
  • the brightness distribution of the first light and the second light emitted by the LCD applied to the augmented reality display system may not be symmetric with respect to the 0 degree side view, and the side view angle when the brightness is the brightest is not 0. degree.
  • the two display modules 12 are respectively located one by one correspondingly above the two fluoroscopic light guiding elements 13.
  • the front surface of the display module 12 and the user's head An angle a is formed, the angle a of the angle a being from 0 to 180 degrees, preferably an obtuse angle.
  • the projection of the display module 12 on the horizontal plane is perpendicular to the normal plane.
  • the position of the see-through light guiding element 13 can be rotated by a certain angle b around a certain axis of rotation perpendicular to the horizontal plane, the angle b of the angle b being 0 degrees to 180 degrees, preferably 0 degrees to 90 degrees.
  • the see-through light guiding elements 13 corresponding to the left and right eyes can be adjusted in pitch by the mechanical structure on the head frame 11 to accommodate the user's interpupillary distance, ensuring comfort and imaging quality in use.
  • the farthest distance between the edges of the two see-through light guiding elements 13 is less than 150 mm, that is, the left edge of the see-through light guiding element 13 corresponding to the left eye to the see-through light guiding element corresponding to the right eye
  • the distance to the right edge of 13 is less than 150 mm.
  • the display modules 12 are connected by a mechanical structure, and the distance between the display modules 12 can also be adjusted, or the same effect can be achieved by adjusting the position of the display content on the display module 12.
  • the headgear frame 11 may be an eyeglass frame structure for hanging on the ear and nose of the user, on which the nose pad 111 and the temple 112 are disposed, and the nose pad 111 and the temple 112 are fixed to the user's head.
  • the temple 112 is a foldable structure, wherein the nose pad 111 is correspondingly fixed on the nose bridge of the user, and the temple 112 is correspondingly fixed on the user's ear.
  • the glasses legs 112 can also be connected by an elastic band, and the elastic band tightens the temples when worn to help the frame to be fixed at the head.
  • the nose pad 111 and the temple 112 are telescopic mechanisms that adjust the height of the nose pad 111 and the telescopic length of the temple 112, respectively.
  • the nose pad 111 and the temple 112 can also be of a detachable structure, and the nose pad 111 or the temple 112 can be replaced after disassembly.
  • the head frame 11 may include a nose pad and a stretch rubber band that is fixed to the user's head by a nose pad and a stretch rubber band; or only a stretch rubber band that is fixed to the user's head by the stretch rubber band.
  • the headgear frame 11 may also be a helmet-type frame structure for wearing on the top and nose of the user's head.
  • the head frame 11 since the main function of the head frame 11 is to be worn on the user's head and to provide support for the optical and electrical components such as the display module 12 and the see-through light guiding element 13, the head frame includes but not Limited to the above manner, under the premise of having the above-mentioned main effects, those skilled in the art can make some modifications to the head frame according to the needs of practical applications.
  • the display module 12 emits a first light ray 121 including left-eye virtual image information, and the first light ray 121 reflected by the concave surface 131 of the see-through light guiding element 13 enters the left eye 14 of the user; similarly, the display module emits Another first light containing the virtual image information of the right eye, another first light reflected by the concave surface of the other see-through light guiding element enters the right eye of the user, thereby forming a visual feeling of the 3D virtual scene in the user's brain,
  • a small display screen is directly disposed in front of the user's right eye, resulting in a small visual area.
  • more display modules are reflected by two fluoroscopy light guiding elements. The first light enters the user's eyes, respectively, and the visual area is large.
  • each of the see-through light guiding elements 13 further has a convex surface disposed opposite to the concave surface; the convex surface of the light guiding type light guiding element 13
  • the concavely transmitted second light containing the external image information enters the user's eyes to form a visual blend of the 3D virtual scene and the real scene.
  • a see-through light guiding element 13 further has a convex surface 132 disposed opposite the concave surface 131, and the second light 151 containing external image information transmitted through the convex surface 132 and the concave surface 131 of the see-through light guiding element 13 enters the user.
  • the left eye 14 is similarly shaped.
  • the other see-through light guiding element further has a convex surface disposed opposite to the concave surface thereof, and the second light containing the external image information transmitted through the convex surface and the concave surface of the see-through light guiding element enters the right side of the user.
  • the user can see the real scene of the outside world, thereby forming a visual experience of mixing the 3D virtual scene and the real scene.
  • a diopter correcting lens 16 is disposed between the human eye and the see-through light guiding element 13, the diopter correcting lens 16 being disposed perpendicular to the horizontal plane.
  • the plane of the diopter correction lens may also be at an angle of 30 degrees to 90 degrees from the horizontal plane.
  • different degrees of diopter correcting lenses may be arbitrarily set.
  • the display module 12 emits a first light ray 121 including left-eye virtual image information, a first light ray 121 reflected through the concave surface 131 of the fluoroscopic light guiding element 13, and a convex surface 132 and a concave surface 131 transmitted through the fluoroscopic light guiding element 13
  • the second light ray 151 of the image information passes through the refractive correction lens 16 before entering the left eye 14 of the user.
  • the refractive correction lens 16 is a concave lens, and the first light 121 and the second light 151 passing therethrough are diverged, so that the focus of the first light 121 and the second light 151 on the left eye 14 are shifted back.
  • the refractive correction lens 16 can also be a convex lens that converges the first light ray 121 and the second light ray 151 thereon to advance the focus of the first light ray 121 and the second light 151 on the left eye 14.
  • the display module emits another first light containing the virtual image information of the right eye, another first light reflected through the concave surface of the other see-through light guiding element, and the convex and concave surface transmitted through the transparent light guiding element.
  • the lens is also corrected by a diopter.
  • the user's eyeball is the apex, and the user's eyeball reaches the virtual display area of the virtual image seen through the see-through light guiding element 13.
  • the side edges form a diagonal field of view.
  • the distance from the farthest end of the head frame to the contact position with the foremost end of the head is c, and the distance length of the c can be adjusted as needed.
  • the angular extent of the diagonal field of view region is inversely proportional to the distance from the most distal end of the head frame 11 to the contact position with the foremost end of the head.
  • the distance from the farthest end of the head frame to the contact position with the foremost end of the head is less than 80 mm under the premise that the diagonal field of view area is greater than 55 degrees.
  • the second display module 12 is connected to the main board 17 by a cable.
  • the main board 17 is also provided with a video interface, a power interface, a communication chip, and a memory.
  • the video interface is used to connect a computer, a mobile phone, or other device to receive a video signal.
  • the video interface may be: hmdi, display port, thunderbolt or usb type-c, micro usb, MHL (Mobile High-Definition Link) and the like.
  • the processor is configured to process data, wherein the video signal is mainly used for decoding and displayed on the display module 12.
  • the power interface is used for external power supply or battery power supply.
  • the power interface includes a USB interface or other interfaces.
  • the communication chip is configured to perform data interaction with the outside world through a communication protocol, specifically, connecting to the Internet through a communication protocol such as WiFi, WDMA, TD-LTE, and then acquiring data through the Internet or connecting with other information processing devices based on the augmented reality; or It is directly connected to other information processing equipment based on augmented reality through a communication protocol.
  • a communication protocol such as WiFi, WDMA, TD-LTE
  • the memory is used for storing data, and is mainly used for storing display data displayed in the display module 12.
  • the information processing apparatus based on the augmented reality includes only the head frame 11, the two display modules 12, the two fluoroscopic light guiding elements 13, and the main board 17, as described above, all 3D virtual scene rendering and image generation corresponding to both eyes are performed. It is performed in an external device connected to an augmented reality-based information processing device.
  • the external device includes: a computer, a mobile phone, a tablet computer, and the like.
  • the information processing device based on the augmented reality receives the video signal of the external device through the video interface, and displays it on the display module 12 after decoding.
  • the interaction with the user is performed by an application software on an external device such as a computer, a mobile phone, a tablet computer, etc.
  • the augmented reality-based information processing device can be interacted by using a mouse keyboard, a touch pad or a button on the external device. Examples of such basic structures include, but are not limited to, large screen portable displays.
  • the information processing device based on the augmented reality can project the display screen at a certain fixed position within the user's field of view. The user needs to adjust the size, position, and the like of the projected screen through software on the device connected to the augmented reality-based information processing device.
  • An information processing apparatus based on the augmented reality provided by the embodiment of the present application reflects the first ray including the left-eye virtual image information and the right-eye virtual image information into the user by the concave surfaces of the two fluoroscopic light guiding elements.
  • the eyes of the eyes form a visual experience of the 3D virtual scene in the user's brain, and the visual area is large.
  • a plurality of sensors are disposed to perform perception on a surrounding environment.
  • an augmented reality-based information processing device provided by the embodiment, the information processing device based on the augmented reality has a total weight of less than 350 grams, and includes: a head frame 21, two display modules 22, and two perspective light guides. Element 23 and main board 24.
  • the display module 22, the see-through light guiding element 23 and the main board 24 are all disposed on the head frame 21.
  • the head frame 21 fixes the display module 22, the see-through light guiding element 23 and the main board 24.
  • the display module 22 is disposed on the upper side of the see-through light guiding element 23, and the light emitted by the display module 22 can be reflected by the see-through light guiding element 23.
  • the main board 24 and the main board 24 are located between the two display modules 22.
  • the main board 24 is provided with a processor for processing virtual image signals and displaying the virtual image information on the display module 22.
  • the head frame 21, the two display modules 22, the two see-through light guiding elements 23, the main board 24 and the head frame 11 described in the first embodiment, the two display modules 12, the two see-through light guiding elements 13, and the main board 17 The specific functions, structures, and positional relationships are the same and will not be described here.
  • a diopter correcting lens is disposed between the human eye and the see-through light guiding element 23, the diopter correcting lens being disposed perpendicular to the horizontal plane.
  • different degrees of diopter correcting lenses may be arbitrarily set.
  • the head frame 21 is further provided with a monocular camera 211, a binocular/multi-view camera 212, an eyeball tracking camera 213, a gyroscope 214, an accelerometer 215, a magnetometer 216, a depth of field sensor 217, an ambient light sensor 218, and/or a distance. Sensor 219.
  • the monocular camera 211, the binocular/multi-view camera 212, the eyeball tracking camera 213, the gyroscope 214, the accelerometer 215, the magnetometer 216, the depth of field sensor 217, the ambient light sensor 218, and/or the distance sensor 219 are all electrically connected to the main board 24 on.
  • the monocular camera 211 is a color monocular camera placed at the front of the head frame 21.
  • the monocular camera 211 faces the other side with respect to the user's face, and the camera can be used for photographing.
  • the information processing device based on the augmented reality may be assisted by using the camera to detect a position known in the environment using computer vision technology.
  • the monocular camera 211 can also be a high-resolution camera for taking photos or taking video; the captured video can also superimpose the virtual object seen by the user through software, and reproduce the user through the augmented reality-based information processing device. What you see.
  • the binocular/multi-view camera 212 may be a monochrome or color camera disposed on the front or side of the head frame 21 and located on one side, two sides or all sides of the monocular camera 211. Further, the binocular/multi-view camera 212 may be provided with an infrared filter. Using the binocular camera, you can further obtain the depth of field information on the image based on the environment image. With a multi-camera camera, you can further expand the camera's viewing angle to get more ambient image and depth of field information.
  • the environmental image and distance information captured by the dual/multi-view camera 212 can be used to: (1) fuse with the data of the gyroscope 214, the accelerometer 215, and the magnetic field meter 216 to calculate the pose of the information processing device based on the augmented reality. (2) Capture user gestures, palm prints, etc. for human-computer interaction.
  • each of the above-mentioned monocular camera or binocular/multi-view camera may be one of an RGB camera, a monochrome camera or an infrared camera.
  • the eyeball tracking camera 213 is disposed on one side of the see-through light guiding element 23, and when the user wears the augmented reality based information processing device, the eyeball tracking camera 213 faces the side opposite to the user's face.
  • the eyeball tracking camera 213 is used to track the focus of the human eye, and to track and specialize the specific parts in the virtual object or virtual screen that the human eye is watching. For example, the specific information of the object is automatically displayed next to the object that the human eye is watching.
  • the area of the human eye can display the high-definition virtual object image, while for other areas, only the low-definition image can be displayed, which can effectively reduce the amount of image rendering calculation without affecting the user experience.
  • the gyroscope 214, the accelerometer 215, and the magnetometer 216 are disposed between the two display modules 22.
  • the relative pose between the user's head and the initial position of the system can be obtained by fusing the data of the gyroscope 214, the accelerometer 215, and the magnetometer 216.
  • the raw data of these sensors can be further fused with the data of the binocular/multi-view camera 212 to obtain the position and attitude of the information processing device based on the augmented reality in a fixed environment.
  • the depth of field sensor 217 is disposed at the front of the head frame 21, and can directly obtain depth information in the environment. Compared to the dual/multi-view camera 212, the depth of field sensor can obtain more accurate, higher resolution depth of field data. Similarly, the use of these data can: (1) fuse with the data of the gyroscope 214, the accelerometer 215, and the magnetometer 216 to calculate the pose of the information processing device based on the augmented reality. (2) Capture user gestures, palm prints, etc. to interact with humans. (3) detecting three-dimensional information of objects around the user.
  • the ambient light sensor 218 is disposed on the head frame 21, and can monitor the intensity of ambient light in real time.
  • the information processing device based on the augmented reality adjusts the brightness of the display module 22 in real time according to the change of the ambient light to ensure the consistency of the display effect under different ambient light.
  • the distance sensor 219 is disposed at a position where the information processing apparatus based on the augmented reality is in contact with the face of the user for detecting whether the information processing apparatus based on the augmented reality is worn on the head of the user. If the user removes the information processing device based on the augmented reality, the power can be saved by turning off the display module 22, the processor, or the like.
  • the augmented reality-based information processing device further includes: an infrared/near-infrared LED electrically connected to the main board 24, wherein the infrared/near-infrared LED is used for
  • the mesh/multi-view camera 212 provides a light source.
  • the infrared/near-infrared LED emits infrared rays, and when the infrared rays reach an object acquired through the binocular/multi-view camera 212, the object reflects the infrared rays, and the photosensitive element on the binocular/multi-view camera 212 receives the reflection.
  • the returned infrared rays are converted into electrical signals, followed by imaging processing.
  • the operations that can be performed include the following:
  • An augmented reality-based information processing device can project a display screen at a certain fixed position within a user's field of view. The user can adjust the size, position, and the like of the projection screen through sensors on the augmented reality-based information processing device.
  • the remote control has a button, a joystick, a touchpad, etc., and is connected to an information processing device based on augmented reality through a wired or wireless manner as a human-computer interaction interface.
  • the device and microphone can be integrated by adding an audio decoding and power amplifying chip to the main board, integrating an earphone jack, an earplug, or a speaker, and allowing the user to interact with the information processing device based on the augmented reality using voice.
  • a video interface and a processor are provided on the motherboard.
  • the augmented reality-based information processing apparatus includes the headgear frame 21, the two display modules 22, the two see-through light guiding elements 23, the main board 24, and the plurality of sensors as described above, all of the 3D virtual scene renderings are performed.
  • the image generation corresponding to both eyes and the processing of data acquired by a plurality of sensors can be performed in an external device connected to the information processing device based on the augmented reality.
  • the external device includes: a computer, a mobile phone, a tablet computer, and the like.
  • the information processing device based on the augmented reality receives the video signal of the external device through the video interface, and displays it on the display module 23 after decoding.
  • the external device receives the data acquired by the plurality of sensors on the information processing device based on the augmented reality, and performs processing to adjust the image displayed by the two eyes according to the data, and is reflected on the image displayed on the display module 23.
  • the processor on the augmented reality based information processing device is only used to support the transmission and display of video signals and the transfer of sensor data.
  • a processor with strong computing power is disposed on the motherboard, and some or all of the computer vision algorithms are completed in the information processing device based on the augmented reality.
  • the information processing device based on the augmented reality receives the video signal of the external device through the video interface, and displays it on the display module 23 after decoding.
  • the external device receives data acquired by a part of the sensors on the information processing device based on the augmented reality, and performs processing to adjust the image displayed by both eyes according to the sensor data, and is reflected on the image displayed on the display module 23.
  • the data acquired by the remaining sensors is processed on an augmented reality based information processing device.
  • data acquired by the monocular camera 211, the binocular/multi-view camera 212, the gyroscope 214, the accelerometer 215, the magnetometer 216, and the depth of field sensor 217 are processed in an augmented reality-based information processing apparatus.
  • the data acquired by the eyeball tracking camera 213, the ambient light sensor 218, and the distance sensor 219 are processed in an external device.
  • a processor on an augmented reality based information processing device is used to support transmission and display of video signals, processing of partial sensor data, and transfer of remaining sensor data.
  • a high-performance processor and an image processor are disposed on the main board to perform all operations in an augmented reality-based information processing device.
  • Augmented Reality displays operate as a stand-alone system without the need to connect an external device.
  • the information processing device based on the augmented reality processes the data acquired by the sensor, the image displayed by the two eyes is adjusted, and then displayed on the display module 23 after rendering.
  • a processor on an augmented reality based information processing device is used for decoding processing and display of video signals and processing of sensor data.
  • the concave surface of the see-through light guiding element is plated with a reflective film.
  • the reflective surface of the see-through light guiding element coated with the reflective film has a reflectance of 20% to 80%.
  • the concave surface of the see-through light guiding element is plated with a polarizing reflective film, and the polarization direction of the polarizing reflective film and the polarization of the first light
  • the angle between the directions is greater than 70° and less than or equal to 90°, for example, the polarization direction of the polarizing reflective film is perpendicular to the polarization direction of the first light, achieving a reflectivity of approximately 100%, and, in addition, due to the second image containing external image information.
  • the light is unpolarized light.
  • the concave surface of the see-through light guiding element is plated with a polarizing reflective film, when the second light passes through the polarizing reflective film, nearly 50% of the second light enters the user's eyes, and the user can still see the outside world. Real scene.
  • the convex surface of the see-through light guiding element is coated with an anti-reflection film.
  • the concave surface of the see-through light guiding element is provided with a pressure sensitive reflective film, and by changing the magnitude of the voltage applied to the pressure sensitive reflective film, the reflectance of the pressure sensitive reflective film can be adjusted between 0 and 100% when the pressure sensitive reflective film is When the reflectance is 100%, the information processing device based on the augmented reality can realize the function of the virtual reality.
  • the other surface of the see-through light guiding element disposed opposite the concave surface A pressure-sensitive black sheet is provided thereon, and the light transmittance of the pressure-sensitive black sheet can be adjusted by changing the magnitude of the voltage applied to the pressure-sensitive black sheet.
  • An information processing apparatus based on the augmented reality provided by the embodiment of the present application reflects the first ray including the left eye virtual image information and the right eye virtual image information by the concave surfaces of the two fluoroscopic light guiding elements.
  • the user's eyes form a visual experience of the 3D virtual scene in the user's brain, and the visual area is large.
  • a plurality of sensors are disposed on the information processing device based on the augmented reality. After the sensor senses the surrounding environment, the perceived result can be reflected in the image displayed in the display module, so that the on-site experience is better and the user experience is better.
  • the embodiment provides an information processing method based on augmented reality, including:
  • the participant wears the augmented reality-based information processing device according to the first or second embodiment to connect the information processing device based on the augmented reality to the notebook computer, based on the enhancement.
  • the actual information processing device acquires display data of the virtual image transmitted through the notebook computer, and then displays it on the display module 12 of the information processing device based on the augmented reality.
  • the display data of the virtual image includes: one or several virtual display screens, various virtual three-dimensional objects, and the like, and the two display modules 12 emit light of the display data including the virtual image and combine the acquired light-guided light-guiding elements 13
  • the light containing the image information of the conference room scene, the two kinds of light are merged in the eyes of the user through the synthesis of the see-through light guiding element 13 on the information processing device based on the augmented reality, and the virtual image can be processed by the human brain of the participant.
  • the content of the displayed data is presented in three dimensions in front of the participants. It can be understood that the augmented reality-based information processing device projects the display data of the virtual image into the conference room live image within the user's field of view.
  • all scene renderings, image generation on both display modules 12 are performed in a laptop connected to an augmented reality based information processing device.
  • the interaction with the display data of the virtual image is performed by the application software on the notebook computer. For example, participants can adjust the size and position of the virtual display screen through the application software on the laptop.
  • partial image generation on the two display modules 12 can be performed in a laptop connected to the augmented reality based information processing device, other portions of the scene rendering and other portions of the two display modules 12.
  • Image generation is performed on an augmented reality based information processing device.
  • all of the scene rendering, all image generation on the two display modules 12 is processed in a processor of the augmented reality based information processing device.
  • the information processing apparatus based on the augmented reality can display the display data of the virtual image from the memory in the information processing apparatus based on the augmented reality in the display module 12 without using the notebook computer.
  • the one or several virtual display screens are mainly used to display text content or two-dimensional images that are only visible to the participants, such as a speech at the time of the report, and a private chat message sent by other participants in real time. Since this information appears directly in the participant's field of vision, it can be easily seen and only the participants can see it. Some information that needs to be transmitted privately during the meeting can ensure the confidentiality while ensuring the received message. Read in time.
  • the various virtual three-dimensional objects such as a three-dimensional model of the product, can be rotated, translated, scaled, partially enlarged, etc. by conventional mouse and keyboard operations.
  • the three-dimensional model angle, position, size and other information can be transmitted to other participants in real time through the communication chip, so that they can see the changes of the model synchronously, making the discussion around these three-dimensional objects very convenient.
  • a three-dimensional object can also be a three-dimensional image of a character.
  • the three-dimensional image can be a fictional image resembling an animated character, or it can be a real three-dimensional image synthesized by one or more cameras.
  • the information processing device based on the augmented reality may also be connected to a mobile device such as a mobile phone or a tablet computer, and input through an external wired and wireless mouse keyboard to control display data.
  • a mobile device such as a mobile phone or a tablet computer
  • the first display mode is between the virtual image and the real image
  • the relative angle and the relative position are not fixed display modes
  • the second display mode is a display mode in which the relative angle between the virtual image and the real image and the relative position are fixed.
  • the third mode is a display mode in which a relative angle between the virtual image and the live image is fixed and the relative position is not fixed.
  • the information processing device based on the augmented reality is only used as a display, and does not have the ability to sense the surrounding environment and the position of the head in the three-dimensional space, when the participant's head rotates or moves, the information processing device based on the augmented reality is in the user.
  • the display data projected in the field of view will also rotate and move in the real space, and the position and posture in the participant's field of view will be fixed only. Referring to FIG. 12, the position and posture of the display data of the virtual display screen, the virtual product, and the like in the coordinate system F H of the information processing apparatus based on the augmented reality are fixed.
  • the processor When the participant's head rotates, the position and posture of the information processing device based on the augmented reality changes in the real space, that is, the relative position and posture of F H and F I change, and the projected display data is in real space coordinates. The position and posture in the system F I will also change.
  • This display mode is called "first display mode”.
  • the processor combines the first ray composition of the live view image including the conference room with the second ray of the virtual image, and displays the image in the first display mode.
  • the azimuth camera 211 can be used to detect the position-known mark in the environment by using the computer vision technology to help the augmented reality-based information processing device to perform positioning.
  • the depth of field information in the environment is obtained by the depth of field sensor 217.
  • the information processing apparatus based on the augmented reality may further obtain the depth information on the acquired image by using the binocular/multi-view camera 212 on the basis of obtaining the environment image.
  • the information processing device based on the augmented reality processes the data obtained by the monocular camera 211, the depth of field sensor 217 or the binocular/multi-view camera 212, and the processor uses the computer vision technology to perform 3D modeling on the surrounding environment to identify the real-time in real time. Different objects in the environment and determine their position and posture in space. In this way, the information processing device based on the augmented reality can analyze which spaces in the vicinity of the participants can better project display data such as virtual display screens, virtual products, and other digital contents.
  • the information processing apparatus based on the augmented reality may also combine the image acquired by the gyroscope 214, the accelerometer 215, and the magnetic field meter 216 with the image and depth of field data obtained by the monocular camera 211, the depth of field sensor 217, or the binocular/multi-view camera 212,
  • the position and posture of the information processing device based on the augmented reality in real space, that is, the relative position and angle relationship T of the coordinate systems F H and F I are calculated.
  • the position and posture in the coordinate system F H are known, and the projected digital content can be obtained in the real space by T ( Position and angle in F I ).
  • the position and posture of the projected content in the information processing device coordinate system F H based on the augmented reality can be calculated through the relationship T. Place a projected content such as a virtual screen here.
  • the information processing device based on the augmented reality can implement the "second display mode".
  • the projected content can be "fixed” in the real space by adjusting the position and posture displayed by the projected content in the augmented reality-based information processing device, so that the participants feel I feel that the virtual object is real.
  • the information processing device based on augmented reality uses a gyroscope, an accelerometer and a magnetic field meter to obtain a relative angle between the user's head and the environment in which it is located, and can realize a "third display mode" in which the virtual object and the environment The relative angle between the two is fixed, but the relative position can be moved.
  • the relationship between the first display mode, the second display mode, and the third display mode and the real environment and the user's head is as follows:
  • first display mode the “second display mode” or the “third display mode” may be used for different virtual images, and may be determined by the system software or by the user.
  • the "first display mode”, the "second display mode” or the “third mode” is implemented by a two-dimensional code set in a live view image or other manually set auxiliary mark.
  • the two-dimensional code set in the real-life image is scanned and recognized by the monocular camera 211, the depth of field sensor 217, or the binocular/multi-view camera 212, and the two-dimensional code includes turning on the first display mode and turning on the second display mode.
  • Information or turn on the information in the third display mode After the information in the two-dimensional code is recognized, it is displayed in a display mode corresponding to the information of the two-dimensional code. For example, if the information in the two-dimensional code is scanned to turn on the information of the first display mode, the display is performed in the first display mode; and the information in the scanned two-dimensional code is the second display mode or the third display mode. The information is displayed in the second display mode or the third mode.
  • the manual mark set in the live image may be scanned and recognized by the monocular camera 211, the depth of field sensor 217 or the binocular/multi-view camera 212, and the artificial mark includes opening the first display mode or turning on the second display mode. information. For example, if the information in the manual mark is the information that the first display mode is turned on, the display is performed in the first display mode; and if the information in the identified manual mark is the information in the third mode of the second display mode, Display in the second display mode or the third mode.
  • the artificial mark on the two-dimensional plane set in the two-dimensional code or other real-life image may also be used to assist the positioning by the augmented reality-based information processing device when displaying in the second display mode: according to the monocular camera 211, the depth of field sensor 217 or The shape and size of the two-dimensional code or the artificial mark captured by the binocular/multi-head camera 212 are compared with the actual size and shape of the two-dimensional code or the artificial mark on the two-dimensional plane, and the mark is compared with the camera. Relative position and angle. Since the position of the mark in the environment is fixed, the relative position and angle relationship T of the information processing device based on the augmented reality and the environment can be calculated therefrom, thereby implementing the second display mode.
  • the information processing method based on the augmented reality further includes: synchronously displaying the virtual image in different augmented reality-based information processing devices.
  • the virtual objects projected by the augmented reality-based information processing device worn by each participant may be set to a synchronous mode, and connected to other augmented reality based on the communication chip.
  • the information processing device synchronously transmits the model, size, posture, and position of the virtual object to the display module 12 in each of the augmented reality-based information processing devices worn by the designated participants. Synchronized participants can also control and adjust the size, posture and position of virtual objects, and simultaneously transmit them to other synchronized display participants to achieve collaborative work in an augmented reality environment.
  • the synchronized display is applicable to both the first display mode and the second display mode.
  • the first display mode the synchronized virtual object is at a position within the field of view of the participant. If the synchronous display is performed in the second display mode, the position of the virtual object in the real space is synchronized. And the participants are in the same conference room, the virtual products appear in the same position in space for everyone, and the experience of working together will be very close to the situation when facing a real product, the effect is realistic.
  • the information processing device based on the augmented reality can track the motion of the user's gesture through the monocular camera 211, the depth of field sensor 217, or the binocular/multi-view camera 212, analyze the intent of the participant, and realize the operation of the virtual displayed content.
  • the monocular camera 211 the depth of field sensor 217, or the binocular/multi-view camera 212 to track the position of the finger of the participant, after the click action of the finger is recognized, an operation command corresponding to the finger click action is executed.
  • the corresponding instruction of the grabbing action is executed, for the virtual screen as a whole, or the virtual screen.
  • the object in the object is dragged.
  • an instruction corresponding to the zoomed or zoomed gesture action is performed on the virtual screen or the virtual screen.
  • the object is scaled.
  • gesture operations on virtual objects can also be synchronized to the heads of designated or all participants. If combined with the second display mode, natural and realistic multiplayer interaction can be achieved.
  • the face recognition of the participant can be performed, and the identity information of the participant corresponding to the recognized face is displayed.
  • the A-based augmented reality information processing apparatus may photograph the surrounding environment by the monocular camera 211 or the binocular/multi-view camera 212, and use computer vision technology to identify the faces of all participants, respectively, with the company internal database or The avatars on the social network page of the participants are compared, and the names, positions, backgrounds, etc. of each participant in front of the participants are identified, and directly projected into the information processing device based on the augmented reality, each participant On the nearby virtual reminder board, participants can quickly understand the important information of the other party even if they meet for the first time, making the meeting more efficient.
  • the augmented reality-based information processing method further includes: acquiring a real-life image of a remote conference room and/or a virtual image displayed in a remote augmented reality-based information processing device, and a local real-life image and / or virtual image is synthesized and displayed.
  • conference room 1 and conference room 2 are located at different locations.
  • A, B are in the conference room 1
  • C is in the conference room 2 for the remote conference.
  • Participants A and B can perform real-time three-dimensional modeling of their environment through the camera set in the conference room and the depth of field sensor 217 on the information processing device based on augmented reality, and pass the three-dimensional image through the communication chip.
  • the Augmented Reality-based information processing device Connected to the Augmented Reality-based information processing device transmitted to the C in real time, processed by the processor and projected in front of the C eye, and the 3D image projected in front of the C eye is the image of the conference room where A and B are located and/or worn by A and B.
  • the virtual reality image displayed on the augmented reality-based information processing device and the image of the conference room where C is located and/or the virtual image displayed on the augmented reality-based information processing device worn by C are processed by the processor.
  • C can transmit its own 3D image to the Augmented Reality-based information processing device worn by A and B in the conference room through the network, and then processed by the processor and projected in front of A and B.
  • C's augmented reality-based information processing device uses the second display mode to model the surrounding environment
  • the virtual images of A and B can be fixed on an empty chair in front of the conference table.
  • a and B can also fix the 3D virtual image of C in an empty chair in the conference room where A and B are located. In this way, for all participants, all participants will feel that they are meeting in the same conference room, which greatly enhances the sense of presence.
  • the eyeball tracking camera 213 can also track the focus of the participant's eyes, and track and special treatment of the virtual object or the specific part of the virtual screen that the participant's eye focus is on, for example, in the participant.
  • the local area in which the eyes are concentrated is displayed automatically, and the specific information of the observed object is displayed.
  • the information processing method based on the augmented reality provided by the embodiment combines the real scene image of the conference room with the virtual image, and the virtual image can provide information prompts for the participants, and assist the participants to perform the conference.
  • the user experience is good and easy to use.

Abstract

一种基于增强现实的信息处理方法，包括：获取包含会议室的实景图像的第一光线(121)；发出包含虚拟图像的第二光线(151)；将包含会议室的实景图像的第一光线(121)与包含虚拟图像的第二光线(151)进行合成。区别于现有技术，基于增强现实的信息处理方法将会议室的实景图像与虚拟图像结合，虚拟图像能够为参会者提供信息提示，辅助参会者进行会议的进行，用户体验好，使用方便。

Description

一种基于增强现实的信息处理方法及设备
本申请要求于2017年2月14日提交中国专利局，申请号为2017100793562，发明名称为“一种基于增强现实的信息处理方法及设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及增强现实技术领域，特别是涉及一种基于增强现实的信息处理方法及设备。
背景技术
目前，无论在政府、企业、各类非赢利性组织，还是各类教育机构，会议主要的形式是参会者集中在会议室内，通过投影仪、电视、音响设备等，将各类数字信息，如文档、幻灯片、视频、图像等，传递给所有参会人员。若有远程参会者，则通常使用如Webex，GoToMeetings等网络会议服务，通过互联网共享双方在本地屏幕上显示的内容，同步双方所见的内容。并通过网络或电话进行视频或音频通话，使双方能够看到对方，拉近距离。
发明人在实现本申请的过程中，发现现有技术至少存在以下问题，比如，对于在会议室内的参会者：
数字信息呈现不够直观。由于数字信息的呈现主要依赖二维的显示屏幕或投影，在很多需要展示三维模型的情况下，比如产品展示、工程部件演示等等，从二维平面式只能从一个或几个角度观察，不够直观。
难以协同控制数字信息。以产品展示为例，当主持人在屏幕上展示产品的三维模型是，仅主持人可以控制模型的展示角度和模型大小，其他参会人员在讨论过程中，如果需要转动三维模型，则需要向主持人描述转动方向，缩放部位等等，无法直接进行操作，非常不便，不利于协同工作。
无法便捷地对每个参会者提供个性化的私密信息服务。举例来说， 在大企业内两个团队第一次开会，由于双方是第一次见面，即便做了自我介绍，也常常会在会议的过程当中无法回忆起对方中某人或某几个人的姓名、职务、背景等等。因此容易产生各种误会和不变。现有的会议解决方案对此完全束手无策。另外，在开会的过程中，特别是对于商务会谈，时常出现需要与某人或某几人私下交流的状况，现有的网络会议服务提供了网络私聊的功能，来尝试解决这一问题。但是，由于开会时大家的注意力不会一直集中在自己面前的电脑屏幕上，而是更多的目视正在说话的参会者，这样，私聊的信息传递会非常不及时，往往被收信方看见时已经为时过晚。而通过其他方式交流，私密性有无法得到保障。
对于远程的参会者，现有的会议方式也有不少缺陷：
临场感有限，讨论时融入感差，无法面对面交流。由于远程参会者和本地参会者主要通过摄像头拍摄现场让对方见到自己，而仅在电脑屏幕显示对方的视频，临场感差，难以融入会议展开讨论。如果会议时仅仅通过网络音频或电话沟通，那情况会更加糟糕，甚至会出现人多的一方展开热烈的讨论，而将电话另一端的参会者完全忽略的情况。
无法清晰的了解正在说话的参会者。假若在每一个远程会议点都只有一名参会者，当某位参会者说话时，现有的网络会议软件均能提示是哪一个会议点的参会者在说话。但是，如果某一个会议室内有多位参会者，现有的会议软件无法分辨是哪位参会者正在说话。对于其他远程会议点的参会者，倘若对其他参会者并不熟悉，即便有网络视频，也常常会难以分辨是谁正在说话。
与在会议室内的参会者一样，远程参会者同样面对数字信息呈现不够直观，难以协同控制数字信息，以及无法便捷地对每个参会者提供个性化的私密信息服务等问题。
发明内容
本申请实施例主要解决的技术问题是提供一种基于增强现实的信息处理方法及设备，具有临场感，内容显示直观。
为解决上述技术问题，本申请实施例提供一种基于增强现实的信息处理方法，包括：获取包含会议室的实景图像的第一光线；发出包含虚拟图像的第二光线；将所述包含会议室的实景图像的第一光线合成与包含虚拟图像的第二光线进行合成。
区别于现有技术，本实施例提供的一种基于增强现实的信息处理方法将会议室的实景图像与虚拟图像结合，虚拟图像能够为参会者提供信息提示，辅助参会者进行会议的进行，用户体验好，使用方便。
附图说明
一个或多个实施例通过与之对应的附图中的图片进行示例性说明，这些示例性说明并不构成对实施例的限定，附图中具有相同参考数字标号的元件表示为类似的元件，除非有特别申明，附图中的图不构成比例限制。
图1a是本申请实施例一提供的一种基于增强现实的信息处理设备的结构示意图；
图1b是图1a所示的透视型导光元件设置在头戴框架上时的示意图；
图1c是图1a所示的显示模块的侧视角度与显示亮度之间的第一关系图；
图1d是图1a所示的显示模块的侧视角度与显示亮度之间的第二关系图；
图1e是图1a所示的显示模块的侧视角度与显示亮度之间的第三关系图；
图2a是佩戴图1a所示的基于增强现实的信息处理设备时显示模块与用户脸部的位置关系示意图；
图2b是旋转图1a所示的显示模块的示意图；
图3是图1a所示的基于增强现实的信息处理设备的成像原理示意 图；
图4是图1a所示的基于增强现实的信息处理设备设置屈光度矫正镜片时的示意图；
图5是图1a所示的基于增强现实的信息处理设备对角线视场区域与头部框架的最远端到用户头部最前端的距离关系的示意图；
图6是图1a所示的基于增强现实的信息处理设备连接外接设备工作时的示意图；
图7是本申请实施例二提供的一种基于增强现实的信息处理设备的结构示意图；
图8是图7所示的基于增强现实的信息处理设备连接外接设备工作时的示意图；
图9是图7所示的基于增强现实的信息处理设备连接外接设备工作时的又一示意图；
图10是图7所示的基于增强现实的信息处理设备工作时的示意图；
图11是本申请第三实施例提供的一种基于增强现实的信息处理方法的应用实例图；
图12是本申请第三实施例提供的一种基于增强现实的信息处理方法中的第一显示模式的示意图；
图13是本申请第三实施例提供的一种基于增强现实的信息处理方法中的第二显示模式的示意图；
图14是本申请第三实施例提供的一种基于增强现实的信息处理方法中的同步显示时的示意图；
图15是本申请第三实施例提供的一种基于增强现实的信息处理方法进行人脸识别时的示意图；
图16是本申请第三实施例提供的一种基于增强现实的信息处理方 法进行远程显示时的示意图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。
此外，下面所描述的本申请各个实施例中所涉及到的技术特征只要彼此之间未构成冲突就可以相互组合。
实施例一
参阅图1a，本申请实施例提供的一种基于增强现实的信息处理设备，所述基于增强现实的信息处理设备的总重量小于350克，其包括：头戴框架11、两个显示模块12、两个透视型导光元件13。其中，透视型导光元件13是一种部分透射、部分反射的光学合成装置。
所述显示模块12及透视形导光元件13皆设置在头戴框架11上，支架11将显示模块12及透视形导光元件13进行固定。显示模块12设置在透视形导光元件13的上侧，显示模块12发出的光线能够经过透视形导光元件13后发生反射。可选地，所述显示模块13还可位于所述透视型导光元件13的侧方。
所述基于增强现实的信息处理设备还包括：主板17，主板17设置在头戴框架11上，且位于二显示模块12之间。所述主板17上设置有处理器，所述处理器用于处理虚拟图像信号并将虚拟图像信息显示在显示模块12上。
本申请实施例中，头戴框架11用于佩戴在用户的头部，每一透视型导光元件13具有一凹面，凹面朝向用户的双眼设置。经由一透视型导光元件13的凹面反射的第一光线进入用户的左眼，以及经由另一透视型导光元件13的凹面反射的另一第一光线进入用户的右眼，以在用户的头脑中形成3D虚拟场景的视觉。其中，第一光线是由显示模块12发射的，且第一光线包含左眼及右眼的虚拟图像信息。
参阅图1b，两个透视型导光元件13设置在头戴框架11上，分别独立地嵌入到头戴框架11上。可选地，可在制作透视型导光元件的原材料上设置两个对应于用户左右眼的区域，所述区域的形状大小与上述的独立设置时的每一透视型导光元件13的形状大小相同；最终的效果为一块大的透视型导光元件上设置有两个对应于用户左右眼的区域。可以理解为在一块大的透视型导光元件的原材料上加工出两个与独立设置时的透视型导光元件13的形状大小相同的区域，即两个透视型导光元件13一体成型。所述设置有对应于用户左右眼区域的透视型导光元件嵌入到头戴框架11上。
需要说明的是，显示模块12可拆卸安装于头戴框架11上，比如，显示模块为手机、平板电脑等智能显示终端；或者，显示模块固定安装于头戴框架上，比如，显示模块与头戴框架集成设计。
头戴框架11上可以安装两个显示模块12，用户的左眼和右眼分别对应地设置一个显示模块12，例如，一个显示模块12用于发射包含左眼虚拟图像信息的第一光线，另一个显示模块12用于发射包含右眼虚拟图像信息的另一第一光线。两个显示模块12可以分别一一对应地位于两个透视型导光元件13的上方，当基于增强现实的信息处理设备佩戴在用户的头部时，两个显示模块12分别一一对应地位于用户的左眼和右眼的上方；显示模块12也可以位于透视型导光元件的侧方，即两个透视型导光元件位于两个显示模块之间，当基于增强现实的信息处理设备佩戴在用户的头部时，两个显示模块分别一一对应地位于用户的左眼和右眼的侧方。
头戴框架11上也可以安装单个显示模块12，该单个显示模块12上有两个显示区域，一个显示区域用于发射包含左眼虚拟图像信息的第一光线，另一个显示区域用于发射包含右眼虚拟图像信息的另一第一光线。
显示模块包括但不限于LCD(Liquid Crystal Display，液晶显示器)、OLED(Organic Light-Emitting Diode，有机发光二极管)、LCOS (Liquid Crystal On Silicon，硅基液晶)等类型的显示器。
参阅图1c，图中的横向轴标识侧视角度，纵向轴表示显示亮度。显示模块12为LCD时，显示模块12的亮度是随着观察者的角度来变化的。对于普通LCD，在显示亮度为50％时的侧观察角度θ一般都比较大。
LCD应用于对于增强现实显示系统时，则比较适用于小的侧观察角度，这样的显示模块12的亮度就会集中在靠近中心的角度区域。因为增强现实显示系统主要使用靠近中心的角度区域，这样的话投影到用户眼中的第一光线及第二光线的亮度会比较高。参阅图1d，应用于增强现实显示系统中的LCD发出的第一光线及第二光线的亮度在显示亮度为50％时的侧观察角度θ一般都比较小。并且，应用于增强现实显示系统中的LCD发出的第一光线及第二光线的亮度的分布关于0度侧视角左右对称，且侧视角度小于60度。即是，用户视角垂直于显示模块12时，显示模块12发出的第一光线及第二光线的亮度的显示亮度为最大，用户视角向两侧偏移时，显示亮度逐渐减小，在侧视角小于60度时，显示亮度为0。
可选地，参阅图1e，应用于增强现实显示系统的LCD的发出的第一光线及第二光线的亮度分布可不关于0度侧视角对称，且显示亮度最亮时的侧视角度不为0度。
参阅图2a，两个显示模块12分别一一对应地位于两个透视型导光元件13的上方，用户佩戴上所述基于增强现实的信息处理设备时，显示模块12与用户头部的正平面形成一夹角a，所述夹角a的角度为0度至180度，优选为钝角。同时，显示模块12在水平面上的投影与正平面垂直。
参阅图2b，在某些实例中，透视形导光元件13的位置可以绕与水平面垂直的某一转轴旋转一定角度b，所述角度b的角度为0度至180度，优选为0度至90度。同时，对应左眼和右眼的透视型导光元件13可以通过头戴框架11上的机械结构调整间距，以适应不同用户的瞳距，保证使用时的舒适度和成像质量。所述两个透视型导光元件13的边缘 之间的最远距离小于150毫米，即对应于左眼设置的透视型导光元件13的左边缘到对应于右眼设置的透视型导光元件13的右边缘的距离小于150毫米。相应的，显示模块12之间通过机械结构连接，所述显示模块12之间的距离也可以进行调整，或者通过调整显示内容在显示模块12上的位置达到同样的效果。
头戴框架11可以是用于挂在用户耳部和鼻梁部的眼镜式的框架结构，其上设置有鼻托111和镜腿112，通过鼻托111与镜腿112固定在用户的头部，所述镜腿112为可折叠结构，其中鼻托111对应固定在用户的鼻梁上，镜腿112对应固定在用户的耳朵上。进一步的，眼镜腿112之间还可以通过松紧带相连，佩戴时松紧带收紧眼镜腿，帮助框架在头部的固定。
可选地，鼻托111和镜腿112为可伸缩机构，可分别调整鼻托111的高度和镜腿112的伸缩长度。同样，鼻托111和镜腿112还可以为可拆卸结构，拆卸后可对鼻托111或者镜腿112进行更换。
可选地，头戴框架11可包括鼻托和伸缩皮筋，通过鼻托与伸缩皮筋固定在用户头部；或者仅包括伸缩皮筋，通过所述伸缩皮筋固定在用户头部。可选地，头戴框架11也可以是用于佩戴在用户头顶和鼻梁部的头盔式框架结构。本申请实施例中，由于头戴框架11的主要作用是用来佩戴在用户的头部以及为显示模块12、透视型导光元件13等光、电元器件提供支撑，头戴框架包括但不限于上述方式，在具备上述主要作用的前提下，本领域技术人员能够根据实际应用的需要对头戴框架作出若干变形。
参阅图3，显示模块12发射包含左眼虚拟图像信息的第一光线121，经由一透视型导光元件13的凹面131反射的第一光线121进入用户的左眼14；同理，显示模块发射包含右眼虚拟图像信息的另一第一光线，经由另一透视型导光元件的凹面反射的另一第一光线进入用户的右眼，从而在用户的大脑中形成3D虚拟场景的视觉感受，另外，不同于谷歌眼镜中通过在用户的右眼前直接设置一块小型显示屏的方式，导致视觉 区域较小，本申请实施例中，通过两个透视型导光元件反射更多的显示模块发射的第一光线分别进入用户的双眼，视觉区域较大。
在本申请实施例中，当基于增强现实的信息处理设备实现增强现实的功能，每一透视型导光元件13还具有一与凹面相背设置的凸面；经由透视型导光元件13的凸面和凹面透射的包含外界图像信息的第二光线进入用户的双眼，以形成混合3D虚拟场景和真实场景的视觉。再次参阅图1a，一透视型导光元件13还具有与凹面131相背设置的凸面132，经由透视型导光元件13的凸面132和凹面131透射的包含外界图像信息的第二光线151进入用户的左眼14，同理，另一透视型导光元件还具有与其凹面相背设置的凸面，经由该透视型导光元件的凸面和凹面透射的包含外界图像信息的第二光线进入用户的右眼，用户能够看到外界的真实场景，从而形成混合3D虚拟场景和真实场景的视觉感受。
参阅图4，可选地，在人眼与透视型导光元件13之间设置一屈光度矫正镜片16，所述屈光度矫正镜片16垂直于水平面设置。可选地，所述屈光度矫正镜片所在平面也可与水平面成30度到90度的夹角。可选地，可任意设置不同度数的屈光度矫正镜片。显示模块12发射包含左眼虚拟图像信息的第一光线121，经由透视型导光元件13的凹面131反射的第一光线121以及经由透视型导光元件13的凸面132和凹面131透射的包含外界图像信息的第二光线151进入用户的左眼14之前，先经过屈光矫正镜片16。所述屈光矫正镜片16为凹透镜，使经过其上的第一光线121以及第二光线151发散，使第一光线121以及第二光线151在左眼14上的焦点后移。同样，所述屈光矫正镜片16还可为凸透镜，使经过其上的第一光线121以及第二光线151汇聚，使第一光线121以及第二光线151在左眼14上的焦点前移。
同理，显示模块发射包含右眼虚拟图像信息的另一第一光线，经由另一透视型导光元件的凹面反射的另一第一光线以及经由该透视型导光元件的凸面和凹面透射的包含外界图像信息的第二光线进入用户的右眼之前，也先经过一屈光度矫正镜片。
参阅图5，基于增强现实的信息处理设备佩戴在用户头部上后，以用户的眼球为顶点，用户的眼球到通过所述透视型导光元件13看到的虚拟图像的虚拟显示区域的两侧边缘构成对角线视场区域。头部框架的最远端到与头部最前端接触位置的距离为c，可根据需要调节所述c的距离长度。所述对角线视场区域的角度大小与所述头部框架11的最远端到与头部最前端接触位置的距离成反比。优选地，在保证对角线视场区域大于55度的前提下，头部框架的最远端到与头部最前端接触位置的距离小于80mm。
参阅图6，二显示模块12通过电缆连接到主板17上。
主板17上还设置有视频接口、电源接口、通信芯片以及存储器。
所述视频接口用于连接计算机、手机、或其他设备接收视频信号。其中所述视频接口可以为：hmdi、display port、thunderbolt或usb type-c，micro usb,MHL(Mobile High-Definition Link)等接口。
所述处理器，用于处理数据，其中主要用于解码视频信号传输并显示在显示模块12上。
所述电源接口，用于外接电源或电池供电。所述电源接口包括USB接口或者其他接口。
所述通信芯片，用于通过通信协议与外界进行数据交互，具体为通过WiFi、WDMA、TD-LTE等通信协议连接互联网，再通过互联网获取数据或者与其它基于增强现实的信息处理设备连接；或者直接通过通信协议与其它基于增强现实的信息处理设备相连。
所述存储器，用于存储数据，主要用于存储显示模块12中显示的显示数据。
当基于增强现实的信息处理设备仅包括如上所述的头戴框架11、二显示模块12、两个透视型导光元件13及主板17时，所有的3D虚拟场景渲染、对应双眼的图像生成均在与基于增强现实的信息处理设备相连的外接设备中进行。所述外接设备包括：计算机、手机、平板电脑等。
具体地，基于增强现实的信息处理设备通过视频接口接收外接设备的视频信号，解码后在显示模块12上显示。同时，与用户的交互通过计算机、手机、平板电脑等外接设备上的应用软件进行，可通过使用外接设备上的鼠标键盘、触摸板或按钮与所述基于增强现实的信息处理设备进行交互。这种基本结构的应用实例包括但不限于大屏幕便携显示器。基于增强现实的信息处理设备可以将显示屏幕投射在用户视野内的某一固定位置。用户需要通过与基于增强现实的信息处理设备相连的设备上的软件进行调整投射屏幕的尺寸、位置等操作。
本申请实施例提供的一种基于增强现实的信息处理设备，通过两个透视型导光元件的凹面更多地将包含左眼虚拟图像信息以及右眼虚拟图像信息的第一光线分别反射进入用户的双眼，从而在用户的大脑中形成3D虚拟场景的视觉感受，视觉区域较大。
实施例二
参阅图7，在实施例一中提供的一种基于增强现实的信息处理设备的基础上，设置多个传感器进行对周边环境进行感知。
本实施例提供的一种基于增强现实的信息处理设备，所述基于增强现实的信息处理设备的总重量小于350克，其包括：头戴框架21、二显示模块22、两个透视型导光元件23及主板24。
所述显示模块22、透视形导光元件23及主板24皆设置在头戴框架21上，头戴框架21将显示模块22、透视形导光元件23及主板24进行固定。显示模块22设置在透视形导光元件23的上侧，显示模块22发出的光线能够经过透视形导光元件23后发生反射。主板24，主板24位于二显示模块22之间，所述主板24上设置有处理器，所述处理器用于处理虚拟图像信号并将虚拟图像信息显示在显示模块22上。
头戴框架21、二显示模块22、两个透视型导光元件23、主板24与实施例一中所述的头戴框架11、二显示模块12、两个透视型导光元件13、主板17的具体功能、结构及位置关系相同，在此不进行赘述。
同样，在人眼与透视型导光元件23之间设置一屈光度矫正镜片，所述屈光度矫正镜片垂直于水平面设置。可选地，可任意设置不同度数的屈光度矫正镜片。
头部框架21上还设置有单目摄像头211、双目/多目摄像头212、眼球追踪摄像头213、陀螺仪214、加速度计215、磁场计216、景深传感器217、环境光传感器218和/或距离传感器219。
单目摄像头211、双目/多目摄像头212、眼球追踪摄像头213、陀螺仪214、加速度计215、磁场计216、景深传感器217、环境光传感器218和/或距离传感器219皆电连接在主板24上。
具体地，所述单目摄像头211为彩色单目摄像头，放置于头部框架21的前部。用户佩戴所述基于增强现实的信息处理设备时，单目摄像头211朝向相对于用户脸部的另一侧，可以使用该摄像头进行拍照。进一步的，还可以对使用该摄像头，运用计算机视觉技术检测环境中的位置已知的标记，帮助所述基于增强现实的信息处理设备进行定位。
所述单目摄像头211还可以为高分辨率的摄像头，用于拍照或者拍摄视频；拍摄所获得的视频还可以通过软件叠加用户所见的虚拟物体，复现用户通过基于增强现实的信息处理设备看到的内容。
所述双目/多目摄像头212可以是单色或彩色的摄像头，其设置在头戴框架21前部或侧面，且位于单目摄像头211的一侧、两侧或者四周。进一步的，所述双目/多目摄像头212可以带有红外滤光片。使用双目摄像头，可以在获得环境图像的基础上，进一步得到图像上的景深信息。使用多目摄像头，则可以进一步扩展相机的视角，获得更多的环境图像与景深信息。双/多目摄像头212捕获的环境图像和距离信息可被用于:(1)与陀螺仪214、加速度计215、磁场计216的数据相融合，计算基于增强现实的信息处理设备的姿态。(2)捕获用户手势、掌纹等用于人机交互。
可选地，上述的单目摄像头或双目/多目摄像头中的每一目均可是RGB摄像头、单色摄像头或红外摄像头中的一种。
所述眼球追踪摄像头213，设置在透视型导光元件23的一侧，用户佩戴所述基于增强现实的信息处理设备时，眼球追踪摄像头213朝向相对于用户脸部的一侧。所述眼球追踪摄像头213用于跟踪人眼焦点，对人眼所注视的虚拟物件或虚拟屏幕中的特定部位进行追踪和特殊处理。比如，在人眼所注视的物件旁边自动显示此物件的具体信息等。另外对人眼注视的区域可以显示高清晰度的虚拟物件图像，而对其他区域则只显示低清晰度图像即可，这样可以有效减少图像渲染的计算量，而不会影响用户体验。
陀螺仪214、加速度计215、磁场计216设置在二显示模块22之间。可以通过融合陀螺仪214、加速度计215和磁场计216的数据，得到用户头部与系统初始位置间相对姿态。这些传感器的原始数据可以进一步和双目/多目摄像头212的数据进行融合，得到基于增强现实的信息处理设备在固定环境中的位置和姿态。
所述景深传感器217设置在头戴框架21的前部，可以直接获得环境中的景深信息。与双/多目摄像头212相比，景深传感器可以获得更准确、分辨率更高的景深数据。类似的，使用这些数据可以：(1)与陀螺仪214、加速度计215、磁场计216的数据相融合，计算基于增强现实的信息处理设备的姿态。(2)捕获用户手势、掌纹等用与人机交互。(3)检测用户周围物体的三维信息。
所述环境光传感器218设置在头戴框架21上，可以实时监控环境光线的强弱。基于增强现实的信息处理设备根据环境光的变化实时的调整显示模块22的亮度，以保证显示效果在不同环境光下的一致性。
所述距离传感器219设置在基于增强现实的信息处理设备与用户面部接触的位置，用于检测基于增强现实的信息处理设备是否佩戴在用户头部。若用户摘下了基于增强现实的信息处理设备，则可以通过关闭显示模块22、处理器等方式节电。
可选地，所述基于增强现实的信息处理设备还包括：红外/近红外光LED，所述红外/近红外光LED电连接在主板24上，所述红外/近红外 光LED用于为双目/多目摄像头212提供光源。具体为，所述红外/近红外LED发出红外线，在红外线到达通过双目/多目摄像头212获取的物体时，所述物体将红外线反射回去，双目/多目摄像头212上的感光元件接收反射回来的红外线并转换成电信号，接着在进行成像处理。
所述基于增强现实的信息处理设备在进行人机交互时，可进行的操作包括如下：
(1)基于增强现实的信息处理设备可以将显示屏幕投射在用户视野内的某一固定位置。用户可通过基于增强现实的信息处理设备上的传感器进行调整投射屏幕的尺寸、位置等操作。
(2)可以通过各类传感器进行手势、掌纹识别，用于人机交互。
(3)可以通过眼球追踪判断用户的意图，对人眼所观察虚拟物件或虚拟屏幕中的特定部位进行相应处理。
(4)还可以在支架上增加实体或触摸按钮、摇杆等，用于人机交互。
(5)可以配有遥控器，遥控器上有按钮、摇杆、触控板等，通过有线或无线的方式与基于增强现实的信息处理设备相连，作为人机交互界面。
(6)可以通过在主板上增加音频解码和功率放大芯片，集成耳塞插孔、耳塞、或喇叭等发生设备与麦克风，允许用户使用语音与基于增强现实的信息处理设备进行交互。
参阅图8，主板上设置有视频接口和处理器。
当基于增强现实的信息处理设备包括如上所述的头戴框架21、二显示模块22、两个透视型导光元件23、主板24以及如上所述的多个传感器时，所有的3D虚拟场景渲染、对应双眼的图像生成以及多个传感器获取的数据的处理均可在与基于增强现实的信息处理设备相连的外接设备中进行。所述外接设备包括：计算机、手机、平板电脑等。
具体地，基于增强现实的信息处理设备通过视频接口接收外接设备的视频信号，解码后在显示模块23上显示。外接设备接收基于增强现实的信息处理设备上的多个传感器获取的数据，进行处理后根据数据对双眼显示的图像进行调整，在显示模块23上显示的图像上进行体现。基于增强现实的信息处理设备上的处理器仅用于支持视频信号的传输与显示以及传感器数据的传递。
参阅图9，主板上设置有运算能力较强的处理器，将部分或全部计算机视觉算法在基于增强现实的信息处理设备内完成。
具体地，基于增强现实的信息处理设备通过视频接口接收外接设备的视频信号，解码后在显示模块23上显示。外接设备接收基于增强现实的信息处理设备上的部分传感器获取的数据，进行处理后根据传感器数据对双眼显示的图像进行调整，在显示模块23上显示的图像上进行体现。其余传感器获取的数据则在基于增强现实的信息处理设备上处理。例如，单目摄像头211、双目/多目摄像头212、陀螺仪214、加速度计215、磁场计216及景深传感器217获取的数据在基于增强现实的信息处理设备中处理。眼球追踪摄像头213、环境光传感器218及距离传感器219获取的数据在外接设备中处理。基于增强现实的信息处理设备上的处理器用于支持视频信号的传输与显示、部分传感器数据的处理以及其余传感器数据的传递。
参阅图10，主板上设置有高性能的处理器以及图像处理器，在基于增强现实的信息处理设备内完成所有的运算。在这种模式下，增强现实显示无需连接外接设备，可作为一个独立的系统运行。
具体地，基于增强现实的信息处理设备将传感器获取的数据进行处理后,对双眼显示的图像进行调整,渲染后在显示模块23上显示。基于增强现实的信息处理设备上的处理器用于视频信号的解码处理与显示以及传感器数据的处理。
在实施例一及实施例二中所述的基于增强现实的信息处理设备实现增强现实的实际应用中，为了增加透视型导光元件的凹面对显示模块 发射的第一光线的反射率，例如，透视型导光元件的凹面镀有反射膜，较佳的，镀有反射膜的透视型导光元件的凹面的反射率是20％-80％。又如，若第一光线是线偏振光，为了增加透视型导光元件的凹面的反射率，透视型导光元件的凹面镀有偏振反射膜，偏振反射膜的偏振方向与第一光线的偏振方向之间的角度大于70°且小于等于90°，比如：偏振反射膜的偏振方向与第一光线的偏振方向垂直，实现近乎为100％的反射率，另外，由于包含外界图像信息的第二光线是非偏振光，若透视型导光元件的凹面镀有偏振反射膜，当第二光线经由该偏振反射膜时，有近乎50％的第二光线进入用户的双眼，用户仍然能够看到外界的真实场景。为了更好地让包含外界图像信息的第二光线进入用户的双眼，透视型导光元件的凸面镀有增透膜。
在实施例一及实施例二中所述的基于增强现实的信息处理设备的实际应用中，为了实现透视型导光元件的凹面对显示模块发射的第一光线的反射率的可控调节，透视型导光元件的凹面设有压敏反射膜，通过改变加载在压敏反射膜上的电压大小，能够调节压敏反射膜的反射率位于0至100％之间，当压敏反射膜的反射率为100％时，基于增强现实的信息处理设备可以实现虚拟现实的功能。
为了实现透视型导光元件的与凹面相背设置的另一表面对包含外界图像信息的第二光线的透光率的可控调节，透视型导光元件的与凹面相背设置的另一表面上设有压敏黑片，通过改变加载在压敏黑片上的电压大小，能够调节压敏黑片透光率的高低。
本申请实施例提供的一种基于增强现实的信息处理设备，通过两个透视型导光元件的凹面更多地将包含左眼虚拟图像信息及包含右眼虚拟图像信息的第一光线分别反射进入用户的双眼，从而在用户的大脑中形成3D虚拟场景的视觉感受，视觉区域较大。同时在基于增强现实的信息处理设备上设置多个传感器，传感器感知周边的环境后，可将感知的结果在显示模块中显示的图像中进行体现，使得临场感受更好，用户体验更佳。
实施例三
本实施例提供一种基于增强现实的信息处理方法，包括：
获取包含会议室的实景图像的第一光线；
发出包含虚拟图像的第二光线；
将所述包含会议室的实景图像的第一光线合成与包含虚拟图像的第二光线进行合成。
具体地，参阅图11，在传统会议方式的基础上，参会者佩戴实施例一或二所述的基于增强现实的信息处理设备，将基于增强现实的信息处理设备连接至笔记本电脑，基于增强现实的信息处理设备获取通过笔记本电脑传送的虚拟图像的显示数据，然后在基于增强现实的信息处理设备的显示模块12上进行显示。虚拟图像的显示数据包括：一块或数块虚拟显示屏，各种虚拟的三维物体等，两个显示模块12发出包含上述虚拟图像的显示数据的光线再结合获取的经过透视型导光元件13的包含会议室景象图像信息的光线，两种光线经过基于增强现实的信息处理设备上的透视型导光元件13的合成后在用户眼睛内融合，经过参会者的人脑处理，可以将虚拟图像的显示数据的内容以三维的形式呈现在参会者的眼前。可以理解为所述基于增强现实的信息处理设备将虚拟图像的显示数据投射在用户视野内的会议室实景图像中。
在此种应用场景中，所有的场景渲染、两个显示模块12上的图像生成均在与基于增强现实的信息处理设备相连的笔记本电脑中进行。与虚拟图像的显示数据的交互通过笔记本电脑上的应用软件进行。比如，参会者可通过笔记本电脑上的应用软件进行调整投射虚拟显示屏幕的尺寸、位置等操作。
同样，部分的场景渲染、两个显示模块12上的部分图像生成可在与基于增强现实的信息处理设备相连的笔记本电脑中进行，其他部分的场景渲染以及两个显示模块12上的其他部分的图像生成在基于增强现实的信息处理设备上进行。或者全部的场景渲染、两个显示模块12上 的全部图像生成在基于增强现实的信息处理设备的处理器中处理。
同样，基于增强现实的信息处理设备还可以不接笔记本电脑使用，从基于增强现实的信息处理设备内的存储器中调用虚拟图像的显示数据在显示模块12中进行显示。
所述一块或数块虚拟显示屏，主要用来显示仅参会者本人可见的文字内容或二维图像，比如做报告时的演讲稿，其他参会者实时发送的私聊信息等。由于这些信息直接出现在参会者的视野内，可以很方便地被看到且仅有参会者能看到，一些开会时需要私下传递的信息能够在保证私密性的同时，确保被收信方及时阅读。
所述各种虚拟的三维物件，比如产品的三维模型等，并可以通过常规的鼠标、键盘操作对其进行转动、平移、缩放、局部放大等操作。三维模型角度、位置、大小等信息都可以通过通信芯片实时地传递给其他参会者，使他们能够同步的看到模型的变化，使得围绕这些三维物件的探讨变得非常便利。三维物体还可以是人物的三维形象。三维形象可以是类似动画人物的虚构形象，也可以是通过一个或多个摄像机拍摄后合成的真实三维形象。
由于参会者依然可以看见周围的环境，因此使用鼠标键盘进行输入不会带来任何困难。
可选地，基于增强现实的信息处理设备也可以连接至手机、平板电脑等移动端设备，通过外接有线、无线鼠标键盘进行输入，对显示数据进行控制。
将获取到的所述会议室的实景图像与所述虚拟图像合成后进行显示时包括第一显示模式、第二显示模式或第三模式；所述第一显示模式为虚拟图像与实景图像之间的相对角度以及相对位置皆不固定的显示模式；所述第二显示模式为虚拟图像与实景图像之间的相对角度以及相对位置皆固定的显示模式。所述第三模式为虚拟图像与实景图像间的相对角度固定，相对位置不固定的显示模式。
由于基于增强现实的信息处理设备仅作为显示器，并不具备感知周围环境和头显在三维空间中位置的能力，当参会者头部发生转动或移动时，基于增强现实的信息处理设备在用户视野内投射的显示数据也会跟着在真实空间中发生转动和移动，仅在参会者本人视野内的位置和姿态固定。参阅图12，虚拟显示屏、虚拟产品等显示数据在基于增强现实的信息处理设备的坐标系F H中的位置和姿态是固定的。当参会者头部发生转动时，基于增强现实的信息处理设备在真实空间中的位置和姿态发生变化，即F H与F I的相对位置和姿态发生变化，投射的显示数据在真实空间坐标系F I中的位置和姿态也会随之改变。这种显示模式称为“第一显示模式”。处理器将所述包含会议室的实景图像的第一光线合成与包含虚拟图像的第二光线进行合成后以第一显示模式进行显示。
在实施例二所述的基于增强现实的信息处理设备的应用时，可以通过单目摄像头211运用计算机视觉技术检测环境中的位置已知的标记，帮助所述基于增强现实的信息处理设备进行定位，通过景深传感器217获得环境中的景深信息。或者，基于增强现实的信息处理设备还可以通过使用双目/多目摄像头212，在获得环境图像的基础上，进一步得到获取的图像上的景深信息。接着，基于增强现实的信息处理设备通过将单目摄像头211、景深传感器217或者双目/多目摄像头212获得的数据进行处理，处理器利用计算机视觉技术对周围环境进行3D建模，实时识别真实环境中不同物体并确定它们在空间中的位置和姿态。这样，基于增强现实的信息处理设备能分析得出参会者附近有哪些空间能够较好地投射虚拟显示屏、虚拟产品以及其他数字内容等显示数据。另外，基于增强现实的信息处理设备还可以通过陀螺仪214、加速度计215、磁场计216获取的数据结合单目摄像头211、景深传感器217或者双目/多目摄像头212获得的图像和景深数据，计算基于增强现实的信息处理设备在真实空间中的位置和姿态，即坐标系F H与F I的相对位置和角度关系T。由于基于增强现实的信息处理设备中投射的虚拟图像的显示数据，如虚拟显示屏、虚拟产品等，在坐标系F H中的位置和姿态已知，通过T可以获得投射数字内容在真实空间(F I)中的位置和角度。相对的，若希望投射内容出现在真实空间(F I)的某个位置和角度，则可通过关系T， 计算出投射内容在基于增强现实的信息处理设备坐标系F H中的位置和姿态，将虚拟屏幕等投射内容放置于此。
这样，基于增强现实的信息处理设备可以实现“第二显示模式”。参阅图13，当参会者头部转动时，可以通过调整投射内容在基于增强现实的信息处理设备内显示的位置和姿态，将投射内容“固定”在真实空间内，让参会者感觉到觉得虚拟物体是真实的。
基于增强现实的信息处理设备使用陀螺仪、加速度计和磁场计，获得用户头部与所处环境间的相对角度，可以实现“第三显示模式”在这种显示模式下，虚拟物体的与环境间的相对角度固定，但相对位置可以发生移动。
综上所述，第一显示模式、第二显示模式以及第三显示模式与真实环境以及使用者头部之间的关系如下表所示：
  与环境相对位置 与环境相对角度 与头部相对位置 与头部相对角度
第一显示模式 不固定 不固定 固定 固定
第二显示模式 固定 固定 不固定 不固定
第三显示模式 不固定 固定 固定 不固定
需要注意的是，“第一显示模式”、“第二显示模式”或“第三显示模式”可以针对不同的虚拟图像混合使用，可以由系统软件决定也可以由使用者自主设置。
所述“第一显示模式”、“第二显示模式”或“第三模式”通过实景图像中设置的二维码或者其他人工设定的辅助标记实现。
具体地，通过单目摄像头211、景深传感器217或者双目/多目摄像头212扫描并识别在实景图像中设置的二维码，所述二维码包含开启第一显示模式、开启第二显示模式的信息或开启第三显示模式的信息。在识别出二维码中的信息后，以所述二维码的信息对应的显示模式进行显 示。如，扫描出二维码中的信息为开启第一显示模式的信息，则以第一显示模式进行显示；又如扫描出的二维码中的信息为开启第二显示模式或者第三显示模式的信息，则以第二显示模式或者第三模式进行显示。
同理，可通过单目摄像头211、景深传感器217或者双目/多目摄像头212扫描并识别在实景图像中设置的人工标记，所述人工标记包含开启第一显示模式或者开启第二显示模式的信息。如，识别出人工标记中的信息为开启第一显示模式的信息，则以第一显示模式进行显示；又如识别出的人工标记中的信息为开启第二显示模式第三模式的信息，则以第二显示模式或者第三模式进行显示。
二维码或其他实景图像中设置的二维平面上的人工标记还可以用于辅助以第二显示模式显示时的基于增强现实的信息处理设备进行定位：根据单目摄像头211、景深传感器217或者双目/多目摄像头212拍摄到的二维码或者人工标记的形状与大小，与该二维码或者人工标记在二维平面上的实际大小与形状进行比对，推算出标记与摄像头之间的相对位置和角度。由于标记在环境中的位置固定，则可以由此计算出基于增强现实的信息处理设备与环境的相对位置和角度关系T，从而实现第二显示模式。
所述一种基于增强现实的信息处理方法还包括：将虚拟图像在不同的基于增强现实的信息处理设备中进行同步显示。具体地，参阅图14，每一个参会者佩戴的基于增强现实的信息处理设备投射的虚拟物体，包括虚拟显示器、虚拟产品等等，均可以设置为同步模式，通过通信芯片连接其它基于增强现实的信息处理设备并将虚拟物体的模型、大小、姿态以及位置，同步传送至每一个或指定参会者佩戴的基于增强现实的信息处理设备内的显示模块12中显示。被同步的参会者也可以对虚拟物体的大小、姿态以及位置进行控制与调整，并同步传送至其他同步显示的参会者，实现在增强现实环境下的协同工作。同步显示对于第一显示模式和第二显示模式均适用。在第一显示模式中，同步虚拟物体在参会者是视野范围内的位置。若在第二显示模式中进行同步显示，则同步虚拟物体在真实空间中的位置。且参会者们同处一个会议室内，虚拟产品 对所有人来说都出现在空间中同一位置，协同工作的体验会非常接近面对一个真实产品时的情况，效果逼真。
进一步的，可对虚拟图像进行操控。基于增强现实的信息处理设备可通过单目摄像头211、景深传感器217或者双目/多目摄像头212跟踪用户的手势的运动，分析参会者的意图，实现对虚拟显示的内容进行操作。
示例的，通过利用单目摄像头211、景深传感器217或者双目/多目摄像头212跟踪参会者的手指的位置，在识别到手指的点击动作之后，执行手指点击动作对应的操作指令。
示例的，通过利用单目摄像头211、景深传感器217或者双目/多目摄像头212识别到参会者的手势的抓取动作，则执行抓取动作对应的指令，对虚拟屏幕整体，或者虚拟屏幕中的物体进行拖曳操作。
示例的，通过利用单目摄像头211、景深传感器217或者双目/多目摄像头212识别到放大或缩小的手势动作，则执行放大或缩小的手势动作对应的指令，对虚拟屏幕或者虚拟屏幕中的物体进行缩放。
这些对虚拟物体的手势操作同样可以被同步到指定或所有参会者的头显内，若与第二显示模式相结合，可以实现自然逼真的多人交互。
进一步的，可对参会者进行人脸识别，显示识别到的人脸对应的参会者的身份信息。参阅图15，A基于增强现实的信息处理设备可以单目摄像头211或者双目/多目摄像头212拍摄周围的环境，运用计算机视觉技术识别所有参会者的人脸，分别与公司内部的数据库或参会者社交网络页面上的头像进行比对，识别出面前每一位参会者的姓名、职位、背景等信息，并直接投影在基于增强现实的信息处理设备内、每一位参会者附近的虚拟提示板上，使参会者即便是第一次见面也能迅速了解对方的重要信息，使会议更加高效。
可选地，所述一种基于增强现实的信息处理方法还包括：获取远程的会议室的实景图像和/或远程的基于增强现实的信息处理设备中显示 的虚拟图像，与本地的实景图像和/或虚拟图像合成后进行显示。参阅图16，会议室一和会议室二分别在不同的地点。A、B在会议室一内，C在会议室二进行远程会议。参会者A、B可以通过在会议室一内设置的摄像头以及基于增强现实的信息处理设备上的景深传感器217，对自己所处的环境进行实时的三维建模，并将三维影像通过通信芯片连接网络实时的传送至C的基于增强现实的信息处理设备中，经处理器处理后投射在C眼前，投射在C眼前的三维影像为A、B所在会议室的图像和/或A、B佩戴的基于增强现实的信息处理设备上显示的虚拟图像与C所在会议室的图像和/或C佩戴的基于增强现实的信息处理设备上显示的虚拟图像经处理器合成后的景象。同样，C可以将自己的三维影像通过网络传送至会议室内的A、B佩戴的基于增强现实的信息处理设备中，经处理器处理后投射在A、B的眼前。若C的基于增强现实的信息处理设备使用第二显示模式，对周围环境进行了建模，则可以将A，B的虚拟图像固定在会议桌前的空椅子上。同样的，A和B也可以将C的三维虚拟影像固定在A和B所在会议室内的空椅子上。这样，对所有参会者来说，都会觉得所有参会者是在同一个会议室内开会，极大的提升临场感。
可选地，还可通过眼球追踪摄像头213跟踪参会者的眼睛的焦点，对参会者眼睛焦点所注视的虚拟物件或虚拟屏幕中的特定部位进行追踪和特殊处理，比如，在参会者眼睛所集中观察的局部区域，自动显示注释，和所观察物体的具体信息等。这些附加显示同样可以有选择地与其他参会者进行同步。
区别于现有技术，本实施例提供的一种基于增强现实的信息处理方法将会议室的实景图像与虚拟图像结合，虚拟图像能够为参会者提供信息提示，辅助参会者进行会议的进行，用户体验好，使用方便。
以上所述仅为本申请的实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整的描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。

Claims (22)

  1. 一种基于增强现实的信息处理方法，其特征在于，包括：
    获取包含会议室的实景图像的第一光线；
    发出包含虚拟图像的第二光线；
    将所述包含会议室的实景图像的第一光线合成与包含虚拟图像的第二光线进行合成。
  2. 根据权利要求1所述的方法，其特征在于，所述虚拟图像包括：一块或数块虚拟显示屏和/或虚拟的三维物体。
  3. 根据权利要求2所述的方法，其特征在于，所述一块或数块虚拟显示屏用来显示文字内容或者二维图像；所述虚拟的三维物体包括产品的三维模型或人物的三维形象；所述人物的三维形象包括类似动画人物的虚构形象或通过一个/多个摄像机拍摄后合成的真实三维形象。
  4. 根据权利要求1-3任一项权利要求所述的方法，其特征在于，将获取到的所述会议室的景实景图像与所述虚拟图像合成后进行显示时包括第一显示模式、第二显示模式或第三显示模式；所述第一显示模式为虚拟图像与实景图像之间的相对角度以及相对位置皆不固定的显示模式；所述第二显示模式为虚拟图像与实景图像之间的相对角度以及相对位置皆固定的显示模式；所述第三模式为虚拟图像与实景图像间的相对角度固定，相对位置不固定的显示模式。
  5. 根据权利要求3所述的方法，其特征在于，所述第一显示模式、第二显示模式或第三显示模式通过实景图像中设置的二维码或者其他人工设定的辅助标记实现。
  6. 根据权利要求1-3任一项权利要求所述的方法，其特征在于，所述方法还包括：获取到的所述会议室实景图像与所述虚拟图像合成后显示在多个基于增强现实的信息处理设备上，每一基于增强现实的信息处理设备通信连接，将虚拟图像在不同的基于增强现实的信息处理设备中 进行同步显示。
  7. 根据权利要求1-3任一项权利要求所述的方法，其特征在于，所述方法还包括：跟踪手势的运动，对虚拟图像进行操作。
  8. 根据权利要求1-3任一项权利要求所述的方法，其特征在于，所述方法还包括：进行人脸识别，显示识别到的人脸对应的参会者的身份信息。
  9. 根据权利要求1-3任一项权利要求所述的方法，其特征在于，获取远程的实景图像和/或虚拟图像，与获取的本地的实景图像和/或虚拟图像合成后进行显示。
  10. 根据权利要求1-3任一项权利要求所述的方法，其特征在于，所述方法还包括：对用户眼睛所注视的虚拟图像的特定部位进行追踪和特定处理。
  11. 一种基于增强现实的信息处理设备，包括：
    透视型导光元件，用于获取包含会议室的实景图像的第一光线；
    显示模块，用于显示虚拟图像并发出包含虚拟图像的第二光线；
    透视型导光元件还用于将所述包含会议室的实景图像的第一光线合成与包含虚拟图像的第二光线进行合成。
  12. 根据权利要求11所述的设备，其特征在于，显示模块显示的虚拟图像包括：一块或数块虚拟显示屏和/或虚拟的三维物体。
  13. 根据权利要求12所述的设备，其特征在于，所述一块或数块虚拟显示屏用来显示文字内容或者二维图像；所述虚拟的三维物体包括产品的三维模型或人物的三维形象；所述人物的三维形象包括类似动画人物的虚构形象或通过一个/多个摄像机拍摄后合成的真实三维形象。
  14. 根据权利要求11-13任一项权利要求所述的设备，其特征在于，所述设备还包括处理器、陀螺仪、加速度计、磁场计、单目摄像头、景深传感器或者双目/多目摄像头，所述处理器用于将所述包含虚拟图像 的第一光线与包含会议室的实景图像的第二光线进行合成后以第一显示模式进行显示；或者处理器结合陀螺仪、加速度计、磁场计、单目摄像头、景深传感器或者双目/多目摄像头的数据后以第二显示模式进行显示；或者处理器结合陀螺仪、加速度计或磁场计的数据后，以第三显示模式进行显示；所述第一显示模式为虚拟图像与实景图像之间的相对角度以及相对位置皆不固定的显示模式；所述第二显示模式为虚拟图像与实景图像之间的相对角度以及相对位置皆固定的显示模式；所述第三模式为虚拟图像与实景图像间的相对角度固定，相对位置不固定的显示模式。
  15. 根据权利要求14所述的设备，其特征在于，所述第一显示模式、第二显示模式或第三显示模式通过实景图像中设置的二维码或者其他人工设定的辅助标记实现。
  16. 根据权利要求14所述的设备，其特征在于，所述单目摄像头或双目/多目摄像头中的每一目均可是RGB摄像头、单色摄像头或红外摄像头中的一种。
  17. 根据权利要求11-13任一项权利要求所述的设备，其特征在于，所述设备还包括：通信芯片，用于与其它基于增强现实的信息处理设备进行通信连接；处理器将所述虚拟图像通过通信芯片发送至其它基于增强现实的信息处理设备上，将虚拟图像在其它基于增强现实的信息处理设备中进行同步显示。
  18. 根据权利要求11-13任一项权利要求所述的设备，其特征在于，所述单目摄像头、景深传感器或者双目/多目摄像头还用于跟踪手势的运动，并识别手势动作。
  19. 根据权利要求11-13任一项权利要求所述的设备，其特征在于，所述方法还包括：单目摄像头或者双目/多目摄像头还用于进行人脸识别，识别到的数据经处理器处理后在显示模块中显示识别到的人脸对应的参会者的身份信息。
  20. 根据权利要求11-13任一项权利要求所述的设备，其特征在于， 通信芯片还用于获取远程的会议室的实景图像和/或虚拟图像，处理器将获取远程的会议室的实景图像和/或虚拟图像与获取的本地的实景图像和/或虚拟图像合成后进行显示。
  21. 根据权利要求11-13任一项权利要求所述的设备，其特征在于，所述设备还包括眼球追踪摄像头，用于对用户眼睛所注视的虚拟图像的特定部位进行追踪和特定处理。
  22. 根据权利要求11-13任一项权利要求所述的设备，其特征在于，所述基于增强现实的信息处理设备与外接设备相连时，可通过使用外接设备上的鼠标键盘、触摸板或按钮与所述基于增强现实的信息处理设备进行交互。
PCT/CN2018/073463 2017-02-14 2018-01-19 一种基于增强现实的信息处理方法及设备 WO2018149266A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710079356.2A CN108427195A (zh) 2017-02-14 2017-02-14 一种基于增强现实的信息处理方法及设备
CN201710079356.2 2017-02-14

Publications (1)

Publication Number Publication Date
WO2018149266A1 true WO2018149266A1 (zh) 2018-08-23

Family

ID=63155182

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/073463 WO2018149266A1 (zh) 2017-02-14 2018-01-19 一种基于增强现实的信息处理方法及设备

Country Status (2)

Country Link
CN (1) CN108427195A (zh)
WO (1) WO2018149266A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111582822A (zh) * 2020-05-07 2020-08-25 维沃移动通信有限公司 基于ar的会议方法、装置及电子设备
CN113676690A (zh) * 2020-05-14 2021-11-19 钉钉控股（开曼）有限公司 一种视频会议的实现方法、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102577370A (zh) * 2010-03-31 2012-07-11 新日铁系统集成株式会社 信息处理系统、会议管理装置、信息处理方法、会议管理装置的控制方法以及程序
CN103391411A (zh) * 2012-05-08 2013-11-13 索尼公司 图像处理设备、投影控制方法及程序
CN104603865A (zh) * 2012-05-16 2015-05-06 丹尼尔·格瑞贝格 一种由移动中的用户佩戴的用于通过锚定虚拟对象充分增强现实的系统
US9165318B1 (en) * 2013-05-29 2015-10-20 Amazon Technologies, Inc. Augmented reality presentation
CN106302132A (zh) * 2016-09-14 2017-01-04 华南理工大学 一种基于增强现实的3d即时通讯系统与方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8760499B2 (en) * 2011-04-29 2014-06-24 Austin Russell Three-dimensional imager and projection device
US9367960B2 (en) * 2013-05-22 2016-06-14 Microsoft Technology Licensing, Llc Body-locked placement of augmented reality objects
CN104915979A (zh) * 2014-03-10 2015-09-16 苏州天魂网络科技有限公司 跨移动平台实现沉浸式虚拟现实的系统
CN105915879B (zh) * 2016-04-14 2018-07-10 京东方科技集团股份有限公司 一种视频显示方法、头戴式显示装置及系统
CN105955453A (zh) * 2016-04-15 2016-09-21 北京小鸟看看科技有限公司 一种3d沉浸式环境下的信息输入方法
CN106056405A (zh) * 2016-05-27 2016-10-26 上海青研科技有限公司 基于虚拟现实视觉兴趣区域的广告定向推送技术
CN106354205A (zh) * 2016-08-25 2017-01-25 安徽协创物联网技术有限公司 一种可穿戴虚拟现实实现系统
CN106339087B (zh) * 2016-08-29 2019-01-29 上海青研科技有限公司 一种基于多维坐标的眼球追踪方法及其装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102577370A (zh) * 2010-03-31 2012-07-11 新日铁系统集成株式会社 信息处理系统、会议管理装置、信息处理方法、会议管理装置的控制方法以及程序
CN103391411A (zh) * 2012-05-08 2013-11-13 索尼公司 图像处理设备、投影控制方法及程序
CN104603865A (zh) * 2012-05-16 2015-05-06 丹尼尔·格瑞贝格 一种由移动中的用户佩戴的用于通过锚定虚拟对象充分增强现实的系统
US9165318B1 (en) * 2013-05-29 2015-10-20 Amazon Technologies, Inc. Augmented reality presentation
CN106302132A (zh) * 2016-09-14 2017-01-04 华南理工大学 一种基于增强现实的3d即时通讯系统与方法

Also Published As

Publication number Publication date
CN108427195A (zh) 2018-08-21

Similar Documents

Publication Publication Date Title
US10554921B1 (en) Gaze-correct video conferencing systems and methods
US10643394B2 (en) Augmented reality
US8395655B2 (en) System and method for enabling collaboration in a video conferencing system
US9122321B2 (en) Collaboration environment using see through displays
US8908008B2 (en) Methods and systems for establishing eye contact and accurate gaze in remote collaboration
US20180199039A1 (en) Reprojecting Holographic Video to Enhance Streaming Bandwidth/Quality
WO2018149267A1 (zh) 一种基于增强现实的显示方法及设备
CN107924589B (zh) 通信系统
US20210281802A1 (en) IMPROVED METHOD AND SYSTEM FOR VIDEO CONFERENCES WITH HMDs
JP2017504277A (ja) ウェアラブルディスプレイ装置を用いてミラービデオチャットを可能にするシステム及び方法
US11941167B2 (en) Head-mounted VR all-in-one machine
CN206497255U (zh) 增强现实显示系统
CN108428375A (zh) 一种基于增强现实的教学辅助方法及设备
US9270933B1 (en) System and method for face-to-face video communication
CN108427193A (zh) 增强现实显示系统
CN108446011A (zh) 一种基于增强现实的医疗辅助方法及设备
TW202141120A (zh) 具可調整影像感測模組的頭戴式裝置及其系統
WO2018149266A1 (zh) 一种基于增强现实的信息处理方法及设备
TWI602436B (zh) 虛擬會議系統
CN107111143B (zh) 视觉系统及观片器
CN110060349B (zh) 一种扩展增强现实头戴式显示设备视场角的方法
CN111158145A (zh) 一种单板反射式ar眼镜的投屏装置
US20220398785A1 (en) Augmented image overlay on external panel
CN108696740A (zh) 一种基于增强现实的直播方法及设备
CN114846434A (zh) 非均匀立体渲染

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18754276

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 19/12/2019)

122 Ep: pct application non-entry in european phase

Ref document number: 18754276

Country of ref document: EP

Kind code of ref document: A1