WO2022111668A1

WO2022111668A1 - Virtual-reality fusion display device

Info

Publication number: WO2022111668A1
Application number: PCT/CN2021/133765
Authority: WO
Inventors: 朱帅帅; 邓焯泳
Original assignee: 华为技术有限公司
Priority date: 2020-11-30
Filing date: 2021-11-27
Publication date: 2022-06-02
Also published as: CN114578554B; CN114578554A

Abstract

A virtual-reality fusion display device (100), comprising a first combiner (110), a second combiner (120), a first modulation assembly (130), a second modulation assembly (140), a blocking pattern plate (152) and an optical engine (154). The first combiner (110) receives first ambient light and transmits the first ambient light to the first modulation assembly (130); the first modulation assembly (130) modulates the polarization state of the first ambient light to form second ambient light and images the second ambient light to the blocking pattern plate (152); the blocking pattern plate (152) modulates the polarization state of the second ambient light according to a blocking relationship between a real object (20) and a virtual object (22), to form third ambient light; the first modulation assembly (130) modulates the polarization state of the third ambient light again to form fourth ambient light, and transmits the fourth ambient light to the second modulation assembly (140); the second modulation assembly (140) is used for transmitting the fourth ambient light and second display light to the second combiner (120); and the second combiner (120) can transmit the fourth ambient light and the second display light into eyes (30) of a wearer so as to achieve a virtual-reality fusion display effect.

Description

A display device that realizes the fusion of virtual and real

This application claims the priority of the Chinese patent application filed on November 30, 2020 with the application number 202011374209.6 and titled "Display Device for Realizing Virtual Reality Integration", the entire contents of which are incorporated into this application by reference.

technical field

The present application relates to the technical field of head-mounted display devices, and in particular, to a display device that realizes virtual-real fusion.

Background technique

Head Mounted Display (HMD) refers to a display device worn by the wearer on the head, which can generally be divided into Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (Mixed Reality). , MR) these three categories. Among them, HMDs such as AR and MR can enable the wearer to observe real objects and virtual content at the same time, but because the HMD's combiner projects the light carrying the real object and the light carrying the virtual content into the wearer's eyes at the same time, Therefore, from the perspective of the wearer, the virtual content cannot effectively block the real object, and there are problems such as ghosting between the virtual content and the real object, which cannot achieve the effect of virtual and real fusion, and the wearer's experience effect is poor.

SUMMARY OF THE INVENTION

The purpose of the present application is to provide a display device that realizes virtual-real fusion, so as to solve the problem that the existing head-mounted display device cannot realize the virtual-real fusion.

In order to solve the above technical problems, the present application provides a display device for realizing virtual and real fusion, including: a first combiner, a second combiner, a first modulation component, a second modulation component, a shading template, and an optomechanical; the first The modulation component is located on the light-emitting side of the first combiner; the second modulation component is located on the light-emitting side of the first modulation component and on the light-incident side of the second combiner. The first combiner is configured to receive first ambient light and transmit the first ambient light to the first modulation component, where the first ambient light carries real object information. The first modulation component is configured to modulate the polarization state of the first ambient light to form a second ambient light, and image the second ambient light to the occlusion template. The occlusion template is used to modulate the polarization state of the second ambient light according to the occlusion relationship between the real object and the virtual object to form a third ambient light. The first modulation component modulates the polarization state of the third ambient light again to form a fourth ambient light, and transmits the fourth ambient light to the second modulation component. The second modulation component is used to transmit the fourth ambient light to the second combiner, and is also used to modulate the polarization state of the first display light to form a second display light, and transmit the second display light to the second combiner; wherein the first display light is generated by the optomechanical and carries the virtual object information; the polarization state of the second display light and the polarization of the fourth ambient light state is different. The second combiner is used to transmit the fourth ambient light and the second display light, so that the fourth ambient light and the second display light are incident on the wearer's eyes. Based on this, when the wearer sees the virtual content and the real object, the virtual content and the real object can be blocked from each other, so as to achieve the effect of virtual and real fusion, so as to provide the wearer with a more natural and realistic visual experience.

In some embodiments, the first modulation component includes a first lens; the first lens is facing the light-emitting side of the first combiner, and is used for imaging the first ambient light to the occlusion template, so that The polarization state of light is modulated by the occlusion template.

In some embodiments, the first modulation component further includes a first polarizer, a first polarizing beam splitter and a half-wave plate. The first polarizer and the first polarizing beam splitter are sequentially located on the light-emitting side of the first lens and away from the first combiner; the half-wave plate is located on the side of the first polarizing beam splitter. On the light-emitting side, the shielding template is located on the back focal plane of the first lens, and the angle between the fast axis direction of the half-wave plate and the polarization direction of the third ambient light is 45°. The first polarizer is used to modulate the polarization state of the first ambient light to form the second ambient light; the polarization beam splitter is used to reflect the second ambient light to the shielding template, and for transmitting the third ambient light; the half-wave plate is used to modulate the polarization state of the third ambient light to form the fourth ambient light.

In some embodiments, the second modulation component includes a second polarizer, a second polarizing beam splitter, and a second lens. The second lens faces the light incident side of the second combiner, the second polarizing beam splitter is located on the light incident side of the second lens, and faces the half-wave plate; the optomechanical and the shielding template are both located on the focal plane of the second lens. The second polarizer is used to modulate the polarization state of the first display light to form the second display light. The second polarizing beam splitter is used for reflecting the fourth ambient light to the second lens, and for transmitting the second display light to the second lens. The second lens is used for collimating the fourth ambient light and the second display light.

In some embodiments, the display device further includes a first zoom lens, the first zoom lens is located on the light incident side of the first combiner; the first zoom lens is used for collimating the first ambient light , so that the first ambient light is transmitted in the first combiner.

In some embodiments, the display device further includes a second zoom lens, the second zoom lens is located on the light-emitting side of the second combiner; the second zoom lens is used for collimating through the second combiner The fourth ambient light and the second display light are transmitted.

In some embodiments, the display device further includes an eye tracking component; the eye tracking component is used to capture the gaze direction of the wearer's eyes.

In some embodiments, the display device further includes a controller that is electrically connected to the first zoom lens, the second zoom lens, and the eye tracking component, respectively; the controller is configured to: Adjust the optical power of the first zoom lens and the second zoom lens according to the direction of the wearer's line of sight. Based on this, the controller can adjust the refractive power of the first zoom lens and the second zoom lens to adaptively adjust the presentation effects of the real object and the virtual object.

In some embodiments, the depth of the wearer's gaze point is obtained according to the wearer's line of sight, and the controller is configured to control the first zoom lens and the second zoom lens, so that the real object and the The virtual image corresponding to the virtual object is set on the virtual image plane where the gaze point is located to improve the VAC problem.

In some embodiments, the display device further includes a depth detector; the depth detector is provided on one side of the first coupler and away from the second coupler; the depth detector is used to obtain the wearer's Depth information of the surrounding real environment. According to the depth information and the virtual content, the occlusion template is further configured to refresh the third ambient light in real time, so as to adaptively change the occlusion relationship between the real object and the virtual object. According to the depth information, the optical machine is further configured to refresh the virtual object information carried by the first display light in real time, so as to adaptively change the occlusion relationship between the virtual object and the real object.

In some embodiments, based on the wearer's degree of myopia, the controller is configured to adjust the optical power of the first zoom lens and the second zoom lens according to the following formula.

Wherein, P ₁ is the optical power of the first zoom lens, P ₂ is the optical power of the second zoom lens, M is the wearer's degree of myopia, and V is the line of sight of the wearer's eyes. the depth of the gaze point. Based on this, the display device can adaptively correct the wearer's myopia.

In some embodiments, based on the depth of the gaze point obtained from the wearer's line of sight, the occlusion template is further used to defocus the incident second ambient light; and/or, the optomechanical for defocusing the first display light. Based on this, the virtual-real fusion effect between the real object and the virtual object can be improved, so as to improve the visual experience of the wearer.

In some embodiments, the display device further includes an electrochromic sheet; the electrochromic sheet is sandwiched between the first coupler and the second coupler, and is used to block the transmission through the light from the first combiner. Alternatively, the display device further includes a shielding piece; the shielding piece is sandwiched between the first coupler and the second coupler, and is used to block light passing through the first coupler.

In some embodiments, the display device is a head-mounted display device or a head-up display device. Among them, the head-up display device can be applied in the smart cockpit of the car. For example, the head-up display device can cooperate with the front windshield of a car to present relevant content on the windshield.

In some embodiments, the included angle between the fast axis direction of the half-wave plate and the polarization direction of the third ambient light may also be about 45°.

In some embodiments, the first lens is a single lens, or the first lens is an assembly of two or more lenses.

In some embodiments, the second lens is a single lens, or the second lens is an assembly of two or more lenses.

In some embodiments, the first combiner is a diffractive optical waveguide, a reflective optical waveguide or a pinhole mirror. The second combiner is a diffractive optical waveguide, a reflective optical waveguide or a pinhole mirror.

In some embodiments, the occlusion template is LCoS or DLP.

In the present application, through the cooperation between the structures such as the first modulation component, the occlusion template, the second modulation component, etc., the wearer can see the virtual content and real objects that are combined with the virtual and the real, so as to provide the wearer with a more natural and realistic vision experience.

Description of drawings

FIG. 1 is a schematic diagram of a conventional AR display device.

FIG. 2 is a schematic diagram of a real object and virtual content under a first positional relationship.

FIG. 3 is a schematic diagram of a real object and virtual content in a second positional relationship.

FIG. 4 is a schematic diagram of a real object and virtual content in a third positional relationship.

FIG. 5 is a frame diagram of a head-mounted display device according to an embodiment of the present application.

FIG. 6 is a schematic structural diagram of a head-mounted display device according to an embodiment of the present application.

FIG. 7 is a schematic structural diagram of a head-mounted display device according to an embodiment of the present application.

FIG. 8 is a schematic diagram of the occlusion template, the virtual content, and the scene seen by the wearer when the wearer looks at the real object.

FIG. 9 is a schematic diagram of the occlusion template, the virtual content, and the scene seen by the wearer when the wearer stares at the virtual object.

FIG. 10 is a schematic diagram of a scene when a wearer observes a real object.

FIG. 11 is a schematic diagram of a scene when a wearer observes real objects and virtual content.

12 is a schematic diagram of a scene when a wearer observes real objects and virtual content through the head-mounted display device of the embodiment.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

Please refer to FIG. 1 , taking an AR head-mounted display device 10 as an example, which generally includes an optical machine 12 , a combiner 14 , an in-coupling grating 16 and an out-coupling grating 18 . Light reflected from a real object 20 existing in the real world may pass through the outcoupling grating 18 and the combiner 14 in sequence to be incident on the wearer's eye 30 . Synchronously, the optomechanical 12 is used to output light carrying the virtual content 22; the light carrying the virtual content 22 can be coupled into the combiner 14 through the coupling grating 16, and is totally reflected in the combiner 14 towards the coupler. out of the direction of the grating 18. Afterwards, the light is coupled out by the outcoupling grating 18 and exits to the wearer's eyes 30, correspondingly enabling the wearer to see virtual content 22 that does not exist in the real world. Based on this, while observing the real object 20 through the head-mounted display device 10 , the wearer can also see the virtual content 22 , so as to provide a different visual experience from electronic devices such as mobile phones and televisions.

It should be understood that, according to the position of the real object 20 on the three-dimensional model and the occlusion relationship between the real object 20 and the virtual content 22, the virtual content 22 can be processed by modeling, rendering, etc. to adaptively present the virtual content twenty two. For example, the virtual content 22 is partially occluded by the real object 20; or, there is no occlusion relationship between the virtual content 22 and the real object 20, and so on. Wherein, the virtual content 22 may include virtual objects and corresponding virtual three-dimensional scenes; or, the virtual content 22 may also only include virtual objects or virtual three-dimensional scenes, which is not limited. For example, the virtual content 22 may be a whale floating in the air, a hot air balloon, a building sign, and the like. However, since the real object 20 is an object existing in the real world, the light carrying the real object 20 is directly incident on the wearer's eyes 30 through the head mounted display device 10 . That is, the light carrying the virtual content 22 and the light carrying the real object 20 are relatively independent, and the virtual content 22 generated by the optical machine 12 cannot block the real object 20 .

For ease of understanding, unless otherwise specified, in each embodiment, a cube is generally used as an example of the real object 20 and a cylinder is used as an example of the virtual content 22 for description.

FIG. 2 is a schematic diagram of a real object and virtual content under a first positional relationship. As shown in FIG. 2 , based on the head-mounted display device 10 , under the first positional relationship, there is no overlap between the real object 20 and the virtual content 22 , so that the wearer can relatively clearly see objects in different directions. Real objects 20 and virtual content 22 .

FIG. 3 is a schematic diagram of a real object and virtual content in a second positional relationship. As shown in FIG. 3 , based on the head-mounted display device 10, under the second positional relationship, when the virtual content 22 is located behind the real object 20, although there is overlap between the two, since the virtual content 22 is composed of generated by the optical machine 12 , so the virtual content 22 can be rendered and other processing so as not to display the part of the virtual content 22 located behind the real object 20 . The virtual content 22 can thus be seen relatively clearly behind the real object 20 in the eyes of the wearer.

FIG. 4 is a schematic diagram of a real object and virtual content in a third positional relationship. As illustrated in FIG. 4 , based on the head mounted display device 10 , under the third positional relationship, when the virtual content 22 is located in front of the real object 20 , ideally, the virtual content 22 is to block a part of the real object 20 . That is, the wearer cannot see part of the real object 20 behind through the virtual content 22 . However, since the head-mounted display device 10 can only perform processing such as rendering on the virtual content 22 of the cylinder, but cannot perform processing such as rendering on the real object 20 of the cube, accordingly, the virtual content 22 located in the front cannot be well aligned. The real object 20 is occluded. Problems such as ghosting may exist between the virtual content 22 and the real object 20 , the sense of violation is strong, and the experience effect of the wearer is also poor.

Based on the above problems, the embodiment of the present application provides an optical see-through display device for virtual and real fusion. When the wearer sees the virtual content and the real object, the virtual content and the real object can be blocked from each other, so as to achieve the fusion of the virtual and the real. effect, so as to provide the wearer with a more natural and realistic visual experience.

It should be understood that, in each embodiment of the present application, the optical see-through display device is mainly illustrated by a head-mounted display device, but is not limited thereto. In some other embodiments, the optical see-through display device may also be a head up display device (Head Up Display, HUD). Among them, the head-up display device can be applied in the smart cockpit of the car. For example, the head-up display device can cooperate with the front windshield of a car to present relevant content on the windshield.

In order to distinguish real objects from virtual content, ambient light and display light are defined in various embodiments of the present application. The ambient light refers to the light reflected by the real object, and the ambient light carries real object information; the ambient light includes, for example, a first ambient light, a second ambient light, and the like. The display light is light carrying virtual content, which can be generated by the optical machine 12; the display light includes, for example, a first display light, a second display light, and the like.

Please refer to FIG. 5 and FIG. 6 simultaneously, a head-mounted display device 100 for realizing virtual-real fusion provided by an embodiment of the present application. The head-mounted display device 100 includes: a first combiner 110 and a second combiner 120, and both the first combiner 110 and the second combiner 120 can transmit light through total reflection. The first combiner 110 is mainly used to transmit ambient light, and the second combiner 120 is mainly used to transmit ambient light and display light. It should be understood that, as illustrated in FIG. 5 , in each embodiment, the optical path of ambient light transmission is extended by the cooperation of the first combiner 110 and the second combiner 120 . During the propagation process of the ambient light, the ambient light can be polarized, blocked, etc., relatively conveniently through the relevant structure, so that the visual information of the real object 20 carried by the ambient light can be changed, so as to adjust the real object 20 adaptively. The mutual occlusion relationship between the object 20 and the virtual content 22 . From the perspective of the wearer, the real object 20 and the virtual content 22 can be blocked from each other, and the visual presentation effect of the head-mounted device will be better.

In some embodiments, the first coupler and the second coupler are spaced apart, but not limited thereto. In some other embodiments, the first coupler and the second coupler can also be arranged in layers.

Please refer to FIG. 5 and FIG. 6 again. In some embodiments, the head-mounted display device 100 provided by the embodiments of the present application may further include: a first modulation component 130 , a second modulation component 140 , a blocking template 152 and an optomechanical 154 . The first modulation component 130 , the second modulation component 140 , the shielding template 152 and the optomechanical 154 may all be disposed on the same side of the first combiner 110 and close to the second combiner 120 . It should be understood that the first coupler 110 has less related structures on the side away from the second coupler 120 , so that the thickness of the head-mounted display device 100 can be better controlled, so as to improve the integration of the head-mounted display device 100 Spend. In addition, the head-mounted display device 100 can also have a small volume, which is convenient for the wearer to wear or carry.

In some embodiments, the first modulation component 130 is located on the light-emitting side of the first combiner 110 , and can transmit and modulate the first ambient light emitted by the first combiner 110 to form the second ambient light. The shielding template 152 can modulate and reflect the second ambient light transmitted through the first modulation component 130 to form the third ambient light. The first modulation component 130 can transmit and modulate the third ambient light modulated by the occlusion template 152 again to form the fourth ambient light. The fourth ambient light can be transmitted to the second modulation component 140 for subsequent processing.

In some embodiments, the shielding template 152 may be disposed on one side of the first modulation component 130 and away from the second combiner 120 .

In some embodiments, the first modulation component 130 may perform imaging, polarization and other processing on the first ambient light, and image (relay) the image of the real object 20 corresponding to the formed second ambient light onto the occlusion template 152; That is, the occlusion template 152 is optically conjugated to the real object 20 . Based on this, the second ambient light can be modulated and reflected through the occlusion template 152 to adjust the visual information of the real object 20 carried by the second ambient light.

In some embodiments, the second modulation component 140 is disposed on one side of the first modulation component 130 and away from the shielding template 152 . Taking the second combiner 120 as a reference, the second modulation component 140 is specifically located on the light incident side of the second combiner 120 and away from the first combiner 110 . Wherein, the second modulation component 140 can at least transmit and collimate the fourth ambient light, so as to transmit the fourth ambient light to the second combiner 120 .

In some embodiments, the optomechanical 154 may be disposed on one side of the second modulation component 140 and away from the second combiner 120 . Based on this, the light machine 154 may emit the first display light carrying the virtual content 22 . It should be understood that the second modulation component 140 can also receive the first display light emitted from the optical machine 154, and can perform polarization, collimation, etc. on the first display light to form the second display light, and the second display light The display light is transmitted to the second combiner 120 . Therefore, after the fourth ambient light and the second display light are combined through the second modulation component 140, they can be incident on the second combiner 120 and transmitted in the second combiner 120 to be incident on the wearer's eyes 30, so that the A visual effect of mutual occlusion is achieved between the real object 20 and the virtual content 22 .

Referring to FIG. 7 , in some embodiments, the first combiner 110 may include a first in-coupling grating 112 , a first substrate 114 and a first out-coupling grating 116 . The first in-coupling grating 112 and the first out-coupling grating 116 are disposed at intervals, and are both disposed on the surface of the first substrate 114 . The first ambient light is coupled into the first substrate 114 after being diffracted by the first coupling grating 112 . In the first substrate 114 , a certain diffraction order of the first ambient light may undergo a phenomenon of total reflection so as to be transmitted toward the direction of the first coupling-out grating 116 . The first ambient light transmitted through the first substrate 114 is diffracted again by the first coupling-out grating 116 to be coupled out from the first combiner 110 and enter the first modulation component 130 . It should be understood that the light incident side of the first combiner 110 may be the side corresponding to the first coupling grating 112 , that is, the side where light enters the first combiner 110 as shown in FIGS. 6 and 7 . The light-emitting side of the first combiner 110 may be the side corresponding to the first outcoupling grating 116 , that is, the side where the light exits the first combiner 110 as shown in FIGS. 6 and 7 .

In some embodiments, the first combiner 110 may be, for example, a diffractive optical waveguide, a reflective optical waveguide, or a pin-hole mirror (Pin-mirror), which is not limited thereto.

In some embodiments, as illustrated in FIG. 7 , the first coupling grating 112 and the first coupling out grating 116 are both reflective diffraction gratings, but not limited thereto. In some other embodiments, at least one of the first coupling-in grating 112 and the first coupling-out grating 116 may also be a transmissive diffraction grating. It should be understood that, based on the type of the first in-coupling grating 112 and the first out-coupling grating 116 , the light-incident side and the light-exiting side of the first combiner 110 may be the same or different. For example, if the first coupling grating 112 is a reflective diffraction grating, the light incident side of the first combiner 110 is the side away from the first coupling grating 112 . For another example, the first coupling grating 112 is a transmissive diffraction grating, and the light incident side of the first combiner 110 is the side where the first coupling grating 112 is disposed.

Referring to FIG. 7 , in some embodiments, the first modulation component 130 may include a first lens 132 , the first lens 132 is facing the first out-coupling grating 116 of the first combiner 110 , and the shielding template 152 is located at the first lens 132 on the back focal plane (also known as the second focal plane). Based on this, after the first ambient light is coupled out from the first combiner 110 , the first lens 132 can image the image of the real object 20 corresponding to the first ambient light onto the occlusion template 152 . The occlusion template 152 needs to modulate and reflect the first ambient light; that is, between the first modulation component 130 and the second modulation component 140 , the occlusion template 152 needs to reflect the light carrying real object information. In this regard, in order to cooperate with the shielding template 152, the first modulation component 130 may further include a first polarizer ((Polarizer) 134 and a first polarization beam splitter (Polarization Beam Splitter, PBS) 136, the first polarizer 134 and The first polarizing beam splitter 136 is sequentially located on the light exit side of the first lens 132 and away from the first combiner 110 ; wherein the shielding template 152 is provided on one side of the first polarizing beam splitter 136 and away from the second modulation component 140 . The first polarizer 134 can modulate the first ambient light into S-polarized polarized light, that is, the second ambient light. Based on this, the first polarized beam splitter 136 can reflect the second ambient light to the shielding template 152 .

In some embodiments, since the occlusion template 152 is located on the back focal plane of the first lens 132 on the optical path, the occlusion template 152 is conjugated to the real object 20 . Through the modulation of the first ambient light by the first modulation component 130 , the second ambient light with a specific polarization state can be imaged onto the occlusion template 152 , and the formed image corresponds to the real object 20 .

In some embodiments, the shielding template 152 may be, for example, a liquid crystal on silicon (LCoS) or a digital light processor (Digital Light Processing, DLP) and other devices to modulate ambient light, which is not limited. Based on this, the occlusion template 152 can perform pixel-level polarization phase modulation on the second ambient light, so as to change the visual information of the real object 20 carried by the second ambient light. For example, the controller of the head-mounted display device can obtain the depth, position and attitude information of the real object 20 and the virtual content 22 output by the optical machine, and then by comparing the information of the virtual content 22 and the real object 20, it can The occlusion relationship between the real object 20 and the virtual content 22 is obtained, which will be described in detail below.

Therefore, through the modulation of the occlusion template 152 , a part of the image corresponding to the real object 20 can be occluded, and the part is the part of the real object 20 that is blocked by the virtual content 22 . Therefore, when the subsequent fourth ambient light is incident on the wearer's eyes 30 , the real object 20 seen by the wearer based on the fourth ambient light is different from the real object seen by the wearer after the head mounted display device 100 is removed. There are certain differences between the objects 20 . That is, a part of the real object 20 that the wearer sees based on the fourth ambient light may be missing, and this part is the part that the virtual content 22 blocks the real object 20 . In this way, when the fourth ambient light and the second display light are incident on the wearer's eyes 30, problems such as ghosting and the like are unlikely to occur between the real object 20 and the virtual content 22, so that the real object 20 and the virtual content 22 can be connected between the real object 20 and the virtual content 22. fusion between.

In some embodiments, the occlusion template 152 is an LCoS as an example, the LCoS can modulate the incident second ambient light, and then reflect the modulated third ambient light.

It should be understood that the LCoS can perform binary phase modulation on the second ambient light; that is, the pixels on the LCoS have two states, one that changes the polarization state of the incident light, and the other state that does not change the incident light. polarization state. For ease of understanding, the states of the pixels on the LCoS are defined as a first state and a second state. Wherein, under the modulation of the pixels in the first state, the corresponding part of the second ambient light can reflect the light in the P polarization state, and based on the optical characteristics of the first polarization beam splitter 136, the light in the P polarization state can pass through the first polarization state. A polarizing beam splitter 136 for subsequent processing. Under the modulation of the pixels in the second state, the corresponding part of the second ambient light reflects the light in the S polarization state. Based on the optical characteristics of the first polarization beam splitter 136 , the light in the S polarization state cannot pass through the first polarization state. The polarizing beam splitter 136 ; that is, the image of the real object 20 corresponding to the part of the light is blocked, and thus cannot be incident on the wearer's eyes 30 .

In some embodiments, the pixels on the LCoS can be switched between the first state and the second state to realize modulation of the second ambient light according to requirements such as usage scenarios and picture presentation effects.

In some embodiments, the state of the pixels on the LCoS changes in real time to refresh the fusion effect between the real object 20 and the virtual content 22 in real time.

Referring to FIG. 7 , in some embodiments, the first modulation component 130 may further include a half-wave plate 138 . The half-wave plate 138 is located on the light-emitting side of the first polarizing beam splitter 136 and away from the shielding template 152 . When the third ambient light is transmitted to the half-wave plate 138 through the first polarization beam splitter 136 again, the half-wave plate 138 can modulate the polarization state of the third ambient light to the S-polarized state, that is, the fourth ambient light is formed, It is transmitted to the second combiner 120 in order to cooperate with the second modulation component 140 .

In some embodiments, the included angle between the fast axis direction of the half-wave plate 138 and the polarization direction of the incident third ambient light is 45°; The angle between the polarization directions is approximately 45°. Based on this, after the incident third ambient light in the P-polarized state exits the half-wave plate 138 , the P-polarized state thereof will be modulated into the S-polarized state, thereby forming the fourth ambient light. The fourth ambient light can facilitate transmission through the second modulation component 140 later.

In some embodiments, as illustrated in FIG. 7 , the first lens 132 is illustrated as a single lens. In some other embodiments, the first lens 132 may also be an assembly of two or more lenses, which is not limited.

Referring to FIG. 7 , in some embodiments, the second modulation component 140 may include a second polarizer 142 , a second polarizing beam splitter 144 and a second lens 146 arranged in sequence. The second polarizer 142 is disposed between the optical machine 154 and the second polarizing beam splitter 144 , and the second polarizer 142 can modulate the first display light emitted by the optical machine 154 into light with a P polarization state, that is, to form a second polarizer. Show light. It should be understood that the second polarizing beam splitter 144 faces the half-wave plate 138 , which can reflect the fourth ambient light in the S-polarized state to the second lens 146 . In addition, the second polarizing beam splitter 144 can also transmit the second display light in the P-polarized state to the second lens 146, and based on this, the fourth ambient light and the second display light can be combined.

In some embodiments, the second lens 146 faces the second polarizing beam splitter 144 and is located between the second polarizing beam splitter 144 and the second combiner 120 . The second lens 146 is facing the light incident side of the second combiner 120, and can perform collimation processing on the combined fourth ambient light and the second display light. It should be understood that on the optical path, the shielding template 152 and the optical machine 154 are both located on the focal plane of the second lens 146. Therefore, after the fourth ambient light and the second display light are collimated by the second lens 146, they can Transmission within the combiner 120 and incidence to the wearer's eye 30 .

Taking the mutual occlusion of the real object 20 and the virtual content 22 as an example, since the second ambient light can be adaptively occluded by modulating a part of the occlusion template 152 , this part is the part of the real object 20 that needs to be blocked by the virtual content 22 . When rendering the virtual content 22 , the optical machine 154 may adaptively not display another part of the virtual content 22 , and the other part is the part of the virtual content 22 blocked by the real object 20 . Based on this, similar to the fitting of the pieces in the puzzle, from the perspective of the wearer, the real object 20 and the virtual content 22 can just be merged with each other, and there is no or less overlap between the two. This can achieve a good visual effect of virtual and real fusion.

In some embodiments, the optomechanical 154 may be, for example, based on Liquid Crystal on Silicon (LCoS), Digital Light Processing (DLP), Micro Organic Light-Emitting Diode (Micro Organic Light-Emitting Diode, Micro -OLED), Micro Light-Emitting Diode (Micro-LED) or laser beam scanner (Laser Beam Scanner, LBS), etc., there is no restriction on this.

In some embodiments, as illustrated in FIG. 7 , like the first lens 132 , the second lens 146 is illustrated as a single lens. In some other embodiments, the second lens 146 may also be an assembly of two or more lenses, which is not limited.

Referring to FIG. 7 , in some embodiments, the second combiner 120 may include a second in-coupling grating 122 , a second substrate 124 and a second out-coupling grating 126 . The second in-coupling grating 122 and the second out-coupling grating 126 are spaced apart, and are both disposed on the surface of the second substrate 124 . Similar to the first combiner 110 , the combined fourth ambient light and the second display light are coupled into the second substrate 124 after diffracted by the second coupling grating 122 . In the second substrate 124, a phenomenon of total reflection of a diffraction order of the fourth ambient light and a phenomenon of total reflection of a diffraction order of the second display light, the fourth ambient light and the second display light are both directed toward the first Two out-coupled grating 126 directions transmit. The light transmitted through the second substrate 124 can be diffracted again by the second outcoupling grating 126 to be coupled out from the second coupler 120 . The fourth ambient light and the second display light coupled out from the second combiner 120 may be incident on the wearer's eye 30 . Based on this, the wearer can see the real object 20 and the virtual content 22 in which the virtual and the real are merged, so as to improve the visual experience.

It should be understood that the light incident side of the second combiner 120 may be the side corresponding to the second coupling grating 122 , that is, the side where light enters the second combiner 120 as shown in FIGS. 6 and 7 . The light-emitting side of the second combiner 120 may be the side corresponding to the second outcoupling grating 126 ; that is, the side where light exits the second combiner 120 as shown in FIGS. 6 and 7 .

In some embodiments, similar to the first combiner 110, the second combiner 120 may be, for example, a diffractive optical waveguide, a reflective optical waveguide, or a pinhole mirror, etc., which is not limited.

In some embodiments, as illustrated in FIG. 7 , similar to the first coupling-in grating 112 and the first coupling-out grating 116 , the second coupling-in grating 122 and the second coupling-out grating 126 are both reflective diffraction gratings, but not This is limited. In some other embodiments, at least one of the second coupling-in grating 122 and the second coupling-out grating 126 may also be a transmissive diffraction grating. It should be understood that, based on the types of the second in-coupling grating 122 and the second out-coupling grating 126 , the light-incident side and the light-exiting side of the second combiner 120 may be the same or different. For example, if the second coupling grating 122 is a reflective diffraction grating, the light incident side of the second combiner 120 is the side away from the second coupling grating 122 . For another example, the second coupling grating 122 is a transmissive diffraction grating, and the light incident side of the second combiner 120 is the side where the second coupling grating 122 is disposed.

Referring to FIG. 7 , in some embodiments, the head mounted display device 100 may further include an electrochromic sheet 162 sandwiched between the first coupler 110 and the second coupler 120 .

It should be understood that the head mounted display device 100 according to the embodiments of the present application needs to perform occlusion processing on the light carrying real object information. That is, when the head mounted display device 100 works normally, the first ambient light will not directly enter the wearer's eyes 30 , and the wearer will not directly see the real object 20 . Based on the above requirements, the electrochromic sheet 162 can block part of the first ambient light passing through the first coupler 110 when the power is on, so as to prevent the part of the first ambient light from directly incident on the wearer's eyes 30 and avoid occurrence of Problems such as ghosting between the real object 20 and the virtual content 22 and the disappearance of the occlusion effect on the real object 20 .

In some embodiments, when the electrochromic sheet 162 is not energized, the electrochromic sheet 162 is in a transparent state, based on which the wearer can see through the second coupler 120, the electrochromic sheet 162 and the first The real object 20 can be seen directly by the combination 110 .

In some other embodiments, the head-mounted display device 100 may not use the electrochromic sheet 162, but use a dark-colored blocking sheet, which can also block part of the first ambient light passing through the first coupler .

Referring to FIG. 7 , in some embodiments, the head mounted display device 100 may further include a depth detector 164 . The depth detector 164 may be disposed on one side of the first coupler 110 and away from the second coupler 120 . It should be understood that the depth detector 164 is used to obtain depth information of the real environment around the wearer. The depth detector 164 may be, for example, a device based on a binocular camera, structured light, time-of-flight (ToF), or the like.

Based on the depth information of the real environment around the wearer and the Simultaneous Localization And Mapping (SLAM) system, a three-dimensional model of the real environment around the wearer can be reconstructed. According to the usage scenario, the virtual content 22 and the real object 20 can also be fused adaptively, so as to achieve a better visual presentation effect.

Referring to FIG. 7 , in some embodiments, the head mounted display device 100 may further include a first zoom lens 166 . The first zoom lens 166 is disposed on one side of the first combiner 110 and away from the second combiner 120 . The first zoom lens 166 is used for collimating the first ambient light, so that the first ambient light is transmitted in the first combiner 110 . It should be understood that the collimated first ambient light can be coupled into the first substrate 114 through the first coupling grating 112 to be transmitted toward the direction of the first coupling grating 116 .

Referring to FIG. 7 , in some embodiments, the head-mounted display device 100 may further include a second zoom lens 168 . The second zoom lens 168 is disposed on one side of the second combiner 120 and is far from the first combiner 110; that is, the two combiners (110, 120) are sandwiched between the first zoom lens 166 and the second zoom lens between 168. It should be understood that after the combined fourth ambient light and the second display light are coupled out through the second coupling-out grating 126, the second zoom lens 168 can collimate the combined fourth ambient light and the second display light, so that the collimated fourth ambient light and the second display light are incident on the wearer's eye 30 .

In some embodiments, the first zoom lens 166 is exemplified as a convex lens, and the second zoom lens 168 is exemplified as a concave lens. Both the first zoom lens 166 and the second zoom lens 168 can be, for example, a liquid crystal zoom lens, a liquid zoom lens, an Alvarez lens, or other devices that can realize real-time zooming.

In some other embodiments, both the first zoom lens 166 and the second zoom lens 168 can be replaced with fixed focal length lenses. For example, the first zoom lens 166 is replaced with a convex lens with a fixed focal length, and the second zoom lens 168 is replaced with a concave lens with a fixed focal length, which is not limited.

Referring to FIG. 7 , in some embodiments, the head mounted display device 100 may further include a controller 170 and an eye tracking component 172 . The controller 170 is electrically connected to the first zoom lens 166, the second zoom lens 168 and the eye tracking component 172, respectively. It should be understood that the eye tracking assembly can capture the direction of the gaze of the wearer's eyes 30 . According to the intersection of binocular vision, the depth of the wearer's gaze point can be obtained. Based on the depth, the controller 170 may adjust the power of the first zoom lens 166 and the second zoom lens 168 to adaptively adjust the presentation effects of the real object 20 and the virtual content 22 .

In some embodiments, as shown in FIG. 7 , the number of controllers 170 is one, but not limited thereto. In some other embodiments, the number of the controllers 170 may also be two, so as to control the first zoom lens 166 and the second zoom lens 168 respectively.

Referring to FIG. 7, in some embodiments, the eye tracking component 172 may include a near-infrared transmitter 172a and a near-infrared receiver 172b. The near-infrared transmitter 172a may emit near-infrared light, and the near-infrared receiver 172b may receive the near-infrared light reflected from the wearer's eye 30 . Based on the pupil corneal reflex method, the line of sight direction of the wearer's eyes can be obtained, and the depth of the wearer's gaze point can be obtained according to the intersection of the wearer's eyes in the respective line-of-sight directions.

It should be understood that based on the depth detector 164 acquiring depth information of the real environment around the wearer, the SLAM system can reconstruct a three-dimensional model of the real environment. Among them, the real environment can be understood as a part of the real world, which is generally updated in real time according to the position and posture of the wearer.

Simultaneously, based on the virtual three-dimensional scene input to the controller 170, the mutual occlusion relationship between the virtual object and the real object 20 can be obtained. Wherein, the virtual three-dimensional scene includes virtual objects, and information such as the position, posture, size, and color of the virtual objects in the virtual three-dimensional scene.

Based on the 3D model of the real environment and the virtual 3D scene, the mutual occlusion relationship between the virtual object and the real object 20 can be obtained by comparing the position, posture, depth and size of the virtual object and the real object 20 . For example, by comparing the position and depth of the virtual object and the real object, it can be determined that the virtual object is located behind the real object, and accordingly, the virtual object will be partially occluded by the real object. Another example: by comparing information such as the position and depth of the virtual object and the real object, it can be determined that the virtual object is located in front of the real object, and accordingly, the real object will be partially occluded by the virtual object. Another example: by comparing the position, posture, size, depth and other information of the virtual object and the real object, it can be determined that the depth of the virtual object and the real object are roughly the same, and the orientation of the virtual object and the real object are different, then the virtual object can be determined. and the real object are set at intervals, and they are not occluded from each other.

It should be understood that according to the situation that the real object 20 is blocked by the virtual object, the occlusion template 152 can modulate the second ambient light, and in the subsequent light path, can intercept the part of the image corresponding to the real object 20 that is blocked by the virtual object; After the occlusion template 152 is modulated, only the portion of the image corresponding to the real object 20 that is not occluded by the virtual object can be output.

According to the virtual three-dimensional scene, images corresponding to the left and right eyes of the wearer are respectively generated. It should be understood that, because the wearer generally obtains the depth information of the object being watched through the difference in the viewing angle between its left and right eyes; based on this, in the virtual three-dimensional scene, virtual cameras can be placed at positions corresponding to the wearer's eyes, and each virtual camera The optical axis of the virtual camera is consistent with the line of sight of the wearer's eye 30 , and the position and size of the entrance pupil of the virtual camera match the pupil of the wearer's eye 30 . The image obtained by the virtual camera shooting the virtual three-dimensional scene is the original virtual content. The positions of the eyes of the wearer may be determined according to the position and posture of the wearer, and the direction of the line of sight of the wearer may be determined according to the eye tracking component 172 .

According to the occlusion relationship between the virtual object and the real object 20, the part of the virtual object occluded by the real object 20 can be obtained, and the virtual content 22 in the above embodiments can be obtained by removing this part from the original virtual content. The virtual content 22 may be emitted by the light machine 154 as the first display light.

As described above, the fourth ambient light can be transmitted to the second combiner 120 through the second modulation component 140. After the first display light emitted by the optical machine 154 is modulated by the second modulation component 140 to form the second display light, the Two display rays can also be transmitted to the second combiner 120 and finally incident on the wearer's eye 30 . Based on this, the wearer can see the real object 20 and the virtual content 22 in which the virtual and the real are merged.

It should be understood that when the wearer looks at the real object 20 in the real world, his eyeballs will turn and face the object, and the human brain can judge the depth of the object through the vergence angle of the eyes, which is called the vergence distance (Vergence Distance). . At the same time, the human eye adjusts the diopter of the lens through the contraction of the ciliary muscle to image the target clearly, so the state of the ciliary muscle contraction can give the brain a depth signal, which is also called the Accommodation Distance.

Based on this, in order to better simulate the real world situation, through the cooperation of structures such as the first zoom lens 166 , the second zoom lens 168 and the eye tracking component 172 , the head-mounted display device of each embodiment of the present application can also Defocusing is performed on the light carrying real object information and the light carrying virtual object information to improve the visual effect of virtual and real fusion. FIG. 8 is a schematic diagram of the occlusion template, the virtual content and the scene seen by the wearer when the wearer looks at the real object. Please refer to FIG. , the virtual object can be defocused and blurred to make the effect of virtual and real fusion more realistic. It should be understood that the blurring degree of the virtual object can be determined according to the distance between the virtual object and the wearer's gaze point. The farther away from the gaze point, the more serious the blurring degree; conversely, the closer the distance to the gaze point, the less blurry degree. As shown in FIG. 8 , in order to realize the matching between the real object 20 and the virtual content 22 , the occlusion template 152 can also be subjected to the same blurring processing, that is, defocus blurring processing is performed on the black cylinder in FIG. 8 .

9 is a schematic diagram of the occlusion template, the virtual content, and the scene seen by the wearer when the wearer looks at the virtual object. Please refer to FIG. 9. Since the wearer's gaze point is located on the virtual content 22, the occlusion template 152 and the virtual content 22 are blocked. None of the content 22 is defocused. It should be understood that since the first ambient light is derived from the real object 20 , when the wearer looks at the virtual content 22 , the image corresponding to the real object 20 will also be automatically defocused and blurred, and no special processing is required.

It should be understood that the defocusing of the virtual content 22 may be achieved by a rendering algorithm. In addition, the way of defocusing can also be understood by analogy with the depth of field in the field of photography.

In some embodiments, based on the cooperation of the controller 170 , the first zoom lens 166 , the second zoom lens 168 , the eye tracking component and other structures, the head-mounted display device 100 can adaptively integrate the virtual and the real while realizing the fusion. Corrects the wearer's possible nearsightedness or farsightedness. Taking myopia as an example, it is assumed that the degree of myopia of the wearer is M (unit is Diopter), the corresponding desired virtual image position is V (unit is Diopter), the refractive power of the first zoom lens 166 is P1, and the second zoom lens 168 The optical power is P2, then the parameters shown above can satisfy the following relationship:

The degree of myopia of the wearer can be input by the wearer himself. And according to the depth of the wearer's gaze point (ie the vergence depth) obtained by the eye tracking component, the depth is defined as the virtual image position V. Therefore, the virtual image position V can be adjusted by adjusting the values of P1 and P2 according to the above relationship, so as to achieve clear imaging for different wearers.

It should be understood that for a wearer with presbyopia, the focusing ability of the lens is weak, and the adjusting ability of eye muscles such as the ciliary muscle is also poor. In this regard, the head-mounted display device 100 according to the embodiments of the present application can obtain the depth of the wearer's gaze point in real time through the eye tracking component 172 . Based on the depth, the first zoom lens 166 can adaptively change its optical power, and the second zoom lens 168 can adaptively change its optical power, so as to replace the wearer's lens to achieve focusing to a certain extent function to achieve clear imaging.

Referring to FIG. 10 , when a person observes a real object 20 in the real world (for example, a cylinder and a cube), the vergence depth and the focus depth are consistent and will not conflict. Therefore, the wearer can better feel the relationship between himself and the real object 20, such as the orientation; for example, the wearer can roughly estimate how many steps are required to walk to reach the position of the cylinder.

However, the principle of VR, AR, MR and other types of head-mounted display devices is roughly to project the image of the optical machine 154 or the display screen to a certain virtual image position. The virtual images corresponding to the left eye and the right eye have a certain parallax. When the human eye looks at an object on the virtual image surface, the eyeball will also turn and face the object, so the brain will obtain the depth of the object according to the vergence angle of the eyes. Please refer to Figure 10. The dotted cylinder and dotted cube are the vergence depths felt by the brain when looking at the corresponding object. Taking a cylinder as an example, the wearer obtains the vergence depth of the cylinder as L1 according to the vergence angle of the eyes. At the same time, the human eye will always focus on the virtual image surface for clear imaging, so the depth of the human eye to the virtual image surface is the focusing depth. As exemplified in FIG. 11 , the solid-line cylinder and solid-line cube are respectively the focusing depth perceived by the brain, and an example of the focusing depth is L2. Taking a cylinder as an example, it can be clearly seen that the vergence depth L1 and the focus depth L2 are not equal. When the above two depth signals collide, it is obviously different from the wearer's observation of real objects in the real world, and the wearer cannot well grasp the distance between himself and the cylinder; this also leads to the wearer's easy to produce eyeballs. Fatigue and vertigo, also known as Vergence Accommodation Conflict (VAC).

Based on this, please refer to FIG. 12 , the head-mounted display device 100 according to the embodiments of the present application is based on the cooperation of the controller 170 , the first zoom lens 166 , the second zoom lens 168 , and the eye tracking component 172 , etc. At the same time, the depth of the wearer's gaze point can be obtained through the eye tracking component 172; that is, it can be known whether the wearer is looking at the real object 20 or the virtual content 22. Based on this, by adjusting the refractive power of the first zoom lens 166 and the second zoom lens 168, a virtual image can be set on the corresponding virtual image plane to improve the VAC problem.

When the wearer is looking at the real object 20 that is far away (ie, the cube in FIG. 12 ), the angle at which the wearer’s eyes converge is small. Based on the depth of the wearer’s gaze point calculated by the eye tracking component 172, the first power of the zoom lens 166 and the second zoom lens 168 to set the virtual image at the first virtual image plane S101. Similarly, when the wearer is gazing at the virtual content 22 (ie, the cylinder in FIG. 12 ) that is closer, the angle at which the wearer’s eyes converge is larger. Based on the depth of the wearer’s gaze point calculated by the eye tracking component 172, The power of the first zoom lens 166 and the second zoom lens 168 may be adjusted to set the virtual image at the second virtual image plane S102. Wherein, compared with the first virtual image surface S101, the second virtual image surface S102 is closer to the wearer. Therefore, from the perspective of the wearer, in the process of staring at the real object 20 or the virtual content 22, the corresponding vergence depth and focus depth are consistent, and the wearer is not prone to eye fatigue and dizziness. VAC problem.

The above are specific embodiments of the present application. It should be pointed out that for those skilled in the art, without departing from the principles of the present application, several improvements and modifications can also be made, and these improvements and modifications may also be regarded as The protection scope of this application.

Claims

A display device for realizing virtual-real fusion, comprising: a first combiner, a second combiner, a first modulation component, a second modulation component, a shading template, and an optomechanical; the first modulation component is located in the second a light-emitting side of a combiner; the second modulating component is located on the light-emitting side of the first modulating component and on the light-incident side of the second combiner;

The first combiner is configured to receive first ambient light and transmit the first ambient light to the first modulation component, and the first ambient light carries real object information;

The first modulation component is configured to modulate the polarization state of the first ambient light to form a second ambient light, and image the second ambient light to the occlusion template;

The occlusion template is used to modulate the polarization state of the second ambient light according to the occlusion relationship between the real object and the virtual object to form a third ambient light;

The first modulation component modulates the polarization state of the third ambient light again to form a fourth ambient light, and transmits the fourth ambient light to the second modulation component;

The second modulation component is used for transmitting the fourth ambient light to the second combiner, and is also used for modulating the polarization state of the first display light to form a second display light, and the second display light Light is transmitted to the second combiner; wherein the first display light is generated by the optomechanical and carries the virtual object information; the polarization state of the second display light and the fourth ambient light different polarization states;

The second combiner is used to transmit the fourth ambient light and the second display light, so that the fourth ambient light and the second display light are incident on the wearer's eyes.
The display device according to claim 1, wherein the first modulating component comprises a first lens; the first lens is facing the light-emitting side of the first combiner, and is used to convert the first environment Rays are imaged onto the occlusion template.
The display device of claim 2, wherein the first modulation component further comprises a first polarizer, a first polarizing beam splitter and a half-wave plate;

The first polarizer and the first polarizing beam splitter are sequentially located on the light-emitting side of the first lens and away from the first combiner; the half-wave plate is located on the side of the first polarizing beam splitter. On the light-emitting side, the shielding template is located on the back focal plane of the first lens, and the angle between the fast axis direction of the half-wave plate and the polarization direction of the third ambient light is 45°;

The first polarizer is used to modulate the polarization state of the first ambient light to form the second ambient light; the polarization beam splitter is used to reflect the second ambient light to the shielding template, and for transmitting the third ambient light; the half-wave plate is used to modulate the polarization state of the third ambient light to form the fourth ambient light.
The display device of claim 3, wherein the second modulation component comprises a second polarizer, a second polarizing beam splitter and a second lens;

The second lens faces the light incident side of the second combiner, the second polarizing beam splitter is located on the light incident side of the second lens, and faces the half-wave plate; the optomechanical and the shielding template are both located on the focal plane of the second lens;

The second polarizer is used to modulate the polarization state of the first display light to form the second display light;

The second polarizing beam splitter is used to reflect the fourth ambient light to the second lens, and is also used to transmit the second display light to the second lens;

The second lens is used for collimating the fourth ambient light and the second display light.
The display device according to any one of claims 1 to 4, wherein the display device further comprises a first zoom lens, and the first zoom lens is located on the light incident side of the first combiner; the The first zoom lens is used for collimating the first ambient light.
The display device according to claim 5, wherein the display device further comprises a second zoom lens, the second zoom lens is located on the light exit side of the second combiner; the second zoom lens is used for The fourth ambient light and the second display light transmitted through the second combiner are collimated.
6. The display device of claim 6, wherein the display device further comprises an eye-tracking component; the eye-tracking component is used to capture the gaze direction of the wearer's eyes.
8. The display device of claim 7, wherein the display device further comprises a controller electrically connected to the first zoom lens, the second zoom lens, and the eye tracking assembly, respectively. connection; the controller is configured to adjust the optical power of the first zoom lens and the second zoom lens according to the direction of the wearer's line of sight.
The display device according to claim 8, wherein the depth of the wearer's gaze point is obtained according to the wearer's gaze direction, and the controller is configured to control the first zoom lens and the second zoom lens, The virtual images corresponding to the real object and the virtual object are set on the virtual image plane where the gaze point is located.
The display device according to claim 8, characterized in that, the display device further comprises a depth detector; the depth detector is provided on one side of the first combiner and is far from the second combiner; The depth detector is used to obtain depth information of the real environment around the wearer;

According to the depth information and the virtual content, the occlusion template is further configured to refresh the third ambient light in real time, so as to adaptively change the occlusion relationship between the real object and the virtual object;

According to the depth information, the optical machine is further configured to refresh the virtual object information carried by the first display light in real time, so as to adaptively change the occlusion relationship between the virtual object and the real object.
The display device according to any one of claims 8 to 10, wherein the controller is configured to adjust the light of the first zoom lens and the second zoom lens according to the following formula based on the degree of myopia of the wearer power;

Wherein, P 1 is the optical power of the first zoom lens, P 2 is the optical power of the second zoom lens, M is the wearer's degree of myopia, and V is the line of sight of the wearer's eyes. the depth of the gaze point.
The display device according to claim 11, wherein, based on the depth of the gaze point obtained from the wearer's line of sight, the occlusion template is further used to defocus the incident second ambient light; And/or, the optical machine is used for defocusing and blurring the first display light.
The display device according to any one of claims 1 to 12, wherein the display device further comprises an electrochromic sheet; the electrochromic sheet is sandwiched between the first coupler and the second between the couplers, and used to block the light passing through the first coupler after power-on; or,

The display device further includes a shielding piece; the shielding piece is sandwiched between the first coupler and the second coupler, and is used to block light passing through the first coupler.
The display device according to any one of claims 1 to 13, wherein the display device is a head-mounted display device or a head-up display device.