WO2018045985A1

WO2018045985A1 - Augmented reality display system

Info

Publication number: WO2018045985A1
Application number: PCT/CN2017/100933
Authority: WO
Inventors: 钟张翼; 毛颖
Original assignee: 深圳梦境视觉智能科技有限公司
Priority date: 2016-09-07
Filing date: 2017-09-07
Publication date: 2018-03-15

Abstract

An augmented reality display system, comprising: a head-mounted frame (11), a display module (12), two see-through light guide elements (13), and a motherboard (17). The display module (12), the two see-through light guide elements (13), and the motherboard (17) are provided on the head-mounted frame (11). The beneficial effect of the embodiments comprises: more first light comprising left-eye virtual image information and second light comprising right-eye virtual image information may be respectively reflected into user's eyes by means of concave surfaces of the two see-through light guide elements (13), thereby forming visual experience of a 3D virtual scene with a large visual area in the brain of the user. In addition, third light comprising external image information enters the user's eyes by means of transmission of convex and concave surfaces of the see-through light guide elements (13), so that the user can see the real scene of the outside world, thereby forming visual experience combining the 3D virtual scene and the real scene.

Description

Augmented reality display system

Technical field

The embodiments of the present application relate to the field of augmented reality technologies, and in particular, to an augmented reality display system.

Background technique

Augmented reality display system is a new technology developed in recent years. It is mainly divided into VR (Virtual Reality) and AR (Augmented Reality) according to specific applications. The principle of AR is to simulate virtual vision through an augmented reality display system, superimposed on the user's normal vision. At present, the AR augmented reality display system has two implementations of optical perspective and video perspective, the main difference being that the optical synthesis device is different.

Wherein, the optical synthesis device in the optical perspective augmented reality display system may be a partially transmissive, partially reflective component through which light from a real environment is partially projected, and virtual image information is projected onto the component. It is reflected into the eyes of the user to synthesize real and virtual image information.

In the process of implementing the present application, the inventors have found that at least the following problems exist in the related art: the field of view of the existing augmented reality display system is usually small, and the user cannot efficiently interact with the virtual image information. It is also impossible to form a 3D virtual scene. In addition, the existing augmented reality display system also has a problem of large volume and no portable belt.

Summary of the invention

The technical problem to be solved by the embodiments of the present application is to provide an augmented reality display system with a large visual field area and capable of forming a 3D virtual scene with a small size and a portable belt.

To solve the above technical problem, an embodiment of the present invention provides an augmented reality display system, including: a head frame, a display module, two see-through light guiding elements, and a main board, the display module, two see-through light guiding elements, and The motherboard is placed on the head frame;

The head worn frame for wearing on a user's head;

a main board, the main board is provided with a processor, the processor is configured to process virtual image information and display the virtual image information on the display module;

The display module is detachably or fixedly mounted on the headset frame for displaying virtual image information, and transmitting the virtual image information in the first light and the second light;

The two fluoroscopic light guiding elements each have a concave surface disposed toward the eyes of the user; the first light reflected by the concave surface of the fluoroscopic light guiding element enters the user a left eye, and a second light reflected through the concave surface of the other of the see-through light guiding elements enters the right eye of the user to form a view of the 3D virtual scene; wherein the first light includes left eye virtual image information, The second light ray includes right eye virtual image information.

The beneficial effect of the embodiment of the present application is that the first ray including the left-eye virtual image information and the second ray including the right-eye virtual image information are respectively reflected into the eyes of the user through the concave surfaces of the two fluoroscopic light guiding elements. Thus, a visual experience of a 3D virtual scene is formed in the user's brain, and the visual area is large. In addition, the third light containing the external image information transmitted through the convex and concave surfaces of the see-through light guiding element enters the eyes of the user, and the user can see the real scene of the outside world, thereby forming a visual feeling of mixing the 3D virtual scene and the real scene.

DRAWINGS

The one or more embodiments are exemplified by the accompanying drawings in the accompanying drawings, and FIG. The figures in the drawings do not constitute a scale limitation unless otherwise stated.

1a is a schematic structural diagram of an augmented reality display system according to Embodiment 1 of the present application;

Figure 1b is a schematic view of the see-through light guiding element shown in Figure 1a when it is placed on the head frame;

Figure 1c is a first relationship diagram between a side view angle and a display brightness of the display module shown in Figure 1a;

Figure 1d is a second relationship diagram between the side view angle and the display brightness of the display module shown in Figure 1a;

Figure 1e is a third relationship diagram between a side view angle and a display brightness of the display module shown in Figure 1a;

2a is a schematic diagram showing the positional relationship between the display module and the user's face when the augmented reality display system shown in FIG. 1a is worn;

Figure 2b is a schematic view showing the rotation of the display module shown in Figure 1a;

3 is a schematic diagram of an imaging principle of the augmented reality display system shown in FIG. 1a;

3a is a schematic structural view of a see-through light guiding element provided with the light shielding layer shown in FIG. 1a;

FIG. 3b is a schematic structural diagram of an augmented reality display system according to an embodiment of the present invention; FIG.

3c is a schematic structural view of an embodiment of a display screen in a display module;

Figure 3d is a schematic structural view of still another embodiment of the display screen in the display module;

3e is a schematic structural view of still another embodiment of a display screen in a display module;

Figure 3f is a schematic structural view of still another embodiment of the display screen in the display module;

Figure 4 is a cross-sectional view of a see-through light guiding member for introducing a concave shape of a concave surface;

Figure 4a is a plan view of a see-through light guiding element for introducing a surface recess value;

Figure 4b is a cross-sectional view of the see-through light guiding element for introducing the concave value of the convex surface;

Figure 5 is a schematic view of the augmented reality display system of Figure 1a when a diopter correction lens is provided;

6 is a schematic diagram showing the distance relationship between the diagonal field of view area and the farthest end of the head frame to the foremost end of the user's head of the augmented reality display system shown in FIG. 1a;

7 is a schematic diagram of the augmented reality display system shown in FIG. 1a connected to an external device;

8 is a schematic structural diagram of an augmented reality display system according to Embodiment 2 of the present application;

9 is a schematic diagram of the augmented reality display system shown in FIG. 8 connected to an external device;

10 is another schematic diagram of the augmented reality display system shown in FIG. 8 when the external device is connected to work;

Figure 11 is a schematic illustration of the augmented reality display system of Figure 8 in operation.

Figure 12 is a schematic view showing the arrangement angle and light reflection of the partial structure of the augmented reality display system shown in Figure 1a.

detailed description

In order to make the objects, technical solutions, and advantages of the present application more comprehensible, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to be limiting.

Further, the technical features involved in the various embodiments of the present application described below may be combined with each other as long as they do not constitute a conflict with each other.

Embodiment 1

Referring to FIG. 1a, an augmented reality display system provided by an embodiment of the present application has a total weight of less than 350 grams, including: a head frame 11, two display modules 12, and two perspective light guides. Element 13. Wherein, the see-through light guiding element 13 is a partially transmissive, partially reflective optical synthesizing device.

The display module 12 and the see-through light guiding elements 13 are all disposed on the head frame 11. The bracket 11 fixes the display module 12 and the see-through light guiding element 13. The display module 12 is disposed in a see-through light guide On the upper side of the element 13, the light emitted by the display module 12 can be reflected by the see-through light guiding element 13. Optionally, the display module 12 may also be located at a side of the see-through light guiding element 13 .

The augmented reality display system further includes a main board 17 disposed on the head frame 11 and located between the two display modules 12. The main board 17 is provided with a processor for processing a virtual image signal and displaying the virtual image information on the display module 12.

In the embodiment of the present application, the head frame 11 is used for wearing on the head of the user, and each of the see-through light guiding elements 13 has a concave surface which is disposed toward the eyes of the user. The first light reflected through the concave surface of the see-through light guiding element 13 enters the left eye of the user, and the right eye of the second light user reflected by the concave surface of the other see-through light guiding element 13 to form in the mind of the user The vision of 3D virtual scenes. The first light is emitted by the display module 12, and the first light includes left eye virtual image information, the second light is emitted by the display module 12, and the second light includes right eye virtual image information.

Referring to FIG. 1b, two see-through light guiding elements 13 are disposed on the head frame 11 and are independently embedded in the head frame 11, respectively. Optionally, two regions corresponding to the left and right eyes of the user may be disposed on the raw material for fabricating the fluoroscopic light guiding element, and the shape and size of the region are different from the shape and size of each of the fluoroscopic light guiding members 13 when independently disposed as described above. The same effect; the final effect is that a large see-through light guiding element is provided with two areas corresponding to the left and right eyes of the user. It can be understood that two regions of the same size as the independently disposed fluoroscopy light guiding elements 13 are processed on the raw material of a large fluoroscopic light guiding element, that is, the two fluoroscopic light guiding elements 13 are integrally formed. The see-through light guiding elements provided to correspond to the left and right eye regions of the user are embedded in the head frame 11.

It should be noted that the display module 12 is detachably mounted on the head frame 11, for example, the display module is an intelligent display terminal such as a mobile phone or a tablet computer; or the display module is fixedly mounted on the head frame, for example, the display module and the head. Wear a frame integrated design.

Two display modules 12 can be mounted on the headgear frame 11. The left eye and the right eye of the user are respectively provided with a display module 12, for example, one display module 12 is configured to emit a first light containing virtual image information of the left eye, and A display module 12 is configured to emit a second light that includes virtual image information of the right eye. The two display modules 12 can be respectively located above the two fluoroscopic light guiding elements 13 in a one-to-one correspondence. When the augmented reality display system is worn on the user's head, the two display modules 12 are respectively located one by one on the left side of the user. Above the eye and right eye; the display module 12 can also be located on the side of the see-through light guiding element, that is, two see-through light guiding elements are located between the two display modules, when the augmented reality display system is worn on the user's head The two display modules are respectively located on the side of the left eye and the right eye of the user in a one-to-one correspondence.

A single display module 12 can also be mounted on the headgear frame 11. The single display module 12 has two display areas, one for emitting a first ray containing left eye virtual image information and the other for transmitting The second light of the virtual image information of the right eye.

The display module includes, but is not limited to, an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), an LCOS (Liquid Crystal On Silicon), or the like.

The headgear frame may be a spectacle frame structure for hanging on the ear and nose of the user, or a helmet frame structure for wearing on the top and nose of the user's head. In the embodiment of the present invention, since the main function of the headwear frame is to be worn on the user's head and to provide support for the optical and electrical components such as the display module and the see-through light guiding component, the head frame includes but is not limited to the above. In a manner, with the above-mentioned main effects, those skilled in the art can make some modifications to the head frame according to the needs of practical applications.

Referring to Figure 1c, the horizontal axis represents the side view angle and the vertical axis represents the display brightness. When the display module 12 is an LCD, the brightness of the display module 12 varies with the angle of the viewer. For a normal LCD, the side observation angle θ at a display luminance of 50% is generally large.

When the LCD is applied to an augmented reality display system, it is more suitable for a small side viewing angle, and the brightness of such a display module 12 is concentrated in an angular area near the center. Because the augmented reality display system mainly uses an angled area near the center, the brightness of the first light and the second light projected into the user's eyes will be relatively high. Referring to FIG. 1d, the brightness of the first light and the second light emitted by the LCD applied to the augmented reality display system is generally small when the display brightness is 50%. Moreover, the distribution of the brightness of the first light and the second light emitted by the LCD applied in the augmented reality display system is symmetric about the 0 degree side view, and the side view angle is less than 60 degrees. That is, when the user's angle of view is perpendicular to the display module 12, the display brightness of the brightness of the first light and the second light emitted by the display module 12 is maximum, and when the user's angle of view is shifted to both sides, the display brightness is gradually reduced, and the side view is gradually reduced. When it is less than 60 degrees, the display brightness is 0.

Optionally, referring to FIG. 1e, the brightness distribution of the first light and the second light emitted by the LCD applied to the augmented reality display system may not be symmetric with respect to the 0 degree side view, and the side view angle when the brightness is the brightest is not 0. degree.

Referring to FIG. 2a, two display modules 12 are respectively located one by one correspondingly above the two fluoroscopic light guiding elements 13. When the user wears the augmented reality display system, the display module 12 and the user's head are flat. The face forms an angle a which is from 0 to 180 degrees, preferably an obtuse angle. At the same time, the projection of the display module 12 on the horizontal plane is perpendicular to the normal plane.

Referring to FIG. 2b, in some examples, the position of the see-through light guiding element 13 can be rotated by a certain angle b around a certain axis of rotation perpendicular to the horizontal plane, the angle b of the angle b being 0 degrees to 180 degrees, preferably 0 degrees to 90 degrees. At the same time, the see-through light guiding elements 13 corresponding to the left and right eyes can be adjusted in pitch by the mechanical structure on the head frame 11 to accommodate the user's interpupillary distance, ensuring comfort and imaging quality in use. The farthest distance between the edges of the two see-through light guiding elements 13 is less than 150 mm, that is, the left edge of the see-through light guiding element 13 corresponding to the left eye to the see-through light guiding element corresponding to the right eye The distance to the right edge of 13 is less than 150 mm. Correspondingly, the display modules 12 are connected by a mechanical structure, and the distance between the display modules 12 can also be adjusted, or the same effect can be achieved by adjusting the position of the display content on the display module 12.

The headgear frame 11 may be an eyeglass frame structure for hanging on the ear and nose of the user, on which the nose pad 111 and the temple 112 are disposed, and the nose pad 111 and the temple 112 are fixed to the user's head. The temple 112 is a foldable structure, wherein the nose pad 111 is correspondingly fixed on the nose bridge of the user, and the temple 112 is correspondingly fixed on the user's ear. Further, the glasses legs 112 can also be connected by an elastic band, and the elastic band tightens the temples when worn to help the frame to be fixed at the head.

Optionally, the nose pad 111 and the temple 112 are telescopic mechanisms that adjust the height of the nose pad 111 and the telescopic length of the temple 112, respectively. Similarly, the nose pad 111 and the temple 112 can also be of a detachable structure, and the nose pad 111 or the temple 112 can be replaced after disassembly.

Alternatively, the head frame 11 may include a nose pad and a stretch rubber band that is fixed to the user's head by a nose pad and a stretch rubber band; or only a stretch rubber band that is fixed to the user's head by the stretch rubber band. Alternatively, the headgear frame 11 may also be a helmet-type frame structure for wearing on the top and nose of the user's head. In the embodiment of the present application, since the main function of the head frame 11 is to be worn on the user's head and to provide support for the optical and electrical components such as the display module 12 and the see-through light guiding element 13, the head frame includes but not Limited to the above manner, under the premise of having the above-mentioned main effects, those skilled in the art can make several modifications to the head frame according to the needs of practical applications.

Referring to FIG. 3, the display module 12 emits a first light ray 121 including left-eye virtual image information, and the first light ray 121 reflected by the concave surface 131 of the see-through light guiding element 13 enters the left eye 14 of the user; similarly, the display module emits a second light comprising the virtual image information of the right eye, the second light reflected by the concave surface of the other see-through light guiding element enters the right eye of the user, thereby forming a 3D virtual scene in the brain of the user Visual perception, in addition, different from Google glasses, by directly setting a small display screen in front of the user's right eye, resulting in a smaller visual area. In the embodiment of the present application, more display is reflected by two see-through light guiding elements. The first light and the second light emitted by the module respectively enter the eyes of the user, and the visual area is large.

In the embodiment of the present application, when the augmented reality display system realizes the function of augmented reality, each of the see-through light guiding elements 13 further has a convex surface disposed opposite to the concave surface; the convex surface and the concave surface transmitted through the transparent light guiding element 13 A third ray containing external image information enters the user's eyes to form a visual blend of the 3D virtual scene and the real scene. Referring again to FIG. 3a, a see-through light guiding element 13 further has a convex surface 132 disposed opposite the concave surface 131, and the third light 151 containing external image information transmitted through the convex surface 132 and the concave surface 131 of the see-through light guiding element 13 enters the user. The left eye 14 is similarly shaped, and the other see-through light guiding element further has a convex surface disposed opposite to the concave surface thereof, and the third light containing the external image information transmitted through the convex surface and the concave surface of the see-through light guiding element enters the right side of the user. In the eye, the user can see the real scene of the outside world, thereby forming a visual experience of mixing the 3D virtual scene and the real scene.

In the embodiment of the present invention, when the head mounted display realizes the function of the virtual reality, the other surface of each of the see-through light guiding elements disposed opposite to the concave surface includes, but is not limited to, a convex shape, in order to block the image information including the external image. The third light enters the user's eyes, that is, the user avoids the real scene of the outside world. As shown in FIG. 3a, the other surface of the see-through light guiding element 13 opposite to the concave surface 131 may be plated or affixed with shading. Layer 16; as shown in FIG. 3b, a hood 171 for blocking the inclusion of the third light containing external image information into the eyes of the user may also be disposed on the head frame 17, so that only the left eye virtual image information is transmitted by the display module. The first light and the second light containing the virtual image information of the right eye enter the eyes of the user, forming a visual experience of the 3D virtual scene in the user's brain, and realizing the function of the virtual reality.

In the embodiment of the present invention, the display module 12 includes a display screen. As shown in FIG. 3c, the display screen may be a display screen 18 having a spherical surface. The radius of curvature of the spherical surface of the display screen 18 is positive, that is, the display The light emitting surface 181 of the screen 18 is convex; as shown in FIG. 3d, the display screen may be a screen 19 having a spherical surface, and the radius of curvature of the spherical surface of the display screen 19 is negative, that is, the light emitting surface 191 of the display screen 19. As shown in FIG. 3e, the display screen may also be a display screen 20 having a cylindrical surface. The radius of curvature of the cylinder surface of the display screen 20 is positive, that is, the light emitting surface 201 of the display screen 20 is convex. As shown in FIG. 3f, the display screen may also be a display screen 21 having a cylindrical surface. The radius of curvature of the cylinder surface of the display screen 21 is negative, that is, the light emitting surface 211 of the display screen 21 is internal. Concave cylinder.

In order to realize that the left-eye virtual image information and the right-eye virtual image information loaded in the first light and the second light emitted by the display module are present on the user's binocular retina with high quality, the concave surfaces of the two see-through light guiding elements need to be able to The aberration of the user's eyes and the aberration caused by the tilting of the see-through light guiding element are balanced, and based on this, the concave surface of the see-through light guiding element is designed according to four special functions, as explained below.

As shown in FIG. 4, in the optical concept, the face depression value refers to the distance of different regions of the surface of the optical element from the center point O of the surface of the optical element in the Z-axis direction. In the embodiment of the present invention, the optical component refers to a see-through light guiding component, the surface of the optical component refers to a concave surface of the see-through light guiding component, and the concave shape of the concave surface of the see-through light guiding component is sag(x, y). As shown in FIG. 4, the projection point coordinate of the concave surface of the see-through light guiding element on the XY coordinate plane is (x, y).

1. The concave surface of the see-through light guiding element is designed according to the following power series polynomial function:

Where c is the basic curvature of the concave and/or convex surface, k is the basic conic coefficient of the concave and/or convex surface, N is the number of polynomials, A _i is the coefficient of the i-th order polynomial, E _i (x, y) is the binary power series polynomial of the standard two variables (x, y).

2. The concave surface of the see-through light guiding element is designed according to the following Chebyshev polynomial function:

Where c is the basic curvature of the concave and/or convex surface, N is the number of polynomials in the x direction, M is the number of polynomials in the y direction, and a _ij is the coefficient of the sum of the fractional parts of the ij order polynomial,

with

Is the normalized coordinates after redefining the x and y coordinates to the interval [-1, 1];

Where max(|x|) is the maximum value of x absolute values, and max(|y|) is the maximum value of y absolute values.

3. The concave surface of the see-through light guiding element is designed according to the following standard Zernike polynomial function:

Where c is the basic curvature of the concave and/or convex surface, k is the basic conic coefficient of the concave and/or convex surface, a _i is the coefficient of the i-th order aspheric variable, N is the number of standard Zernike polynomials, ρ And φ are the polar coordinates corresponding to the x coordinate and the y coordinate, respectively, the interval range of ρ is [0, 1], and the interval range of φ is [0, 2π].

It is the i-th standard Zernike polynomial.

4. The concave surface of the see-through light guiding element is designed according to the following Anamorphic function:

Wherein c _x is the basic curvature of the concave and/or convex surface in the x direction, k _x is the basic conic coefficient of the concave and/or convex surface in the x direction, and c _y is the concave and/or convex surface in the y direction The basic curvature, k _y is the basic conic coefficient of the concave and/or convex surface in the y direction, α ₄ is the fourth higher order coefficient of axial symmetry, and β ₄ is the fourth higher order coefficient of axial asymmetry, α ₆ is the sixth higher order coefficient of axial symmetry, β ₆ is the sixth higher order coefficient of axial asymmetry, α ₈ is the eighth higher order coefficient of axial symmetry, and β ₈ is the eighth highest of axial asymmetry. The order factor, α ₁₀ is the 10th higher order coefficient of axial symmetry, and β ₁₀ is the 10th higher order coefficient of axial asymmetry.

As shown in FIG. 4b, in the above optical concept, the optical element refers to a see-through light guiding element, and the surface of the optical element refers to a convex surface of the see-through light guiding element, and the concave shape of the convex surface of the see-through light guiding element is Sag(x, y), as shown in Fig. 4a, the projection point coordinate of the convex surface of the see-through light guiding element on the XY coordinate plane is (x, y), in order to ensure that the third light containing the external image information enters the user's eyes When the interference is reduced, the convex surface of the see-through light guiding element is designed according to any one of the power series polynomial function, the Chebyshev polynomial function, the standard Zernike polynomial function, and the Anamorphic function.

Referring to Figure 5, optionally, a diopter correcting lens 16 is disposed between the human eye and the see-through light guiding element 13, the diopter correcting lens 16 being disposed perpendicular to the horizontal plane. Optionally, the plane of the diopter correction lens may also be at an angle of 30 degrees to 90 degrees from the horizontal plane. Alternatively, different degrees of diopter correcting lenses may be arbitrarily set. The display module 12 emits a first light ray 121 including left-eye virtual image information, a first light ray 121 reflected through the concave surface 131 of the fluoroscopic light guiding element 13, and a convex surface 132 and a concave surface 131 transmitted through the fluoroscopic light guiding element 13 The third light ray 151 of the image information passes through the refractive correction lens 16 before entering the left eye 14 of the user. The refractive correction lens 16 is a concave lens, and the first light 121 and the third light 151 passing therethrough are diverged, so that the focus of the first light 121 and the third light 151 on the left eye 14 are shifted back. Similarly, the refractive correction lens 16 can also be a convex lens that converges the first light ray 121 and the third light ray 151 thereon to advance the focus of the first light ray 121 and the third light 151 on the left eye 14.

Similarly, the display module emits a second light containing the virtual image information of the right eye, the second light reflected through the concave surface of the other see-through light guiding element, and the external image information transmitted through the convex and concave surfaces of the see-through light guiding element. Before the third light enters the user's right eye, the lens is first corrected by a diopter.

Referring to FIG. 6, after the augmented reality display system is worn on the user's head, the user's eyeball is the apex, and the user's eyeball is formed on both sides of the virtual display area of the virtual image seen through the see-through light guiding element 13. Diagonal field of view. The distance from the farthest end of the head frame to the contact position with the foremost end of the head is c, and the distance length of the c can be adjusted as needed. The angular extent of the diagonal field of view region is inversely proportional to the distance from the most distal end of the head frame 11 to the contact position with the foremost end of the head. Preferably, the distance from the farthest end of the head frame to the contact position with the foremost end of the head is less than 80 mm under the premise that the diagonal field of view area is greater than 55 degrees.

Referring to FIG. 7, the second display module 12 is connected to the main board 17 by a cable.

The main board 17 is also provided with a video interface and a power interface.

The video interface is used to connect a computer, a mobile phone, or other device to receive a video signal. The video interface may be: hmdi, display port, thunderbolt or usb type-c, micro Usb, MHL (Mobile High-Definition Link) and other interfaces.

The processor is configured to decode the video signal transmission and display it on the display module 12.

Power connector for external power or battery power. The power interface includes a USB interface or other interfaces.

When the augmented reality display system includes only the headgear frame 11, the two display modules 12, the two see-through light guiding elements 13 and the main board 17, as described above, all 3D virtual scene rendering, image generation corresponding to both eyes are enhanced and enhanced. The reality display system is connected to an external device. The external device includes: a computer, a mobile phone, a tablet computer, and the like.

Specifically, the augmented reality display system receives the video signal of the external device through the video interface, and displays it on the display module 12 after decoding. At the same time, the interaction with the user is performed by an application software on an external device such as a computer, a mobile phone, a tablet computer, etc., and can interact with the augmented reality display system by using a mouse keyboard, a touch pad or a button on the external device. Examples of such basic structures include, but are not limited to, large screen portable displays. The augmented reality display system can project the display screen at a fixed location within the user's field of view. The user needs to adjust the size, position, and the like of the projection screen through software on the device connected to the augmented reality display system.

The augmented reality display system provided by the embodiment of the present application reflects the first ray including the left-eye virtual image information and the second ray including the right-eye virtual image information by the concave surfaces of the two fluoroscopic light guiding elements. The user's eyes are entered, thereby forming a visual experience of the 3D virtual scene in the user's brain, and the visual area is large.

Embodiment 2

Referring to FIG. 8, on the basis of an augmented reality display system provided in the first embodiment, a plurality of sensors are disposed to perform sensing on the surrounding environment.

The present invention provides an augmented reality display system. The augmented reality display system has a total weight of less than 350 grams, and includes a head frame 21, two display modules 22, two see-through light guiding elements 23, and a main board 24.

The display module 22, the see-through light guiding element 23 and the main board 24 are all disposed on the head frame 21. The head frame 21 fixes the display module 22, the see-through light guiding element 23 and the main board 24. The display module 22 is disposed on the upper side of the see-through light guiding element 23, and the light emitted by the display module 22 can be reflected by the see-through light guiding element 23. The main board 24, the main board 24 is located between the two display modules 22, the main board A processor is provided 24 for processing the virtual image signal and displaying the virtual image information on the display module 22.

The head frame 21, the two display modules 22, the two see-through light guiding elements 23, the main board 24 and the head frame 11 described in the first embodiment, the two display modules 12, the two see-through light guiding elements 13, and the main board 17 The specific functions, structures, and positional relationships are the same and will not be described here.

Similarly, a diopter correcting lens is disposed between the human eye and the see-through light guiding element 23, the diopter correcting lens being disposed perpendicular to the horizontal plane. Alternatively, different degrees of diopter correcting lenses may be arbitrarily set.

The head frame 21 is further provided with a monocular camera 211, a binocular/multi-view camera 212, an eyeball tracking camera 213, a gyroscope 214, an accelerometer 215, a magnetometer 216, a depth of field sensor 217, an ambient light sensor 218, and/or a distance. Sensor 219.

The monocular camera 211, the binocular/multi-view camera 212, the eyeball tracking camera 213, the gyroscope 214, the accelerometer 215, the magnetometer 216, the depth of field sensor 217, the ambient light sensor 218, and/or the distance sensor 219 are all electrically connected to the main board 24 on.

Specifically, the monocular camera 211 is a color monocular camera placed at the front of the head frame 21. When the user wears the augmented reality display system, the monocular camera 211 faces the other side with respect to the user's face, and the camera can be used for photographing. Further, the augmented reality display system can be used to locate the use of the camera using a computer vision technology to detect known locations in the environment.

The monocular camera 211 can also be a high-resolution camera for taking photos or taking video; the captured video can also be superimposed by software to superimpose the virtual object seen by the user, and the user can see through the augmented reality display system. content.

The binocular/multi-view camera 212 may be a monochrome or color camera disposed on the front or side of the head frame 21 and located on one side, two sides or all sides of the monocular camera 211. Further, the binocular/multi-view camera 212 may be provided with an infrared filter. Using the binocular camera, you can further obtain the depth of field information on the image based on the environment image. With a multi-camera camera, you can further expand the camera's viewing angle to get more ambient image and depth of field information. The ambient image and distance information captured by the dual/multi-view camera 212 can be used to: (1) fuse with the data of the gyroscope 214, the accelerometer 215, and the magnetometer 216 to calculate the pose of the augmented reality display system. (2) Capture user gestures, palm prints, etc. for human-computer interaction.

Optionally, each of the above-mentioned monocular camera or binocular/multi-view camera may be one of an RGB camera, a monochrome camera or an infrared camera.

The eyeball tracking camera 213 is disposed on one side of the see-through light guiding element 23, and when the user wears the augmented reality display system, the eyeball tracking camera 213 faces the side opposite to the user's face. The eyeball tracking camera 213 is used to track the focus of the human eye, and to track and specialize the specific parts in the virtual object or virtual screen that the human eye is watching. For example, the specific information of the object is automatically displayed next to the object that the human eye is watching. In addition, the area of the human eye can display the high-definition virtual object image, while for other areas, only the low-definition image can be displayed, which can effectively reduce the amount of image rendering calculation without affecting the user experience.

The gyroscope 214, the accelerometer 215, and the magnetometer 216 are disposed between the two display modules 22. The relative pose between the user's head and the initial position of the system can be obtained by fusing the data of the gyroscope 214, the accelerometer 215, and the magnetometer 216. The raw data of these sensors can be further fused with the data of the binocular/multi-view camera 212 to obtain the position and attitude of the augmented reality display system in a fixed environment.

The depth of field sensor 217 is disposed at the front of the head frame 21, and can directly obtain depth information in the environment. Compared to the dual/multi-view camera 212, the depth of field sensor can obtain more accurate, higher resolution depth of field data. Similarly, the use of these data can be: (1) fused with the data of the gyroscope 214, the accelerometer 215, and the magnetometer 216 to calculate the pose of the augmented reality display system. (2) Capture user gestures, palm prints, etc. to interact with humans. (3) detecting three-dimensional information of objects around the user.

The ambient light sensor 218 is disposed on the head frame 21, and can monitor the intensity of ambient light in real time. The augmented reality display system adjusts the brightness of the display module 22 in real time according to the change of the ambient light to ensure the consistency of the display effect under different ambient light.

The distance sensor 219 is disposed at a position where the augmented reality display system is in contact with the user's face for detecting whether the augmented reality display system is worn on the user's head. If the user removes the augmented reality display system, the power can be saved by turning off the display module 22, the processor, and the like.

Optionally, the augmented reality display system further comprises: an infrared/near-infrared LED electrically connected to the main board 24, wherein the infrared/near-infrared LED is used for binocular/multiple The camera 212 provides a light source. Specifically, the infrared/near-infrared LED emits infrared rays, and when the infrared rays reach an object acquired through the binocular/multi-view camera 212, the object reflects the infrared rays, and the photosensitive element on the binocular/multi-view camera 212 receives the reflection. The returned infrared rays are converted into electrical signals, followed by imaging processing.

The operations that the augmented reality display system can perform when performing human-computer interaction include the following:

(1) The augmented reality display system can project the display screen at a fixed position within the user's field of view. The user can adjust the size, position, and the like of the projection screen through sensors on the augmented reality display system.

(2) Gesture and palmprint recognition can be performed by various sensors for human-computer interaction.

(3) The user's intention can be judged by the eyeball tracking, and the virtual object or the specific part in the virtual screen observed by the human eye is processed accordingly.

(4) It is also possible to add entities or touch buttons, rockers, etc. to the bracket for human-computer interaction.

(5) It can be equipped with a remote control. There are buttons, joysticks, touchpads, etc. on the remote control, which are connected to the augmented reality display system by wired or wireless means as a human-computer interaction interface.

(6) The device and the microphone can be integrated by adding an audio decoding and power amplifying chip to the main board, integrating an earphone jack, an earplug, or a speaker, and allowing the user to interact with the augmented reality display system using voice.

Referring to Figure 9, a video interface and a processor are provided on the motherboard.

When the augmented reality display system includes the headgear frame 21, the two display modules 22, the two see-through light guiding elements 23, the main board 24, and the plurality of sensors as described above, all of the 3D virtual scene renderings, corresponding eyes The image generation and the processing of the data acquired by the plurality of sensors can be performed in an external device connected to the augmented reality display system. The external device includes: a computer, a mobile phone, a tablet computer, and the like.

Specifically, the augmented reality display system receives the video signal of the external device through the video interface, and displays it on the display module 23 after decoding. The external device receives the data acquired by the plurality of sensors on the augmented reality display system, and performs processing to adjust the image displayed by the two eyes according to the data, and is reflected on the image displayed on the display module 23. The processor on the augmented reality display system is only used to support the transmission and display of video signals and the transmission of sensor data.

Referring to FIG. 10, a processor with strong computing power is disposed on the motherboard, and some or all of the computer vision algorithms are completed in the augmented reality display system.

Specifically, the augmented reality display system receives the video signal of the external device through the video interface, and displays it on the display module 23 after decoding. The external device receives the data acquired by a part of the sensors on the augmented reality display system, and performs processing to adjust the image displayed by the two eyes according to the sensor data, and is reflected on the image displayed on the display module 23. The data acquired by the remaining sensors is on the augmented reality display system. deal with. For example, data acquired by the monocular camera 211, the binocular/multi-view camera 212, the gyroscope 214, the accelerometer 215, the magnetometer 216, and the depth of field sensor 217 are processed in an augmented reality display system. The data acquired by the eyeball tracking camera 213, the ambient light sensor 218, and the distance sensor 219 are processed in an external device. The processor on the augmented reality display system is used to support the transmission and display of video signals, the processing of part of the sensor data, and the transfer of the remaining sensor data.

Referring to Figure 11, the motherboard is equipped with a high-performance processor and an image processor to perform all operations in the augmented reality display system. In this mode, Augmented Reality displays operate as a stand-alone system without the need to connect an external device.

Specifically, after the augmented reality display system processes the data acquired by the sensor, the image displayed by the two eyes is adjusted, and is displayed on the display module 23 after being rendered. The processor on the augmented reality display system is used for decoding processing and display of video signals and processing of sensor data.

In the practical application of implementing the augmented reality in the augmented reality display system described in the first embodiment and the second embodiment, in order to increase the reflectivity of the first light and the second light emitted by the concave display module by the concave light guiding element, For example, the concave surface of the see-through light guiding element is plated with a reflective film. Preferably, the reflective surface of the see-through light guiding element coated with the reflective film has a reflectance of 20% to 80%. For example, if the first light and the second light are linearly polarized light, in order to increase the reflectivity of the concave surface of the see-through light guiding element, the concave surface of the see-through light guiding element is plated with a polarizing reflective film, and the polarization direction of the polarizing reflective film is The angle between the polarization directions of a light and the second light is greater than 70° and less than or equal to 90°. For example, the polarization direction of the polarized reflective film is perpendicular to the polarization directions of the first light and the second light, achieving nearly 100% reflection. In addition, since the third light containing the external image information is unpolarized light, if the concave surface of the see-through light guiding element is plated with the polarizing reflective film, when the third light passes through the polarizing reflective film, there is nearly 50% of the third light. Entering the user's eyes, the user can still see the real scene from the outside world. In order to better allow the third light containing external image information to enter the user's eyes, the convex surface of the see-through light guiding element is coated with an anti-reflection film.

In the practical application of the augmented reality display system described in the first embodiment and the second embodiment, in order to realize the controllable adjustment of the reflectance of the first light and the second light emitted by the concave display module of the see-through light guiding element The concave surface of the see-through light guiding element is provided with a pressure sensitive reflective film, and by changing the magnitude of the voltage applied to the pressure sensitive reflective film, the reflectance of the pressure sensitive reflective film can be adjusted between 0 and 100% when the pressure sensitive reflective film When the reflectivity is 100%, the augmented reality display system can realize the function of virtual reality.

In order to realize the other surface pair of the see-through light guiding element disposed opposite to the concave surface, the external image is included The controllable adjustment of the transmittance of the third light of the interest, the pressure sensitive black sheet is disposed on the other surface of the see-through light guiding element disposed opposite to the concave surface, and by changing the voltage applied to the pressure sensitive black sheet, Adjust the light transmittance of the pressure sensitive black sheet.

Since the importance of different regions is different in the physiological field of view of the human eye, in order to enable the first light and the second light reflected by the concave surface of the see-through light guiding element to enter an important region of the physiological field of view of the human eye, preferably As shown in FIG. 12, the display module 12 is disposed at an angle of between 5° and 70° with respect to the horizontal direction; the reflected light 521 entering the upper edge of the user's left eye 14 in the first ray. The angle with the incident ray 522 is less than 90°; the angle of the reflected ray 531 and the incident ray 532 entering the lower edge of the user's left eye 14 in the first ray is greater than 35°; the first ray entering the user's left eye 14 The angle between the reflected light between the upper edge of the field of view and the lower edge of the field of view and the incident light is between 35° and 90°. It should be noted that those skilled in the art can adjust the angle 2 and the angle 3 by adjusting the placement angle of the display module 12 with respect to the horizontal direction and the placement angle of the see-through light guiding element 13 according to the needs of the actual application. In order to achieve the best effect, the effective utilization of the left eye virtual image information and the right eye virtual image information is improved, and the user experience is improved.

The augmented reality display system provided by the embodiment of the present application reflects the first ray including the left-eye virtual image information and the second ray including the right-eye virtual image information by the concave surfaces of the two fluoroscopic light guiding elements. The user's eyes are entered, thereby forming a visual experience of the 3D virtual scene in the user's brain, and the visual area is large. At the same time, a plurality of sensors are arranged on the augmented reality display system. After the sensor senses the surrounding environment, the perceived result can be reflected in the image displayed in the display module, so that the on-site experience is better and the user experience is better.

The above description is only the embodiment of the present application, and thus does not limit the scope of the patent application, and the equivalent structure or equivalent process transformation made by using the specification and the drawings of the present application, or directly or indirectly applied to other related technologies. The fields are all included in the scope of patent protection of this application.

Claims

An augmented reality display system, comprising: a head frame, a display module, two see-through light guiding elements, and a main board, wherein the display module, two see-through light guiding elements, and a main board are disposed on the wearing On the frame; among them,

The head worn frame for wearing on a user's head;

The main board, the main board is provided with a processor, the processor is configured to process virtual image information and display the virtual image information on the display module;

The display module is detachably or fixedly mounted on the headset frame for displaying virtual image information, and transmitting the virtual image information in the first light and the second light;

The two fluoroscopic light guiding elements each have a concave surface disposed toward the eyes of the user; the first light reflected by the concave surface of the fluoroscopic light guiding element enters the user a left eye, and a second light reflected through the concave surface of the other of the see-through light guiding elements enters the right eye of the user to form a view of the 3D virtual scene; wherein the first light includes left eye virtual image information, The second light ray includes right eye virtual image information.
The augmented reality display system according to claim 1, wherein each of said see-through light guiding elements further has a convex surface disposed opposite said concave surface; and a convex surface via each of said see-through light guiding elements And the concavely transmitted third light containing the external image information enters the user's eyes to form a visual of the mixed 3D virtual scene and the real scene.
The augmented reality display system according to claim 2, wherein the see-through light guiding element is rotatable by an angle b about a rotation axis perpendicular to a horizontal plane, the angle b being an angle of 0 to 180 degrees.
The augmented reality display system according to claim 3, wherein the display module is located above the see-through light guiding element, or the display module is located lateral to the see-through light guiding element.
The augmented reality display system according to any one of claims 1 to 4, characterized in that the concave and/or convex surface of the see-through light guiding element is designed according to the following power series polynomial function:

Where c is the basic curvature of the concave and/or convex surface, k is the basic conic coefficient of the concave and/or convex surface, N is the number of polynomials, A i is the coefficient of the i-th order polynomial, E i (x, y) is the binary power series polynomial of the standard two variables (x, y).
The augmented reality display system according to any one of claims 1 to 4, characterized in that the concave and/or convex surface of the see-through light guiding element is designed according to the following polynomial function:

Where c is the basic curvature of the concave and/or convex surface, N is the number of polynomials in the x direction, M is the number of polynomials in the y direction, and a ij is the coefficient of the sum of the fractional parts of the ij order polynomial,
with
Is the normalized coordinates after redefining the x and y coordinates to the interval [-1, 1];

Where max(|x|) is the maximum value of x absolute values, and max(|y|) is the maximum value of y absolute values.
The augmented reality display system according to any one of claims 1 to 4, characterized in that the concave and/or convex surface of the see-through light guiding element is designed according to the following standard polynomial function:

Where c is the basic curvature of the concave and/or convex surface, k is the basic conic coefficient of the concave and/or convex surface, a i is the coefficient of the i-th order aspheric variable, N is the number of standard Zernike polynomials, ρ And φ are the polar coordinates corresponding to the x coordinate and the y coordinate, respectively, the interval range of ρ is [0, 1], and the interval range of φ is [0, 2π].
It is the i-th standard Zernike polynomial.
The augmented reality display system according to any one of claims 1 to 4, characterized in that the concave and/or convex surface of the see-through light guiding element is designed according to the following function:

Wherein c x is the basic curvature of the concave and/or convex surface in the x direction, k x is the basic conic coefficient of the concave and/or convex surface in the x direction, and c y is the concave and/or convex surface in the y direction The basic curvature, k y is the basic conic coefficient of the concave and/or convex surface in the y direction, α 4 is the fourth higher order coefficient of axial symmetry, and β 4 is the fourth higher order coefficient of axial asymmetry, α 6 is the sixth higher order coefficients axially symmetrical, β 6 is axially asymmetric higher order coefficient of 6, α 8 is axially symmetric higher order coefficients 8, β 8 is an axial section of an asymmetric high-8 The order factor, α 10 is the 10th higher order coefficient of axial symmetry, and β 10 is the 10th higher order coefficient of axial asymmetry.
The augmented reality display system according to claim 1 or 2, characterized in that

The headgear frame is a spectacle frame structure for hanging on a user's ear and a nose bridge, and the headgear frame is provided with a nose pad and two temples, the two temples being of a foldable structure. Wherein the nose pads are correspondingly fixed on the nose bridge of the user, and the two temples are correspondingly fixed on the user's ear; or

An elastic band is disposed between the two temples, and the arms can be contracted by the elastic band when worn; or

The headgear frame includes a nose pad and an elastic band, the nose pad correspondingly being fixed to a nose bridge of a user, the elastic band being fixed to a user's head.
The augmented reality display system according to claim 1 or 2, wherein a distance between the see-through light guiding elements is adjustable by a mechanical mechanism provided on the head frame to fit a user's flaw Distance; or,

The distance between the display modules can be adjusted by mechanical structure to adapt to the user's interpupillary distance; or,

The location at which content is displayed on the display module can be adjusted to fit the user's interpupillary distance.
The augmented reality display system of claim 1 or 2, wherein the augmented reality display system has a total weight of less than 350 grams.
The augmented reality display system according to claim 1 or 2, wherein the augmented reality system uses the farthest end of the head frame to ensure that the diagonal field of view area is greater than 55 degrees. The distance from the front end of the head of the household is less than 80 mm.
The augmented reality display system according to claim 1 or 2, wherein the farthest distance between the edges of the two see-through light guiding elements is less than 150 mm.
The augmented reality display system according to claim 1 or 2, wherein the two see-through light guiding elements are independently disposed on the head mounted frame, or the two see-through light guiding elements are integrated After molding, it is disposed on the head frame.
The augmented reality display system according to claim 1 or 2, wherein the augmented reality display system further comprises: a diopter correction lens, wherein the first light and the second light diverge after passing through the diopter correction lens, or A light and a second light converge after passing through the diopter correcting lens.
The augmented reality display system according to any one of claims 1 to 4, characterized in that the head frame is further provided with a monocular camera, a binocular/multi-view camera, an eyeball tracking camera, a gyroscope, an accelerometer, and a magnetic field meter. Depth of field sensor, ambient light sensor, infrared/near-infrared LED and/or distance sensor.
The augmented reality display system according to claim 16, wherein the monocular camera, the binocular/multi-view camera, the eyeball tracking camera, the gyroscope, the accelerometer, the magnetic field meter, the depth of field sensor, the ambient light sensor, and the infrared The near-infrared LED and the distance sensor are electrically connected to the main board.
The apparatus according to claim 17, wherein each of said monocular camera or binocular/multi-view camera is one of an RGB camera, a monochrome camera or an infrared camera.
The augmented reality display system of claim 17 wherein:

The monocular camera is placed at the front of the head frame for detecting a position known in the environment using computer vision technology to assist the augmented reality display system in positioning;

The binocular/multi-view camera is disposed around the head frame for further obtaining depth of field information on the image and obtaining more environment images on the basis of obtaining the environment image;

The depth of field sensor is disposed at a front portion of the head frame for directly obtaining depth information in the environment;

The ambient light sensor is disposed on the frame for monitoring the intensity of ambient light in real time.

The distance sensor is disposed at a portion of the head frame that is in contact with the user's head for detecting whether the augmented reality display system is worn on the user's head;

The infrared/near-infrared LED is used to provide a light source for a binocular/multi-view camera.
The augmented reality display system according to claim 17, wherein the monocular camera is a high-resolution camera for taking photos or capturing video; and the captured video can also be superimposed by software to supervise the virtuality seen by the user. The object reproduces what the user sees through the augmented reality display system.
The augmented reality display system according to claim 19, wherein the gyroscope, the accelerometer and the magnetometer are disposed on one side of the display module, and the gyroscope, the accelerometer and the magnetometer obtain the user after detecting the data. The relative posture between the head and the initial position of the augmented reality display system can be further integrated with the data of the binocular/multi-view camera to obtain the position and posture of the augmented reality display system in a fixed environment.
The augmented reality display system according to claim 19, wherein the eyeball tracking camera is disposed on one side of the see-through light guiding element for tracking the focus of the human eye, the virtual object or the virtual screen that the human eye is looking at. Tracking and special treatment of specific parts in the process.
The augmented reality display system according to claim 1 or 2, wherein the concave surface of the see-through light guiding element is plated with a reflective film.
The augmented reality display system according to claim 23, wherein the reflectance of the concave surface of the see-through light guiding member plated with the reflective film is 20% to 80%.
The augmented reality display system according to claim 1 or 2, wherein the concave surface of the see-through light guiding element is provided with a pressure sensitive reflective film.
The augmented reality display system according to claim 1 or 2, characterized in that the other surface of the see-through light guiding element disposed opposite to the concave surface is provided with a pressure sensitive black sheet.
The augmented reality display system according to claim 1 or 2, wherein the distribution of the brightness of the first light and the second light emitted by the display module is symmetric about the 0 degree side view, and the side view angle of the display module is less than 60. degree.
The augmented reality display system according to claim 1 or 2, wherein the brightness distribution of the first light and the second light emitted by the display module is not symmetric about the 0 degree side view, and the side view when the brightness is the brightest The angle is not 0 degrees.
The augmented reality display system according to any one of claims 1 to 2, wherein the first light and the second light are linearly polarized light, and the concave surface of the see-through light guiding element is plated with a polarizing reflective film. The angle between the polarization direction of the polarizing reflective film and the polarization direction of the first light and the second light is greater than 70°.
The augmented reality display system according to any one of claims 2-4 or 29, wherein the convex surface of the see-through light guiding element is plated with an anti-reflection film.
The augmented reality display system according to claim 1, wherein the other surface of each of the see-through light guiding elements disposed opposite to the concave surface is plated or adhered with a light shielding layer.
The augmented reality display system according to claim 1, wherein the headgear is provided with a hood for blocking a third light containing external image information from entering the eyes of the user.
The augmented reality display system according to any one of claims 1 to 4, wherein a surface of the display screen in the display module is a spherical surface; a radius of curvature of the spherical surface is positive, or a radius of curvature of the spherical surface Negative.
The augmented reality display system according to any one of claims 1 to 4, wherein a surface of the display screen in the display module is a cylinder; a radius of curvature of the cylinder is positive, or the cylinder The radius of curvature is negative.