KR20220073183A - Augmented reality optical apparatus - Google Patents

Abstract

이다. The present invention relates to an augmented reality optical device. It includes a transflective concave mirror that reflects to and transmits light from an external object, and the refractive index of the transflective concave mirror is n, and when the outer diameter and the inner diameter are formed to be spaced apart by a thickness (t), the radius of the outer diameter (R1) ) and the radius of the inner diameter (R2)

to be.

Description

Augmented Reality Optical Device {AUGMENTED REALITY OPTICAL APPARATUS}

The present invention relates to an augmented reality optical device that simultaneously provides a virtual image and a real image to a user.

A conventional bithbath type AR/MR optical system has a structure that allows an observer to view a virtual image and a stereoscopic image at the same time. In this case, a concave mirror (CM) is used as an external window for viewing a virtual image and a stereoscopic image at the same time. The inner surface of the concave mirror is formed with a transflective coating so that a user can see a virtual image, and there is an outer surface separated by a certain thickness. The minimum thickness of the concave mirror is determined according to the ease of the glass manufacturing process (related to the yield).

In this case, the location of the virtual image can be formed at a certain depth according to the design of the optical system, and the AR/MR optical system simultaneously sees the real object and the virtual image, so the depth of the real object must be accurately recognized by the user. This is particularly important when utilized as AR/MR optical systems used in the medical field and mutual interaction.

In this case, the depth of an external real object is perceived differently by the viewer according to the curvature information of the concave mirror and the thickness thereof.

In general, the bithbath type AR/MR optical system has the advantage of being simple, compact, and inexpensive to manufacture. However, when the position of the virtual image is not the front, the depth of the virtual image formed at the central position is different. This is particularly problematic when working while viewing a virtual image and the real thing at the same time.

Conventional US Patent Publication (US 10,459,230) and US Patent Publication (US 2019/0197790) disclose a bird bath type AR device, but there is no mention of depth distortion improvement of the virtual image according to the concave mirror, and the observer's Depth correction of the virtual image according to the gaze direction is not considered.

That is, the existing bird-bass-type AR device had a problem in that the perceived depth of the real thing and the actual depth did not match, and the depth of the virtual image was perceived differently by the observer depending on the gaze direction of the observer. This problem causes a serious problem when interacting while simultaneously viewing real and virtual images of 250 mm to 1000 mm, which are considered to be applications of AR/MR optical systems.

Therefore, by improving the bird bath-type AR device of the prior art, it is possible to comfortably work while viewing the real and virtual images while viewing the real and virtual images at the same time, even if the observer changes the gaze depth or direction. There is a need for an existing device.

US Registered Patent Publication US 10,459,230 US Patent Publication US 2019/0197790

It is an object of the present invention to provide an augmented reality optical device that reduces distortion of the depth of a real object so that an observer can accurately recognize the depth of an external object, and provides an optimal virtual image according to gaze information of the observer.

이다.An embodiment of the augmented reality optical device of the present invention for solving the above problems is a display that provides a virtual image, a beam splitter that is inclined on a path of the display light, and is disposed between the beam splitter and an external object , a transflective concave mirror that reflects the display light reflected from the beam splitter back to an observer and transmits light from an external object, wherein a refractive index of the transflective concave mirror is n, and an outer diameter and an inner diameter are thickness t ), the difference between the radius of the outer diameter (R ₁ ) and the radius of the inner diameter (R ₂ ) is

to be.

Preferably, an eye tracking module for detecting gaze information including focal depth and gaze angle information of the observer's eye, and calculating a virtual image formation depth based on the gaze information, and the display side according to the virtual image formation depth It may further include a display control unit for adjusting the separation distance (d) between the display from one end of the transflective concave mirror.

Preferably, it further comprises a data storage unit for storing the separation distance (d) value for forming a constant virtual image depth with respect to the gaze information, wherein the display control unit is the separation distance (d) value data from the data storage unit may be received to adjust the separation distance d according to the gaze information measured by the eye tracking module.

Preferably, the augmented reality optical device is respectively disposed with respect to the right eye and the left eye of the observer, the first eye tracking module of the first augmented reality optical device detects first gaze information of the observer's right eye, and the second augmented reality optical device The second eye tracking module of the device may detect second gaze information of the observer's left eye.

Preferably, the separation distance d includes a first separation distance d ₁ according to the first gaze information and a second separation distance d ₂ according to the second gaze information, and the display control unit includes the The first separation distance d ₁ of the first display provided in the first augmented reality optical device and the second separation distance d ₂ of the second display provided in the second augmented reality optical device are individually adjusted Thus, depth distortion of the virtual image can be reduced.

According to the present invention, by adjusting the radius of the inner and outer diameters of the concave mirror, the observer can accurately recognize the depth of the external object, and by adjusting the distance between the display and the optical system, the virtual information formation position according to the observer's gaze depth is constant can be maintained.

1 is a side view schematically showing an augmented reality optical device according to an embodiment of the present invention.
FIG. 2 is a graph showing the difference in radius between the inner and outer diameters of the concave mirror of FIG. 1 .
3A and 3B are diagrams showing results of spot radius according to the field of the concave mirror according to the present invention.
4 is a schematic diagram for explaining a configuration for adjusting the separation distance between the display and the concave mirror according to the virtual image formation depth of the augmented reality optical device according to an embodiment of the present invention.
5 is a diagram illustrating a state in which a real object and a virtual image are simultaneously observed in the observer's eye in FIG. 4 .
6 is a graph exemplarily illustrating an optimal virtual image formation depth according to a separation distance of a display in FIG. 4 .
FIG. 7 is a view showing the degree to which an image point of a real object formed on the retina of the eye falls within the diffraction resolution for each field in the case of gazing at the center and the side in FIG. 4 .
8 is a graph illustrating a spot radius characteristic according to the field of FIG. 7 .
9 is a diagram illustrating a relationship between an observer's gaze angle and an RMS spot radius for each field of a virtual image formed by measuring the observer's gaze direction.
10 is an augmented reality optical device according to an embodiment of the present invention, according to the depth of the virtual image formation for the observer's left and right eyes to explain individually adjusting the separation distance between the display and the transflective concave mirror of each device It is a conceptual diagram for

Hereinafter, detailed contents for practicing the present invention will be described with reference to the accompanying drawings. In the description of the present invention, when it is determined that the subject matter of the present invention may be unnecessarily obscured as it is obvious to those skilled in the art with respect to related known functions, the detailed description thereof will be omitted.

1 is a side view schematically showing an augmented reality optical device according to an embodiment of the present invention. FIG. 2 is a graph showing the difference in radius between the inner and outer diameters of the concave mirror of FIG. 1 .

Referring to FIG. 1 , the augmented reality optical device according to the present invention includes a display 10 , a beam splitter 20 , and a concave mirror 30 . The display 10 forms a virtual image and provides it to an observer. The beam splitter 20 is disposed obliquely on the path of the light exiting the display. The beam splitter 20 is disposed below the display 10 between the observer and an object in the field of view the observer is looking at, and reflects the display light to the concave mirror 30 . The concave mirror 30 is formed as a transflective concave mirror, and is disposed between the beam splitter 20 and an object in the field of view of the observer. The concave mirror 30 reflects the display light reflected from the beam splitter 20 back to the viewer.

Accordingly, when the observer observes the object 1 in the field of view through the augmented reality optical device, it is possible to observe the real world object coming through the concave mirror 30 and the beam splitter 20, and also The virtual image formed on the display passes through the beam splitter 20, is reflected from the inner diameter of the concave mirror 30, enters the eye of the observer, and is focused on the retina (to be described later in FIGS. 4 and 5).

The beam splitter 20 does not have refraction power, but the transflective concave mirror 30 has a thickness t and has the form of a spherical lens consisting of a radius of an outer diameter (R ₁ ) and a radius of an inner diameter (R ₂ ). have When the material of the transflective concave mirror 30 is glass, the ratio of the thickness to the size of the transflective concave mirror 30 may be designed to be 1/10 or more in order to facilitate precision processing in consideration of the yield. In this case, when the size of the transflective concave mirror 30 is 30 to 50 mm, t is 3 to 5 mm or more.

In the present invention, when the transflective concave mirror 30 has a thickness t, the refractive power (k) of the transflective concave mirror 30 is 0 to minimize the distortion of perception of depth of an external real object. The radius of the outer diameter of the mirror 30 (R ₁ ) and the radius of the inner diameter (R ₂ ) are derived in detail.

The condition that the optical refractive power (k) of the transflective concave mirror 30 is 0 is as follows.

Equation (1) for calculating the optical power of a thick lens:

If t=0, it becomes the same as the general thin lens formula.

Equation (2) under the condition that k=0:

Referring to FIG. 2 and the above formula, the difference (ΔR) between the radius (R ₁ ) of the outer diameter and the radius (R ₂ ) of the inner diameter of the transflective concave mirror 30 is proportional to the thickness t of the transflective concave mirror 30 . to increase linearly.

When the transflective concave mirror 30 is, for example, N-BK7 (refractive index n = 1.52), the thickness is t = 4 mm, and the radius of the inner diameter (R ₂ ) is R ₂ = 49.6 mm, a specific embodiment is As follows.

When the radius of the outer diameter (R ₁ ) and the radius of the inner diameter (R ₂ ) are the same (R ₁ = R ₂ ), the refractive power exists and the optimal distance varies.

=1.368mm이 되어 R₁= 50.968로 설정하면 된다. 실제로 광학 전산모사 툴을 최적화한 결과는 R₁= 50.613mm로, 수식 (2)에 의해 계산된 결과와 0.7% 정도의 오차로 무시할 수 있는 정도의 차이이다.If the refractive power is set to 0 by adjusting the radius of the outer diameter (R ₁ ) and the inner radius (R ₂ ), applying Equation (2),

=1.368mm, just set R ₁ = 50.968. In fact, the result of optimizing the optical simulation tool is R ₁ = 50.613 mm, which is a negligible difference from the result calculated by Equation (2) with an error of about 0.7%.

3A and 3B are diagrams showing results of spot radius according to the field of the concave mirror according to the present invention. 3A is a case in which the radius of the outer diameter (R ₁ ) and the radius of the inner diameter (R ₂ ) are the same, and FIG. 3B is the case of Equation (2).

Referring to FIGS. 3A and 3B, when the refractive power of the CM according to the present invention is corrected to be 0 when the focus of the observer's eye is focused on the real object, the image points of the real object formed on the retina of the eye in all fields are It can be seen that it enters the diffraction resolution (Airy Disk). This means that the depth of an actual object is accurately recognized by the eye without deterioration of image quality. 3 shows an embodiment of the present invention, and the field range is ±17.67 degrees in the horizontal direction and ±10.27 degrees in the vertical direction.

4 is a schematic diagram for explaining a configuration for adjusting the separation distance between the display and the concave mirror according to the virtual image formation depth of the augmented reality optical device according to an embodiment of the present invention. 5 is a diagram illustrating a state in which a real object and a virtual image are simultaneously observed in the observer's eye in FIG. 4 .

Referring to FIG. 4 , the augmented reality optical apparatus according to an embodiment of the present invention may further include an eye tracking module 40 , a display control unit 50 , and a data storage unit 60 .

The eye tracking module 40 may detect gaze information including information on a focal depth and gaze angle of the observer's eye. The eye tracking module 40 may calculate the depth of the virtual image formation through the gaze information. The gaze information is gaze information including information on the focal depth and gaze angle of the observer's eyeball when the observer gazes at a specific point of the environment image 3 .

The display control unit 50 determines the distance between the display 10 and an extension line extending parallel to the display 10 from one end of the transflective concave mirror 30 close to the display 10 according to the virtual image formation depth ( d) can be adjusted.

In the virtual image formed on the display 10 , the display light passes through the beam splitter 20 and is reflected from the inner diameter of the transflective concave mirror 30 , enters the eye, and is focused on the retina.

The radius R ₂ of the inner diameter of the transflective concave mirror 30 is determined by the focal length required for optical design in consideration of the size of the display 10, the field of view (FOV) of the virtual image, and eye relief. . At this time, the depth of the image based on the virtual image coincident with the optical axis (gazing angle 0 degrees) can be adjusted by adjusting the separation distance d of the display 10 .

The data storage unit 60 stores a distance (d) value for forming a constant virtual image depth with respect to the gaze information of the observer. This may be stored in the form of a lookup table. The display control unit 50 receives the separation distance (d) value data from the data storage unit 60, and according to the gaze information of the observer measured by the eye tracking module 40, the separation distance (d) of the display 10 can be adjusted. At this time, the display control unit 50 is provided with a lookup table including the gaze information and virtual image depth information of the observer according to the separation distance (d) value of the display 10, and, accordingly, the separation distance ( d) can be adjusted.

Referring to FIG. 5 , the real object light and the display light overlap and converge on the observer's pupil 2 , and the observer sees an image in which the real image 3 and the virtual image 4 are superimposed. The point at which the real image 3 and the virtual image 4 shown to the observer converge to the pupil 2 may vary according to the observer's eye gaging distance gd.

6 is a graph exemplarily illustrating an optimal virtual image formation depth according to a separation distance of a display in FIG. 4 .

Referring to FIG. 6 , when the observer stares at the front, the optimal virtual image formation depth is linearly changed in diopter units according to the change in the separation distance d of the display 10 . Using this characteristic, the display control unit 50 may change the depth of the virtual image by receiving feedback from the observer's gaze depth from the data storage unit 60 . That is, when the observer's gaze depth is measured, the display position value corresponding to the gaze depth is adjusted in the augmented reality optical device, but the separation distance (d) value of the display 10 may be adjusted according to the data of the lookup table.

FIG. 7 is a view showing the degree to which an image point of an actual object formed on the retina of the eye falls within the diffraction resolution for each field in the case of gazing from the center and the side in FIG. 4 . 8 is a graph illustrating a spot radius characteristic according to the field of FIG. 7 . Fig. 8 (a) is a graph showing the relationship between the observer's gaze angle and the RMS spot radius, and Fig. 8 (b) is a graph showing the relationship between the virtual image formation depth according to the observer's gaze angle.

Referring to FIG. 7 , when the eye gaze is focused on 1.5D (667mm), for example, and the center of the virtual image is adjusted by adjusting the display position, the focus is exactly at the center, but the It can be seen that the image quality is greatly deteriorated.

Referring to (a) of FIG. 8 , in consideration of normal visual acuity, when the RMS spot radius is within 5 μm, it can be said that the image quality is not deteriorated within the field range, and accordingly, within the field corresponding to 0 degrees to 10 degrees. In the image, it can be said that defocus does not occur even if the observer focuses on the image in front.

Referring to (b) of FIG. 8, the image blur according to the field of the virtual image tends to increase as the depth of the virtual image created at the same display position, that is, the separation distance d of the display 10, increases as the field increases. appears because there is Figure 8 (b) is a case in which the focus is fixed to the display position at 667 mm (1.5D) in the zero-field, and it can be seen that the depth of the virtual image formation is rapidly increased when the field is 15 degrees or more. .

9 is a diagram illustrating a relationship between an observer's gaze angle and an RMS spot radius for each field of a virtual image formed by measuring the observer's gaze direction. Fig. 9(a) is a case of a zero-field, Fig. 9(b) is a case of a 7 degree-field, Fig. 9(c) is a case of a 14 degree-field, and Fig. 9(d) is a case of a 17 degree-field - Indicates each case of the field.

9 (a) to (d), in the augmented reality optical device according to an embodiment of the present invention, the eye tracking module 40 measures the gaze direction of the observer, and the gaze of the observer is placed on the display control unit 50 The information data is provided, and the display control unit 50 adjusts the position of the display 10, that is, the separation distance d of the display 10, even at the same gaze depth according to the field of the virtual image so that the virtual image has the same depth. to always be formed in At this time, the observer's gaze depth is fixed to 1.5D (~667mm), and the optimal gaze angle of the observer can be set to 0 degrees, 7 degrees, 14 degrees, and 17 degrees.

When a range in which the RMS spot radius is smaller than the airy radius due to diffraction is viewed as a field region that can be viewed without image quality degradation, a region that can be seen clearly can be maintained even when the gaze angle changes in consideration of the characteristics of the eyeball. The fovea of the eyeball is a region with the best visual field on the retina, and the region that can be clearly seen corresponds to a range of ±2.5 degrees with respect to this region as the center. Although the angular range decreases as the gaze angle increases, the range of the fovea degree of the retina can be adjusted to a well-focused area even at a gaze angle of 17 degrees.

According to the present invention, it can be particularly usefully used when the virtual images are formed at positions in which the gaze directions of the virtual images of both eyes are different from each other. This is because there is a problem in that the disparity image matching becomes difficult if there is a difference in the sense of depth between the two images when the left and right images are different from each other in both eyes.

10 is an augmented reality optical device according to an embodiment of the present invention, according to the depth of the virtual image formation for the observer's left and right eyes to explain individually adjusting the separation distance between the display and the transflective concave mirror of each device It is a conceptual diagram for

Referring to FIG. 10 , the augmented reality optical device according to an embodiment of the present invention may include a plurality of the augmented reality optical devices described above. The plurality of augmented reality optical devices may include a first augmented reality optical device 100 and a second augmented reality optical device 200 respectively disposed for the right eye and the left eye of the observer. The displays 110 and 210, the beam splitters 120 and 220, and the transflective concave mirrors 130 and 230 included in the first and second augmented reality optical devices 100 and 200, respectively, have the above-described augmented reality optical device. The detailed configuration of the display 10 , the beam splitter 20 , and the transflective concave mirror 30 can be applied.

The first augmented reality optical device 100 includes a first eye tracking module 140 that detects first gaze information of the observer's right eye, and the second augmented reality optical device 200 includes a second gaze of the observer's left eye. A second eye tracking module 240 for detecting information may be included.

The first eye tracking module 140 may detect first gaze information of the observer's right eye, and the second eye tracking module 240 may detect second gaze information of the observer's left eye. When the observer gazes at the virtual image point p through the right eye and the left eye, the images converge to the pupils 2-1 and 2-2 of the observer's right and left eyes. In this case, the gaze information of each of the right eye and the left eye becomes the first gaze information and the second gaze information.

The separation distance d of the displays 110 and 210 includes a first separation distance d ₁ according to the first gaze information and a second separation distance d ₂ according to the second gaze information.

The display control unit 50 includes a first separation distance d ₁ of the first display 110 provided in the first augmented reality optical device 100 and a second display provided in the second augmented reality optical device 200 . Depth distortion of the virtual image may be reduced by individually adjusting the second separation distance d ₂ of 210 .

For example, when the image point gaze angles of the virtual images of the right and left eyes are θ ₁ and θ ₂ from the optical axis and θ ₂ > θ ₁ , the first separation distance ( d ₁ ) and the second separation distance d ₂ of the second display 210 may be adjusted differently. In this case, the second separation distance d ₂ may be set to a value smaller than the first separation distance d ₁ .

The scope of protection in this field is not limited to the description and expression of the embodiments explicitly described above. In addition, it is added once again that the protection scope of the present invention cannot be limited due to obvious changes or substitutions in the technical field to which the present invention pertains.

10: display 20: beam splitter
30: transflective concave mirror 40: eye tracking module
50: display control unit 60: data storage unit
100: first augmented reality optical device
110: first display 140: first eye tracking module
200: the second augmented reality optical device
210: second display 240: second eye tracking module
n: refractive index of the transflective concave mirror
t: thickness of the transflective concave mirror
R ₁ : Radius of outer diameter R ₂ : Radius of inner diameter
d: separation distance of display
d ₁ : the first separation distance of the first display
d ₂ : the second separation distance of the second display

Claims (5)

a display providing a virtual image;
a beam splitter inclined on a path of the display light; and
a transflective concave mirror disposed between the beam splitter and an external object to reflect the display light reflected from the beam splitter back to an observer and transmit light from the external object;
When the refractive index of the transflective concave mirror is n, and the outer diameter and the inner diameter are formed to be spaced apart by the thickness (t), the difference between the radius of the outer diameter (R ₁ ) and the radius of the inner diameter (R ₂ ) is

Characterized in that, augmented reality optical device.
According to claim 1,
an eye tracking module for detecting gaze information including focal depth and gaze angle information of the observer's eye, and calculating a virtual image formation depth based on the gaze information; and
Augmented reality optical device, characterized in that it further comprises a display control unit for adjusting the separation distance (d) between the display from one end of the transflective concave mirror on the display side according to the depth of the virtual image formation, characterized in that it further comprises.
3. The method of claim 2,
Further comprising a data storage unit for storing the distance (d) value for forming a constant virtual image depth with respect to the gaze information,
Augmented reality optics, characterized in that the display control unit receives the separation distance (d) value data from the data storage unit and adjusts the separation distance (d) according to the gaze information measured by the eye tracking module Device.
4. The method of claim 3,
The augmented reality optical device is respectively disposed for the right eye and the left eye of the observer,
The first eye tracking module of the first augmented reality optical device detects first gaze information of the observer's right eye, and the second eye tracking module of the second augmented reality optical device detects second gaze information of the observer's left eye Characterized in, augmented reality optical device.
5. The method of claim 4,
The separation distance d includes a first separation distance d ₁ according to the first gaze information and a second separation distance d ₂ according to the second gaze information,
The display control unit is the first separation distance (d ₁ ) of the first display provided in the first augmented reality optical device and the second separation distance (d ₂ ) of the second display provided in the second augmented reality optical device ) by individually adjusting the depth distortion of the virtual image, characterized in that the augmented reality optical device.

Images