KR20230065846A - Smart glasses - Google Patents

Abstract

The present invention relates to smart glasses capable of adjusting the sense of depth of a virtual object, and the smart glasses having a left eye lens on which a left eye virtual image is formed and a right eye lens on which a right eye virtual image is formed, the smart glasses included in the left eye virtual image and an image controller configured to adjust a depth of a virtual object visible to a user by a combination of the left and right eye virtual objects by adjusting a position of a left eye virtual object and a position of a right eye virtual object included in the right eye virtual image, respectively.

Description

Smart glasses {SMART GLASSES}

The present invention relates to smart glasses, and more particularly, to smart glasses capable of adjusting a sense of depth of a virtual object visible to a user.

Augmented Reality (AR) refers to a technology that synthesizes virtual objects or information in a real environment to make them look like objects existing in the original environment.

Mixed Reality (MR) or Hybrid reality refers to creating a new environment or new information by combining the virtual world and the real world. In particular, it is called mixed reality when it refers to real-time interaction between reality and virtual reality in real time.

At this time, the created virtual environment or situation stimulates the user's five senses and allows them to freely move in and out of the boundary between reality and imagination by allowing them to have a spatial and temporal experience similar to the real one. In addition, users are not only immersed in these environments, but also interact with things implemented in these environments, such as manipulating or giving commands using real devices.

Recently, research on equipment used in these technical fields has been actively conducted. On the other hand, since the position of the pupil is different for each user, the interpupillary distance (IPD) is different. Since the path through which light enters the eyes is pre-determined in the manufacturing process of the product, each user with a different pupil distance has a different sense of depth for the virtual object. can feel different.

An object of the present invention is to provide smart glasses capable of adjusting a sense of depth of a virtual object so that all users having different interpupillary distances can feel the same sense of depth.

Smart glasses having a left eye lens on which a left eye virtual image is formed and a right eye lens on which a right eye virtual image is formed according to an embodiment of the present invention, the position of the left eye virtual object included in the left eye virtual image and the right eye virtual image included in the and an image controller configured to adjust a sense of depth of a virtual object visible to a user by a combination of the left and right eye virtual objects by adjusting each position of the right eye virtual object.

In an embodiment, the image controller adjusts the position of the left eye virtual object and the right eye virtual object, respectively, based on a user's input.

In another embodiment, a first distance measurement unit for measuring a first distance that is the inter-pupillary distance; and a second distance measurer configured to measure a second distance between one of the left eye lens and the right eye lens and the pupil, wherein the image control unit controls the left eye and the second distance based on the first distance and the second distance. Adjust the position of the right eye virtual object.

In the embodiment, the feedback receiver may further include a feedback receiving unit configured to receive a response from a user regarding a sense of depth of a virtual object having an adjusted sense of depth, and the image control unit may, based on the user's response, control the virtual object Determines whether or not to further adjust the sense of depth.

Smart glasses having a left eye lens on which a left eye virtual image is formed and a right eye lens on which a right eye virtual image is formed according to an embodiment of the present invention include a left eye lens on which a left eye virtual object is displayed; a right eye lens displaying a right eye virtual object; and an interval adjuster configured to change a sense of depth of a virtual object formed by a combination of the left eye virtual object and the right eye virtual object by adjusting a distance between the left eye lens and the right eye lens.

Smart glasses having a left eye lens on which a left eye virtual image is formed and a right eye lens on which a right eye virtual image is formed according to an embodiment of the present invention include a left eye lens on which a left eye virtual object is displayed; a right eye lens displaying a right eye virtual object; and an angle adjuster configured to change a sense of depth of a virtual object formed by a combination of the left eye virtual object and the right eye virtual object by changing an angle of at least one of the left eye lens and the right eye lens in a forward direction.

According to the present invention, the sense of depth of the virtual object of the image controller can be adjusted not only when the focus is adjusted, but also when smart glasses are being used, and the sense of depth of the virtual object can be adjusted even when the gaze direction is not the front center. possible.

According to the present invention, since the distance from the left eye lens to the pupil of the left eye or the distance from the right eye lens to the pupil of the right eye is considered in adjusting the sense of depth of the virtual object, the sense of depth of the virtual object can be more accurately adjusted.

1 is a diagram for explaining smart glasses according to a first embodiment of the present invention.
2 is a diagram for explaining a sense of depth of a virtual object according to a pupillary distance.
FIG. 3 is a diagram for explaining a process in which a sense of depth of a virtual object displayed to a user having a pupillary distance of k2 shown in FIG. 2 is adjusted by an image controller.
FIG. 4 is a diagram for explaining location correction information stored in the storage unit shown in FIG. 1 .
5 is a diagram for explaining smart glasses according to a second embodiment of the present invention.
FIG. 6 is a diagram for explaining the frame portion shown in FIG. 5 .
7A and 7B are diagrams for explaining a method for adjusting a depth of a virtual object of the smart glasses shown in FIG. 5 .
8 is a diagram for explaining smart glasses according to a third embodiment of the present invention.
FIG. 9 is a view for explaining the frame portion shown in FIG. 8 .
10A and 10B are views for explaining a method for adjusting a depth of a virtual object of the smart glasses shown in FIG. 8 .

The configuration and operation of the embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining smart glasses according to a first embodiment of the present invention.

The smart glasses may be any one of augmented reality glasses, virtual reality glasses, and mixed reality glasses.

Referring to FIG. 1 , smart glasses 10 include a frame unit 100, a lens unit 200, a sensor unit 300, a controller 400, a light output unit 500, and a storage unit 600. .

The frame unit 100 may include a front frame 110 and a pair of side frames 120 extending rearward from both sides of the front frame 110 . The frame unit 100 may support the lens unit 200, the sensor unit 300, the controller 400, the light output unit 500, the storage unit 600, and a battery (not shown).

Specifically, the front frame 110 supports the left eye lens 200L and the right eye lens 200R corresponding to both eyes of the user. For example, the front frame 110 may be provided in the form of a spectacle frame surrounding a lens. Unlike this, it may be provided in the form of supporting only the top or bottom of the lens, or supporting only one side of the lens.

The side frame 120 may include a sensor unit 300, a control unit 400, and a storage unit 600, and the light output unit 500 may be provided on the front frame 110 or the side frame 120. there is.

The lens unit 200 includes a left eye lens 2OOL and a right eye lens 200R.

A left eye virtual image is formed on the left eye lens 200L, and a right eye virtual image is formed on the right eye lens 200R. The left eye virtual image includes at least one left eye virtual object, and the right eye virtual image corresponds to the left eye virtual image and includes at least one right eye virtual object. A virtual object having a sense of depth is visible to the user by a combination of a left eye virtual object included in the left eye virtual image and a right eye virtual object included in the right eye virtual image.

The light output unit 500 radiates a virtual image to the lens unit 200 to form a virtual image on the lens unit 200 . The light output unit 500 includes a light output unit 500L for the left eye and a light output unit 500R for the right eye. The light emitting unit 500L for the left eye radiates a virtual image for the left eye to the left lens 200L, and the light emitting unit 500R for the right eye radiates a virtual image for the right eye to the lens 200R for the right eye. The virtual image for the left eye and the virtual image for the right eye include at least one virtual object formed of any one or a combination of figures, letters, and images such as points, lines, and planes corresponding to each other.

The sensor unit 300 includes one or more sensors for sensing at least one of information within the smart glasses 10, surrounding environment information surrounding the smart glasses 10, and user information. The sensor unit 300 includes first to third distance measurement units 310 , 320 , and 330 and an image sensor unit 340 .

The first distance measuring unit 310 measures a first distance, which is the user's interpupillary distance (IPD). At this time, the first distance measurement unit 310 may be configured as an image sensor, and a known method may be used to measure the distance of the inter-pupillary distance (IPD).

The second distance measurement unit 320 measures the distance between the pupil and the lens. Specifically, the second distance measurement unit 320 measures a second distance, which is a distance between the left eye lens and the left eye pupil or a distance between the right eye lens and the right eye pupil.

The third distance measuring unit 330 measures a third distance from the smart glasses 10 to an object where the virtual object is located.

The image sensor 440 periodically photographs each of the user's left and right eyes, generates a left eye image and a right eye image, and transmits the images to the image controller 410 . In addition, the image sensor 440 may periodically photograph the surrounding environment in the direction the user is looking to generate an external image and transmit it to the image controller 410 . Accordingly, the image controller 410 may determine a gaze point and an object corresponding to the gaze point based on the periodically transmitted left eye image and right eye image.

The controller 400 performs a function of controlling the smart glasses 10 . The control unit 400 includes an image control unit 410, a gaze tracking unit 420, a target determination unit 430, a display possibility determination unit 440, and a feedback receiving unit 450.

2 is a diagram for explaining a sense of depth of a virtual object according to a pupillary distance.

The image controller 410 adjusts the sense of depth of the virtual object based on the first to third distances measured by the sensor unit 300 .

Referring to FIG. 2 , when a user with a pupil distance k1 wears smart glasses, a left eye virtual object lv1 included in the left eye virtual image formed on the left eye lens 200L and a right eye virtual image formed on the right eye lens 200R A virtual object ck1 having a sense of depth m1 is shown to the user by a combination of right eye virtual objects rv1 included in .

Meanwhile, when a user with a pupillary distance of k2 wears smart glasses, the left eye virtual object lv1 included in the left eye virtual image formed on the left eye lens 200L and the right eye included in the right eye virtual image formed on the right eye lens 200R A virtual object ck2 having a sense of depth l1 is shown to the user by the combination of the virtual objects rv1.

That is, as the interpupillary distance increases from k1 to k2, the sense of depth of the virtual object decreases from m1 to l1. The sense of depth of the virtual object appears differently for each person with a different interpupillary distance, and accordingly, the sense of immersion felt by each person may be different.

FIG. 3 is a diagram for explaining a process in which a sense of depth of a virtual object displayed to a user having a pupillary distance of k2 shown in FIG. 2 is adjusted by an image controller.

Referring to FIG. 3 , in order to change the sense of depth of the third virtual object felt by a user with a pupillary distance k2, the image controller 410 moves the position of the left eye virtual object lv1 to the left, and moves the right eye virtual object (lv1) to the left. rv1) may be moved to the right side to make the depth sense m1 of the virtual object ck3 felt by the user with the interpupillary distance k2 and the user with k1 equal to each other. That is, the image controller 410 may change the sense of depth of the virtual object by changing the distance between the left eye virtual object and the right eye virtual object.

Specifically, when setting the distance from the position (lv0) where the line from the left eye pupil to the left eye lens meets and the position of the left eye virtual object (lv1) before correction on the left eye lens is set as re1', and the line from the left eye pupil to the left eye lens is lowered The distance from the meeting position (lv0) to the position of the left eye virtual object (lv2) after correction on the left eye lens is set as re1''. In this case, the left eye virtual object lv1 before correction and the right eye virtual object rv1 before correction are set for a user having a specific interpupillary distance.

Then, the distance from the meeting point (rv0) from the right eye pupil when lowering the right eye lens surface to the position of the right eye virtual object (rv1) on the right eye lens before correction is set as re2', The distance from the meeting point (rv0) to the position of the right eye virtual object (rv2) after correction on the right eye lens is set as re2''.

In addition, from the point where the line segment connecting the pupils of both eyes is drawn from the location of the virtual object (ck2) before modification seen to the user by the combination of the left eye virtual object (lv1) before modification and the right eye virtual object (rv1) before modification, the left eye, The distances to each of the right eyes are set to k21 and k22. At this time, re11' is calculated as a value obtained by dividing k21 by the sense of depth (m1) of the virtual object (ck2) before modification and multiplying by k21.

And de means the distance between the left eye lens and the left eye pupil or the distance between the right eye lens and the right eye pupil.

Position difference (re1) between the left eye virtual object (lv1) before correction and the left eye virtual object (lv2) after correction on the left eye lens, and the position difference between the right eye virtual object (rv1) before correction and the right eye virtual object (rv2) after correction (rv2) on the right eye lens ( re2) is obtained by Equations 1 to 4 below (abs means an absolute value). And Equation 4 is an equation calculated to correspond to Equation 3.

Position difference (re1) between the left eye virtual object (lv1) before correction and the left eye virtual object (lv2) after correction on the left eye lens, and the position difference between the right eye virtual object (rv1) before correction and the right eye virtual object (rv2) after correction on the right eye lens (re2) may be implemented by shifting the image data emitted by the light emitting unit in units of pixels. That is, the image controller shifts the left-eye virtual object and the right-eye virtual object in units of pixels to adjust the sense of depth of the virtual object, and considers the degree of shifted pixels according to the positions of the left-eye virtual object and the right-eye virtual object on the lens. You can adjust the depth of the virtual object.

According to the present invention, the depth control of the virtual object of the image controller has the advantage that the depth control is possible not only when the focus is adjusted, but also when the smart glasses are being used. That is, it is possible to adjust the sense of depth of the virtual object even when the viewing direction is not the front center.

According to the present invention, since the distance from the left eye lens to the pupil of the left eye or the distance from the right eye lens to the pupil of the right eye is considered in adjusting the sense of depth of the virtual object, the sense of depth of the virtual object can be more accurately adjusted.

The image controller 410 may manually adjust the depth of the virtual object under the control of the user.

Alternatively, the sense of depth of the virtual object may be automatically adjusted based on the position correction information stored in the storage unit 600 .

FIG. 4 is a diagram for explaining location correction information stored in the storage unit shown in FIG. 1 .

Referring to FIG. 4 , the position correction information includes pixel position correction degree values for each pixel position of a left eye image and a right eye image for each of a plurality of inter-pupillary distances k. The pixel position correction degree value is a position correction degree of a pixel to display an image, means an image shifting degree, and represents values corresponding to re1 and re2 in Equations 3 and 4.

For example, when the interpupillary distance is K-3, when the left eye virtual object is expressed at the m-th pixel and the right eye virtual object is expressed at the n-th pixel, in the position correction information stored in the storage unit 600, the left eye When the position correction degree of the virtual object is -Δ5 and the position correction degree of the right eye virtual object is +Δ2, the image controller 410 displays the left eye virtual object at the m-5 th pixel in the case of the left eye, In the case of the right eye, the right eye virtual object is corrected to be displayed at the n+2 th pixel.

At this time, as described above, the image controller 410 can adjust the position of a pixel including the virtual object output by the light emitter 210 to change the sense of depth of the virtual object, and the image controller 410 also controls the virtual object The size, color, shape, position, and depth of the image can be modified. In particular, the image controller 410 may allow the user to intuitively recognize the depth of the virtual object by changing the color of the text according to the depth of the virtual object.

The gaze tracking unit 420 determines the user's gaze point using the image sensor 440 .

The target determination unit 430 determines an object corresponding to the gaze point based on the gaze point of the user determined by the gaze tracking unit 420 and the external image transmitted from the image sensor 440 . At this time, the object determination unit 430 may determine the size, color, and shape of the object.

The display possibility determination unit 440 analyzes the virtual object based on the target information such as the size, color, and shape of the object determined by the target determination unit 430, that is, whether the virtual object contains letters or not By analyzing the virtual object information, it is determined whether the virtual object can be displayed on the target.

When the virtual object can be displayed on the target, the image controller 410 may not change the size, color, shape, etc. of the virtual object. If it is determined that the virtual object cannot be displayed on the target, the image controller 410 may change the size, color, shape, etc. of the virtual object and display the virtual object.

On the other hand, the displayability determination unit 440 determines whether a character is included in the virtual object, and when the character is expressed on the object and the character size is less than a certain size, the image control unit 410 may display the virtual object to the user. It is possible to display an alarm indicating that the virtual object is not displayed, and display the user's location where the virtual object can be displayed or the location of an object where the virtual object can be displayed.

The feedback receiving unit 450 receives a response from the user about the sense of depth of the virtual object whose depth sense is adjusted. In response, the image controller 410 may determine whether to additionally adjust the sense of depth of the virtual object based on the user's response. In addition, when the image controller 410 determines that additional adjustment of the depth of the virtual object is necessary based on the user's response, the image controller 410 generates a plurality of virtual objects having different depths that can be selected by the user. Thus, the light emitting unit 500 may emit light to the left eye lens and the right eye lens, respectively.

The storage unit 600 includes position correction information of a pixel according to a first distance, which is a plurality of inter-pupillary distances, or a first distance, which is a plurality of inter-pupillary distances, and a second distance, which is a distance between a plurality of eyes and lenses. Position correction information of pixels according to a first distance, which is a plurality of inter-pupillary distances, a second distance, which is a distance between a plurality of eyes and a lens, and a third distance, which is a distance to an object. can do.

5 is a diagram for explaining smart glasses according to a second embodiment of the present invention, and FIG. 6 is a diagram for explaining the frame portion shown in FIG. 5 . 7A and 7B are diagrams for explaining a method for adjusting a depth of a virtual object of the smart glasses shown in FIG. 5 .

5 and 6, the smart glasses 10 include a frame unit 100, a lens unit 200, a sensor unit 300, a controller 400, a light output unit 500, and a storage unit 600. include A description overlapping with that in the first embodiment is omitted.

The frame unit 100 includes a front frame 110 including a left safety frame 110L supporting the left eye lens 200L and the right eye lens 200R and a right safety frame 110R supporting the right eye lens 200R, and The left side frame and the right frame (120L, 120R) extending from the outer portion of the front frame 110, and the left extension and right extension (130L, 130L, 130R).

The left and right extensions 130L and 130R are fixed to the motion controller 140 to maintain the shape of the glasses.

The distance adjusting unit 140 performs a function of adjusting the distance between the left extension part 130L and the right extension part 130R.

The smart glasses of the first embodiment adjust the depth of the virtual object by adjusting the positions of the left eye virtual object in the left eye virtual image and the right eye virtual object in the right eye virtual image, but the smart glasses 10 of the second embodiment have a left eye lens (200L) and the distance between the lenses, which is the distance between the right eye lenses 200R, is physically adjusted to adjust the sense of depth of the virtual object.

The first distance measurement unit 310 of the sensor unit 300 measures the first distance, which is the interpupillary distance of the user, and transmits the measurement to the control unit 400 .

The operation control unit 411 controls the distance adjusting unit 140 to adjust the distance between lenses based on the position correction information stored in the storage unit 600 .

The storage unit 600 includes position correction information including an inter-lens distance for each of a plurality of inter-pupillary distances. Unlike the first embodiment, the storage unit of the second embodiment includes information about the distance to the right eye lens or the left eye lens for each interpupillary distance.

In the case of FIG. 7A, the left eye lens 200L and the right eye lens 200R are simultaneously adjusted to adjust the depth of the virtual object. In the case of FIG. 7B, the left eye lens 200L and the right eye lens 200R are shown. It shows smart glasses that adjust the depth of a virtual object by adjusting the position of any one of the lenses.

Referring to FIGS. 7A and 7B , as the distance between the left eye lens 200L and the right eye lens 200R decreases, the sense of depth of the virtual object decreases, and as the distance increases, the sense of depth of the virtual object deepens.

8 is a diagram for explaining smart glasses according to a third embodiment of the present invention, and FIG. 9 is a diagram for explaining the frame portion shown in FIG. 8 . 10A and 10B are diagrams for explaining a method for adjusting the depth of a virtual object.

8 and 9, the smart glasses 10 include a frame unit 100, a lens unit 200, a sensor unit 300, a controller 400, a light output unit 500, and a storage unit 600. includes A description overlapping with that in the first embodiment is omitted.

The frame unit 100 includes a front frame 110 including a left front frame 110L supporting the left eye lens 200L and a right front frame 110R supporting the right eye lens 200R, and The left side frame and the right frame (120L, 120R) extending from the outer portion of the front frame (110), the left eye front frame (110L) and the right eye front frame (110R) extending in a form facing each other, respectively, and It includes right extension portions 130L and 130R.

The left and right extensions 130L and 130R are fixed to the angle adjuster 141 to maintain the shape of the glasses.

The angle adjusting unit 141 performs a function of adjusting a sense of depth of a virtual object by adjusting an angle between the left eye lens 200L and the right eye lens 200R.

The smart glasses of the first embodiment adjust the depth of the virtual object by adjusting the positions of the pixels on which the left and right eye virtual objects are displayed, but the smart glasses 10 of the third embodiment have a left eye lens 200L and a right eye lens ( 200R) adjusts the depth of the virtual object by physically adjusting the angle between the lenses.

The first distance measurement unit of the sensor unit 300 measures the first distance, which is the interpupillary distance of the user, and transmits the measurement to the control unit 400 .

The operation control unit 411 controls the angle adjusting unit 140 to adjust the angle between the lenses based on the angle correction information stored in the storage unit 600 .

The storage unit 600 includes angle correction information including an inter-lens angle for each of a plurality of inter-pupillary distances. Unlike the first embodiment, the storage unit of the second embodiment includes information about angle adjustment information of the lens for each pupillary distance.

Referring to FIGS. 10A and 10B , when the angle formed by the left eye lens 200L and the right eye lens 200R is changed based on 180 degrees, the depth of the virtual object is changed. The feeling is different.

Meanwhile, in the case of FIG. 10A, when the angle between the lenses formed by the left eye lens 200L and the right eye lens 200R is 180 degrees, the depth of the virtual object is measured as l1 and m1, but as shown in FIG. 10b, the left eye lens 200L and By adjusting the angle of the lens formed by the right eye lens 200R, the sense of depth of the virtual object can be adjusted.

Certain or other embodiments of the present invention described above are not mutually exclusive or distinct from each other. Certain or other embodiments of the present invention described above may be used in combination or combination of respective components or functions.

For example, configuration A described in a specific embodiment and/or drawing may be combined with configuration B described in another embodiment and/or drawing. That is, even if the combination between the components is not directly explained, it means that the combination is possible except for the case where the combination is impossible.

The above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

10: smart glasses
100: frame part
110: front frame
120: side frame
200: lens unit
300: sensor unit
310, 320, 330: first, second, third distance measuring unit
340: image sensor unit
400: control unit
410: video control unit
420: gaze tracking unit
430: target determination unit
440: displayable determination unit

Claims (6)

In smart glasses having a left eye lens on which a left eye virtual image is formed and a right eye lens on which a right eye virtual image is formed,
An image of adjusting the depth of a virtual object visible to a user by a combination of the left and right eye virtual objects by adjusting the position of the left eye virtual object included in the left eye virtual image and the right eye virtual object included in the right eye virtual image, respectively. Smart glasses comprising a control unit. According to claim 1,
The smart glasses according to claim 1 , wherein the image controller adjusts a position of the left eye virtual object and a position of the right eye virtual object, respectively, based on a user's input. According to claim 1,
a first distance measurement unit that measures a first distance that is the inter-pupillary distance; and
A second distance measuring unit for measuring a second distance between one of the left eye lens and the right eye lens and the pupil;
The smart glasses according to claim 1 , wherein the image controller adjusts positions of the left and right eye virtual objects based on the first distance and the second distance. According to claim 3,
A feedback receiver configured to receive a response from a user about the sense of depth of the virtual object whose sense of depth is adjusted, from the user;
The smart glasses according to claim 1 , wherein the image controller determines whether to additionally adjust the depth of the virtual object based on the user's response. a left eye lens displaying a left eye virtual object;
a right eye lens displaying a right eye virtual object;
and an interval adjuster configured to change a sense of depth of a virtual object formed by a combination of the left eye virtual object and the right eye virtual object by adjusting a distance between the left eye lens and the right eye lens. a left eye lens displaying a left eye virtual object;
a right eye lens displaying a right eye virtual object; and
and an angle adjustment unit configured to change a sense of depth of a virtual object formed by a combination of the left eye virtual object and the right eye virtual object by changing an angle of at least one of the left eye lens and the right eye lens in a forward direction.

Images