KR20220040174A - See-through optics system applied to near eye display device and eye tracking method in see-through optics system - Google Patents

Abstract

Disclosed are an eye tracking method in a see-through optical system and a near eye image display device using the same. According to the present invention, the see-through optics system for matching a virtual image and an actual image comprises: a first reflection unit making a first image, which is a part of the actual image, to penetrate and changing the optical path of a second image which is the rest of the actual image; a second reflection unit changing the optical path of the second image again; an image collection unit collecting the second image; a control unit generating the virtual image based on the second image, and emitting the virtual image from a display panel; an expansion lens unit expanding the virtual image emitted from the display panel; and a third reflection unit making the virtual image expanded through the expansion lens unit form on the retina of a user. The control unit tracks the eye of the user other than the second image, and generates the virtual image based on the tracked eye. Accordingly, the present invention can make a structural design possible for a configuration for forming image light of a virtual image on the retina of a user and a configuration for collecting image light of the eye image of the user to be integrated within the see-through optics system without separately preparing such configurations or separately mounting a camera, integrate overlapped parts, reduce weight, increase user convenience, and prevent declines in product reliability.

Description

See-through optical system applied to near eye display device and eye tracking method in see-through optics system

The present invention relates to a see-through optical system applied to a near-eye image display device and a gaze tracking method in a see-through optical system, and more particularly, to VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality). Reality: It relates to a see-through optical system for Mixed Reality) and a gaze tracking method in the see-through optical system.

In the near-eye image display device for augmented/mixed reality, the see-through optical system allows the image beam emitted from the display LCD (LCoS), OLED, LED, and laser module, etc. to pass through the designed optical system lens according to the configured optical lens configuration. By sending it to the , the user can watch the video.

At this time, while the user's eyes see the image information desired by the user or the see-through (see-through) front object, the eyes of the pupils are changed in various directions. These gazes can be divided into a change in the direction of the gaze and a change in the gaze angle when viewed based on both eyes, and tracking the user's gaze by identifying the change in the gaze direction or gaze angle is VR, It becomes a very important element in AR and MR services.

That is, eye tracking achieves the original purpose of selecting video content corresponding to the user's gaze and providing it to the user, as well as for achieving the secondary purpose of reducing dizziness or motion sickness caused by the service. It is also a necessary issue.

However, existing services not only add weight by separately installing a camera for eye tracking, but also seriously reduce convenience, and have a problem that reliability can be deformed because there is no integral structural design.

Korean Patent Publication No. 10-2020-0023992 (Surgical Smart Tracker and AR Display System)

The present invention has been devised to solve the above problems, and an object of the present invention is to provide an image based on information on the tracked gaze, as well as to enable gaze tracking within the near-eye image display device. An object of the present invention is to provide a see-through optical system and a gaze tracking method in a see-through optical system applied to a near-eye image display device that enables the see-through optical system to operate in conjunction with each other according to the tracked gaze.

In order to achieve the above object, a see-through optical system for matching a virtual image and a real image according to an embodiment of the present invention transmits a first image that is a part of the real image and changes the optical path of a second image that is a part of the real image a first reflector; a second reflector for re-changing the optical path of the second image; an image collection unit for collecting the second image; a control unit generating the virtual image based on the second image and outputting the virtual image from a display panel; a magnifying lens unit for enlarging the virtual image output from the display panel; and a third reflector for allowing the virtual image enlarged through the magnifying lens unit to form an image on the user's retina, wherein the control unit tracks the user's gaze other than the second image and based on the tracked gaze Create a virtual image.

Here, the first reflector transmits a third image that is a part of the user's gaze image and changes an optical path of a fourth image that is a part of the user's gaze, and the third reflector and the second reflector include the optical path of the fourth image may be changed so that the gaze image is collected by the image collection unit.

Here, the controller may track the user's gaze based on the gaze image collected by the image collection unit, and track the gaze direction of the user based on the coordinates of the pupil in the eyeball in the gaze image.

Here, the control unit compares the left eye gaze image collected by the left eye image collection unit and the right eye gaze image collected by the right eye image collection unit to analyze the distance difference between the intraocular pupils, and based on the distance difference between the intraocular pupils to track the user's gaze.

Here, the controller determines a subject that the user gazes on based on the tracked gaze direction and the tracked gaze angle, and controls the display panel so that a virtual image corresponding to the determined subject is displayed on the display panel. can do.

And, the control unit controls the hinge unit based on the tracked gaze direction and the tracked gaze angle, and the hinge unit moves according to the control of the control unit so that the see-through optical system is rotated and translated simultaneously. , the see-through optical system may be positioned on the user's gaze direction.

In addition, the first reflector, the second reflector, and the third reflector change the optical path of the second image and transmit it to the image collecting unit, and change the optical path of the fourth image. It may serve to transmit the image to the image collection unit.

The image collection unit may separate the fourth image from the received image based on a preset sample gaze image and extract the second image by excluding the fourth image portion from the received image.

In addition, the see-through optical system may be mounted on a head-mounted display device and used for medical purposes.

Meanwhile, in the eye tracking method in a see-through optical system for matching a virtual image and a real image according to an embodiment of the present invention for achieving the above object, a first image that is a part of the real image is transmitted and a second part that is a part of the real image is transmitted. changing the optical path of the image; re-changing the optical path of the second image; collecting the second image; generating the virtual image based on the second image; allowing the virtual image to be output from a display panel; enlarging the output virtual image; and changing the path of the virtual image so that the enlarged virtual image is formed on the user's retina, wherein the generating of the virtual image includes, in addition to the second image, tracking the user's gaze and tracking the user's gaze. The virtual image may be generated based on .

Accordingly, there is no need to separately provide a configuration that serves to form the image light of the virtual image on the user's retina and a configuration that serves to collect the image light of the user's gaze image, as well as installing a camera separately. By enabling the structural design to be integrated in the see-through optical system without the need to reduce the weight, increase convenience, and prevent deterioration of product reliability through the integration of redundant components.

1 is a diagram illustrating an example of a near-eye image display device according to an embodiment of the present invention.
2 is a view showing a see-through optical system identified from the bottom of the near-eye image display device of the present invention.
3 is a view provided to explain the internal configuration of the see-through optical system.
4 is a view provided to explain a process in which a gaze image is collected by an image collecting unit through a see-through optical system.
5 is a view provided to explain the gaze angle and IPD in a state before the gaze is changed.
6 is a view provided to explain the gaze angle and IPD in a state after the gaze is changed.
7 is a diagram illustrating a state in which the see-through optical system is moved as the gaze is changed.
FIG. 8 is a view provided to explain a structure for moving the see-through optical system as the line of sight is changed.
9 is a flowchart provided to explain a gaze tracking method in a see-through optical system according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

1 is a diagram illustrating an example of a near-eye image display device according to an embodiment of the present invention, and FIG. 2 is a diagram illustrating a see-through optical system viewed from the bottom of the near-eye image display device of the present invention.

1 and 2, a head-mounted display device is shown as an example of a near-eye image display device, and in the case of such a head-mounted display device, a user mounts it on the head, and thus a real image through a see-through optical system inside the head-mounted display device. and a virtual image are provided to the user.

As shown in FIG. 2 , it can be seen that the see-through optical system 100 is designed to be positioned in front of both eyes of the user through the bottom of the head-mounted display device, and the see-through optical system 100 is formed between the left eye and the left eye. Each of the right eyes is provided (100-1, 100-2).

3 is a view provided to explain the internal configuration of the see-through optical system 100 .

The see-through optical system 100 according to an embodiment of the present invention refers to a device that generates a virtual image so that a real image that a user looks at and the generated virtual image are matched.

The see-through optical system 100 according to the present embodiment does not block the user's field of view so that the front object (S: subject) that the user is looking at can be observed, and related to the observed object (S: subject) By generating and outputting a virtual image, it is possible to provide augmented reality to the user.

For this purpose, the see-through optical system 100 according to the present embodiment includes a first reflection unit 110 , a second reflection unit 120 , an image collection unit 130 , a control unit 140 , a display panel 150 , and a magnification. It is composed of a lens unit 160 , a third reflection unit 170 , and a polarizing film 180 .

The first reflector 110 is a polyhedron formed of a glass material or a plastic material having a predetermined transmittance, so that image light of an actual image incident to the first reflector 110 is partially transmitted or partially reflected. Accordingly, the image light for the front object (S: subject) passes through the first reflector 110 to form an image on the user's retina.

In particular, some light is transmitted and some light is reflected on the inclined surface of the first reflecting unit 110. In order to enable such partial transmission and partial reflection, the inclined surface is used to distinguish between image information passed or An appropriate filter (a band-pass filter, a polarization filter) may be provided to improve visibility.

The second reflector 120 is formed of a glass material or a plastic material having a predetermined transmittance like the first reflector 110 , and the image light of the real image reflected from the first reflector 110 and incident therein is Make it partially transmitted or partially reflected.

That is, the image light of the real image reflected by the first reflection unit 110 is reflected by the second reflection unit 120 and transmitted to the image collection unit 130 .

In addition, the second reflector 120 transmits the image light of the user's gaze image transmitted through the first reflector 110 and is reflected by the second reflector 120 to the image collecting unit 130 .

That is, the image collection unit 130 can collect both the image light of the real image and the image light of the gaze image. The image light of the real image is used to select the contents of the virtual image, and the image light of the gaze image is applied to an object that matches the user's gaze direction and gaze angle when multiple real images are generated according to the user's gaze direction and gaze angle. It will be used to enable the content of the corresponding virtual image to be provided.

The image light of the real image and the image light of the gaze image collected together in this way are used separately/separated from each other by the image collecting unit 130. In particular, the image collecting unit 130 is received based on a preset sample gaze image. It is possible to obtain the image light of the real image by separating and extracting only the gaze image from the received image and excluding the part of the gaze image separated and extracted from the received image.

In particular, some light is transmitted and some light is reflected on the inclined surface of the second reflection unit 120. In order to enable such partial transmission and partial reflection, the inclined surface is used to distinguish or An appropriate filter (a band-pass filter, a polarization filter) may be provided to improve visibility.

The image collection unit 130 is composed of a module capable of collecting images, such as a camera, and serves to collect information on the actual image and information on the user's gaze as described above.

In particular, when the image collection unit 130 receives the image of the user's gaze through the first reflector 110 and is reflected by the second reflector 120, through the image of the user's gaze, When a single eye is used as a standard, in which direction the pupil is deviated from the center of the user's eyeball, that is, the direction of the gaze is determined.

For example, when the user sees the sky direction, the pupil will be biased toward the north, and the image collection unit 130 tracks the direction of the user's gaze by collecting information that the pupil has moved in the north direction. do.

On the other hand, the image collection unit 130 also determines how much the distance between the pupils of the user's eyes is narrowed based on both eyes through the image of the user's gaze.

For example, as the distance between the user's pupils is further reduced, the gaze angle becomes larger, so that it is possible to determine that the user is gazing at a closer subject in a situation in which two subjects overlap. In this case, it is possible to determine which subject is being gazed upon even when two or more subjects overlap through an operation on the gaze angle.

In this way, the image collection unit 130 serves to collect images of the gaze to make it possible to determine the direction and the gaze angle of the gaze.

The control unit 140 receives an image of an object in front from the image collection unit 130 in real time, and generates a virtual image corresponding to the image based on the received image.

In addition, the control unit 140 receives the images for both eyes of the user from the image collection unit 130 in real time, calculates the gaze direction and the gaze angle from the received images for both eyes, and based on the calculated results It tracks the user's gaze and serves to provide a virtual image that matches the gaze.

The controller 140 transmits the image light of the virtual image to the display panel 150 so that the image light of the virtual image generated above is emitted from the display panel 150 .

The display panel 150 is used to receive and output the image light of a virtual image from the controller 140, such as LCD (Liquid Crystal Display), OLED (Organic Light Emitting Diodes), LCoS (Liquid Crystal on Disply), etc. It may be implemented in various devices or elements.

The magnifying lens unit 160 is used for the purpose of magnifying the image light of the virtual image emitted from the small-sized display panel 150 .

When the display panel 150 is provided in a small size, the image light of the virtual image emitted from the display panel 150 is smaller than the image light of the real image that the user looks at.

Accordingly, the magnifying lens unit 160 is provided so that the image light of the emitted virtual image can be enlarged to a preset angle of view and provided to the user.

To this end, the magnifying lens unit 160, as shown, may be formed of two or more lens groups, and each lens forming the lens group includes a single-sided concave lens, a single-sided convex lens, a double-sided concave lens, a double-sided convex lens, and the like. can be done

The image light of the virtual image enlarged by the magnifying lens unit 160 is incident on the third reflecting unit.

The third reflection unit 170 is formed of a glass material or a plastic material having a predetermined reflectance, is emitted from the display panel 150 and is incident on the third reflection unit 170 after passing through the magnifying lens unit 160 . Image light of all virtual images is reflected along a preset path.

The image light of the virtual image reflected by the third reflector 170 is re-reflected by the first reflector 110 to form an image on the user's retina.

The polarizing film 180 is used for the purpose of preventing the light irradiated from the light source from being incident on the image collecting unit 130 , and for this purpose, the polarizing film 180 is formed on the front surface of the light source (not shown) and the image collecting unit 130 . ), and between the second reflecting unit 120 and the magnifying lens unit 160 , and the like.

At this time, in order to prevent the light irradiated from the light source (not shown) from being incident on the image collecting unit 130 , the polarizing film 180 provided on the front surface of the light source (not shown) and the front surface of the image collecting unit 130 . The polarizing film 180 provided on the polarization film 180 is implemented to absorb different types of polarization components.

Specifically, when one is a P-wave polarizing film, the other may be an S-wave polarizing film, or vice versa.

4 is a view provided to explain a process in which a gaze image is collected by the image collecting unit 130 through the see-through optical system 100 .

As described above, the first reflector 110 is formed of a glass or plastic material having a predetermined transmittance. In this case, the preset transmittance is a light reflectance of 50% for partially transmitting or partially reflecting image light incident to the first reflecting unit 110, and the first reflecting unit 110 has a light reflectance of 50%. surface treated.

The first reflector 110 has a predetermined transmittance so that an actual image of a front object (subject: S) is provided to the user's retina, but also has a reflectance inverse of the transmittance.

According to this combination of reflectance and transmittance, the image light of the gaze image is reflected by the first reflector 110, moves to the third reflector 170 along the path of R10, and is reflected by the third reflector 170 again. The path of R10 is moved in reverse, and it is transmitted through the first reflector 110 and moved to the magnifying lens unit 160 along the path of R20.

Thereafter, the image light of the user's gaze image is reflected by the second reflection unit 120 and transmitted to the image collection unit 130 .

Through this process, the image collection unit 130 can determine the direction of the user's gaze by identifying the position of the pupil in the collected image.

In addition, since the see-through optical system 100 is provided in both eyes, information on the left eye gaze image and information on the right eye gaze image collected by the image collecting unit 130 provided in each see-through optical system 100 are collected. It is possible to calculate the user's gaze angle.

The image collection unit 130 transmits the image light of the left eye gaze image and the image light of the right eye gaze image reached in the above to the controller 140 . Specifically, the left eye image collection unit 130-1 collects the left eye gaze image and transmits it to the controller 140 , and the right eye image collection unit 130-2 collects the right eye gaze image and transmits it to the controller 140 . will do

The controller 140 generates a virtual image to be provided to the user based on the left eye gaze image and the right eye gaze image received from the image collection unit 130 . Of course, the control unit 140 is based on the actual image received from the image collection unit (130). That is, the virtual image corresponding to the subject is generated by determining whether the object located at which gaze direction and at which gaze direction in the real image is the subject actually gazed by the user.

Thereafter, the controller 140 transmits the generated virtual image to the display panel 150 .

The display panel 150 emits the image light of the virtual image so that the image light of the virtual image received from the controller 140 can reach the user.

The emitted image light is partially transmitted through the second reflection unit 120 and passes through the magnifying lens unit 160 , and is expanded to a preset angle of view while passing through the magnifying lens unit 160 . As described above, the image light of the virtual image output in a small size from the display panel 150 may be implemented to be the same as the image light size of the real image that the user looks at.

The image light of the enlarged virtual image passes through the first reflector 110 and reaches the third reflector 170 .

Since the third reflecting unit 170 is a condensing lens having a light reflectivity of 100%, the image light of the virtual image that has passed through the first reflecting unit 110 and reached is directed to the first reflecting unit 110 at a preset angle of view. will be re-reflected.

Then, the image light of the virtual image re-reflected by the first reflector 110 is reflected once again toward the user's retina by the first reflector 100 to finally form an image on the user's retina.

As described above, the first reflector 110 , the second reflector 120 , and the third reflector 170 serve to form the image light of the virtual image on the user's retina as well as the user's gaze image. The image light is collected by the image collecting unit 130 in parallel, and a structural design is possible to be integrated in the see-through optical system 100 without separately mounting a camera, through the integration of redundant parts. It is possible to reduce weight, increase convenience, and prevent deterioration of product reliability.

That is, when a camera is separately mounted to track the user's gaze, a state in which the front of the user's gaze can always be transmitted to keep an eye on the subject must be provided, so various configurations may be involved to change the optical path. In addition, these components have to be used in a general see-through optical system that does not track the user's gaze. By unifying components for providing a virtual image in the see-through optical system method and components for tracking the gaze Compared to the case of provision, a more optimized design is possible, and weight/convenience/reliability improvement is also possible.

On the other hand, the above configurations used in the see-through optical system should be changed according to the user's gaze, and in the past, a method such as the user directly adjusting the distance between the see-through optical systems was used.

Hereinafter, when a virtual image is formed on the user's retina, based on the information on the tracked gaze, a method for automatically adjusting the see-through optical system to transmit the virtual image more accurately and effectively will be described.

5 is a view provided to explain the gaze angle and IPD in a state before the gaze is changed, FIG. 6 is a diagram provided to explain the gaze angle and IPD in a state after the gaze is changed, and FIG. 7 is It is a view illustrating a state in which the see-through optical system is moved as the line of sight is changed, and FIG. 8 is a view provided to explain a structure for moving the see-through optical system as the line of sight is changed.

When the user's gaze gazes at the subject S, when the gaze angle is θ1°, the angle between the see-through optical system 100-1 for the left eye and the see-through optical system 100-2 for the right eye is 180-θ1° It becomes θ2°. At this time, when the interpupillary distance (IPD) of the user becomes d1, the distance between the user's eyes and the subject S becomes d2 proportional to d1.

In such a situation, when the user's gaze is moved to a closer subject, the user's pupils are more converged with each other, and accordingly, the IPD is further reduced to d3 as shown in FIG. 6 .

That is, as the distance to the subject is reduced to d4, the IPD is also reduced to d3 to correspond thereto. As such, the reduction in the distance between the IPD and the subject means that the gaze angle has increased to θ3°, and accordingly the see-through optical system 100-1 for the left eye and the see-through optical system 100-2 for the right eye It means that the angle between the angles should also be changed to 180-θ3°.

That is, the angle θ4° between the see-through optical system 100-1 for the left eye and the see-through optical system 100-2 for the right eye is further reduced when the user's gaze is moved to a closer subject.

In order to reduce the angle between the see-through optical system 100-1 for the left eye and the see-through optical system 100-2 for the right eye, as shown in FIGS. 6 and 7, the see-through optical system 100-1 for the left eye and the right eye It is necessary to rotate the see-through optical system 100 - 2 in opposite directions.

That is, the image collection unit 130 determines the user's gaze angle together with the user's gaze direction by tracking the user's gaze through the see-through optical system 100-1 for the left eye and the see-through optical system 100-2 for the right eye. And, by rotating the see-through optical system 100-1 for the left eye and the see-through optical system 100-2 for the right eye according to the determined gaze angle, the left eye and the right eye are aligned with the user's gaze directions.

At this time, by simply rotating the see-through optical system 100-1 for the left eye and the see-through optical system 100-2 for the right eye based on the center, the left-eye see-through optical system is perpendicular to the direction in which both eyes of the user gaze. (100-1) and the see-through optical system 100-2 for the right eye cannot be positioned.

That is, when the gaze angle increases as the user gazes at a closer subject, the see-through optical system 100-1 for the left eye and the see-through optical system 100-2 for the right eye rotate in opposite directions and gather more together. The see-through optical system 100-1 for the left eye and the see-through optical system 100-2 for the right eye can be positioned on the straight line distance between the user's both eyes and the subject.

For this purpose, as shown in FIG. 8 , the see-through optical system 100-1 for the left eye and the see-through optical system 100-2 for the right eye are connected to each other through a hinge part 190 at the center, and the hinge part 190 ) moves left and right, so that the see-through optical system 100-1 for the left eye and the see-through optical system 100-2 for the right eye move closer and further away from each other at the same time as they rotate.

That is, the control unit 140 receives information on both eyes of the user, determines the gaze angle of the user, and controls the hinge unit 190 so that the hinge unit 190 operates according to the gaze angle, so that only the left eye see-through optics The system 100-1 and the see-through optical system 100-2 for the right eye can be controlled.

9 is a flowchart illustrating a gaze tracking method in a see-through optical system for matching a virtual image and a real image according to an embodiment of the present invention.

First, in order to provide a virtual image corresponding to the user's gaze, the first image, which is a part of the real image, is transmitted and the optical path of the remaining part of the second image is changed.

Thereafter, when the optical path of the second image is changed again, this second image is transmitted to the image collecting unit, and the control unit is the user based on the second image collected by the image collecting unit and the user's gaze image collected by the image collecting unit. After tracking the gaze, a virtual image corresponding to the real image matching the gaze of the user is generated.

In addition, the controller changes the path of the virtual image so that the virtual image is output from the display panel and the virtual image enlarged through the magnifying lens unit is formed on the user's retina.

The see-through optical system for matching the virtual image with the real image and the eye tracking method in the see-through optical system can be used in various fields, and in particular, can be used in cases requiring high concentration such as medical surgery, When used for medical purposes, it may be used in the form of a head-mounted display worn on the user's head to minimize sound movement.

In the above, preferred embodiments of the present invention have been illustrated and described, but the present invention is not limited to the specific embodiments described above, and it is common in the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims. Various modifications are possible by those having the knowledge of, of course, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: see-through optical system
100-1: see-through optical system for left eye
100-2: see-through optical system for right eye
110: first reflector 120: second reflector
130: image collection unit 140: control unit
150: display panel 160: magnifying lens unit
170: third reflection unit 180: polarizing film
190: hinge

Claims (10)

In the see-through optical system for matching a virtual image and a real image,
a first reflector for transmitting a first image, which is a part of the real image, and changing an optical path of a second image, which is a part of the remaining image;
a second reflector for re-changing the optical path of the second image;
an image collecting unit for collecting the second image;
a control unit generating the virtual image based on the second image and outputting the virtual image from a display panel;
a magnifying lens unit for enlarging the virtual image output from the display panel; and
A third reflector for allowing the virtual image enlarged through the magnifying lens unit to form an image on the user's retina;
The control unit is
The see-through optical system, characterized in that by tracking a user's gaze other than the second image, and generating the virtual image based on the tracked gaze. According to claim 1,
The first reflector,
The third image, which is a part of the user's gaze image, is transmitted and the optical path of the fourth image, which is the remaining part, is changed,
The third reflector and the second reflector,
The see-through optical system, characterized in that by re-changing the optical path of the fourth image, the gaze image is collected by the image collecting unit. 3. The method of claim 2,
The control unit is
The user's gaze is tracked based on the gaze image collected by the image collection unit,
A see-through optical system, characterized in that the user's gaze direction is tracked based on the coordinates of the pupil in the eye in the gaze image. 4. The method of claim 3,
The control unit is
By comparing the left eye gaze image collected by the left eye image collection unit and the right eye gaze image collected by the right eye image collection unit, the distance difference between the pupils in the eye is analyzed, and the user's gaze angle is determined based on the difference in the distance between the pupils in the eyeball. A see-through optical system characterized in that it is tracked. 5. The method of claim 4,
The control unit is
determining the subject the user gazes at based on the tracked gaze direction and the tracked gaze angle;
The see-through optical system of claim 1, wherein the virtual image corresponding to the determined subject is displayed on the display panel by controlling the display panel. 6. The method of claim 5,
The control unit is
Control the hinge unit based on the tracked gaze direction and the tracked gaze angle,
The hinge part,
The see-through optical system, characterized in that by moving according to the control of the control unit so that the see-through optical system is rotated and translated at the same time, the see-through optical system is positioned in the user's gaze direction. 7. The method of claim 6,
The first reflector, the second reflector, and the third reflector,
A see-through optical system, characterized in that it changes the optical path of the second image and transmits it to the image collection unit, and changes the optical path of the fourth image to transmit it to the image collection unit. 8. The method of claim 7,
The image collection unit,
Separating and extracting the fourth image from the received image based on a preset sample gaze image,
and extracting the second image by excluding the fourth image portion from the received image. 9. The method of claim 8,
The see-through optical system,
A see-through optical system, characterized in that it is mounted on a head-mounted display device and used for medical purposes. In the gaze tracking method in a see-through optical system for matching a virtual image and a real image,
transmitting a first image, which is a part of the real image, and changing an optical path of a second image, which is a part of the remaining image;
re-changing the optical path of the second image;
collecting the second image;
generating the virtual image based on the second image;
allowing the virtual image to be output from a display panel;
enlarging the output virtual image; and
changing the path of the virtual image so that the enlarged virtual image is formed on the user's retina;
The step of generating the virtual image comprises:
A gaze tracking method, characterized in that by tracking a user's gaze other than the second image, and generating the virtual image based on the tracked gaze.

Images