KR101213596B1 - Line of sight tracking system integrated with head tracker and eye tracker and mehtod thereof - Google Patents

Abstract

The present invention relates to an eye tracking system in which a head tracker and an eye tracker are integrated, and after recognizing a structure of a user's helmet, estimating the operation of the helmet, calculating helmet position information and helmet direction information on a first three-dimensional coordinate system, A head tracker for synchronizing and transmitting the calculated helmet position information and helmet direction information according to the received synchronization signal, and calculating the gaze direction information on a second three-dimensional coordinate system by tracking the gaze direction of the user, and calculating the calculated gaze direction information. Receives the eye tracker to synchronize and transmit according to the received synchronization signal, and the helmet position information, helmet direction information and gaze direction information, gaze of the user using the received helmet position information, helmet direction information and gaze direction information A third three-dimensional direction in which the first three-dimensional coordinate system and the second three-dimensional coordinate system are integrated; A head tracker including an integrated controller for calculating and displaying on a coordinate system and an eye tracker constitute an integrated eye tracking system. According to this, by tracking the position and direction of the helmet and the direction of the user's gaze to calculate the gaze direction of the user, there is an effect that can estimate the precise and rapid gaze direction. In addition, by integrating and using various algorithms at the time of head tracking or eye tracking, it is possible to compensate for the shortcomings of using any one algorithm and to perform head tracking or eye tracking more accurately and quickly.

Description

Line of sight tracking system with integrated head tracker and eye tracker and method thereof {LINE OF SIGHT TRACKING SYSTEM INTEGRATED WITH HEAD TRACKER AND EYE TRACKER AND MEHTOD THEREOF}

The present invention relates to a gaze tracking system and a method thereof, and more particularly, to a gaze tracking system and a method incorporating a head tracker and an eye tracker.

In weapons systems such as Apache helicopters, eye tracking systems are used to aim the target. The eye tracking system tracks the pilot's line of sight so that the target can be aimed quickly and quickly.

The conventional gaze tracking system tracks only the movement of the pilot's head or tracks the eyes of the pupil to estimate the pilot's gaze direction. However, in the case of the head tracker aiming the target by the movement of the head, there is a problem that the gaze point cannot be changed quickly because the eye movement is not considered. In the case of an eye tracker aiming at a target by tracking only the movement of the pupil, the eye of the pilot can be tracked using the equipment mounted on the helmet, but the gaze point also changes when the head moves.

On the other hand, conventional head trackers include optical head trackers and inertial head trackers. The optical head tracker operates by estimating the position and direction of the helmet by optically recognizing the distance between the feature points attached to the helmet, and has the advantage of calculating the position and direction of the helmet relatively accurately. However, the output speed is relatively slow. On the contrary, since the optical head tracker is operated in a manner of estimating the position and direction of the helmet by detecting the movement of the helmet by an inertial sensor, this has the advantage of estimating the position and direction of the helmet relatively quickly. . However, the accuracy in calculating the position and orientation of the helmet is poor.

As described above, both methods have advantages and disadvantages, but there is no known method that takes advantage of both methods.

On the other hand, the existing eye tracker also does not take all the advantages of each eye tracking algorithm by calculating the gaze direction according to any one eye tracking algorithm.

It is an object of the present invention to provide a gaze tracking system incorporating a head tracker and an eye tracker.

Another object of the present invention is to provide a gaze tracking method using an integrated head tracker and an eye tracker.

The eye tracking system incorporating the head tracker and the eye tracker for achieving the above object of the present invention, after recognizing the structure of the user's helmet, estimates the operation of the helmet, helmet position information and helmet direction on the first three-dimensional coordinate system Information is calculated, the head tracker which synchronizes and transmits the calculated helmet position information and helmet direction information according to the received synchronization signal, and tracks the gaze direction of the user to calculate gaze direction information on a second three-dimensional coordinate system, and calculates An eye tracker for synchronizing the transmitted gaze direction information according to the received synchronization signal, and receiving the helmet position information, the helmet direction information, and the gaze direction information, and using the received helmet position information, the helmet direction information, and the gaze direction information. The gaze direction of the user through the first three-dimensional coordinate system and the second three-dimensional coordinate system The claim can be configured to include an integrated controller for calculating the three-dimensional coordinate system and displayed. Here, the head tracker is configured to estimate the operation of the helmet by using an inertial sensor mounted on the helmet, and to use information on the estimated operation of the helmet to calculate the helmet position information and helmet direction information. Can be. The head tracker may be configured to recognize the feature points of the plurality of infrared LEDs attached to the helmet through an infrared camera to calculate the helmet position information and helmet direction information. The feature points may be configured to form a plurality of triangular patterns. The head tracker may be configured to calculate helmet position information and helmet direction information by recognizing a distance between feature points forming the triangle pattern. On the other hand, the eye tracker calculates the gaze direction information by using an infrared pupil image acquisition method, a multiple pupil recognition technique, and a gaze point estimation method, but the infrared pupil image acquisition method, a multiple pupil recognition technique, and a gaze point estimation method The eye gaze direction information may be calculated by proportionally summing according to a ratio of the amount of each measured value by. The eye tracker may be configured to recognize the infrared rays of the object reflected from the dichroic filter through an infrared camera to track the eye gaze of the user.

In the eye tracking method using an integrated head tracker and an eye tracker for achieving the above object of the present invention, the integrated controller generates a predetermined synchronization signal and transmits the generated synchronization signal to the head tracker and the eye tracker, respectively. The head tracker calculating helmet position information and helmet direction information of the user's helmet on a first three-dimensional coordinate system, and the eye tracker calculating eye gaze direction information of the user on a second three-dimensional coordinate system; The tracker receives the sync signal and transmits the helmet position information and helmet direction information to the integrated controller according to the received sync signal, and the eye tracker receives the sync signal and the eye direction information according to the received sync signal. Sending the to the unified controller, wherein the unified controller Receive helmet position information, helmet direction information and eye gaze direction information, and the first three-dimensional coordinate system and the second three-dimensional coordinate system integrate the gaze direction of the user by using the received helmet position information, helmet direction information and gaze direction information Calculating and displaying on the third, three-dimensional coordinate system. Here, the head tracker calculates the helmet position information and helmet direction information of the user's helmet on the first three-dimensional coordinate system, and the eye tracker calculating the gaze direction information of the user on the second three-dimensional coordinate system, the head The tracker may be configured to estimate the operation of the helmet using an inertial sensor mounted on the helmet, and use information on the estimated operation of the helmet to calculate the helmet position information and the helmet direction information. The head tracker calculates helmet position information and helmet direction information of the user's helmet on a first three-dimensional coordinate system, and the eye tracker calculates the gaze direction information of the user on a second three-dimensional coordinate system. It may be configured to recognize the feature points consisting of a plurality of infrared LED attached to the helmet through an infrared camera to calculate the helmet position information and helmet direction information. The feature points may be configured to form a plurality of triangular patterns. The head tracker may calculate helmet position information and helmet direction information of the user's helmet on a first three-dimensional coordinate system, and the eye tracker calculates the gaze direction information of the user on a second three-dimensional coordinate system. Recognize the distance between the feature points forming the triangular pattern may be configured to calculate the helmet position information and helmet direction information. On the other hand, the head tracker to calculate the helmet position information and helmet direction information of the user's helmet on the first three-dimensional coordinate system, the eye tracker calculating the eye gaze direction information of the user on the second three-dimensional coordinate system, the infrared pupil The gaze direction information is calculated using an image acquisition method, a multiple pupil recognition method, and a gaze point estimation method, and according to a ratio of the amount of each measurement value by the infrared pupil image acquisition method, the multiple pupil recognition method, and the gaze point estimation method. The proportional sum may be configured to calculate the gaze direction information. The head tracker calculates helmet position information and helmet direction information of the user's helmet on a first three-dimensional coordinate system, and the eye tracker calculates the gaze direction information of the user on a second three-dimensional coordinate system. The infrared ray of the object reflected from the dichroic filter may be recognized through an infrared camera to track the user's gaze direction.

Regarding the eye tracking system and the method as described above, in more detail, according to the eye tracking system and method integrated with the head tracker and the eye tracker, the position and direction of the helmet and the direction of the user's eye gaze to track the user By calculating the gaze direction of, the gaze direction can be estimated accurately and quickly. In addition, by integrating and using various algorithms at the time of head tracking or eye tracking, it is possible to compensate for the shortcomings of using any one algorithm and to perform head tracking or eye tracking more accurately and quickly.

1 is a block diagram of an eye tracking system incorporating a head tracker and an eye tracker according to an exemplary embodiment of the present invention.
2 is a block diagram of a head tracker according to an embodiment of the present invention.
3 is a block diagram of an eye tracker according to an embodiment of the present invention.
4 is a physical view of an eye tracking system according to an exemplary embodiment of the present invention.
5 is an estimation diagram of helmet position information and helmet direction information when the helmet moves forward according to an embodiment of the present invention.
FIG. 6 is an estimation diagram of helmet position information and helmet direction information when a yaw is rotated by 45 ° according to an embodiment of the present invention. FIG.
7 is a conceptual diagram illustrating the operation of an eye tracker according to an embodiment of the present invention.
8 is a physical view of an eye tracker according to an embodiment of the present invention.
9 is a diagram illustrating a pupil recognition and a gaze direction estimation result of an eye tracker according to an exemplary embodiment of the present invention.
10 is a flowchart of a gaze tracking method using an integrated head tracker and an eye tracker according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for like elements in describing each drawing.

The terms first, second, A, B, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an eye tracking system incorporating a head tracker and an eye tracker according to an exemplary embodiment of the present invention. 2 is a block diagram illustrating a head tracker according to an embodiment of the present invention, and FIG. 3 is a block diagram illustrating an eye tracker according to an embodiment of the present invention.

1 to 3, the eye tracker system 100 (hereinafter, referred to as an eye tracker system) incorporating a head tracker and an eye tracker according to the present invention may include a head tracker 110, an eye tracker 120, It can be configured to include an integrated controller 130. Here, the integrated controller 130 may be configured to include the synchronization signal generator 131, the signal detector 132, the information collector 133, and the information output unit 134. The head tracker 110 calculates the infrared LED 111 attached to the outside of the helmet, the inertial sensor 112 mounted on the inside of the helmet, the first infrared camera 113, the second infrared camera 114, and the helmet position and direction. It may be configured to include the unit 115. The eye tracker 120 may be configured to include a dichroic filter 121 mounted at a position in front of a user's eye of the helmet, an infrared camera 122 mounted at a front surface of the helmet, and a gaze direction calculator 123. .

The eye tracking system 100 estimates the gaze direction of the user more accurately and quickly by using the output values of the head tracker 110 and the eye tracker 120. The head tracker 110 is different from the conventional method. Both the head tracking by the optical method and the head tracking by the inertial sensor can be performed and used, and the eye tracker 120 can also be used by performing all the various eye tracking techniques. Hereinafter, each structure is demonstrated in detail.

First, the head tracker 110 recognizes the structure of the user's helmet, estimates the operation of the helmet, calculates helmet position information and helmet direction information on a first three-dimensional coordinate system, and calculates the calculated helmet position information and helmet direction information. And may be configured to transmit in synchronization according to the received synchronization signal. The head tracker 110 is a device for estimating the position and direction of a helmet worn by a user (pilot). At this time, both the optical method and the method by the inertial sensor are used. The optical method is a method of recognizing the position and direction of the helmet by calculating the positions and distances of the plurality of infrared LEDs 111 attached to the outside of the head tracker 110. Therefore, the infrared LEDs 111 attached to the helmet are feature points, and their positions must be specified in advance. On the other hand, the inertial sensor is a method in which the inertial sensor mounted inside the helmet detects the movement of the helmet and estimates the operation of the helmet in advance. The optical method is superior in the accuracy of the results, while the computational amount is complex and slow. However, the inertial sensor has a small amount of calculation and a fast calculation speed, but the accuracy is relatively low.

In the present invention, both methods are combined and used. That is, the head tracker 110 estimates the operation of the helmet using the inertial sensor 112 mounted on the helmet, and calculates the helmet position information and the helmet direction information based on the estimated operation of the helmet. It can be configured to use. In other words, the structure of the helmet is first recognized by the optical method, but after that, the motion of the helmet is sensed by the inertial sensor and used to estimate the position and direction of the helmet. That is, the helmet position information and the helmet direction information are calculated by the optical method, but by estimating the operation of the helmet according to the method by the inertial sensor, the calculation amount by the optical method can be reduced and the calculation speed can be increased.

Meanwhile, the sync signal should be periodically received in advance. As will be described later, the synchronization signal is received from the integrated controller 130, the calculated helmet position information and helmet direction information is transmitted to the integrated controller 130 in synchronization with the received synchronization signal.

Here, the head tracker 110 recognizes the feature points composed of a plurality of infrared LEDs 111 attached to the helmet through the first infrared camera 113 and the second infrared camera 114 to determine the helmet position information. And calculate helmet direction information. The infrared LED 111 is distributed to the outside of the helmet as a whole so that the first infrared camera 113 and the second infrared camera 114 can recognize the structure of the helmet even when the position or direction of the helmet changes.

In this case, the feature points may be configured to form a plurality of triangular patterns. Preferably, a total of 15 feature points, that is, the infrared LED 111 may be attached to one helmet, and the feature points may be configured to form five triangular patterns.

The head tracker 110 recognizes the structure of the helmet using the feature points recognized by the first infrared camera 113 and the second infrared camera 114. In particular, the helmet position and direction calculator 115 may be configured to calculate the helmet position information and the helmet direction information by recognizing the distance between the feature points forming the triangle pattern. The feature points may be configured to be distributed in a pattern of various shapes, but in the case of the triangular pattern, it is very useful because the amount of calculation for calculating the distance between the feature points is small.

Next, the eye tracker 120 may be configured to track the gaze direction of the user to calculate gaze direction information on a second three-dimensional coordinate system, and to synchronize and transmit the calculated gaze direction information according to the received synchronization signal. The eye tracker 120 may be configured to track a user's gaze direction using all of various eye tracking algorithms.

For example, the eye tracker 120 may be configured to calculate the gaze direction information by using an infrared pupil image acquisition method, a multiple pupil recognition technique, and a gaze point estimation method. The gaze direction information may be calculated by proportionally summing according to the ratio of the amount of each measured value by the pupil recognition technique and the gaze point estimation method. Each eye tracking algorithm has its advantages and disadvantages, and the efficiency can vary under unique conditions. Thus, in more detail, the eye tracker 120 applies all three eye tracking algorithms above, but the eye tracker 120 proportionally adds the eye tracking directions by the Kalman filter in the order of the measured values by the most well-known algorithm. May be configured to yield information. Thus, more accurate gaze direction information can be obtained.

Meanwhile, the synchronization signal should also be periodically received in advance. This sync signal is the same sync signal as that received by the head tracker 110 described above. The synchronization signal is received from the integrated controller 130, and the calculated gaze direction information is also synchronized to the received synchronization signal and transmitted to the integrated controller 130. Accordingly, the integrated controller 130 may receive the output signal of the synchronized head tracker 110 and the output signal of the eye tracker 120.

On the other hand, the eye tracker 120 may be configured to recognize the infrared ray of the object reflected from the dichroic filter 121 through the infrared camera 122 to track the eye gaze direction of the user. Infrared light is useful because it does not irritate the user's eyes and does not obstruct the field of view.

Next, the integrated controller 130 receives the helmet position information, helmet direction information and gaze direction information, and uses the received helmet position information, helmet direction information and gaze direction information to determine the gaze direction of the user in the first direction. The three-dimensional coordinate system and the second three-dimensional coordinate system may be configured to calculate and display on an integrated third three-dimensional coordinate system. That is, the integrated controller 130 may calculate or estimate the user's gaze direction by reflecting both the output value of the head tracker 110 and the output value of the eye tracker 120, unlike the conventional method, so that the target can be aimed more accurately. Make sure More specifically.

The synchronization signal generator 131 of the integrated controller 130 generates the synchronization signal and periodically transmits the synchronization signal to the head tracker 110 and the eye tracker 120. The synchronization signal generator 131 basically supports synchronization signal generation in the case of a real-time operating system. However, in the case of a non-real-time operating system, the synchronization signal generator 131 is determined for each thread such as a communication thread, an information collection thread, and a synchronization signal transmission thread. It can be configured to be driven in order.

When the head tracker 110 and the eye tracker 120 transmit an output signal in synchronization with the synchronization signal, the signal detector 132 of the integrated controller 130 receives the output signal. And check if the received output signal is properly received. It may be configured to check for transmission errors using a checksum.

Meanwhile, the information collecting unit 133 of the integrated controller 130 integrates the helmet position information and the helmet direction information of the head tracker 110 and the gaze direction information of the eye tracker 120 in one same three-dimensional space coordinate system. Thus, the gaze direction of the user is accurately estimated.

The information output unit 134 of the integrated controller 130 outputs the gaze direction of the user so that the user may aim the target.

4 is a physical view of an eye tracking system according to an exemplary embodiment of the present invention.

4 (a) shows a helmet and a plurality of infrared LEDs 111 attached to the outside of the helmet, a dichroic filter 121 and an infrared camera 122 mounted to the front of the helmet. 4B illustrates a first infrared camera 113 and a second infrared camera 114 that are installed toward the cockpit in front of the cockpit. And (c) of FIG. 4 shows a cockpit.

5 is an estimation diagram of helmet position information and helmet direction information when the helmet moves forward according to an embodiment of the present invention, and FIG. 6 is a helmet during rotation of yaw 45 ° of the helmet according to the embodiment of the present invention. It is an estimation figure of position information and helmet direction information.

Referring to FIG. 5, it can be seen that a result graph of eye tracking is shown when the helmet translates. Here, the green dotted line indicates information about the gaze direction of the user to be calculated. The red dot islands indicate information on the gaze direction of the user calculated by the optical method, and the blue solid lines indicate information about the gaze direction of the user calculated when both the optical method and the method by the inertial sensor are applied. As can be seen from FIG. 5, it can be seen that the user's gaze direction can be calculated more quickly and accurately when the inertial sensor is added than when only the conventional optical method is used.

Referring to FIG. 6, it can be seen that a result graph of eye tracking is shown when the helmet rotates. As shown in FIG. 5, the green dotted line indicates information on the gaze direction of the user to be calculated. The red dot islands indicate information on the gaze direction of the user calculated by the optical method, and the blue solid lines indicate information about the gaze direction of the user calculated when both the optical method and the method by the inertial sensor are applied. In FIG. 6, it can be seen that the user's gaze direction can be calculated more accurately and quickly when the optical method and the method by the inertial sensor are integrated. In the Roll graph, the blue solid line oscillation represents the inertia correction operation.

7 is a conceptual diagram illustrating the operation of an eye tracker according to an embodiment of the present invention.

Referring to FIG. 7, the dichroic filter secures the front visibility of the user, and reflects only infrared rays so that the infrared camera can track the direction of the user's eyes.

8 is a physical view of an eye tracker according to an embodiment of the present invention.

8A and 8B illustrate a user's wearing of the eye tracker 120. And (c) of FIG. 8 shows a state in which the user gazes at each gaze point after wearing the eye tracker 120 to perform gaze point correction experiment.

9 is a diagram illustrating a pupil recognition and a gaze direction estimation result of an eye tracker according to an exemplary embodiment of the present invention.

FIG. 9 illustrates a result of tracking the eyeline direction by recognizing the direction of the pupil of the user in FIG. 8 (c).

10 is a flowchart of a gaze tracking method using an integrated head tracker and an eye tracker according to an embodiment of the present invention. The gaze tracking method may be implemented using the gaze tracking system of FIG. 1. Hereinafter, the detailed configuration will be described.

First, the integrated controller 130 may be configured to generate a predetermined synchronization signal and transmit the generated synchronization signal to the head tracker 110 and the eye tracker 120, respectively (S110).

Next, the head tracker 110 calculates helmet position information and helmet direction information of the user's helmet on the first three-dimensional coordinate system, and the eye tracker 120 calculates the gaze direction information of the user on the second three-dimensional coordinate system. It may be configured to (S120). Here, the head tracker 110 estimates the operation of the helmet using an inertial sensor mounted on the helmet, and uses the information on the estimated operation of the helmet to calculate the helmet position information and helmet direction information. It can be configured to. And the head tracker 110 recognizes the feature points consisting of a plurality of infrared LED 111 attached to the helmet through the first infrared camera 113 and the second infrared camera 114, the helmet position information and helmet It may be configured to calculate the direction information. In this case, the feature points may be configured to form a plurality of triangular patterns. The head tracker 110 may be configured to recognize helmet distance information and helmet direction information by recognizing a distance between feature points forming the triangle pattern. Meanwhile, the eye tracker 120 calculates the gaze direction information by using an infrared pupil image acquisition method, a multiple pupil recognition technique, and a gaze point estimation method, but the infrared pupil image acquisition method, a multiple pupil recognition technique, and a gaze point estimation method. The eye gaze direction information may be calculated by proportionally summing according to a ratio of the amount of each measured value by. In addition, the eye tracker 120 may be configured to recognize the infrared ray of the object reflected from the dichroic filter 121 through the infrared camera 122 to track the eye gaze direction of the user.

Next, the head tracker 110 receives the synchronization signal and transmits the helmet position information and the helmet direction information to the integrated controller 130 according to the received synchronization signal, and the eye tracker 120 synchronizes the synchronization signal. Receiving a signal may be configured to transmit the gaze direction information to the integrated controller 130 according to the received synchronization signal (S130).

Next, the integrated controller 130 receives the helmet position information, the helmet direction information and the gaze direction information, and sets the gaze direction of the user by using the received helmet position information, the helmet direction information and the gaze direction information. The first three-dimensional coordinate system and the second three-dimensional coordinate system may be configured to calculate and display on the integrated third three-dimensional coordinate system (S140).

Although described with reference to the embodiments above, those skilled in the art will understand that the present invention can be variously modified and changed without departing from the spirit and scope of the invention as set forth in the claims below. Could be.

100: eye tracking system 110: head tracker
111: infrared LED 112: inertial sensor
113: first infrared camera 114: second infrared camera
115: helmet position and direction calculation unit 120: eye tracker
121: dichroic filter 122: infrared camera
123: gaze direction calculator 130: integrated controller
131: synchronization signal generator 132: signal detector
133: information collecting unit 134: information output unit

Claims (14)

The position information and posture information of the helmet calculated by recognizing feature points composed of a plurality of infrared LEDs attached to the helmet of the user, and the operation of the helmet detected using the inertial sensor mounted on the helmet A head tracker which calculates helmet position information and helmet direction information on the first three-dimensional coordinate system and synchronizes the calculated helmet position information and helmet direction information according to the received synchronization signal;
An eye tracker that tracks the gaze direction of the user to calculate gaze direction information on a second three-dimensional coordinate system, and synchronizes the calculated gaze direction information according to the received synchronization signal;
Receiving the helmet position information, helmet direction information and gaze direction information, the gaze direction of the user using the received helmet position information, helmet direction information and gaze direction information to the first three-dimensional coordinate system and the second three-dimensional coordinate system An eye tracking system incorporating a head tracker and an eye tracker comprising an integrated controller for calculating and displaying on an integrated third three-dimensional coordinate system. delete delete The method of claim 1,
And the feature points form a plurality of triangular patterns, wherein the eye tracker and eye tracker are integrated. 5. The method of claim 4,
The head tracker is,
And a head tracker and an eye tracker integrated with each other to calculate helmet position information and helmet direction information by recognizing a distance between the feature points forming the triangle pattern. The method of claim 5,
The eye tracker is,
The gaze direction information is calculated using an infrared pupil image acquisition method, a multiple pupil recognition method, and a gaze point estimation method, and the ratio of the amount of each measurement value by the infrared pupil image acquisition method, the multiple pupil recognition method, and the gaze point estimation method. The eye tracker integrated with the head tracker and the eye tracker, characterized in that to calculate the eye gaze direction information by proportional summing. The method according to claim 6,
The eye tracker is,
An eye tracking system incorporating a head tracker and an eye tracker, which tracks the gaze direction of the user by recognizing infrared rays of an object reflected from a dichroic filter through an infrared camera. Generating, by the integrated controller, a predetermined synchronization signal, and transmitting the generated synchronization signal to the head tracker and the eye tracker, respectively;
The head tracker estimated position information and posture information of the helmet calculated by recognizing the feature points consisting of a plurality of infrared LEDs attached to the user's helmet through an infrared camera, and the inertial sensor mounted on the helmet Calculating helmet position information and helmet direction information of the user's helmet on a first three-dimensional coordinate system by using an operation of the helmet, and the eye tracker calculating the gaze direction information of the user on a second three-dimensional coordinate system;
The head tracker receives the sync signal and transmits the helmet position information and helmet direction information to the integrated controller according to the received sync signal, and the eye tracker receives the sync signal and the gaze according to the received sync signal. Transmitting direction information to the unified controller;
The integrated controller receives the helmet position information, helmet direction information and gaze direction information, and uses the received helmet position information, helmet direction information and gaze direction information to determine the gaze direction of the user in the first three-dimensional coordinate system and the first direction. 2. A gaze tracking method integrating and using a head tracker and an eye tracker, comprising calculating and displaying on a third three-dimensional coordinate system having an integrated two-dimensional coordinate system. delete delete 9. The method of claim 8,
And the feature points form a plurality of triangular patterns, wherein the eye tracker and eye tracker are integrated. The method of claim 11,
The head tracker calculates the helmet position information and helmet direction information of the user's helmet on the first three-dimensional coordinate system, and the eye tracker calculates the gaze direction information of the user on the second three-dimensional coordinate system,
And a head tracker and an eye tracker integrated by using the head tracker to calculate the helmet position information and the helmet direction information by recognizing the distance between the feature points forming the triangle pattern. The method of claim 12,
The head tracker calculates the helmet position information and helmet direction information of the user's helmet on the first three-dimensional coordinate system, and the eye tracker calculates the gaze direction information of the user on the second three-dimensional coordinate system,
The gaze direction information is calculated using an infrared pupil image acquisition method, a multiple pupil recognition method, and a gaze point estimation method, and the ratio of the amount of each measurement value by the infrared pupil image acquisition method, the multiple pupil recognition method, and the gaze point estimation method. And calculating the gaze direction information by proportionally summing according to the gaze tracking method. The method of claim 13,
The head tracker calculates the helmet position information and helmet direction information of the user's helmet on the first three-dimensional coordinate system, and the eye tracker calculates the gaze direction information of the user on the second three-dimensional coordinate system,
The eye tracker recognizes the infrared rays of the object reflected from the dichroic filter through an infrared camera and tracks the eye gaze of the user.

Images