KR20220068440A - Method and apparatus for displaying eye movement - Google Patents

Abstract

The present invention relates to a method and apparatus for displaying eye movement. According to an embodiment of the present invention, a method for displaying eye movement comprises: a step of obtaining a video related to eye movement; a step of extracting an eye image in a frame unit from the video; a step of selecting at least one representative image from eye images extracted in the frame unit; a step of searching for an image corresponding to a new eye image of the at least one representative image when the new eye image is obtained; and a step of comparing and displaying a searched image with the new eye image. According to the present invention, a change in the range of eye movement could be objectively and accurately displayed.

Description

METHOD AND APPARATUS FOR DISPLAYING EYE MOVEMENT

The present invention is directed to a method and apparatus for displaying eye movements that can objectively, accurately, and efficiently display changes in the range of motion of the eyeball.

The human eyeball is moved by four rectus muscles (medial rectus, superior rectus, inferior rectus, external rectus) and two oblique muscles (superior and inferior oblique). The position of the eyes is determined by the viscosity of the orbit and the elasticity of the muscles, and when the eyes are moved, both eyes move in the same direction at the same time. However, problems with eye movement may occur due to various reasons such as aging or muscle abnormalities. Problems of such eye movement may include an imbalance of eye movement, abnormal eye muscle, eye movement disorder, strabismus, and the like.

In order to diagnose and manage these problems, it is very important to check the range of motion of the eyeball. In particular, if the contents and extent of each type of eye movement can be accurately confirmed, it is possible to diagnose whether a related neural circuit is damaged or an ongoing disease or the like.

Accordingly, there is a need for a method for objectively, accurately, and efficiently displaying changes in the range of motion of the eyeball.

The present invention can provide an eye movement display method and apparatus capable of objectively, accurately and efficiently displaying a change in an eye movement range.

The eye movement display method according to an embodiment of the present invention includes: obtaining a moving picture related to eye movement; extracting an eyeball image from the moving image frame by frame; Selecting a representative image, when acquiring a new eyeball image, searching for an image corresponding to the new eyeball image from among the at least one or more representative images, Comprising the steps of comparing and displaying the searched image with a new eyeball image can do.

According to an embodiment, the step of obtaining a video related to the eye movement includes obtaining a video by photographing an eye movement, obtaining a video captured from a video recording device capturing the eye movement, or a video stored in a storage device may include the step of obtaining

According to an embodiment, the selecting of the at least one representative image may include aligning the directions of each image based on a head movement with respect to the eyeball images extracted in units of frames.

According to an embodiment, the selecting of the at least one or more representative images includes at least one or more set eye movement sections, at least one set eye position, or at least one or more cardinal positions of gaze. It may include selecting one or more representative images.

According to an embodiment, the generating of the representative image may include selecting other images from among the images captured in the at least one or more set eye movement sections, the at least one set eye positions, or the at least one or more basic positions of the gaze. The method may include selecting the at least one or more representative images based on the degree of similarity to .

According to an embodiment, the step of searching for an image corresponding to the new eyeball image includes at least one first feature extracted from the new eyeball image and at least one first feature extracted from the at least one or more representative image. The method may include comparing and determining an image having a similarity of the extracted first features greater than a threshold value as an image corresponding to the new eyeball image.

According to an embodiment, the determining of the at least one representative image as an image corresponding to the new eyeball image may include, when a plurality of representative images having a similarity higher than a threshold value exist, at least one different from the first characteristic The method may include comparing second features and determining an image having a similarity of the extracted second features greater than a threshold value as an image corresponding to the new eyeball image.

According to an embodiment, the extracted feature may include at least one of a pupil, an eyeliner, an iris, a size of an eyeball, a size of a cornea, and a position of an eyelid crevice.

According to an embodiment, the comparing of the at least one first feature extracted from the new eyeball image with the at least one first feature extracted from the at least one or more representative images includes using an artificial neural network. can be performed.

According to an embodiment, the comparing and displaying the searched image and the new eyeball image may include overlapping the searched image and the new eyeball image and displaying the displayed image.

According to an embodiment, at least one of the found image and the new eyeball image may be displayed translucently.

According to an embodiment, the comparing and displaying the searched image and the new eyeball image may include displaying at least one of a movement angle of the eyeball, a change in the position of the corneal limbus, and a change in the central position of the eyeball.

The eye movement display device according to an embodiment of the present invention includes a display unit, a video acquisition unit, at least one memory for storing a program for displaying eye movements, and executing the program, so that eye movement related through the video acquisition unit When acquiring a moving picture, extracting an eyeball image from the moving picture in units of frames, selecting at least one representative image from the extracted eyeball images in units of frames, and acquiring a new eye image, the at least one representative image It may include at least one processor that searches for an image corresponding to the new eyeball image among them, and controls to compare and display the searched image with the new eyeball image through the display unit.

According to an embodiment, the video acquisition unit includes a video capture unit for acquiring a video by photographing an eye movement, a transceiver for receiving a video captured from a video recording device capturing an eye movement, and a storage for storing the video It may include at least one or more of the parts.

According to an embodiment, the at least one processor may control to align the directions of each image based on a head movement with respect to the eyeball images extracted in units of frames.

According to an embodiment, the at least one processor is configured to select the at least one or more representative images in at least one or more set eye movement sections, at least one or more set eye positions, or at least one or more cardinal positions of gaze. You can control what you choose.

According to an embodiment, the at least one processor may be configured to communicate with other images among images captured in the at least one or more set eye movement sections, the at least one set eye positions, or the at least one or more basic positions of the gaze. It is possible to control the selection of the at least one or more representative images based on the similarity.

According to an embodiment, the at least one or more processors compare at least one first feature extracted from the new eyeball image with at least one first feature extracted from the at least one or more representative image, and 1 It is possible to control an image having a similarity greater than a threshold value to be determined as an image corresponding to the new eyeball image.

According to an embodiment, the at least one processor compares a first feature with at least one other second feature, and compares the first feature with at least one other second feature, when there are a plurality of representative images having the similarity higher than a threshold value. An image having a similarity greater than a threshold value may be determined as an image corresponding to the new eyeball image.

According to an embodiment, the extracted feature may include at least one of a pupil, an eyeliner, an iris, a size of an eyeball, a size of a cornea, and a position of an eyelid crevice.

According to an embodiment, the at least one or more processors may use an artificial neural network.

According to an embodiment, the at least one processor may control the searched image and the new eyeball image to be displayed in an overlapping manner.

According to an embodiment, the at least one processor may display at least one of the searched image and the new eyeball image semi-transparently.

According to an embodiment, the at least one processor may control to display at least one of a movement angle of the eyeball, a change in the position of the corneal limbus, and a change in the central position of the eyeball.

According to the present invention, it is possible to objectively and accurately display the change in the range of motion of the eyeball.

1 is a view showing before and after an eye movement range.
2 is a flowchart illustrating an eye movement display method according to an embodiment of the present invention.
3 is a view for explaining a method of aligning the direction of an image according to an embodiment of the present invention.
4 is a view for explaining an eyeball position according to an embodiment of the present invention.
5 is a view for explaining an eye movement section according to an embodiment of the present invention.
6 is a diagram for comparing and displaying a searched image and a new eyeball image according to an exemplary embodiment.
7 is a view for explaining a change in the position of the corneal limbus according to an embodiment of the present invention.
8 is a diagram illustrating the configuration of an eye movement display device according to an embodiment of the present invention.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present disclosure pertains can easily implement them. However, the present disclosure may be implemented in several different forms and is not limited to the embodiments described herein. In addition, in order to clearly explain the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented in various numbers of hardware and/or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors, or may be implemented by circuit configurations for a predetermined function. Also, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as an algorithm running on one or more processors. In addition, the present disclosure may employ prior art for electronic configuration, signal processing, and/or data processing, and the like.

In addition, the connecting lines or connecting members between the components shown in the drawings only exemplify functional connections and/or physical or circuit connections. In an actual device, a connection between components may be represented by various functional connections, physical connections, or circuit connections that are replaceable or added.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

1 is a view showing before and after an eye movement range.

Although the eye movement appears to be a linear movement moving up and down, left and right, it is actually a rotational movement in which the center of rotation is located in the center of the eyeball. The eyeball rotates around an axis lying on the equator perpendicular to the line of sight. In the case of a normal eyeball, the eyeball moves within a normal range, but aging, muscle abnormalities, damage to nerve circuits, or ongoing diseases such as exotropia, esotropia, paralytic strabismus, thyroid eye disease, myasthenia gravis, Brown syndrome, chronic progression If you have external ocular muscle paralysis, Duane retraction syndrome, orbital rupture fracture, problems with eye movement may occur. If a problem occurs in eye movement, the range of eye movement may be limited or excessive eye movement may occur. These eye movement abnormalities can be corrected through treatment. As such, in order to recognize that a problem has occurred in eye movement or to correct the eye movement abnormality through treatment, a method for confirming changes in the range of eye movement is required.

Referring to FIG. 1, FIG. 1(a) shows a state in which the eye movement range is limited, and FIG. 1(b) shows an eye movement close to a normal range. When comparing FIG. 1(a) and FIG. 1(b) side by side, the difference in eye movement range can be roughly confirmed, but when comparing only each screen separately or sequentially, the difference in eye movement range can be objectively And it is not easy to know exactly.

Accordingly, there is a need for a method for objectively, accurately, and efficiently displaying changes in the range of motion of the eyeball.

2 is a flowchart illustrating an eye movement display method according to an embodiment of the present invention.

First, in step 210, a video related to eye movement is acquired. In an embodiment, for the video related to eye movement, a video may be obtained by photographing an eye movement, a video captured from a video recording device capturing the eye movement may be obtained, or a video stored in a storage device may be obtained. For example, a video may be obtained by photographing an eye movement using a camera, or a video in which the eye movement is photographed may be received from an external video recording device. In addition, it is also possible to acquire a video related to eye movement that has already been captured and stored in an external or internal storage device. However, the present invention is not limited thereto, and a moving picture related to the eye movement may be obtained in various ways, and a moving picture related to the eye movement may be obtained by using two or more methods. For example, a video may be obtained by photographing an eye movement, and the captured video may be received from an external device.

Thereafter, in step 220, an eyeball image is extracted from the video in frame units. For example, in the case of a 60 fps (frame per second) video, 60 eyeball images can be extracted every second. In this case, the eyeball image may be extracted for each specific frame unit. For example, one image may be extracted per one frame, one image may be extracted per ten frames, and one image may be extracted per 60 frames.

In this way, instead of using the moving image itself, the change in the range of motion of the eyeball can be efficiently displayed by extracting and using an image from the moving image. More specifically, a video has a large data size, so it is difficult to store and manage it. Accordingly, efficiency can be increased by using an image having a relatively small size.

In operation 230, at least one representative image is selected from the eyeball images extracted in units of frames. In operation 220 , a plurality of eyeball images may be extracted from the video in frame units. Among them, a representative image to be used to display the change in the range of motion of the eye may be selected.

In an embodiment, in order to select a representative image, directions of each image may be aligned based on a head movement with respect to the eye images extracted in units of frames. When shooting a video related to eye movement, a person's head, ie, the head, may move without being fixed for various reasons. When the head moves, the direction of the human eye also changes according to the movement of the head. It will be described with reference to FIG. 3 .

3 is a view for explaining a method of aligning the direction of an image according to an embodiment of the present invention.

Referring to FIG. 3, in the case of FIG. 3(b), the central axis 30 is in a vertical state, but the directions of the central axes 31 and 32 in FIGS. 3(a) and 3(c) are not vertical but left and right. You can check that it is wrong. In this way, when the eyeball image in a state in which the central axis is misaligned is used, it is difficult to accurately display the change in the range of motion of the eyeball. In particular, since the range of motion of the eyeball is not large, it may be difficult to accurately display the change in the range of motion of the eyeball even when the central axis is slightly shifted. Accordingly, in one embodiment, in order to accurately display the change in the movement range of the sphere, the direction of each image may be aligned based on the head movement with respect to the eyeball images extracted in units of frames.

Returning to the description of FIG. 2 , in one embodiment, a representative image may be selected from at least one or more set eye movement sections, at least one set eye position, or at least one or more cardinal positions of gaze. . It will be described with reference to FIGS. 3 and 4 .

4 is a view for explaining an eyeball position according to an embodiment of the present invention.

The basic position of the gaze is a central position in which the eyeball is straight ahead, that is, the visual axis is parallel to the midsagittal plane of the head, adduction, abduction, superior and inferior As such, it includes the position of the eye after only one pure horizontal or vertical rotation has occurred, and the position of the eye after a combined horizontal and vertical rotation from the central position. That is, the 9 cardinal positions of gaze may include a central position, nine positions above, below, left, right, upper left, lower left, upper right, or lower right of the central position.

The eyeball can move to the basic positions of these nine gazes by one rotation around one axis. In one embodiment, the eyeball image at this basic position may be selected as the representative image.

5 is a view for explaining an eye movement section according to an embodiment of the present invention.

When shooting a video related to the eyeball, it is possible to record the movement of the eyeballs for each position, but also a state in which the eyeball rotates in a clockwise or counterclockwise direction. In this case, at least one representative image may be selected in the set eye movement section.

Referring to FIG. 5 , in FIG. 5( a ), the eyeball is in an upturned state, and in FIG. 5( b ), the eyeball is abducted (or adducted). When taking a picture of the eyeball rotating in a clockwise or counterclockwise direction, the eye movement section may include an eye movement section 51 including a case in which the eyeball is inverted, and the eyeball is abducted (or adducted) It may include an eye movement section 52 including a case where the In one embodiment, at least one or more representative images may be selected from at least one or more set eye movement sections as described above.

Returning to the description of FIG. 2 , it is also possible to select a representative image from at least one set eyeball position in addition to at least one set eye movement section or at least one or more basic positions of gaze.

In an embodiment, the representative image may be selected based on similarity with other extracted images. For example, an image having the highest similarity to other images may be selected as the representative image. A plurality of representative images may be selected, and in this case, a plurality of representative images may be selected in an order of high similarity. That is, at least one representative image may be selected based on the similarity with other images among images captured in at least one or more set eye movement sections, at least one set eye position, or at least one basic position of gaze.

Thereafter, in step 240 , when a new eyeball image is obtained, an image corresponding to the new eyeball image is searched for among at least one or more representative images. In an embodiment, comparing at least one first feature extracted from a new eyeball image with at least one first feature extracted from at least one or more representative images, and selecting an image in which the similarity of the extracted first features is greater than a threshold value It may be determined as an image corresponding to the new eyeball image. In this case, the extracted features may include a pupil, an eyeliner, an iris, a size of an eyeball, a size of a cornea, a position of an eyelid crevice, and the like.

In addition, in an embodiment, when there are a plurality of representative images having a similarity higher than a threshold value, the first characteristic and at least one other second characteristic are compared, and an image in which the similarity of the extracted second characteristics is greater than the threshold value is selected. It may be determined as an image corresponding to the new eyeball image. That is, an image corresponding to a new eyeball image may be searched by changing the criterion in order to increase the search accuracy.

In this case, an artificial neural network may be used to compare at least one first feature extracted from a new eyeball image with at least one first feature extracted from at least one or more representative images. Such an artificial neural network may be implemented to compare features through machine learning, and may be implemented as a convolutional neural network (CNN) or a modified/improved artificial neural network.

In this way, by implementing a search for a pre-stored eye image based on a new eye image, the identity of the target image can be recognized based on the similarity between the representative images without collecting personal information of the subject to be photographed.

Finally, in step 250, the searched image and the new eyeball image are compared and displayed. In an embodiment, the searched image and the new eyeball image may be overlapped and displayed. In this case, at least one of the searched image and the new eyeball image may be displayed translucently. It will be described with reference to FIG. 6 .

6 is a diagram for comparing and displaying a searched image and a new eyeball image according to an exemplary embodiment.

FIG. 6 is a view showing the overlapping of FIGS. 1(a) and 1(b). For comparison between Fig. 1 (a) and Fig. 1 (b), Fig. 1 (a) was treated to be semi-transparent, and Fig. 1 (a) treated to be semi-transparent was displayed overlaid with Fig. 1 (b). In FIG. 6 in which the image processing is performed in this way, the change in the movement range 60 of the eyeball can be accurately identified.

Returning to the description of FIG. 2 again, as described above, the method of overlapping the searched image and the new eye image is an example of a method of comparing and displaying the searched image and the new eye image, and is not limited thereto. You can compare the image with the new eye image and display it.

In an embodiment, when the searched image and the new eyeball image are compared and displayed, the movement angle of the eyeball, the change in the position of the corneal limbus, the change in the central position of the eye, etc. may be displayed. Although the difference in the range of motion of the eye can be intuitively known through the comparison of the images, according to this embodiment, it is also possible to provide the comparison result in a more objective way. It will be described with reference to FIG. 7 .

7 is a view for explaining a change in the position of the corneal limbus according to an embodiment of the present invention.

Referring to FIG. 7 , the position of the corneal limbus in the searched image and the new eyeball image may be compared. More specifically, a point of the corneal limbus located on a straight line passing through the center of the pupil in the searched image and the new eyeball image may be compared ( 70 ). For example, by overlapping the retrieved image and the new eyeball image as shown in FIG. 7 , the extension line of a straight line passing through the pupil center and the pupil center in the comparison image, and the corneal limbus in the reference image and the corneal limbus in the comparison image, respectively The point of intersection can be measured.

According to an embodiment of the present invention, it is possible to objectively and accurately display the change in the range of motion of the eyeball.

8 is a diagram illustrating the configuration of an eye movement display device according to an embodiment of the present invention.

Referring to FIG. 8 , the apparatus for displaying eye movements according to an embodiment may include a display 810 , a video acquisition unit 820 , a memory 830 , and a processor 840 . However, the present invention is not limited thereto, and the eye movement display device may be implemented by more or fewer than all of the elements illustrated in FIG. 8 .

Although the eye movement display device is illustrated as including one processor in FIG. 8 , the embodiment is not limited thereto and may include a plurality of processors. At least some of the operations and functions of the processor 840 described below may be performed by a plurality of processors. The eye movement display device illustrated in FIG. 8 may perform the eye movement display method according to various embodiments of the present disclosure, and the descriptions of FIGS. 1 to 7 may be applied. Accordingly, content overlapping with the above will be briefly described.

The display unit 810 displays information processed by the eye movement display device. The display unit 810 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display ( 3D display) and electrophoretic display (electrophoretic display) may include at least one. Meanwhile, when the display unit 810 and the touchpad form a layer structure to form a touch screen, the display unit 810 may be used as an input device in addition to an output device. In an embodiment, the display unit 810 may compare and display the searched image with a new eyeball image.

The video acquisition unit 820 acquires a video related to eye movement. In an embodiment, the video acquisition unit 820 includes a video recording unit for acquiring a video by photographing an eye movement, a transceiver for receiving a video captured from a video recording device capturing an eye movement, and storing the video It may include a storage unit for The moving picture capturing unit may include a camera sensor that captures a subject and converts an optical signal into an electrical signal, and a signal processor that converts an analog image signal into digital data. Here, the camera sensor may be a charge coupled device (CCD) sensor or a complementary metal semiconductor (CMOS) sensor, and the signal processing unit may be implemented as a digital signal processor (DSP). In addition, the camera sensor and the signal processing unit may be implemented integrally or may be implemented separately. In an embodiment, the video capturing unit may record eye movements. The transceiver may receive a moving picture of an eye movement from an external device. In addition, the transceiver may include both wired and wireless transmission units. The storage unit may store a moving picture related to eye movement received from an external device or captured by the moving picture capturing unit.

The memory 830 may be installed and stored in various types of data such as programs and files such as applications. The processor 840 may access and use data stored in the memory 830 or store new data in the memory 830 . In one embodiment, the memory 830 may install and store a program and data for displaying eye movements.

The processor 840 controls the overall operation of the eye movement display device, and may include at least one processor such as a CPU or GPU. The processor 840 may control other components included in the eye movement display device to perform an eye movement display operation. Also, the processor 840 may execute a program stored in the memory 830 , read a stored file, or store a new file.

In an embodiment, the processor 840 executes a program stored in the memory 830 to obtain a moving image related to eye movement through the moving image acquisition unit 810, extracts an eyeball image from the moving image frame by frame, and When at least one representative image is selected from the eyeball images extracted as a unit and a new eyeball image is acquired, an image corresponding to the new eyeball image is searched for among the at least one or more representative images, and through the display unit 820 You can control to compare the searched image with the new eye image and display it.

In an embodiment, the processor 840 may control to align the direction of each image based on a head movement with respect to the eyeball images extracted on a frame-by-frame basis. The processor 840 may control to select the at least one representative image from at least one or more set eye movement sections, at least one set eye position, or at least one or more cardinal positions of gaze. In addition, the processor 840 is configured to display the at least one or more representatives based on the similarity with other images among images captured in at least one or more set eye movement sections, at least one set eye position, or at least one or more basic positions of gaze. You can control the selection of images.

In an embodiment, the processor 840 compares the at least one first feature extracted from the new eyeball image with the at least one first feature extracted from the at least one or more representative images, and the degree of similarity between the extracted first features is a threshold value. An image larger than the value may be controlled to be determined as an image corresponding to a new eyeball image. In addition, the processor 840 compares the first feature with at least one other second feature when there are a plurality of representative images having a similarity higher than a threshold value, and selects an image in which the similarity of the extracted second features is greater than a threshold value. It may be determined as an image corresponding to the new eyeball image. In this case, the processor 840 may use an artificial neural network. In an embodiment, the extracted feature may include at least one of a pupil, an eyeliner, an iris, a size of an eyeball, a size of a cornea, and a position of an eyelid crevice.

In an embodiment, the processor 840 may control the image searched through the display 820 and the new eyeball image to be displayed in an overlapping manner. The processor 840 may semi-transparently display at least one of the searched image and the new eyeball image.

According to an embodiment, the processor 840 may control to display at least one of a movement angle of the eyeball, a change in the position of the corneal limbus, and a change in the central position of the eye.

Meanwhile, the above-described embodiment can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable medium. In addition, the structure of the data used in the above-described embodiment may be recorded in a computer-readable medium through various means. In addition, the above-described embodiment may be implemented in the form of a computer program product including a recording medium including instructions executable by a computer, such as a program module executed by a computer. For example, methods implemented as a software module or algorithm may be stored in a computer-readable recording medium as computer-readable codes or program instructions.

Computer-readable media may be any recording media that can be accessed by a computer, and may include volatile and nonvolatile media, removable and non-removable media. The computer readable medium may include a magnetic storage medium, for example, a ROM, a floppy disk, a hard disk, and the like, and an optically readable medium, for example, a storage medium such as a CD-ROM or DVD, but is not limited thereto. . Additionally, computer-readable media may include computer storage media and communication media.

In addition, a plurality of computer-readable recording media may be distributed in network-connected computer systems, and data stored in the distributed recording media, for example, program instructions and codes, may be executed by at least one computer. have.

Although embodiments of the present disclosure have been described with reference to the above and the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can practice the present invention in other specific forms without changing its technical spirit or essential features. You can understand that there is Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

810: display unit
820: video acquisition unit
830: memory
840: processor

Claims (18)

acquiring a video related to eye movement;
extracting an eyeball image in units of frames from the video;
selecting at least one representative image from the eyeball images extracted for each frame;
when acquiring a new eyeball image, searching for an image corresponding to the new eyeball image from among the at least one or more representative images; and
Comprising the step of comparing and displaying the searched image and the new eyeball image, the eye movement display method. According to claim 1,
The step of acquiring a video related to the eye movement is,
A method comprising the step of acquiring a video by photographing an eye movement, acquiring a video captured from a video recording device recording an eye movement, or acquiring a video stored in a storage device. According to claim 1,
The step of selecting the at least one or more representative images comprises:
and aligning the direction of each image based on a head movement with respect to the eye images extracted in units of frames. According to claim 1,
The step of selecting the at least one or more representative images comprises:
Selecting the at least one representative image from at least one or more set eye movement sections, at least one set eye position, or at least one or more cardinal positions of gaze. 5. The method of claim 4,
The step of generating the representative image comprises:
Selecting the at least one representative image based on the similarity with other images among the at least one or more set eye movement sections, the at least one or more set eye positions, or images captured at the at least one or more basic positions of the gaze A method comprising steps. According to claim 1,
The step of searching for an image corresponding to the new eyeball image,
comparing the at least one first feature extracted from the new eyeball image with the at least one first feature extracted from the at least one or more representative images; and
and determining an image in which the similarity of the extracted first features is greater than a threshold value as an image corresponding to the new eyeball image. 7. The method of claim 6,
The step of determining the at least one representative image as an image corresponding to the new eyeball image comprises:
comparing a first feature with at least one second feature different from the first feature when there are a plurality of representative images having the similarity higher than a threshold value; and
and determining an image in which the similarity of the extracted second features is greater than a threshold value as an image corresponding to the new eyeball image. 7. The method of claim 6,
The extracted features are
A method comprising at least one of a pupil, an eyeliner, an iris, a size of an eyeball, a size of a cornea, and a position of an eyelid crevice. 7. The method of claim 6,
Comparing the at least one first feature extracted from the new eyeball image with the at least one first feature extracted from the at least one or more representative images,
A method performed using an artificial neural network. According to claim 1,
Comparing and displaying the searched image with a new eye image includes:
and overlapping the retrieved image with the new eye image. 11. The method of claim 10,
displaying at least one of the retrieved image and the new eye image as translucent. According to claim 1,
Comparing and displaying the searched image with a new eye image includes:
A method of displaying at least one of a movement angle of the eyeball, a change in the position of the corneal limbus, and a change in the central position of the eyeball. display unit;
video acquisition unit;
at least one memory for storing a program for displaying eye movements; and
By executing the program, a video related to eye movement is obtained through the video acquisition unit, an eye image is extracted from the video frame by frame, and at least one representative image is selected from the eyeball images extracted for each frame, and , when acquiring a new eyeball image, searches for an image corresponding to the new eyeball image among the at least one or more representative images, and compares and displays the searched image with the new eyeball image through the display unit. At least one or more An eye movement display device comprising a processor. 14. The method of claim 13,
The video acquisition unit,
A device comprising at least one of a video recording unit for capturing an eye movement to obtain a video, a transceiver for receiving a video captured from a video recording device for capturing an eye movement, and a storage unit for storing the video. 14. The method of claim 13,
The at least one or more processors,
An apparatus for controlling to align the direction of each image based on a head movement with respect to the eye images extracted in units of frames. 14. The method of claim 13,
The at least one or more processors,
At least one first feature extracted from the new eyeball image is compared with at least one first feature extracted from the at least one or more representative images, and an image in which the degree of similarity of the extracted first features is greater than a threshold is selected as the new Controlling the determination to be an image corresponding to the eyeball image, the device. 14. The method of claim 13,
The at least one or more processors,
Controlling the searched image and the new eyeball image to overlap and display. 14. The method of claim 13,
The at least one or more processors,
An apparatus for controlling to display at least one of a movement angle of the eyeball, a change in the position of the corneal limbus, and a change in the central position of the eyeball.

Images