KR102563365B1 - System and method for ambulatory monitoring of eye alignment in strabismus - Google Patents

Abstract

The present invention relates to a system and method for monitoring occurrence of strabismus in daily life, the system comprising: a wearable device including an eye tracker and a camera module; a data analyzer configured to receive and analyze data from the wearable device; and a feedback output unit for determining and outputting feedback based on a result output from the data analysis unit, wherein the data analysis unit obtains strabismus expression data based on the gaze positions of the right and left eyes of the wearer obtained by the eye tracker, , The strabismus expression data includes information on the time of occurrence of strabismus, the frequency of occurrence of strabismus, and the duration of occurrence of strabismus.
According to the present invention, it is possible to accurately determine the severity of intermittent strabismus, it is possible to present a new paradigm for treatment such as surgery, and customized treatment and/or new non-surgical treatment according to the analysis of the strabismus expression environment is possible.

Description

System and method for monitoring strabismus in everyday life {System and method for ambulatory monitoring of eye alignment in strabismus}

The present invention relates to a system and method for monitoring strabismus expression in daily life, and more particularly, to a strabismus expression monitoring system and method capable of determining the severity of strabismus and analyzing the strabismus-inducing environment.

Strabismus is a visual disorder in which the two eyes are not aligned and looking at different points.

The angle of strabismus is an objective measure of the degree of strabismus, and the angle of strabismus indicates how much the other eye rotates.

Conventionally, there are various methods of measuring such an angle of view by obtaining pupil position data using a camera.

However, for practical treatment of strabismus, objective data on the occurrence frequency are required, but there is a limitation in that such data cannot be obtained only by measuring the angle of strabismus according to the prior art.

In particular, in the case of intermittent strabismus, the frequency of occurrence of strabismus is very important in determining its severity and deciding whether to perform surgery, but conventionally, there is no objective measurement method for how frequent strabismus is in everyday life and under what circumstances it usually appears.

Currently, we estimate the usual frequency of occurrence by asking parents about the frequency of occurrence or by looking at how quickly they can overcome when strabismus is induced in the clinic, but this is very inaccurate and cannot reflect the frequency of normal occurrence. there is

Therefore, there is currently a need for a technique capable of monitoring the expression of strabismus in daily life for effective treatment of strabismus.

KR 10-1761635 B1

An object of the present invention is to provide a system and method capable of monitoring the occurrence of strabismus in daily life.

In addition, an object of the present invention is to provide a system and method capable of analyzing the occurrence time, frequency, and duration of strabismus, determining the severity based on this, and analyzing the strabismus-inducing environment.

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the description below.

In order to achieve the above object, a system for monitoring strabismus expression in daily life according to an embodiment of the present invention includes a wearable device including an eye tracker and a camera module; a data analyzer configured to receive and analyze data from the wearable device; and a feedback output unit for determining and outputting feedback based on a result output from the data analysis unit, wherein the data analysis unit obtains strabismus expression data based on the gaze positions of the right and left eyes of the wearer obtained by the eye tracker, , The strabismus expression data includes information on the time of occurrence of strabismus, the frequency of occurrence of strabismus, and the duration of occurrence of strabismus.

Here, the data analysis unit may determine the severity based on the strabismus expression data.

Here, the data analysis unit may analyze the image captured by the camera module through artificial intelligence, automatically classify the wearer's activity of the wearable device, and determine the strabismus-inducing environment.

Here, the wearable device may further include a display unit, and the feedback output unit may output feedback to the display unit.

Here, a remote medical center capable of communicating with the data analyzer and the feedback output unit may be further included, the feedback output unit may receive feedback from the remote medical center, and the feedback may be displayed on the display unit.

Here, the wearable device may be a glasses-type device further including an augmented reality module.

Here, gaze positions of the wearer's right and left eyes are measured at regular intervals by an eye tracker, and the strabismus expression data may include daily variation data of the maximum angle of deviation.

A method for monitoring strabismus expression in daily life according to another embodiment of the present invention includes obtaining gaze position data and image data of a right eye and a left eye of a wearer of a wearable device from an eye tracker and a camera module of the wearable device, respectively; obtaining strabismus expression data by analyzing gaze position data in a data analyzer; Analyzing the image data in a data analysis unit to determine a strabismus-induced environment; and determining feedback based on the strabismus expression data and the strabismus inducing environment in a feedback output unit and outputting the feedback to the wearable device.

Here, the data analysis unit may analyze the image data through artificial intelligence and automatically classify the activity of the wearer to determine the strabismus-inducing environment.

According to the present invention, it is possible to accurately determine the severity of intermittent strabismus and present a new paradigm for treatment such as surgery, as it is possible to acquire objective data on the frequency of occurrence of intermittent strabismus in daily life and the form of strabismus.

In addition, it is possible to know whether strabismus appears when the patient performs any activity, and through this, customized treatment and/or new non-surgical treatment for the patient capable of minimizing the occurrence of strabismus is possible.

In addition, since it can be used in a telemedicine, it is possible to provide convenient and immediate medical services for strabismus treatment to more patients.

1 is a block diagram showing a schematic configuration of a strabismic expression monitoring system in daily life according to an embodiment of the present invention.
2 is a diagram showing a schematic configuration of a wearable device of a system for monitoring strabismus in daily life according to an embodiment of the present invention.
3 and 4 are graphs showing changes in the eye opening angle measured using the eye tracker of the wearable device according to time (minutes) according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. At this time, it should be noted that in the accompanying drawings, the same components are indicated by the same reference numerals as much as possible. In addition, detailed descriptions of well-known functions and configurations that may obscure the subject matter of the present invention will be omitted. For the same reason, in the accompanying drawings, some components are exaggerated, omitted, or schematically illustrated.

In addition, throughout the specification, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated. Also, throughout the specification, "on" means to be located above or below the target part, and does not necessarily mean to be located on the upper side relative to the direction of gravity.

First, with reference to FIGS. 1 and 2 , a system for monitoring strabismus expression in daily life according to an embodiment of the present invention will be described.

1 is a block diagram showing a schematic configuration of a strabismic expression monitoring system in daily life according to an embodiment of the present invention.

2 is a diagram showing a schematic configuration of a wearable device of a system for monitoring strabismus in daily life according to an embodiment of the present invention.

A system for monitoring strabismus in daily life according to an embodiment of the present invention includes a wearable device 100 , a data analysis unit 200 and a feedback output unit 300 .

In one embodiment, the data analysis unit 200 and/or the feedback output unit 300 may be some components built into the wearable device 100, but are not limited thereto.

The wearable device 100 is a device including an eye tracker 110, a camera module 120, and a display unit 130, and may be a device in the form of glasses worn by a patient during daily life.

Here, the eye tracker 110 may acquire gaze data of each eye by measuring the gaze positions of the right eye and the left eye of the wearer in daily life at regular intervals.

The camera module 120 is a camera built into the wearable device 100 and may obtain images of the wearer's gaze activity and gaze environment.

The wearable device 100 further includes a display unit 130 so that the wearer can view feedback for immediate strabismus treatment through the screen of the display unit 130 .

The wearable device 100 according to an embodiment of the present invention may be a device in the form of glasses to which augmented reality/virtual reality technologies are grafted, including a data storage device, a battery, and analysis software itself.

The data analyzer 200 may receive data from the wearable device 100, and includes a severity determiner 210 and a strabismus inducing environment analyzer 220.

The severity determination unit 210 may calculate a vergence angle (refer to FIGS. 3 and 4) based on the gaze position of each eye measured by the eye tracker 110, and through this, strabismus expression data can be obtained.

The strabismus expression data includes information on the time of occurrence of strabismus, the frequency of occurrence of strabismus, and the duration of occurrence of strabismus.

More specifically, the severity determination unit 210 analyzes the gaze positions of the right eye and the left eye obtained by the eye tracker 110, respectively, and the gaze difference angle (blind opening angle) of the section in which the gaze is mismatched, the occurrence time (strabismus expression) time point), frequency, duration, etc. can be obtained.

Here, the severity determination unit 210 may include a storage unit for storing a change record of the maximum divergence angle for a predetermined time among the divergence angles measured by the eye tracker 110 .

According to an embodiment of the present invention, the severity determination unit 210 determines the severity based on the number of occurrences of intermittent strabismus and the duration of occurrence of strabismus during a wearing time when the wearer wears the wearable device 100 .

The strabismus-inducing environment analysis unit 220 analyzes the strabismus-inducing environment through image analysis of the gaze environment at the time of occurrence of strabismus.

Examples of strabismus inducing environments include distant gaze, near work, outdoor activities, indoor activities, social interactions, and time of day.

First, the strabismus inducing environment analysis unit 220 may extract a gaze image (an image obtained by recording an environment the wearer is viewing) corresponding to occurrence of strabismus from an image obtained by the camera module 120 .

The strabismus-inducing environment analysis unit 220 may further include a machine learning unit, and analyze which of the strabismus-inducing environment classifications the extracted image corresponds to through deep learning.

The classification of the strabismus-inducing environment includes gaze distance when strabismus is induced (e.g., distant gaze, near work), strabismus-induced place (e.g., outdoors, indoors), and strabismus-induced behavior (e.g., study/reading, work activity/housework). , social interaction, exercise-related activities, viewing mass media, personal hygiene, eating, etc.), and/or time to induce strabismus.

That is, the machine learning unit can automatically classify the strabismus-inducing environment through artificial intelligence through various classifications.

Therefore, the strabismus-inducing environment analysis unit 220 can automatically classify the activity of the wearer through artificial intelligence to grasp the activity pattern at the time of strabismus, accurately and quickly determine the strabismus-inducing environment, and based on this, appropriate You can provide feedback.

The feedback output unit 300 may output a treatment plan and/or action instructions optimized according to the degree of severity and the strabismus-inducing environment on the display unit 130 as feedback.

The feedback output unit 300 may include a feedback database (DB) 310 .

As mentioned above, the wearable device 100 may be a device in the form of glasses to which augmented reality/virtual reality technology is grafted, and more specifically, an augmented reality module may be provided in the display unit 130 .

Accordingly, the feedback output unit 300 visually outputs feedback to the display unit 130, so that instructions for treatment can be effectively delivered to the wearer.

Through this, it is possible to induce the patient to recognize that strabismus has been expressed and to control it.

In one embodiment of the present invention, the display unit 130 may periodically measure the viewing angle by inducing strabismus while looking at a predetermined distance based on augmented reality.

That is, strabismus may be intentionally induced, and the strabismus state may be analyzed so that feedback may be provided to the patient.

As an example of feedback, the wearer may have the wearer measure the strabismus angle for a certain period of time through alternating occlusion of both eyes after setting the long-distance (6 meters) and near-distance (33 cm) fixation targets to check the diurnal variation of the strabismus angle.

Through this, it is possible to accurately monitor changes in the angle of strabismus when developing strabismus.

That is, in case of using the wearable device 100 such as AR glasses, when occurrence of strabismus is detected, the feedback output unit 300 may display feedback for controlling strabismus on the screen of the display unit 130. .

When the wearer wears the AR glasses according to an embodiment of the present invention, the occurrence of strabismus is continuously monitored, and when strabismus develops, immediate feedback is provided so that the wearer can pay attention to overcoming strabismus.

Here, the feedback is feedback for minimizing occurrence of strabismus, and may be action instructions previously stored in the feedback DB 310 .

In another embodiment, the feedback may serve as a means for causing strabismus to accurately measure the angle of strabismus.

More specifically, in order to accurately measure the viewing angle, the feedback output unit 300 looks at a small target located at a distance of 4 to 6 meters after the wearer performs calibration on the eye tracker 110. In this state, the wearer may be instructed to cover each eye alternately for about 3 seconds, repeating about 5 times.

In this case, it is possible to measure the angle of view at a distance through the difference in the gaze direction of each eye.

Here, the eyes are covered by the upper side of the wearable device 100 in the form of glasses, so that the eye tracker 110 can continuously track the pupil position of the eyes that are covered.

Then, in the same way, the feedback output unit 300 induces the wearer to gaze at a small target at a distance of about 33 cm, and alternately covers both eyes so that the eye tracker 110 can measure the near angle of deviation. .

In one embodiment of the present invention, the feedback output unit 300 may further include a remote medical center 320, and the feedback may include feedback that may be provided from a separate component such as the remote medical center 320. can

In one embodiment of the present invention, the remote medical center 320 is a component capable of communicating with the data analysis unit 200 and the feedback output unit 300, enabling doctors and patients to perform medical activities in a far away place, It is preferable to be configured so that the patient's condition can be grasped by means of communication and appropriate medical treatment can be performed.

Also, the feedback output unit 300 may receive feedback from the remote medical center 320 and display the feedback on the display unit 130 .

According to the present invention, not only such a system but also a method for monitoring strabismus expression in daily life according to another embodiment of the present invention can be provided.

The method may include obtaining data from the wearable device 100; Analyzing data by the data analysis unit 200; and outputting feedback from the feedback output unit 300 based on the analyzed data.

In the step of obtaining data from the wearable device 100, gaze position data and images of the right and left eyes of the wearer of the wearable device 100 are obtained from the eye tracker 110 and the camera module 120 of the wearable device 100, respectively. Acquire data.

Next, the data analysis unit 200 analyzes gaze position data to obtain strabismus expression data, and the data analysis unit 200 analyzes the image data to determine the strabismus inducing environment.

Finally, the feedback output unit 300 determines feedback based on the strabismus expression data and the strabismus inducing environment and outputs the feedback to the wearable device 100 .

Here, the data analysis unit 200 may analyze the image data through artificial intelligence and automatically classify the activity of the wearer to determine the strabismus-inducing environment.

Next, the change of the eye opening angle will be described in more detail with reference to FIGS. 3 and 4 .

3 and 4 are graphs showing changes in the eye opening angle measured using the eye tracker of the wearable device according to time (minutes) according to an embodiment of the present invention.

3 and 4, the horizontal gaze angle of each eye is obtained using the vectors x, z among the gaze position information vectors x, y, and z provided by the eye tracker 110, and the difference between the binocular values The vergence angle can be derived using

If the divergence angle is negative, it means near work, and if the divergence angle is positive, it means divergence, so it can be identified as the occurrence of intermittent exotropia.

3 is a graph of 5 minutes of data obtained from an actual patient, and a graph of an eye opening angle through data obtained from a patient with intermittent exotropia of 40 prismatic diopters.

A positive value indicates the occurrence of intermittent exotropia. For 5 minutes, the patient moved around the room and focused mainly on distance.

Intermittent exotropia occurred for a total of 2.98 minutes in 5 minutes, and the number (frequency) of intermittent exotropia was 29 times, and the longest duration (duration) was about 40 seconds.

Figure 4 is a graph showing the 5-minute data obtained during the actual patient eating, the eye opening each graph obtained during the meal of the intermittent exotropia patient.

A negative divergence angle means that the eyes are converged, and most of the values show a large negative value because the patient is looking at a close distance while eating.

It can be seen that the occurrence frequency and time of strabismus were significantly reduced compared to distance gaze.

As shown in FIGS. 3 and 4, the degree of strabismus manifestation, such as the total time of occurrence of strabismus, the frequency of occurrence, and the average duration of occurrence of strabismus, can be monitored through the divergence angle, and the severity of intermittent exotropia can be determined.

In addition, it is also possible to provide feedback to the patient by specifying the behavior at the time of occurrence of strabismus through an image captured together by the camera module 120 .

On the other hand, the embodiments of the present invention disclosed in the present specification and drawings are only presented as specific examples to easily explain the technical content of the present invention and help understanding of the present invention, and are not intended to limit the scope of the present invention. It is obvious to those skilled in the art that other modified examples based on the technical idea of the present invention can be implemented in addition to the embodiments disclosed herein.

100: wearable device
110: eye tracker
120: camera module
130: display unit
200: data analysis unit
210: severity determination unit
220: strabismus inducing environment analysis unit
300: feedback output unit
310: Feedback DB
320: telemedicine center

Claims (9)

A wearable device including an eye tracker and a camera module;
a data analyzer configured to receive and analyze data from the wearable device; and
A feedback output unit configured to determine and output feedback based on a result output from the data analysis unit; includes;
The data analysis unit obtains strabismus expression data based on the gaze positions of the right and left eyes of the wearer obtained by the eye tracker, and the strabismus expression data includes strabismus expression time point, strabismus expression frequency, and strabismus expression duration time information,
The data analysis unit analyzes the image captured by the camera module through artificial intelligence to automatically classify the activity of the wearer of the wearable device to determine a strabismus-inducing environment,
Strabismus expression monitoring system in daily life. The method of claim 1,
The data analysis unit determines the severity based on the strabismus expression data,
Strabismus expression monitoring system in daily life. delete The method of claim 1,
The wearable device further includes a display unit,
The feedback output unit outputs the feedback to the display unit,
Strabismus expression monitoring system in daily life. The method of claim 4,
Further comprising a remote medical center capable of communicating with the data analysis unit and the feedback output unit;
The feedback output unit receives feedback from the remote medical center and displays the feedback on the display unit.
Strabismus expression monitoring system in daily life. The method of claim 1,
The wearable device is a device in the form of glasses further including an augmented reality module,
Strabismus expression monitoring system in daily life. The method of claim 1,
The gaze positions of the right and left eyes of the wearer are measured at regular intervals by the eye tracker, and the strabismus expression data includes daily variation data of the maximum angle of deviation.
Strabismus expression monitoring system in daily life. acquiring gaze position data and image data of a right eye and a left eye of a wearer of the wearable device, respectively, from an eye tracker and a camera module of the wearable device;
obtaining strabismus expression data by analyzing the gaze position data in a data analyzer;
analyzing the image data through artificial intelligence in the data analyzer and automatically classifying the activity of the wearer to determine a strabismus-inducing environment; and
Including, determining feedback based on the strabismus expression data and the strabismus inducing environment in a feedback output unit and outputting the feedback to the wearable device,
A method for monitoring the occurrence of strabismus in daily life. delete