KR20200109614A - Training and exercise system based on virtual reality or augmented reality for dizziness and disequilibrium - Google Patents

Abstract

A virtual reality or augmented reality-based training and exercise system for ameliorating dizziness and sense of balance, according to the present invention, is a device worn on the head of a user, and comprises: a gaze tracking device which tracks a user′s gaze, and which displays a virtual reality or augmented reality screen; a body tracking device which is equipped with a movable footrest where the user can walk and which is configured to detect the user′s foot pressure through the movable footrest, detect a movement of the user in a space, and collect somesthetic sensibility information of the user; and a content providing server which provides virtual reality content or augmented reality content to the gaze tracking device so that the virtual reality content or augmented reality content is displayed, and which is configured to collect gaze stimulation information and auditory stimulation information from the gaze tracking device, collect pressure information, movement information, and somesthetic sensibility information from the body tracking device, measure, evaluate, and analyze user′s dizziness and sense of balance by linking the collected plurality of pieces of stimulation information about sensory organs, and provide content designed to enable the user to perform exercise and training programs according to the analyzed user′s symptoms and balance sensory functions, to the gaze tracking device. According to the present invention, the virtual reality or augmented reality-based training and exercise system for ameliorating dizziness and sense of balance can prescribe and provide customized exercise and training programs optimized to a patient′s current condition.

Description

{Training and exercise system based on virtual reality or augmented reality for dizziness and disequilibrium}

The present invention relates to an exercise and training system for improving dizziness and a sense of balance, and more particularly, based on virtual reality or augmented reality, due to a sense of balance and imbalance of a person with dizziness. It relates to an exercise and training system for improving gait disorders.

The function to balance the body is an essential function for a person, such as breathing or beating the heart, and having problems with this function causes great inconvenience in life.

In the inner ear, there is a sensory organ that senses the horizontal, vertical, and rotational movements of the head and informs the brain. This is called the vestibular organ. The vestibular organ is located in the hard bones of both ears and is composed of microscopic membranes and fluids, and the movement of the head detected by the vestibular organ enters the brain through the vestibular nerve, and regulates the body so that the body does not tilt to either side by appropriate control of the cerebellum. .

If there is a problem with one ear, the cerebellum has a function that adjusts the body to balance it again, and this function is called jeonsangsang. This delicate nerve control is easily affected by congenital anomalies of the vestibular organs-inflammation-vascular ischemia-tumors, as well as ear bone fractures or microscopic head trauma, resulting in large and small dizziness. The dizziness generated in this way is not only unpleasant, but can lead to postural anxiety, gait problems, and falls.

Although treatment is different for each disease that causes dizziness, it is a basic treatment principle to ensure that all information interactions work properly. For this purpose, vestibular rehabilitation exercises are currently prescribed in clinical practice, and the exercises should be performed for all patients with acute and chronic dizziness and imbalance.

Dizziness is a very common symptom and can lead to direct and indirect loss of medical bills by leading to falls due to imbalance. (Based on Sweden, 5 million euros per year for direct medical expenses for falls per 100,000 people) As a result of receiving data from the National Nutrition and Health Survey for 10,000 people in 2010-2012, 24% of those over 40 and 1/3 of those over 65 are within a year. I had a dizzying experience, and most of them complained of severe disparity that caused them to fall.

Dizziness is a subjective symptom, so objective measurement is difficult. Depending on the person, the type of symptoms (blank, round, unbalanced, etc.) or severity (very, medium, slightly) is subjectively expressed, so a problem occurs in a certain sensory nervous system or in the central nervous system that integrates. It's hard to estimate about whether or not. Some dizziness due to sudden brain lesions (cerebral infarction or cerebral hemorrhage) can reveal structural defects through general magnetic resonance imaging (MRI), but many other sensory nervous system and brain abnormalities are " It is impossible to access pathophysiology without taking a functional approach.” Measurement of “function” can be summarized as the process of applying “constant and controllable stimuli” to the human body in various ways and then confirming the “response”. After all, there is a need for a device that can deliver an ideal stimulus to the sensory nervous system that is closest to the daily environment and can control its properties, and the process of confirming the response is important. The sensory stimulation device used is very primitive, and the researcher cannot arbitrarily control the nature and quantity of the stimulation, so it is difficult to directly infer the response of the corresponding central nervous system. Dynamic test equipment, rotating chair test, gaze movement stimulus) is a level that can measure the response to the rotation stimulus of a one-dimensional axis The conventional gaze movement stimulus is two-dimensional, holding a cylinder close and turning it, or rotating a rotating chair in the field. It is possible only by the test method (an expensive test that can give rotational stimulation based on the x-axis at a fixed frequency).

Due to the development of IT technology, the head mount display applying virtual reality is a device (Visual Simulation System) that can provide gaze movement stimulation that medical staff can arbitrarily control in the exhibition field in a safe and controlled environment. A recent foreign study reported that simple visual stimulation and rehabilitation in dizziness patients is effective in improving symptoms. These are only a few of the virtual reality technologies that can be applied, and they are a new tool that can be used to induce brain activity and measure functions by providing visual stimulation consisting of various combinations of angle, intensity, color, and speed. In addition, it is easy to wear on the human body, and there are no restrictions on the experimental place, so it is very useful when resources are insufficient. In addition to the visual stimulation, the 4D stimulation device (Somatosensory & Warking Simulation System) that stimulates somatosensory sensation can be said to be the most suitable for measuring the response of nervous system functions. In addition, we believe that this study is free from concerns about ethical issues arising from the use of experimental animals and inconsistencies from human application through the safe application of ideal stimuli to the human body, while giving the most natural stimulation to the human brain function. It is easy to access basic science.

The process in which signals from the visual, vestibular and somatosensory systems are integrated may differ depending on the individual's brain function. As a very simple example, while riding a roller coaster, severe excitement of the vestibular system and signals from the somatosensory system come in for a short period of time, and the degree and time of feeling imbalance and dizziness immediately after getting off the coast are severely individual. This is evidence that can be confirmed in daily life that the process of integration of the brain can be different even in a normal human body. Regardless of the disease, the phenotype of each individual may be different, so a process of approaching diagnosis and treatment appropriately for the quantitative abnormality seen in each individual is necessary (customized approach). Therefore, research that adjusts the degree of artificial stimulus and examines the relationship with the quantitative value of abnormal sensations is essential not to match a person to the diagnosis name, but to apply it to a customized diagnosis, treatment, and prediction of the prognosis tailored to the reduced function of the individual. .

Korean Patent Registration 10-1828679

The present invention was conceived to solve the above problems, and using virtual reality or augmented reality technology, virtual reality for evaluating functional decline and improving symptoms according to the pathological characteristics of each individual with imbalance and dizziness symptoms Or, the purpose is to provide an exercise and training system for improving dizziness and balance based on augmented reality.

The object of the present invention is not limited to the above-mentioned object, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

The exercise and training system for improving dizziness and balance based on virtual reality or augmented reality of the present invention for achieving such an object is a device worn on the user's head, and tracks the user's gaze, and a virtual reality or augmented reality screen The displayed gaze tracking device and a moving footrest for the user to walk are provided, and the footrest pressure of the user is sensed through the moving footrest, the user's motion in space is detected, and the user's somatosensory information is collected. Provides virtual reality content or augmented reality content to the body tracking device and the gaze tracking device for display, collects gaze stimulation information and auditory stimulation information from the gaze tracking device, and pressure information and motion information from the body tracking device And somatosensory information is collected, and stimulus information of a number of collected sensory organs is linked, through which the user's vestibular function is measured, evaluated, and analyzed, and the vestibular rehabilitation exercise program is performed according to the analyzed vestibular function of the user. And a content providing server that provides the desired content to the gaze tracking device.

The gaze tracking device has a gaze tracking unit for tracking a user's gaze and a structure worn on the user's head, and includes an HMD unit for displaying a virtual reality or augmented reality screen and outputting sound. have.

The body tracking device is a treadmill structure that allows a user to walk in all directions, and the body tracking device is a movable footrest that can be rotated in a horizontal direction, and an inclination can be adjusted so as to incline up, down, front, back, left and right, and the movement A pressure sensor for detecting the user's foot pressure on the footrest, a motion sensor for detecting the user's motion in space, a departure prevention unit provided around the user to prevent the user from leaving the moving footrest, and the HMD It may include an input unit for inputting a control command on the screen displayed on the unit, a stop button for stopping driving, and a driving unit for driving the moving footrest.

When the stop button is touched, the content providing server measures gaze tracking data, pressure data, and movement data up to that point, and provides content for performing exercise and training programs for improving dizziness and balance. I can.

According to the present invention, the patient's symptoms, head movement and gait are monitored while changing the integrated stimulus to the multi-sensory organs of natural visual, auditory, and somatosensory similar to daily life, and through this, customized exercise optimized for the patient's current state. There is an effect that can be provided by prescribing and training programs.

In addition, according to the present invention, the patient can be immersed in a safe and controlled virtual environment as if playing a game and repeat customized exercise and training, and accordingly, continuous training is possible, and such training records are saved as real-time data. It has the effect that efficient and systematic management is possible.

1 is a diagram illustrating a configuration of an exercise and training system for improving dizziness and a sense of balance based on virtual reality or augmented reality according to an embodiment of the present invention.
2 is a block diagram showing the configuration of a gaze tracking device according to an embodiment of the present invention.
3 shows an example of using an exercise and training system for improving dizziness and a sense of balance according to an embodiment of the present invention.
4 is a diagram showing the configuration of a body tracking device according to an embodiment of the present invention.
5 is a view for explaining a four-dimensional sense of balance control device.
6 is a view for explaining a process of applying the four-dimensional sense of balance control device of FIG. 5 to a dizziness patient.
7 to 20 illustrate contents of an exercise and training system for improving dizziness and a sense of balance according to an embodiment of the present invention.
21 is an example of a monitor screen showing patient-related information according to an embodiment of the present invention.
22 is a block diagram illustrating a content configuration according to an embodiment of the present invention.
23 to 43 illustrate a content UI according to an embodiment of the present invention.

In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

In addition, in the description with reference to the accompanying drawings, the same reference numerals are assigned to the same components regardless of the reference numerals, and redundant descriptions thereof will be omitted. In describing the present invention, when it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted.

1 is a diagram illustrating a configuration of an exercise and training system for improving dizziness and a sense of balance based on virtual reality or augmented reality according to an embodiment of the present invention.

Referring to FIG. 1, an exercise and training system for improving dizziness and a sense of balance according to the present invention includes a gaze tracking device 100, a body tracking device 200, and a content providing server 300.

The gaze tracking device 100 is a device worn on the user's head, tracks the user's gaze, and displays a virtual reality or augmented reality screen.

The body tracking device 200 is provided with a moving footrest that allows the user to walk, senses the pressure of the user's soles through the moving footrest, detects the user's motion in space, and collects the user's somatosensory information.

The content providing server 300 provides virtual reality content or augmented reality content to the gaze tracking device 100 to be displayed. In addition, gaze stimulation information and auditory stimulation information are collected from the gaze tracking device 100, and pressure information, motion information, and somatosensory information are collected from the body tracking device 200. In addition, an exercise and training program for improvement in accordance with the analyzed symptoms and functions of the user by linking the collected stimulus information of a number of sensory organs, measuring, evaluating and analyzing the degree of dizziness and balance-sensing function of the user. Contents for performing the process are provided to the gaze tracking device 100.

2 is a block diagram showing the configuration of a gaze tracking device according to an embodiment of the present invention.

Referring to FIG. 2, the gaze tracking device 100 includes an gaze tracking unit 110 and a head mounted display (HMD) unit 120.

The gaze tracking unit 110 serves to track a user's gaze. For example, the eye tracking unit 110 may be implemented with Tobii Eye Tracking Retrofit hardware, specifically TobiiEyeChip ASIC processing, 10 IR illuminators for each eye, and one Eye Tracking sensor for each eye. have.

The HMD unit 120 has a structure that is worn on the user's head, causes a virtual reality or augmented reality screen to be displayed, and outputs sound.

3 shows an example of using an exercise and training system for improving dizziness and a sense of balance according to an embodiment of the present invention.

Referring to FIG. 3, it is possible to check the shape of the body tracking device 200 in which the gaze tracking device 100 is worn on the user's head and the moving footrest is provided.

4 is a diagram showing the configuration of a body tracking device according to an embodiment of the present invention.

Referring to FIG. 4, the body tracking device 200 includes a moving footrest 210, an input unit 220, a stop button 230, a motion sensor 240, a departure prevention unit 250, a driving unit 260, and a pressure sensor. It includes 270.

The movable footrest 210 is a footrest provided to allow the user to walk, has a treadmill structure that allows the user to walk in all directions, can rotate in a horizontal direction, and can adjust the inclination so that it is inclined vertically, back and forth. .

The input unit 220 is provided for a user to input a control command on a screen displayed on the HMD unit 120 and may be implemented in the form of a joystick.

The stop button 230 is a button provided to stop driving. The stop button 230 serves as a kind of emergency stop function. For example, if the user feels dizziness while driving, the stop button 230 may be pressed to immediately stop driving. In an embodiment of the present invention, the stop button 230 may be implemented as a magnetic button.

The motion sensor 240 serves to detect a user's motion in space. In an embodiment of the present invention, the motion sensor 240 may be implemented as a gyro sensor, a three-axis (X, Y, Z) acceleration sensor, a three-axis (X, Y, Z) geomagnetic sensor.

The departure prevention unit 250 is a type of safety bar provided around the user in order to prevent the user from being separated from the moving footrest 210.

The driving unit 260 serves to drive the moving footrest 210. In an embodiment of the present invention, the driving unit 260 may be implemented as a 6-axis motion controller.

The pressure sensor 270 serves to detect the pressure of the user's soles on the moving footrest 210.

In the present invention, when the stop button 230 is touched, the content providing server 300 measures and analyzes/evaluates gaze tracking data, pressure data, and motion data up to that point, and performs exercise and training programs based on this. Content can be provided.

The present invention can control various gaze movement stimuli and somatosensory stimuli by using virtual reality or augmented reality technology, and by measuring the reaction of the human body using this, it is possible to find out the mechanism by which vision, vestibular sensation, and somatosensory sensation are integrated within the central nervous system. And reveal the compensatory mechanisms corresponding to the pathological imbalance.

In this regard, the present invention proposes a four-dimensional sense of balance control device capable of controlling gaze motor stimulation and somatosensory stimulation, a measurement method thereof, and a training program for improving dizziness or balance using the same.

In order to stand in an upright balance, many sensory systems of the body play various roles in real time. Vision, vestibular system, and somatosensory system accurately deliver sensations, and the central nervous system Only by integrating in multiple ways can you maintain balance and maintain posture. Even if there are errors in only some of the processes, it can be difficult to balance and feel dizzy. In order to measure these pathological conditions, it is necessary to fundamentally understand the central nervous system integration process, and a process of quantitatively confirming the reaction of the brain while artificially presenting various stimuli to each sensory nervous system. Currently, it is possible to apply sensory stimulation to the human body in various harmless forms using IT technologies such as virtual reality and augmented reality.

The HMD using virtual reality is a device that can provide gaze movement stimuli that can be controlled arbitrarily by medical staff in a safe and controlled environment. Through this, it provides visual stimulus consisting of various combinations of angle, intensity, color, and speed. It can provide, and can be used for induction of brain activity and measurement of function.

The process in which signals from the visual, vestibular and somatosensory systems are integrated may differ depending on the individual's brain function. As a very simple example, while riding a roller coaster, severe excitement of the vestibular system and signals from the somatosensory system come in for a short period of time, and the degree and time of feeling imbalance and dizziness immediately after getting off the coast are severely individual. This is evidence that can be confirmed in daily life that the process of integration of the brain can be different even in a normal human body. As such, each individual's phenotype may be different for any disease, so a process of approaching diagnosis and treatment appropriate for the quantitative abnormality seen in each individual is essential. Therefore, adjusting the degree of artificial stimulus and examining the quantitative values and associations of abnormal sensations is essential not to match the person to the diagnosis name, but to apply it to diagnosis, treatment, prognosis, and prediction tailored to the reduced function of the individual.

5 is a view for explaining a four-dimensional sense of balance control device.

In the embodiment of FIG. 5, the four-dimensional sense of balance control device is designed to control a program of visual stimulation with a virtual reality or augmented reality-based head mount display (HMD), and the somatosensory input is used to reduce the confusion of the somatosensory system. It alters the nature and movement of the pad that causes it. And, the shape of the pad is a hard/soft type, and the movement of the pad is a sinusoidal movement in the x and y axes. And, the stimulus corresponding to the output is controlled by various control factors. ① The nature of the stimulus (line, point, color, size, number), ② the distance between the subject and the stimulus (near and far target), and the rotational axis ( 3D rotation around the x, y, z axis), ③ angular velocity (angular velocity around each axis) and frequency.

6 is a view for explaining a process of applying the four-dimensional sense of balance control device of FIG. 5 to a dizziness patient.

Referring to FIG. 6, an algorithm of a four-dimensional sense of balance control device developed for dizziness patients of various disease groups can be applied, and a response of a pathological state can be confirmed and a mechanism can be identified through subjective/objective measurement.

Dizziness disease group (e.g., A. unilateral vestibular disease, B. bilateral vestibular disease, C. chronic subjective dizziness, D. central nervous system disease, etc.) Data on abnormalities in the vestibular system are collected through the vestibular root oil power plant. At this time, the collected data includes the abnormal nystagmus/eye movement gain value, waveform/abnormal rapid movement number, direction, and frequency.

And, by applying the algorithm of the developed 4D sense of balance control device, the subjective imbalance of the 4D stimulus is measured through the questionnaire diagnosis method.

In addition, the relationship between the proposed control factors (property, distance, axis, speed, etc.) of the various stimuli and the collected vestibular abnormality data analysis values is revealed, and the difference with the normal group is confirmed through functional MRI. In addition, the relationship between the damaged vestibular function and the anatomical location is revealed, and the pathological mechanisms for sensory integration abnormalities are identified. And, through feedback, errors in the algorithm are corrected and improved.

Through the experiment of the present invention, a device capable of arbitrarily controlling various stimuli of the nervous system, particularly visual stimuli that can be artificially controlled, can be developed. And, by using this, a sensory integration mechanism of the central nervous system is established that enables the body to balance in response to various external stimuli. As a result, basic scientific results can be achieved that can confirm the function of the human body's discreet central nervous system in response to various stimuli.

In addition, development can be promoted with applied engineering. Based on the integration mechanism of the central nervous system of the normal group, the relationship with abnormal functions of the vestibular nervous system of various disease groups is revealed, and it can be applied to clinical diagnosis and treatment. And, according to the nature of the algorithm that can control various stimuli, it is possible to provide a personalized approach, so it can help in the ultimate treatment of symptoms. The development of a device that gives stimulation expected in the present invention can be used as a medical device by adding a function that can be measured later.

And, through the present invention, socioeconomic ripple effects can be expected. IT technology is developing beyond imagination, but it is true that its application is slow in conservative medical reality. On the other hand, the possibility of responding to diseases and revealing unknown neurological mechanisms by linking the medical and IT industries is endless, and as the hardware industry is developing as well as programs and algorithms, access to mobile devices is required in the future. It is expected to be possible. Based on this, home care and real-time treatment of diseases can be made possible.

The present invention proposes a immersive experience-type virtual/augmented reality content production and implementation technology of medical, virtual reality, augmented reality, and gamification convergence.

In order to increase the effectiveness, commercialization, and feasibility of exercise and training systems for improving dizziness and balance based on virtual or augmented reality, computerized measurement that can be applied according to the patient's condition by fusion of self-experience type ICT technology such as body and sensory measurement An integrated system that can efficiently operate and manage the data generated through the implementation of core element technologies such as evaluation models, software, and contents reflecting processes that can apply various treatment methods based on virtual reality must be established.

In addition, it is necessary to develop highly immersive, high-impact experience-type rehabilitation training contents to which virtual reality or augmented reality technology and gamification based on vestibular rehabilitation medical expertise are applied.

That is, in the present invention, the requirements for the exercise and training system contents for improving the sense of balance and the virtual reality or augmented reality based dizziness derived through patient interviews with professional medical staff and verified treatment protocols can be applied, and functions can be improved. Suggest the content that you have.

In the present invention, by combining gamification based on a treatment protocol, we propose an exercise and training scenario that improves dizziness and balance in which the patient can train with a goal consciousness and high concentration, and creates an attractive, narrative, and vivid virtual world. Plan immersive content to engage patients. Then, we develop and propose interactive storytelling scenarios and storyboards.

In the present invention, a virtual reality or augmented reality-based exercise and training system content design for improving dizziness and balance is designed to provide a fun and immersive user experience, and a content optimized for a user-friendly virtual space interface is designed. Designing exercise and training content that combines physical, visual, and cognitive challenges, and designing game-based interaction designs that increase compliance with exercise and training and increase voluntary participation by combining immersive virtual environment and game technology. Through training guide, result scoring, status display, etc., we implement an intuitive UI that makes it easier and easier to accept the exercise and training process, and allows the user to directly check his or her condition and continuously modify the movement during training. Design interactions that provide inducing visual and auditory feedback. In addition, the difficulty level is adjusted step by step based on medical values so that symptoms can be relieved and cured.

The contents of the present invention proposed according to this planning intention are as follows.

7 to 20 illustrate dizziness and balance improvement training content according to an embodiment of the present invention.

7 to 18 illustrate visual dizziness and balance-sensing improvement training content for real-time intensity control, and FIGS. 19 and 20 illustrate physical balance dizziness and balance-improving training content.

FIG. 7 is a content for vertical/horizontal eye gaze, and FIG. 8 is a content for vertical/horizontal optokinetic nystagmus.

9 is a content for horizontal vestibulo-ocular reflex, and FIG. 10 is a content for vertical vestibulo-ocular reflex.

In the contents of FIGS. 7 to 10, movement of the eyeball or head in a sitting position is performed, a visual effect is added to increase the attention of the object to be observed, and the difference between the object to be observed and the direction of the eyeball (eye tracking) is It is displayed in real time so that it can be checked visually. For example, the content time can be set to 2 minutes, and the difficulty level of the content can be divided into 3 levels.

11 is an example of content for horizontal eyeball gaze exercise, ① displaying a sports car horizontally across the screen, ② displaying the direction of the eyeball (eye tracking) in the form of a camera finder, and ③ allowing the sports car and the camera finder to match. Visual exercise is carried out, and when it is coincident and when it is not, it supports real-time judgment with visual and auditory effects, and ④ the time when the sports car is in the circle (the direction of the object and the eyeballs coincide) compared to the total time. It is measured and converted into a score. ⑤ After training, the final score is displayed on the screen.

Through these contents, the weakening of the function of fixing the eyeball at a certain position is improved, overcoming visual vertigo through desensitization through repetitive visual stimulation, and minimizing confrontation by other visual stimulation and somatosensory through recovery of vestibular function. , It can prevent maladaptation of the central nervous system and prevent dizziness from becoming chronic.

12 and 13 are contents for upper limb exercise, in which the arm and upper body are moved to perform the exercise. In this content, a motion such as moving a hand or grabbing an object can be performed using the HMD controller, and the screen is configured so that the action to be taken can be clearly determined only with visual information. For example, the content time can be set to 2 minutes, and the content difficulty level can be divided into 3 stages.

14 is an example of a concrete implementation content of the upper limb movement, in which a mart environment in which a product showcase is arranged is created in VR. In this content, a random product shines and induces user interaction, and if the product is caught and placed in the cart, it is a success, and if not, it is determined as a failure, and this is converted into a score. Then, the final score is displayed on the screen after training ends.

Through the contents for upper limb movement, symptoms of not only people with central vestibular disorders but also peripheral vestibular disorders can be improved, and somatic sensations, vestibular sensations, and visual functions can be treated so that they can be smoothly integrated in the brain.

15 illustrates content for a standing exercise, checks whether a correct standing posture is steadily maintained, measures whether the posture is correct, and provides visual and auditory feedback in real time to enable voluntary correction.

In the content of FIG. 15, a virtual space consisting of a wall and a floor is displayed, and a guide display is provided on the screen to correct a posture by measuring a posture in real time. Posture stability and mobility can be improved through these contents.

16 illustrates content for an object tracking exercise, and FIG. 17 illustrates physiological considerations of an object tracking exercise.

In the contents of FIG. 16, movement of the eyeball or head in a standing position is performed, visual effects are added to increase the attention of the object to be observed, and the difference between the object to be observed and the direction of the eyeball (eye tracking) can be visually checked. To be displayed in real time.

FIG. 18 is an illustration of content for head fixation horizontal/vertical eyeball smooth follow-up exercise, displaying a butterfly horizontally/vertically crossing the screen, displaying the direction of the eyeball (eye tracking) as a collection network, and collecting the butterfly Visual movement is performed so that the network is consistent, and when it is coincident and when it is not, it supports real-time judgment with visual and auditory effects. In addition, the time when the butterfly is in the collection network (the direction of the object and the eyeballs coincide) is measured and converted into a score, and the final score is displayed on the screen after training is completed. This can contribute to improving the reduced sense of balance and preventing chronic symptoms.

19 is an illustration of content for body balance exercise, displaying a virtual space consisting of a wall and a floor, displaying a separate guide such as a footstep on the floor so that movement can be performed according to the walking speed and direction, and posture in real time Measure and display a guide on the screen to correct if the posture is wrong. Stability and efficiency can be increased by stimulating the substitution of such contents.

20 is a table tennis game content, where NPC serves and a ball flies in an arbitrary direction. If the user hits the ball with a racket, it flies in the direction of the NPC, otherwise, the NPC gets points due to failure, and the player gets points if the NPC makes a mistake and cannot hit the ball. The player who gets 10 points first wins, and the final score is displayed on the screen after the game is over. Through these game contents, visual stimulation, vestibular reflex stimulation that occurs while moving to hit the ball, hand movement, and posture maintenance can be implemented to maximize dizziness and balance improvement training effect.

According to the evaluation model established in this way, treatment methods and treatment procedures can be established, and implementation schedule and results for each training process can be performed so that individual patients can be repeatedly trained to verify the effectiveness of dizziness and balance improvement training. A manager environment that can track variability can be implemented. 21 is an example of a monitor screen showing patient-related information according to an embodiment of the present invention.

22 is a block diagram illustrating a content configuration according to an embodiment of the present invention.

Referring to FIG. 22, in the present invention, an exercise or a game to improve dizziness and a sense of balance may be performed in the order of selecting content, selecting a background, selecting a training method, and setting a difficulty level. An example of a content UI based on such content configuration is as follows.

23 to 43 illustrate a content user interface (UI) according to an embodiment of the present invention.

23 is a UI for selecting a background when dizziness training content is selected. In the embodiment of FIG. 23, a home house and a seabed may be selected as backgrounds. The home house is a living room with no separate movement, and the seabed is a background screen in which fish move.

24 illustrates a training content target concept and illustrates a robot concept in the form of a monitor. In the embodiment of FIG. 24, the concept of a robot in the form of a monitor floats in the air, sprinkles particles 243 along a moving trajectory, and displays instructions or information. In addition, by changing the color 241 of the eye part, it is possible to indicate whether or not the decision is made. For example, it is basically displayed in yellow, if successful, it turns green, and if it fails, it turns red. In addition, the monitor screen 242 counts 1 for every success starting from 10 times, and when the mission is completed, “training success” is displayed as a text, and when the training is completed, the firecracker effect is exploded and the animation of dancing left and right is executed (244).

FIG. 25 is an illustration of game content, a concept of shooting a randomly flying UFO within a limited time with space in the background, and the speed of the UFO increases as the difficulty increases.

26 is a UI for selecting content during dizziness training and shooting games. If'dizziness training' is selected here, as shown in FIG. 27, a UI for selecting a background for training among'living room' and'undersea' is provided.

28 and 29 are UIs for selecting the type of training, which can be selected from'target fixation' and'target movement', and'head left and right turn','head up and down turn','head left and right tilt' You can choose from.

FIG. 30 is a training guide pop-up UI, and shows a demonstration motion continuously looped by animation according to the type of exercise.

31 is a game guide pop-up UI.

FIG. 32 is a training demonstration animation UI, which expresses training demonstrations such as <head turning left and right> and <head tilting left and right> through animation.

FIG. 33 is a pop-up UI for training completion, and results such as score and training time are displayed, and FIG. 34 is a pop-up UI for game completion, and results such as score and training time are displayed.

FIG. 35 is a top view, showing the movement of the target in the target movement mode when the head is rotated 30 degrees left or right. At this time, when the user reaches the red point and the blue point, each of them is presented as successful. However, when the point is reached, the speed is slowed down, or when it returns to the one point without reaching another point, it produces a failure.

FIG. 36 is a side view, showing the movement of the target in the target movement mode when the head moves up and down by 30 degrees. At this time, when the user reaches the red point and the blue point, each of them is presented as successful. However, when the point is reached, the speed is slowed down, or when it returns to the one point without reaching another point, it produces a failure.

FIG. 37 shows a tilting of the head by 20 degrees to the left and right, and is an equation for determining success once if a red point and a blue point are taken within a limited time for each difficulty level.

38 illustrates a random 30-degree movement, and if red points and blue points are taken in order within a limited time for each difficulty level, it is determined that they have succeeded once, and then returns to the yellow point to start the next movement. At this time, it occurs randomly among the vertical, left and right perspective movements, and the perspective movement is assumed to be achieved once when the character movement is completed without a separate determination.

39 is a concept of a space content background, in which objects of various sizes are displayed, move in various trajectories, and move at a constant speed, and each object is produced as a background and particles.

40 is a screen UI for selecting a training difficulty in the case of space training, and a moving direction and speed of a star may be selected.

41 is a training guide pop-up UI when the moving direction of a star is set to the Y axis and the speed is set to normal, and FIG. 42 is a pop-up UI for completing space training.

43 shows that the target moves based on the X-axis, Y-axis, and Z-axis in space training, and the target may randomly move in the forward or reverse direction.

The present invention has been described above using several preferred embodiments, but these embodiments are illustrative and not limiting. Those of ordinary skill in the art to which the present invention pertains will understand that various changes and modifications can be made without departing from the spirit of the present invention and the scope of the rights presented in the appended claims.

100 eye tracking device 200 body tracking device
300 Content Providing Server 110 Eye Tracking Department
120 HMD part 210 Mobile footrest
220 Input 230 Stop button
240 Motion sensor 250 Separation prevention part
260 drive unit 270 pressure sensor

Claims (4)

A device worn on a user's head, comprising: a gaze tracking device that tracks a user's gaze and displays a virtual reality or augmented reality screen;
A body tracking device provided with a movable footrest allowing a user to walk, and sensing a user's foot pressure through the movable footrest, detecting a movement of the user in space, and collecting somatosensory information of the user; And
Provides virtual reality content or augmented reality content to the gaze tracking device to be displayed, collects gaze stimulation information and auditory stimulation information from the gaze tracking device, and receives pressure information, motion information, and somatosensory information from the body tracking device. It collects and links the collected stimulus information of multiple sensory organs, measures, evaluates, and analyzes the user's dizziness and balance through this, and performs exercise and training programs according to the analyzed user's symptoms and sense of balance. A training system for improving dizziness and a sense of balance, including a content providing server that provides content to the gaze tracking device.
The method according to claim 1,
The eye tracking device,
A gaze tracking unit for tracking a user's gaze; And
A training system for improving dizziness and balance, characterized in that the structure is worn on the user's head, and includes an HMD unit for displaying a virtual reality or augmented reality screen and outputting sound.
The method according to claim 2,
The body tracking device,
As a footrest provided to allow a user to walk, a treadmill structure that allows a user to walk in all directions, a movable footrest that can be rotated in a horizontal direction, and an inclination adjustable to incline in the vertical, front, rear, left and right directions;
A pressure sensor for sensing a user's foot pressure on the moving footrest;
A motion sensor for detecting a motion of a user in space;
A departure prevention unit provided around the user to prevent the user from leaving the moving footrest;
An input unit for a user to input a control command on a screen displayed on the HMD unit;
A stop button to stop driving; And
A training system for improving dizziness and a sense of balance, characterized in that comprising a driving unit for driving the moving footrest.
The method of claim 3,
When the stop button is touched, the content providing server measures and analyzes/evaluates gaze tracking data, pressure data and movement data up to that point, and provides content to perform dizziness and balance improvement training based on this. Dizziness and balance improvement training system, characterized in that.

Images