KR102275379B1 - Device, method and program for visual perception training by brain connectivity - Google Patents

Abstract

The present invention relates to a brain connectivity-based visual perception training apparatus, method and program.
The brain connectivity-based visual perception training method according to an embodiment of the present invention comprises: providing, by a computer, a spatial motion image to a head-mounted display device (S200; providing a spatial motion image); providing, by the computer, a visual perception training image simultaneously with the spatial movement image or subsequent to the spatial movement image (S400; providing a visual perception training image); and receiving, by the computer, the result of performing the first task of the visual perception training image from the user (S1000). Hereinafter, detailed description of each step is described.

Description

Brain connectivity-based visual perception training device, method and program {DEVICE, METHOD AND PROGRAM FOR VISUAL PERCEPTION TRAINING BY BRAIN CONNECTIVITY}

The present invention relates to an apparatus, method, and program for visual perception training based on brain connectivity, and more particularly, to an apparatus, method and program for improving visual perception function that is lowered by brain function deterioration or damage.

In general, stroke is a single disease and ranks first for disability in Korea. Among the brain function impairments due to the stroke, visual field impairment is a major disability that occurs in a high proportion of 20-30% of all patients. This visual field impairment is generally accepted as a permanent defect, and the reality is that there is no attempt to rehabilitate the visual field disorder itself. Patients with brain damage to the visual cortex following a stroke characteristically show hemianopia on the opposite side of the lesion . For example, a patient who has had a stroke in the left visual cortex has a visual field impairment on the right side. In this case, the patient cannot recognize objects or movements in the right visual field when both eyes are open and looking straight ahead.

However, some patients with visual field impairment due to a stroke occurring in the visual cortex may notice a movement or shape in an invisible field of view. While answering, it is possible to match the letters more than the chance level with probability, and even show an appropriate response to a visual stimulus. These patients do not perceive stimuli visually, but the brain notices movements or shapes. This phenomenon, discovered by Lawrence Weiskrantz in the 1970s, is called blindsight, which refers to the phenomenon of noticing even though it cannot be seen. The existence of the above-mentioned blindness phenomenon suggests that visual information received through the retina can be processed through other places than the visual cortex.

Accordingly, there is a need for a visual perception training method and apparatus for improving a visual perception function by performing visual perception learning (training) through repetitive provision of visual perception stimulation.

An object of the present invention is to provide a visual perception training apparatus, method, and program based on brain connectivity that provides a visual perception training image and increases the visual perception function improvement effect by training the hippocampus having connectivity with the visual cortex.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In the brain connectivity-based visual perception training method according to an embodiment of the present invention, a computer provides a spatial motion image to a head-mounted display device, wherein the spatial motion image is an image in which a user moves a path in a virtual space, Spatial moving image providing step; providing, by a computer, a visual perception training image simultaneously with the spatial movement image or subsequent to the spatial movement image; and receiving, by the computer, a result of performing the first task of the visual perception training image from the user.

In addition, in another embodiment, in the step of providing the visual perception training image, when the visual perception training image is provided following the spatial movement image, the user in the virtual space moves a plurality of existing in the virtual space through path movement. When entering any one of the spaces, the computer generates and provides the visual perception training image based on the conditions given to each space.

In addition, in another embodiment, the computer requesting a second task to enter the same space as the previous trial to the user; and if the user succeeds in the second task, calculating, by the computer, the result of performing the second task in the result of performing the first task to calculate a final result.

In addition, in another embodiment, in the step of providing the visual perception training image, when the visual perception training image is provided at the same time as the spatial movement image, the visual perception training image is generated on the spatial movement image at the unit time interval. It is characterized in that it is provided by combining the frames of.

Also, in another embodiment, the first task is displayed on the user's visual perception training area in the visual perception training image, and is to be performed when the user is walking, and the result of performing the first task is, It is whether the user performs a movement corresponding to the first task.

In addition, in another embodiment, the providing of the visual perception training image comprises combining the first image frame included in the spatial movement image and the second image frame included in the visual perception training image to generate a final output image. It is characterized in that, the first image frame is changed by the user's head or pupil movement, and the second image frame is fixed without being changed by the user's head or pupil movement.

Also, according to another embodiment, the spatial motion image is generated based on data sensed by the user's actual walking by the gait sensing device.

In addition, in another embodiment, the visual perception training image is to display a task image in the user's visual perception training area in the visual perception training image, and the computer sounds in a direction corresponding to the position where the task image is displayed. outputting; further includes.

In addition, in another embodiment, the computer acquiring the user's gaze direction; and determining a display position of the task image of the visual perception training image in a training area within a viewing range based on the gaze direction.

The brain connectivity-based visual perception training providing program according to another embodiment of the present invention is combined with a computer, which is hardware, to execute the above-mentioned brain connectivity-based visual perception training method, and is stored in the medium.

Brain connectivity-based brain connectivity-based visual perception training computing device according to another embodiment of the present invention, a communication unit for providing a spatial motion image to a head-mounted display device; A control unit for providing a visual perception training image simultaneously with the spatial movement image or following the spatial movement image, and receiving the result of performing the first task of the visual perception training image from the user to calculate the final result of visual perception training; includes; However, the space movement image is an image in which a user moves a path in a virtual space.

In another embodiment, the spatial motion image is provided as the user moves a path in the virtual space, and the spatial motion image is provided as the user enters any one of a plurality of divided spaces existing in the virtual space. Subsequently, when the visual perception training image is provided, the control unit generates the visual perception training image according to the visual perception training image providing condition given to each divided space entered by the user, but the visual perception training image providing condition is the level of reward provided when visual perception training images are provided in each segmented space or when visual perception training images are performed in each segmented space.

Also, in another embodiment, the control unit requests the user for a second task to enter the same space as the previous trial, and when the user succeeds in the second task, the second task execution result is displayed as the first task execution result It is characterized in that the final result is calculated by reflecting it.

In addition, in another embodiment, the control unit simultaneously provides the spatial movement image and the visual perception training image for displaying the first task in the visual perception training area according to the path movement, and during the movement of the path, the first It is characterized in that the first task performance result is calculated by receiving the user's movement corresponding to the test.

In another embodiment, the controller generates the spatial motion image based on data sensed by the user's actual gait by the gait sensing device.

A head-mounted display device according to another embodiment of the present invention provides a spatial motion image on the screen of the head-mounted display device, and provides a visual perception training image including a first task at the same time or at the same time as the spatial motion image. It is provided following the spatial movement image, and the final result of visual perception training is calculated based on the result of performing the first task on the visual perception training image, or the result of the first task is transmitted to a computing device that calculates the final result. , The first task performance result is generated by determining whether a user operation received from a user matches the first task, and the user operation is input through a controller connected to the head mounted display device, and the space is A video is an image of a user moving a path in a virtual space.

The brain connectivity-based visual perception training apparatus according to another embodiment of the present invention provides a spatial motion image that provides a user's path movement in a virtual space, and displays the visual perception training image simultaneously with the spatial motion image or The spatial movement image is then provided, and a result of performing the first task of the visual perception training image is received from the user.

According to the present invention as described above, by training the visual cortex and the hippocampus having brain connectivity together, it is possible to provide a higher visual perception function improvement effect than when performing visual perception training for the visual cortex.

In addition, since spatial movement and visual perception training can be provided sequentially or simultaneously, it is possible to provide various training forms for training the visual cortex and the hippocampus together.

In addition, by providing the second task of finding the same space, the spatial shape in which the spatial movement is performed is memorized, thereby enhancing the effect of stimulating the hippocampus.

In addition, in the process of performing spatial movement, the first task corresponding to the visual perceptual stimulus can be provided as an instruction to perform a specific motion, so that training can be provided in the form of interesting content. That is, it is possible to provide fun to the user during training while increasing the visual perception function improvement effect (ie, treatment effect) compared to the existing one.

1 is a flowchart of a visual perception training method based on brain connectivity according to an embodiment of the present invention.
2 is an exemplary diagram of a radial maze that is a virtual space for generating a spatial motion image according to an embodiment of the present invention.
3(a) is an exemplary view of a field of view seen by a normal person.
3(b) and 3(c) are exemplary views of the visual field range seen by a patient with visual perception impairment due to brain damage.
4 is an exemplary view of a visual perception training image provided in a specific space after performing spatial movement according to an embodiment of the present invention.
5 is a flowchart of a brain connectivity-based visual perception training method, further including a second task execution process of entering the same space as the previous trial according to an embodiment of the present invention.
6 is a flowchart of a brain connectivity-based visual perception training method, further comprising the process of providing a sound output in the task image output direction according to an embodiment of the present invention.
7 is a flowchart of a brain connectivity-based visual perception training method, further comprising the process of adjusting an output position of a task image according to a gaze direction according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments published below, but can be implemented in various different forms, and only these embodiments make the publication of the present invention complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly defined in particular.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components.

In the present specification, the computer includes all of various devices capable of providing a result to a user by performing arithmetic processing. For example, computers include desktop PCs, notebooks (Note Books) as well as smart phones, tablet PCs, cellular phones, PCS phones (Personal Communication Service phones), synchronous/asynchronous A mobile terminal of International Mobile Telecommunication-2000 (IMT-2000), a Palm Personal Computer (PC), a Personal Digital Assistant (PDA), and the like may also be applicable. Also, when a head mounted display (HMD) device includes a computing function, the HMD device may be a computer. In addition, the computer may correspond to a server that receives a request from a client and performs information processing.

In the present specification, the 'visual perception training region' means an area in which visual perception function should be improved by visual perception training within the field of view. For example, in the case of a patient with visual field impairment due to brain injury, the 'visual perception training region' may be a specific region that is not visible within the visual field due to brain injury.

In this specification, a 'spatial movement image' means an image expressing movement of a user in a virtual space.

In the present specification, a 'visual perception training image' refers to an image in which a task is presented in the visual perception training area in order to perform visual perception training.

In this specification, the 'first task' means a task provided in the visual perception training image 200 .

As used herein, the 'second task' is a task provided to a user through a spatial motion image, and refers to a task for evaluating the user's spatial memory. Referring to the drawings, brain connectivity according to embodiments of the present invention Based on the visual perception training apparatus, method and program will be described in detail.

1 is a flowchart of a visual perception training method based on brain connectivity according to an embodiment of the present invention.

Referring to Figure 1, the brain connectivity-based visual perception training method according to an embodiment of the present invention, a computer providing a spatial motion image 100 to a head mounted display device (S200; providing a spatial motion image step) ; providing, by a computer, the visual perception training image 200 simultaneously with the spatial movement image 100 or subsequent to the spatial movement image 100 (S400; providing a visual perception training image); and receiving, by the computer, the result of performing the first task 210 of the visual perception training image 200 from the user (S1000). Hereinafter, detailed description of each step is described.

The visual cortex is a region located in the occipital lobe directly involved in visual information processing within the cerebral cortex. The sensory area of the cerebrum cortex receives and processes signals from the body senses such as touch and pain, and each sensory organ such as sight, hearing, taste, and smell, so that it can be recognized and perceived. If there is a problem in one of these pathways, perception of that sensation may become impossible. Specifically, the process in which a person recognizes a sensory stimulus (for example, a visual stimulus) is a process in which a sensory stimulus (for example, a visual stimulus) is input to a sensory organ (for example, the eye) to perform the Sensation. After that, the sensory area of the cerebrum (eg, the visual cortex) perceives, and the sensory representation (eg, the visual representation) is associated with the memory representation to derive meaning is performed. At this time, if the visual cortex is damaged when the visual stimulus is received by the sensory organ, there is no abnormality in the sensory process of the sensory organ, but the visual stimulus cannot be perceived and the object cannot be seen.

The hippocampus (Hippocampus) is a part of the cerebral cortex that protrudes from the bottom surface of the lateral subventricle of the cerebrum. It is an invisible area in close neural communication with the hypothalamus. The hippocampus is a structure that plays an important role in long-term memory and spatial navigation. Among patients who cannot see objects due to damage to the visual cortex, there are patients in which the hippocampus compensates the damaged visual cortex to recognize objects. That is, the hippocampus (Hippocampus) has connectivity through close neural communication with the cerebral visual cortex.

Therefore, by stimulating the visual cortex and the hippocampus together, it is possible to improve the damaged or decreased visual perception ability. Specifically, in the human brain, brain regions that perform egocentric processing of space-time (e.g., primary visual region, parietal lobe, frontal lobe) and allocentric space-time (Allocentric: space-time) The hippocampus, which processes the environment itself), has brain connectivity. In other words, the hippocampus, which is responsible for spatiotemporal memory, affects visual perception. Accordingly, as navigation training for stimulating the hippocampus in charge of spatiotemporal memory is provided together with visual perception training providing visual stimulation, the effect of visual perception training can be improved.

The computer provides the spatial moving image 100 on the head mounted display screen (S200; spatial moving image providing step). The space movement image 100 is an image in which a user moves a path in a virtual space. The space movement image 100 may be an image that is changed as the user moves through the virtual space regardless of a user's manipulation. That is, the spatial movement image 100 may provide only visual movement of space without accompanying movement of the user. Through this, the computer can stimulate the hippocampus of the user's brain by providing a visual spatial change.

In another embodiment, the space moving image 100 may be an image moved in a virtual space by a user's movement. For example, the spatial motion image 100 is generated based on data sensed by the user's actual gait by the gait sensing device. The gait sensing device may be a virtual reality (VR) treadmill device that allows a user to walk. Through this, the computer can stimulate the hippocampus through movement in the virtual space that is the same as recognizing the space-time environment while moving in the real space by allowing the user to actually walk.

The spatial motion image 100 may be implemented in various types of spaces. As an embodiment, the spatial motion image 100 may be implemented in a form in which a user moves in a virtual space in the form of a radial maze, as shown in FIG. 2 . That is, in the virtual space in the form of a radial maze, there is a central region where the user starts moving, and a plurality of paths that the user can move extend from the central region. In a virtual space in the form of a radial maze, an identification mark (for example, a specific picture frame placed on the wall of the virtual space, a vase erected at a specific location in the virtual space, etc.) can be placed in the space for the user to distinguish each path. have. That is, the computer allows the user to distinguish a specific path of the radial maze based on the identification mark. In addition, as another embodiment, the spatial movement image 100 may be an image in which the user moves in the form of a general maze. In this case, the structure of the general maze may be changed after a specific training cycle is completed. For example, since the user must perform training to remember the maze structure while the user performs one training session, the general maze structure is maintained, and the general maze structure can be changed when a new training starts after the first training session ends. .

The spatial motion image 100 may be implemented in a form in which frames that have already been generated and stored are provided according to the lapse of time or a user's manipulation. In addition, the spatial motion image may be implemented as a real-time image calculated based on the user's location and the user's gaze direction in a specific virtual space.

The computer provides the visual perception training image 200 simultaneously with the spatial movement image 100 or subsequent to the spatial movement image 100 (S400; providing the visual perception training image 200). The visual perception training image 200 is an image for providing a task for training (ie, the first task 210 ) to an area requiring visual perception training (ie, a visual perception training area) within the field of view.

When the user is a patient with a visual perception disorder due to a brain abnormality, as shown in FIG. 3 , an abnormality occurs in the visual cortex, so that a partial region of the visual field may not be visible. The computer provides a task to the visual perception training area that is not visible within the visual field through the visual perception training image 200 and allows the corresponding task to be matched to treat the visual perception disorder.

In addition, if the user is a non-visual perceptual impairment, the visual perceptual function may fall within the normal range, but the visual perceptual function may not be fast enough to respond to a visual stimulus. In this case, the visual perception function may be improved by performing visual perception training.

The first task 210 is to provide a problem that the user needs to fit into a specific area within the screen (ie, the visual perception training image 200). The first task 210 may be provided on the screen in various forms. In one embodiment, the computer presents the first task 210 with the problem of distinguishing different figures or patterns. For example, the computer may provide the problem of distinguishing horizontal stripes or vertical stripes in the visual perception training area as the first task 210 .

In addition, in the first task 210 , in another embodiment, when the visual perception training image 200 is provided together with the spatial movement image 100 , the task to be performed within the spatial movement image 100 is may be applicable. For example, when the spatial movement image 100 of the virtual space is changed through the user's actual walking movement, the computer instructs the visual perception training area in the visual perception training image 200 to change direction (eg, a specific direction). to the first task 210 ) may be provided to the first task 210 ). The computer may determine whether the first task 210 is recognized through whether the user performs a movement corresponding to the first task 210 . For example, an arrow that guides direction change in the visual perception training area (for example, an area invisible due to damage to the visual cortex or an area in which visual perception ability is deteriorated) in the process of a user walking in a virtual space In the case of displaying guidance information indicating the distance remaining until the direction change is performed, the computer performs the first task when the user performs a route change in a direction different from the guided direction or performs a route change at a location different from the guide point. may be considered not to be recognized.

In addition, in another embodiment, in the step of providing the visual perception training image 200 ( S400 ), when the visual perception training image 200 is provided following the spatial movement image 100 , the user in the virtual space Upon entering any one of a plurality of spaces existing in the virtual space through path movement, the computer generates and provides the visual perception training image 200 based on the conditions assigned to each space. The spatial motion image 100 may include a plurality of divided spaces, and as the user remembers the structure of the virtual space, the hippocampus is moved along the path in the spatial motion image 100 to enter a specific space. stimulate For example, when a user moves a path on a radial maze, the computer may provide a visual perception training image in a space (ie, a divided space) at the end of each path.

In this case, each of the divided spaces may be provided with a condition for providing the visual perception training image 200 , respectively. The conditions for providing the visual perception training image 200 may correspond to whether the visual perception training image is provided in a divided space within the virtual space that the user moves, the level of rewards provided when the visual perception training image is provided, and the like. For example, the computer may provide a visual perception training image to only a part of a plurality of divided spaces. When a reward is provided as a result of performing visual perception training, the computer sets so that a reward cannot be obtained when the user enters a partitioned space that does not provide a visual perception training image. Also, for example, the computer may provide a visual perception training image for each divided space, but may set different rewards obtained as the visual perception training is performed.

For example, when performing spatial movement on a radial maze in a virtual space, the computer provides a reward only at the end of a path that meets a specific condition (that is, a condition set for each training) among a plurality of radial paths. visual perception training is performed, and through this, the user searches for and remembers a path that meets the conditions.

In addition, for example, if the route search is repeatedly performed, the user searches the route corresponding to the reward provision condition first, and then starts the route search again in the center of the radial maze or performs the first test. The hippocampus may be stimulated as the movement is performed by memorizing the same or different paths corresponding to the reward provision condition in the trial to find another partition space from the provided location). That is, when the user repeatedly performs the second task of searching for a route, the computer provides a higher reward if the same search for a route on which a reward is provided, or prevents a reward from being provided on a route that has already been moved Thus, it is possible to induce the user's spatial memory.

As the computer provides the visual perception training image 200 according to the provision condition within the space entered by the user, it may provide the user with a training process for stimulating the visual cortex following the training process for stimulating the hippocampus. , in another embodiment, in the step of providing the visual perception training image 200 ( S400 ), when the visual perception training image 200 is provided at the same time as the spatial movement image 100 , the spatial movement image 100 ) by combining the frames of the visual perception training image 200 generated at the unit time interval on the image. For example, the computer generates a final output image by combining the first image frame included in the spatial motion image 100 and the second image frame included in the visual perception training image 200 .

In this case, as an embodiment, the first image frame is changed by the user's head or pupil movement, and the second image frame is fixed without being changed by the user's head or pupil movement. . Even if the user moves his/her head or gazes in the process of moving in the virtual space, the damaged visual cortex region can be trained only when the first task 210 is displayed on the visual perception training region. When the position of the first task 210 is also changed as the user moves the head or the pupil, the user can check the first task 210 as the first task 210 is displayed in a normal area within the field of view. be able to Therefore, in order to fix the first task 210 to the visual perception training area, the computer sets the second image frame included in the visual perception training image 200 to be fixed regardless of the head movement or the eye movement.

The computer receives the result of performing the first task 210 of the visual perception training image 200 from the user (S1000). When the computer provides the visual perception training image 200 after providing the spatial movement image 100 through the head mounted display screen, the computer provides the visual perception training image 200 provided in the space entered by the user. It is determined whether a correct answer is input for the first task 210 . That is, the first task 210 is a problem (eg, a problem of selecting a figure displayed in the visual perception training area from among several figures) provided at a specific location on a two-dimensional or three-dimensional space, as in FIG. 4 . The result of performing the first task 210 is whether the input value for the problem displayed in the visual perception training area is the correct answer.

When the spatial movement image 100 and the visual perception training image 200 are provided together, the first task 210 is displayed in the visual perception training area in the visual perception training image 200, and is performed when the user walks. The result of performing the first task 210 is whether the walking user performs a movement corresponding to the first task 210 . That is, when the spatial movement image 100 and the visual perception training image 200 are simultaneously provided, the computer may allow the user to input an answer to a problem in the visual perception training image 200 .

In one embodiment, the computer receives an input through a controller that holds the correct answer to the first task 210 displayed on one side while the user moves in the virtual space corresponding to the spatial motion image 100 in both hands. In the case of selecting a figure included in the first task 210 among two figures, the computer matches the first figure to the left-hand controller and the second figure to the right-hand controller to determine the correct answer according to the type of controller the user operates can be input.

In addition, in another embodiment, when the first task 210 is an operation or movement to be performed while moving in a virtual space corresponding to the spatial motion image 100 , the computer displays the first task 210 on the screen. It is determined whether the first task 210 is visually recognized based on whether the user performs an operation or movement according to the first task 210 after being provided on the screen. For example, when the operation or movement is the user's gait direction change, the computer determines whether the user has performed the direction change or not, and recognizes the visual recognition of the first task 210 . Also, for example, the computer determines the user's movement type through a vision sensor or a motion sensor that recognizes the user's movement, and determines whether the operation corresponds to the first task 210 .

The computer calculates a visual perception training performance result based on the performance result of the first task 210, and identifies the user's visual perception function state.

In addition, as another embodiment, as in FIG. 5 , when providing the visual perception training image 200 following the spatial movement image 100 , the computer performs the second task of entering the same space as the previous trial. requesting the user (S500); and when the user succeeds in the second task, the computer reflects the result of the second task to the result of the first task 210 to calculate a final result (S600). In addition to performing spatial movement, the computer can stimulate the hippocampus, which is associated with the visual perception function, while the computer is responsible for spatial memory by memorizing the path traveled through space to the user and then re-performing it. That is, the second task is a task to memorize and move the same path as the previous trial in the virtual space within the spatial moving image 100 .

The computer requests the user for the second task to re-enter the same space as the previous trial (S500). For example, the computer may request to find the same space after changing the location where the space movement starts in the same virtual space having a plurality of partition spaces. That is, the computer can stimulate the hippocampus by making it remember the spatial structure identified while moving in the virtual space in the previous implementation process, and it can improve the visual perception function by stimulating the hippocampus through brain connectivity.

Thereafter, when the user succeeds in the second task, the computer reflects the result of the second task in the result of the first task 210 to calculate the final result ( S600 ). That is, if the second task of finding the same space is successful, the hippocampus is stimulated through performing spatial memory, so the computer determines whether the second task succeeds or not with the result of performing the first task 210 for performing visual perception training in a specific space. It is reflected together to calculate the final result of visual perception training.

Further, in another embodiment, as shown in FIG. 6 , the computer outputting sound in a direction corresponding to the position where the first task 210 (ie, task image) is displayed (S700) further includes; . A patient whose visual field is not partially visible due to brain damage may not be able to visually recognize the task image 210 in the visual perception training image 200, so it can guide the direction in which the visual stimulation is provided through other senses. have. The computer provides sound in a direction corresponding to the position where the task image 210 is displayed through three-dimensional sound output, and provides a visual perception function through an attempt to recognize the task image 210 in the field of view in the direction in which the sound is provided. This can be improved.

In addition, in another embodiment, as shown in Figure 7, the computer obtaining the user's gaze direction (S800); and determining a display position of the task image 210 of the visual perception training image 200 in a training area within a viewing range based on the gaze direction (S900). It is necessary to prevent the first task 210 (that is, the task image) from being placed in the normal area instead of being placed in the visual perception training area (eg, the visual perception impaired area) due to the user's gaze movement. To this end, the user's gaze direction is measured in real time, and the position of the first task 210 is adjusted in the visual perception training area based on the gaze direction (ie, the viewing center of the viewing direction). The gaze direction may be determined by a head movement and an eyeball movement. The computer may recognize the user's head movement through a motion sensor provided in the head mounted display device. In addition, the computer may detect the user's eye movement through an eye-tracker provided in the head-mounted display device.

Specifically, the computer acquires the gaze direction (ie, the central direction of the viewing range viewed by the user) through the motion sensor or the eye tracker in the head mounted display device (S800). Thereafter, the computer determines the visual perception training area based on the gaze direction and adjusts the display position of the first task 210 (ie, the task image 210 ) in the virtual space ( S900 ). For example, if the upper left area with respect to the center of view is an area that is not visible due to brain damage, if the gaze direction is changed by the user's head or pupil movement, the computer recognizes the changed gaze direction and determines the gaze direction. After determining the user's viewing range as a reference, the first task 210 is displayed by adjusting the range corresponding to the upper left of the viewing range based on the viewing direction as the visual perception training area.

A head-mounted display device according to another embodiment of the present invention provides a spatial motion image on the screen of the head-mounted display device, and provides a visual perception training image including a first task at the same time or at the same time as the spatial motion image. It is provided following the spatial motion image, and the final result of visual perception training is calculated based on the result of performing the first task on the visual perception training image, or the result of the first task is transmitted to a computing device that calculates the final result. . The result of performing the first task is generated by determining whether a user operation received from the user matches the first task. The user operation is input through a controller connected to the head mounted display device. The controller may include a device gripped and used by the user, a gait sensing device for the user to sit and walk, and the like. The space movement image is an image in which a user moves a path in a virtual space.

The brain connectivity-based visual perception training computing device according to another embodiment of the present invention provides a spatial motion image 100 to a head mounted display device or a client device to which the head mounted display device is connected, and a visual perception training image ( 200) is provided simultaneously with the spatial movement image 100 or subsequent to the spatial movement image 100, and a result of performing the first task 210 of the visual perception training image 200 is received from the user. The space movement image 100 is an image in which a user moves a path in a virtual space.

Brain connectivity-based visual perception training computing device according to another embodiment of the present invention, a communication unit; and a control unit.

The control unit generates a space movement image 100 for a specific virtual space or extracts a pre-stored space movement image 100 in real time. The space movement image 100 is an image in which a user moves a path in a virtual space. In one embodiment, the spatial movement image 100 may be an image that is changed as the user moves in a virtual space regardless of a user's manipulation. That is, the spatial movement image 100 may provide only visual movement of space without accompanying movement of the user. Through this, the computer can stimulate the hippocampus of the user's brain by providing a visual spatial change. In another embodiment, the spatial movement image 100 may be an image that is moved in a virtual space by a user's movement. For example, the spatial motion image 100 is generated based on data sensed by the user's actual gait by the gait sensing device. The gait sensing device may be a virtual reality (VR) treadmill device that allows a user to walk. Through this, the computer can stimulate the hippocampus through movement in the virtual space that is the same as recognizing the space-time environment while moving in the real space by allowing the user to actually walk.

In addition, as another embodiment, the controller provides the spatial motion image 100 based on various types of spaces. For example, the controller may generate an image in which the user moves in the form of a radial maze or a general maze as the spatial movement image 100 .

Also, as another embodiment, when the general maze structure is applied to the virtual space, the controller may change the maze structure after a specific training period ends. For example, since the user must perform training to remember the maze structure while the user performs one training session, the general maze structure is maintained, and the general maze structure can be changed when a new training starts after the first training session ends. .

In addition, as another embodiment, the controller may implement the already generated and stored spatial movement image 100 frame in a form that provides it according to the lapse of time or a user's manipulation. Also, as another embodiment, the controller may implement the spatial motion image 100 as a real-time image calculated based on the user's location and the user's gaze direction in a specific virtual space.

In addition, the control unit serves to provide the visual perception training image 200 simultaneously with the spatial movement image 100 or subsequent to the spatial movement image 100 . The visual perception training image 200 is an image for providing a task for training (ie, the first task 210 ) to an area requiring visual perception training (ie, a visual perception training area) within the field of view. The first task 210 is to provide a problem that the user needs to fit into a specific area within the screen (ie, the visual perception training image 200).

The first task 210 may be provided on the screen in various forms. In one embodiment, the computer presents the first task 210 with the problem of distinguishing different figures or patterns. For example, the computer may provide the problem of distinguishing horizontal stripes or vertical stripes in the visual perception training area as the first task 210 .

In addition, in the first task 210 , in another embodiment, when the visual perception training image 200 is provided together with the spatial movement image 100 , the task to be performed within the spatial movement image 100 is may be applicable. For example, when the spatial movement image 100 of the virtual space is changed through the user's actual walking movement, the control unit instructs the visual perception training area in the visual perception training image 200 to change direction (eg, a specific direction). to the first task 210 ) may be provided to the first task 210 ).

In addition, in another embodiment, when the control unit provides the visual perception training image 200 following the spatial movement image 100 , the user moves in the virtual space through a plurality of paths existing in the virtual space. When entering any one of the spaces, the computer generates and provides the visual perception training image 200 based on the conditions given to each space. The spatial motion image 100 may include a plurality of divided spaces, and as the user remembers the structure of the virtual space, the hippocampus is moved along the path in the spatial motion image 100 to enter a specific space. stimulate For example, when a user moves a path on a radial maze, the computer may provide a visual perception training image in a space (ie, a divided space) at the end of each path. In this case, each of the divided spaces may be provided with a condition for providing a visual perception training image, respectively. A detailed description of the previously described conditions for providing visual perception training images will be omitted. By providing the visual perception training image 200 according to the provision condition in the space entered by the user, the controller may provide the user with a training process for stimulating the visual cortex following the training process for stimulating the hippocampus.

In another embodiment, the spatial motion image is provided as the user moves a path in the virtual space, and the spatial motion image is provided as the user enters any one of a plurality of divided spaces existing in the virtual space. Subsequently, when the visual perception training image is provided, the control unit generates the visual perception training image according to the visual perception training image provision condition given to each divided space entered by the user. The condition for providing the visual perception training image is whether the visual perception training image is provided in each divided space or the level of a reward provided when the visual perception training image is performed in each divided space.

Also, in another embodiment, the control unit requests the user for a second task to enter the same space as the previous trial, and when the user succeeds in the second task, the second task execution result is displayed as the first task execution result It is characterized in that the final result is calculated by reflecting it.

In addition, in another embodiment, when the controller provides the visual perception training image 200 at the same time as the spatial movement image 100 , a message generated on the spatial movement image 100 at the unit time interval It serves to combine and provide the frames of each training image 200 . For example, the control unit generates a final output image by combining the first image frame included in the spatial movement image 100 and the second image frame included in the visual perception training image 200 . In this case, as an embodiment, the first image frame is changed by the user's head or pupil movement, and the second image frame is fixed without being changed by the user's head or pupil movement. . Even if the user moves his/her head or gazes in the process of moving in the virtual space, the damaged visual cortex region can be trained only when the first task 210 is displayed on the visual perception training region.

In addition, the control unit receives the result of performing the first task 210 of the visual perception training image 200 from the user. The control unit receives a first test execution result from a controller connected to the head-mounted display apparatus or a client apparatus connected to the head-mounted display apparatus. The controller may be a device operated while the user holds it, or a device that performs a movement such as walking while the user is seated. For example, the controller generates the spatial motion image based on data sensed by the user's actual walking by the gait sensing device.

Specifically, the controller performs the first task input through the controller holding the correct answer to the first task 210 displayed on one side while the user moves in the virtual space corresponding to the spatial motion image 100 in both hands. results can be received. Also, the controller may receive an operation or movement to be performed while moving in the virtual space corresponding to the spatial motion image 100 as a result of performing the first task. That is, the control unit simultaneously provides the spatial movement image and the visual perception training image displaying the first task in the visual perception training area according to the path movement, and the user corresponding to the first test during the path movement. A result of performing the first task is calculated by receiving the movement. For example, when the visual perception training image is provided as an image for instructing the user to change the walking direction, the controller may receive an operation or movement of the user to change the direction in a specific direction at a specific time as a result of performing the first task. .

In addition, the control unit serves to calculate a result of performing the visual perception training based on the result of performing the first task. In addition, the control unit serves to determine the user's visual perception functional state based on the performance result of the visual perception training.

In one embodiment, when the first task 210 is an operation or movement to be performed while moving in a virtual space corresponding to the spatial motion image 100, the controller controls the first task 210 to be provided on the screen. Afterwards, it is determined whether the user has visually recognized the first task 210 based on whether the user performs an operation or movement according to the first task 210 . In addition, in another embodiment, when the visual perception training image 200 is provided after providing the spatial movement image 100 through the head mounted display screen, the control unit provides the visual perception training provided within the space entered by the user. It is determined whether a correct answer is input for the first task 210 of the image 200 .

The communication unit provides the spatial movement image 100 to a head mounted display device or a client device to which the head mounted display device is connected. The computing device is a server device, and when the head-mounted display device is connected to the client device to display the provided image, the communication unit transmits the spatial motion image 100 to the client device. When the computing device is a client device to which the head-mounted display device is connected, the computing device generates a spatial motion image 100 and provides it to the head-mounted display device. Brain connectivity-based according to another embodiment of the present invention The visual perception training apparatus provides a spatial motion image providing a path movement of the user in a virtual space, and provides a visual perception training image simultaneously with the spatial motion image or following the spatial motion image, and the visual perception from the user Receive the first task execution result of the training image.

In one embodiment, the visual perception training apparatus includes a control unit. The control unit serves to provide a spatial motion image providing a user's path movement in a virtual space, and to provide a visual perception training image simultaneously with the spatial motion image or following the spatial motion image.

The brain connectivity-based visual perception training method according to an embodiment of the present invention described above may be implemented as a program (or application) and stored in a medium in order to be executed in combination with a computer, which is hardware.

The above-described program is C, C++, JAVA, machine language, etc. that a processor (CPU) of the computer can read through a device interface of the computer in order for the computer to read the program and execute the methods implemented as a program It may include code (Code) coded in the computer language of Such code may include functional code related to a function defining functions necessary for executing the methods, etc., and includes an execution procedure related control code necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, the code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer should be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.

The storage medium is not a medium that stores data for a short moment, such as a register, a cache, a memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on various servers accessible by the computer or in various recording media on the computer of the user. In addition, the medium may be distributed in a computer system connected by a network, and a computer readable code may be stored in a distributed manner.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims (17)

A computer providing a spatial motion image to the head-mounted display device, wherein the spatial motion image is an image of a user moving a path in a virtual space, a spatial motion image providing step;
A visual perception training image providing step, in which the computer provides the visual perception training image simultaneously with the spatial movement image or following the spatial movement image; and
Receiving, by a computer, the result of performing the first task of the visual perception training image from the user;
The visual perception training image providing step generates and provides a final output image by combining the first image frame included in the spatial movement image and the second image frame included in the visual perception training image,
The first image frame is changed by the movement of the user's head or pupil,
The second image frame is fixed without being changed by the movement of the user's head or pupil, brain connectivity-based visual perception training method. According to claim 1,
The visual perception training image providing step,
When providing the visual perception training image following the spatial movement image,
When the user enters any one of the spaces in the virtual space through path movement, the visual perception training image is generated and provided based on the condition given to the space entered by the user, brain connectivity-based of visual perception training method. 3. The method of claim 2,
requesting a second task to enter the same space as the previous trial to the user; and
When the user succeeds in the second task, calculating the final result by reflecting the second task performance result in the first task performance result; further comprising, a brain connectivity-based visual perception training method. According to claim 1,
The visual perception training image providing step,
When providing the visual perception training image at the same time as the spatial movement image,
A brain connectivity-based visual perception training method that combines and provides frames of visual perception training images generated at unit time intervals on the spatial movement image. 5. The method of claim 4,
The first task is
As a task to be performed by the user while walking, displayed in the visual perception training area of the user in the visual perception training image,
The results of the first task are:
Whether the walking user performs a movement corresponding to the first task, a brain connectivity-based visual perception training method. According to claim 1,
The spatial motion image is
Brain connectivity-based visual perception training method that is generated based on data sensed by the user's actual walking by a gait sensing device. According to claim 1,
The visual perception training image is,
Displaying a task image in the visual perception training area of the user in the visual perception training image,
Outputting a sound in a direction corresponding to the position where the task image is displayed; further comprising, a brain connectivity-based visual perception training method. According to claim 1,
obtaining a user's gaze direction; and
Determining the display position of the task image of the visual perception training image in a training area within the viewing range based on the gaze direction; further comprising, brain connectivity-based visual perception training method. In combination with a computer that is hardware, any one of claims 1 to 8 stored in the medium to execute the method, brain connectivity-based visual perception training providing program. A communication unit that provides a spatial motion image to a client device connected to the head mounted display device or the head mounted display device - The spatial motion image is an image in which a user moves a path in a virtual space, and the user moves in a virtual space in the virtual space provided when performing the route movement;
A control unit for providing a visual perception training image simultaneously with the spatial movement image or following the spatial movement image, and receiving the result of performing the first task of the visual perception training image from the user to calculate the final result of visual perception training; includes; and,
The control unit is
When the visual perception training image is provided following the spatial movement image as the user enters any one of the spaces in the virtual space, the visual perception training image is provided according to the conditions for providing the visual perception training image to the space entered by the user. Create each training video,
The provision condition of the visual perception training image is whether a visual perception training image is provided in each space or the level of a reward provided when performing a visual perception training image in each space, a visual perception training computing device. delete delete delete delete delete delete delete

Images