KR101965393B1 - Visual perception training device, method and program for visual perception training using head mounted device - Google Patents

Abstract

The present invention relates to a computing device for providing visual perception training, a method and a program for providing visual perception training based on a head mounted display device.
In a method of providing visual perception training based on a head mounted display device according to an embodiment of the present invention, a computer provides a first object on a first screen of the head mounted display device, and a second screen of the head mounted display device. Providing a second object (S200); Receiving, by the computer, an operation of adjusting a positional relationship between the first object and the second object from the user (S400; user operation receiving step); Changing a position of the first screen and the second screen by the computer according to the manipulation (S600; changing the screen position); And outputting the visual perceptual content in the visual field range of the patient including the visual damage region based on the visual center direction (S800).

Description

VISUAL PERCEPTION TRAINING DEVICE, METHOD AND PROGRAM FOR VISUAL PERCEPTION TRAINING USING HEAD MOUNTED DEVICE}

The present invention relates to a computing device for providing visual perception training, a method and a program for providing visual perception training based on a head mounted display device.

In general, stroke is the number one cause of domestic disability as a single disease, and among the brain function damages caused by the stroke, visual disturbance is a major disorder appearing at a high rate of 20-30% of all patients. Such visual disturbances are generally accepted as permanent defects, and there are no attempts at rehabilitation of visual disturbances. . For example, a patient who has a stroke in the left visual cortex shows half-side vision disorder on the right side, where the patient does not recognize objects or movements in the right field of vision when both eyes are opened and looking straight ahead.

However, some patients with vision impairment due to stroke in the visual cortex may notice some movement or shape in the invisible field of vision. While answering, there are chances to match letters above the chance level and even respond appropriately to visual stimuli. These patients do not visually perceive the stimulus, but the brain notices movement or form. This phenomenon, discovered by Lawrence Weiskrantz in the 1970s, is called blindsight, which refers to a phenomenon that is not noticed. The presence of the blind vision suggests that visual information coming through the retina can be processed through a place other than the visual cortex.

Therefore, there is a need for a visual perceptual training method and apparatus for improving visual perceptual function by performing visual perceptual learning (training) through repetitive visual perception stimulation.

In the case of conventional visual perception training, the distance and height from the patient's eye position to the monitor are set, and the visual perception training screen is displayed on the monitor. It is done in such a way. Therefore, there is a problem that it is difficult for the patient to continue the training outside the hospital, and it is difficult to flexibly change the difficulty adaptively according to the change of the visual perception ability of the patient.

The present invention provides a computing device for providing visual perception training, wherein the patient initializes a visual perception training condition suitable for the patient by using a head mounted display device, calculates a training level suitable for the patient's condition, and provides visual perception training. The present invention provides a method and program for providing visual perception training based on a head mounted display device.

Problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In a method of providing visual perception training based on a head mounted display device according to an embodiment of the present invention, a computer provides a first object on a first screen of the head mounted display device, and a second screen of the head mounted display device. Providing a second object; A user operation receiving step of receiving, by a computer, a user operation for adjusting the positional relationship between the first object and the second object; Changing a position of the first screen and the second screen by the computer according to the operation; And outputting visual perception content to a field of view of a patient including a visual damage region based on the visual center direction, wherein the first screen is any one of a left eye screen and a right eye screen of the head mounted display device. The second screen is not the first screen among the left eye screen and the right eye screen of the head mounted display device.

In another embodiment, the user operation receiving step, characterized in that the user to match the position of the first object and the second object on the target point, at least one of the first object and the second object One is an exercise object.

In another embodiment, the method may further include storing, by the computer, output conditions of the first screen and the second screen for a specific user.

In another embodiment, the visual perception content outputting step is to provide any one of a plurality of images at least one point in a two-dimensional or three-dimensional space for each trial, and the computer inputs the user from each trial. Comparing the response to the provided image type.

In another embodiment, the visual perceptual content outputting step may include: convergence matching the first screen and the second screen to a center point; And displaying an object corresponding to visual perceptual content at a specific depth in a specific two-dimensional space based on the depth point of the center point.

In another embodiment, the method further includes calculating a corrected answer rate for each point on the two-dimensional or three-dimensional space to set the visual perception damage area of the user.

In another embodiment, the method may further include determining, by the computer, at least one of a task providing frequency, a task difficulty level, and an output image size of the task based on a periodically calculated visual perception damage area.

Further, in another embodiment, the user can detect the movement of the head or the pupil direction to detect the center of view and provide a task based on a relative position based on the center of view.

The head mounted display apparatus based visual perception training providing program according to another embodiment of the present invention is coupled to a computer which is hardware to execute the aforementioned method for providing visual perceptual training based on the demount display apparatus and is stored in a medium. .

A computing device providing visual perception training according to an embodiment of the present invention provides a first object on a first screen of a head mounted display device, and provides a second object on a second screen of the head mounted display device. Receiving a user operation for adjusting a positional relationship between the first object and the second object from a user, changing the positions of the first screen and the second screen according to the user operation, and based on a viewing center direction; The visual perception content is output to a field of view of a patient including a visual damage region, wherein the first screen is any one of a left eye screen and a right eye screen of the head mounted display device, and the second screen is the head mounted display. It is not the first screen among the left eye screen and the right eye screen of the device.

According to the present invention as described above, has the following various effects.

According to an embodiment of the present invention, a user may perform visual perception training using a visual perception training device regardless of a place.

In addition, according to an embodiment of the present invention, the user may automatically adjust to the first screen and the second screen position suitable for the user's eye state in the process of playing the initial game.

In addition, according to an embodiment of the present invention, unlike the conventional analog visual perception training method, there is an effect that can accurately and flexibly adjust the difficulty of visual perception training according to the visual impairment state of the patient.

1 is a flowchart of a method for providing visual perception training based on a head mounted display device according to an embodiment of the present invention.
2 is an exemplary view showing that the position of the first screen and the second screen is changed according to an embodiment of the present invention.
3 is a flowchart of a method for providing visual perception training based on a head mounted display device, which provides visual perception stimulation on a three-dimensional space according to an embodiment of the present invention.
4 is a flowchart of a method for providing visual perception training based on a head mounted display device further including a process of evaluating a user's response according to an embodiment of the present invention.
FIG. 5 is a flowchart illustrating a method of providing visual perception training based on a head mounted display device, further comprising storing a screen output condition for a specific user according to an embodiment of the present invention.
FIG. 6 is a flowchart illustrating a method of providing visual perception training based on a head mounted display device further including calculating a visual perception damage area according to an exemplary embodiment of the present invention.
7 is an exemplary diagram of a percentage distribution of correct answers according to an embodiment of the present invention.
8 is a flowchart of a method for providing visual perception training based on a head mounted display device, further comprising adjusting a difficulty level based on a visual perception damage area according to an embodiment of the present invention.
9 is a structural diagram of a computing device providing visual perceptual training according to an embodiment of the present invention.
FIG. 10 is an exemplary diagram for providing visual perception training images to a head mounted display device according to an embodiment of the present invention.
11 is an exemplary view comparing the visual field seen in normal people and patients with visual impairment due to brain injury.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms, and only the embodiments are intended to complete the disclosure of the present invention, and the general knowledge in the art to which the present invention pertains. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used in a sense that can be commonly understood by those skilled in the art. In addition, the terms defined in the commonly used dictionaries are not ideally or excessively interpreted unless they are specifically defined clearly.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, "comprises" and / or "comprising" does not exclude the presence or addition of one or more other components in addition to the mentioned components.

In the present specification, the 'computer' includes all the various devices capable of performing arithmetic processing. For example, a computer can be a desktop PC, a notebook, as well as a smartphone, a tablet PC, a cellular phone, a PCS phone (Personal Communication Service phone), synchronous / asynchronous The mobile terminal of the International Mobile Telecommunication-2000 (IMT-2000), a Palm Personal Computer (PC), a Personal Digital Assistant (PDA), and the like may also be applicable. The computer may also be a server computer that receives information from a client. Hereinafter, in the present specification, a computer may be represented as a terminal or a client.

In the present specification, the 'first screen' refers to a left eye screen or a right eye screen of the head mounted display device.

In the present specification, the 'second screen' means not the first screen among the left eye screen and the right eye screen of the head mounted display device.

In the present specification, 'visual perception training image' means an image provided on a screen for visual perception training. The visual perception training image includes an output image displayed at a specific location for visual perception stimulation.

In the present specification, the 'first problem' refers to a problem provided on a screen for providing visual perception cognitive training to a patient having visual perception impairment.

In the present specification, the 'first response' means a response to a first problem provided at a specific time.

In the present specification, the 'second problem' refers to a problem provided to fix a user's gaze direction to a screen center. For example, the second problem may be to display any one of a plurality of objects (for example, 또는 or ㅌ) that is distinguished in the center of the screen at a specific time in the center of the screen.

In the present specification, the 'second response' refers to a response received from the user with respect to the second problem.

In the present specification, the 'output image' refers to an image provided at a specific position on the screen by the first problem. For example, the 'output image' may be a horizontal or vertical striped image displayed in a specific quadrant on the screen.

In the present specification, the 'screen image' is a screen itself on which one or more output images are arranged according to the first problem. The screen image may be output to a general display device or to a head mounted display device. When a screen image is provided to the head mounted display device, the screen image may be generated as a left eye image and a right eye image, respectively.

In the present specification, the 'central field view part' means a central area within a screen provided to a user.

In the present specification, the 'near viewable part' refers to a remaining area of the screen except for the center viewable part.

In the present specification, a 'task display unit' refers to a region in which an output image which is a task of memory training is provided in a screen provided to a user. The 'desk display unit' may include only the peripheral field of view except for the central field of view, or may include both the central field of view and the surrounding field of view.

In the present specification, the 'head mounted display device' refers to a device that provides a visual perception training image to a user's eyes. The 'headmount display device' may be a device including a computing function or a device used in connection with a computer (eg, a PC or a smartphone).

1 is a flowchart of a method for providing visual perception training based on a head mounted display device according to an embodiment of the present invention.

Referring to FIG. 1, in a method of providing visual perception training based on a head mounted display device according to an embodiment of the present invention, a computer provides a first object on a first screen of the head mounted display device and displays the head mounted display. Providing a second object on a second screen of the device (S200); Receiving, by a computer, a user operation for adjusting the positional relationship between the first object and the second object (S400; user operation receiving step); Changing a position of the first screen and the second screen by the computer according to the user operation (S600; screen position changing step); And outputting the visual perceptual content in the visual field range of the patient including the visual damage region based on the visual center direction (S800). Hereinafter, a detailed description of each step will be described.

Patients with visual perception problems due to stroke, etc., do not see some areas of the visual field unlike normal people. Unlike the normal visual field of FIG. 11 (a), when the lower right side of the visual field is not visible because of abnormalities in the visual cortex of the left brain, as shown in FIG. You will not be able to visually identify children. As shown in FIG. 11 (c), when the visual cortex is severely damaged, the patient cannot properly identify an object in a region other than the central portion.

In the case of a patient who has a problem with visual perception function due to a stroke or the like (for example, a patient who does not see some areas within the visual field range), the patient visits a hospital to receive visual perception training to improve visual perception function. Was performed. Specifically, the visual perception training image for visual perception stimulation is provided on the monitor screen, and the training was performed with the head disposed at a predetermined distance and height. In other words, it was possible to proceed with the training in a state where detailed conditions were met, so it was possible to proceed only by visiting the hospital.

Recently, as the head mounted display device capable of providing a virtual reality image has been spread, it is possible to provide an image of a desired shape to a patient. Training can be performed only by adjusting the conditions where both eyes meet. There is a need for a method for realizing a screen presentation state corresponding to a patient having visual perception impairment in a part of the field of view.

In general, when looking at an object in the real space, both eyes converge toward the object, but when using a head mounted display device, the left eye and right eye images are provided respectively, so convergence of both eyes is automatic. There is a problem that cannot be done with. As the virtual reality contents are used by users with normal visual perception ability, the brain processes the information and can view it normally even without convergence matching of both eyes. However, in patients with visual impairment, it may be difficult to perform convergence matching in the brain, and the visual perception training effect may be reduced because the visual perception stimulus of training for improving visual perception cognition is not properly provided. Can be. Thus, in order to provide optimal visual perception training to the visual perception patient, an initial setting process that focuses both eyes (ie, convergence) is required.

The computer provides a first object on the first screen of the head mounted display device, and provides a second object on the second screen of the head mounted display device (S200). That is, the computer provides the object on the left eye screen and the right eye screen of the head mounted display device, respectively. The first object and the second object may be the same or different.

The computer receives a user operation for adjusting the positional relationship between the first object and the second object from the user (S400; user operation receiving step). That is, the user adjusts the position of the first object and the second object by inputting a user input for adjusting the position when the position of the first object and the second object does not match.

For example, when the first object and the second object are the same output image, the user may input an operation for adjusting the same output image displayed on the first screen and the second screen to exactly overlap. Also, for example, when the first object and the second object are different output images (for example, when the first object and the second object are different images that must be matched at the same position), the computer can be configured as a first object from the user. An operation of adjusting the point where the object and the second object to correspond to meet at a specific position (that is, the target point) may be received. That is, in the user operation receiving step (S400), the user receives an input operation for matching the position of the first object and the second object on the target point.

In another embodiment, at least one of the first object and the second object may be generated as an exercise object. Through this, a game may be implemented to perform the initial setting according to the patient eye condition before the patient performs visual perception training. For example, the first object is an arrow image and the second object is a target image, and the center of the second object is set as the target point, so that the arrow image, which is the first object, moves toward the target image, which is the second object. Can be generated. If the arrow on the first screen does not fit in the center of the target on the second screen, the user inputs an input operation for adjusting the arrow image to reach the target.

The computer may receive an operation to move from side to side or up and down from the user (ie, patient). For example, a user may receive an operation of moving the first object or the second object from side to side or up and down through the virtual reality controller. In addition, the computer may receive an operation of rotating the first object or the second object from the user through the virtual reality controller. In addition, the computer may receive an operation for adjusting the sizes of the first object and the second object provided on the screens of both eyes (ie, the first screen and the second screen) from the user.

Thereafter, the computer changes the positions of the first screen and the second screen according to the operation (S600; screen position changing step). That is, as shown in FIG. 2, the computer adjusts the position of the first screen and the second screen based on the input operation so that the first object and the second object meet at the target point. The computer moves the first screen and the second screen by a certain shifting distance.

As a result, the computer may cause the first screen and the second screen to be displayed at a position where the eyes of both eyes meet. That is, the computer may perform convergence matching between the first screen and the second screen (that is, the left eye display screen and the right eye display screen) according to a user's operation. That is, in general, the head mounted display apparatus physically adjusts the distance between the left eye display and the right eye display in order to match the position where the eyes of both eyes are directed, but by operating the first and second objects to correspond. The convergence matching is possible by adjusting the position of the first screen and the second screen in software.

Thereafter, the computer outputs visual perceptual content to the visual field of the patient including the visual damage region based on the visual center direction (S800; visual perceptual content outputting step). That is, the computer may provide visual perception content (that is, visual perception training image) in the viewing range while convergence matching is performed so that binocular images are superimposed properly in the viewing range of the patient.

The computer can provide visual perception stimulation to the field of view in a variety of ways. In one embodiment, the computer may provide a visual perception stimulus (ie, an output image) while changing the position of the entire visual field range (ie, the task display area). That is, the computer can provide visual perception stimuli to both the normal region and the visual impairment region within the viewing range. The visual perception stimulus provided to the normal region within the visual field range stimulates the brain region with respect to the visual damage region, thereby restoring the visual damage region. Thus, the computer can provide visual perception stimulation for the entire range within the field of view.

In another embodiment, the visual perceptual stimulus may be intensively provided to a region within the visual damage region. As will be described later, the computer can calculate the visual impairment area based on the training performance of the patient, and provide visual impulse stimulation only within the visual impairment area or generate the visual impulse stimulation at a higher rate than the normal area in the visual damage area Can provide. In addition, the computer may provide visual perception stimuli (ie, output images) at different points at different points according to the user's visual perception level even within the visual impairment region.

In addition, the computer may provide visual perceptual training images in various forms. In one embodiment, the computer may provide training (first problem) to select the type of screen image provided to the patient. In other words, the computer provides one of the plurality of images at least one point in the two-dimensional or three-dimensional space for each trial.

The first problem is provided by changing at least one of the number, type, arrangement position, and size of the output image, and the number of choices can be increased by varying the change condition. Thus, the computer provides training to select which of the plurality of screen images that can be generated according to the changing condition of the first problem.

In one embodiment, the computer may provide training (ie, a first problem) for selecting which image is displayed at a specific point on the screen among a plurality of predetermined output images. For example, if the output image includes a horizontal stripe image and a vertical stripe image, the computer provides a horizontal stripe image or a vertical stripe image for each trial in the task providing area and receives a visually confirmed image form from the patient. can do.

In another embodiment, the first problem may be varied by changing the arrangement position of the output image included in the screen for each trial providing the first problem on the screen. For example, the computer may generate another screen image by changing the position on the screen where the same output image is displayed. Changing the position of the output image means that when the task display unit is divided into a plurality of divided regions, the divided region in which the output image of the first problem is displayed or a point where the output image is arranged in the same divided region is different. It may be. The computer may allow the patient to select which screen image to present to the patient (ie, the arrangement of one or more output images).

In addition, when the task display area is implemented in a three-dimensional space, when the output image can be generated at various depths, the computer may generate various problem forms by changing the depth at which the output image is displayed. For example, the computer may display the output image by dividing it into an area near or far from the reference depth (eg, the depth at which the second problem is indicated when the second problem is provided). The computer may allow the patient to select which screen image to present to the patient (ie, the arrangement of one or more output images).

Further, in another embodiment, the computer may change the type of the output image (for example, the classification of the figure and the character image, etc.) or the shape (for example, the horizontal stripe pattern and the vertical stripe pattern) to solve different first problems. Can be generated. For example, the computer may generate different problems by displaying output images of different shapes (for example, horizontal stripes and vertical stripes) at the same location of the task display unit.

Further, in another embodiment, the computer may create different first problems as the size of the output image is changed while displaying the same output image type / shape at the same location.

In another embodiment, as illustrated in FIG. 3, the visual perceptual content outputting step S800 may include: convergence matching the first screen and the second screen at a center point (S810); And displaying the object corresponding to the visual perceptual content at a specific depth in a specific two-dimensional space based on the depth point of the center point (S820). That is, the computer may provide visual perception stimuli at various depths after convergence matching at the center point in the three-dimensional space.

Specifically, in case of providing a second problem (for example, a problem of inputting what is displayed on the screen among ㅋ or ㅌ) to fix the patient's gaze direction to the center point of the three-dimensional space, the computer has a second problem. The position in the three-dimensional space for displaying the output image included in the first problem is determined based on the arrangement position in the three-dimensional space. In the 3D space, the images provided to the left and right eyes (that is, the left eye image and the right eye image) form perspective based on a point at which convergence is formed. Accordingly, the computer converges the left eye image and the right eye image with respect to the second problem while arranging the second problem to be described later at a specific depth, and arranges the output image according to the first problem at a specific depth to be disposed at the reference depth. Based on the positional relationship with the second problem, a screen in which the output image is displayed at various depths can be implemented.

Further, as another embodiment, as shown in Figure 4, the computer compares the response input from the user for each trial and the provided image type (S1000); further includes. That is, the computer receives the first response, which is a response to the first task, and determines whether the first response corresponds to the correct answer. For example, when the horizontal stripes and the vertical stripes are provided as the output image included in the first problem, the computer determines whether the image displayed at a specific position on the screen is the correct answer. The computer may calculate the correct answer rate by accumulating the correct answer results for a plurality of trials, and may calculate the correct answer rate for each point in a two-dimensional or three-dimensional space that is a patient's field of view. The point-to-point correct rate may be used to generate a gradient map for determining a visual perception state of a patient as described below.

In another embodiment, as illustrated in FIG. 5, the computer may store the output conditions of the first screen and the second screen for a specific user (S700). That is, the position conditions of the first screen and the second screen corresponding to the eye conditions of the specific user (ie, the patient) are calculated and stored. Accordingly, when several patients perform visual perception training using one head mounted display device, visual perception training can be performed immediately by selecting setting information for a specific user.

Further, as another embodiment, as shown in Figure 6, by calculating the correct answer rate for each point on the two-dimensional or three-dimensional space to set the user's visual perception damage area (S1200); The computer can obtain a correct answer rate for each point by providing visual perception stimuli (ie, output images) at various points of the task providing area. As shown in FIG. 7, the computer generates a gradation map in which the color of each point is expressed differently according to the distribution rate or the correct answer rate of the task providing area based on the correct rate for each point. Then, the computer calculates the visual perceptual damage region by connecting points having a specific value based on the percentage distribution or the gradient map. Through this, the computer can recognize the change in the degree of visual impairment of the patient only by the patient's training performance results without performing a separate test (for example, fMRI test).

In another embodiment, as shown in FIG. 8, the computer determines at least one of a task providing frequency, a task difficulty level, and an output image size of the task based on a periodically calculated visual perception damage area. It further comprises (S1400). That is, the computer may determine the visual perception cognitive ability state of the patient to determine the difficulty of the visual perception training image (ie, the task). Various methods may be applied to adjust the difficulty of the visual perceptual training image. For example, the computer may increase the difficulty by lowering the brightness or saturation of the output image provided on the screen or reducing the size of the output image. In addition, the computer may adjust the difficulty level by adjusting the number of output images appearing in the screen image. As the number of output images increases, as the arrangement relationship between the output images varies, the number of choices to be selected by the patient may increase, thereby increasing difficulty.

In addition, visual perception stimulation should be provided to the visual impairment area based on the patient's gaze-centered direction, which is to confirm the visual perception stimulus by changing the eye direction to confirm through the normal area where the visual perception impulse is invisible. Need to be prevented.

To this end, in an embodiment, the second problem may be provided at the center of the viewing range so that the first problem, which is a visual perception stimulus, is fixed while the eyeline direction is fixed at the center, and the first response may be input. Specifically, the computer provides a second problem displayed on the central field of view within the screen, and receives a second response to the second problem from the user. The second problem serves to fix the gaze direction of the user at a specific position. Preferably, the second problem is provided at the same time as the first problem provided on one side of the task display unit so that the gaze direction is fixed in the center direction at the time of checking the first problem. That is, a screen image for simultaneously displaying the first problem and the second problem on the screen is provided so that the user checks the first problem while the user watches the center view portion where the second problem is displayed.

In another embodiment, the computer senses the user's motion or pupil movement to search for a view center of view, and provides a task based on a relative position based on the view center direction. That is, the computer may measure the gaze direction of the user and determine and provide an arrangement position of the output image according to the first problem based on the gaze direction. For example, since the user is provided with the screen image through the head mounted display device, when the user moves his head or moves the eyeball while wearing the head mounted display device, the direction shown for the same screen image is changed. Therefore, the computer recognizes the user's bow movement or eye movement, acquires the gaze direction, and determines the arrangement position of the output image based on the gaze direction.

Specifically, the computer measures the user's motion based on the motion data acquired through the motion sensor included in the head mounted display device. In addition, the computer measures the eye movement based on eye movement data acquired by an eye movement sensor (ie, an eye tracker) included in the head mounted display device. The computer then calculates the visual direction (ie, the center direction of the visual field range) when the user's bow movement or eye movement is present. The computer then displays the output image about the visual direction (ie, the direction of the center of the viewing range) to generate the screen image provided for the particular trial. That is, the output image arrangement position information in the first problem is a relative position determined based on the visual direction (ie, the center direction of the viewing range), and the computer displays the screen image of each trial based on the output image arrangement position information in the first problem. Create Through this, the computer can smoothly provide memory training while the user performs free head movements and eye movements.

9 is a structural diagram of a computing device providing visual perceptual training according to an embodiment of the present invention.

FIG. 10 is an exemplary diagram for providing visual perception training images to a head mounted display device according to an embodiment of the present invention.

Referring to FIG. 10, a computing device (or visual perceptual training device) 100 that provides visual perception training according to an embodiment of the present invention may include a display 110, a memory 120, a controller 130, and a user input unit. All or part of 140. Since the computing device 100 providing visual perceptual training according to an embodiment of the present invention performs the same visual perceptual training method according to embodiments of the present invention, the descriptions thereof will be omitted.

The visual perceptual training device 100 may be a portable terminal. The visual perceptual training apparatus 100 may be mounted on a user's head and used in a manner in which the display 110 and the user's eyes face each other. The visual perception training apparatus 100 may be a head mounted device such as a head mounted display (HMD), glass, goggles, or a display device such as a mobile phone that can be mounted on the head mounted device.

The head mounted display device includes a first screen and a second screen. The first screen and the second screen are left eye screens and right eye screens of the head mounted display device.

The memory 120 stores the visual perception training program. The visual perception training program may be a native program pre-installed in the visual perception training apparatus 100 or a program downloaded and installed from an external server. The memory 120 may be an external memory connected to or inserted into the visual perceptual training device 100.

When the visual perception training program is executed, the controller 130 may output the visual perception training image corresponding to the visual perception training program to the display 110.

The screen of the display 110 may include a first screen unit corresponding to the left eye of the patient and a second screen unit corresponding to the right eye of the patient. The controller 130 may adjust the parallax and focus of the visual perception training image by adjusting a distance and an overlap between the first image output through the first screen unit and the second image output through the second screen unit.

The visual perception training image may include a pre-image for adjusting the focus of the visual perception training image before performing the visual perception training in earnest.

The user input unit 140 may include a user input for executing the visual perception training program, a user input for adjusting the focus of the visual perception training image during the visual perception training, a user input for correcting a problem included in the visual perception training image, and the like. Can be received.

The user may manually adjust the focal length through the user input unit 140 while viewing the preliminary image, or may automatically adjust the focal length while playing the game through the preliminary image produced in the game format. The user may manually adjust the focus through the user input unit 140 even during the visual perception training.

For example, a method of adjusting focus through a preliminary image produced in a game format will be described below.

The controller 130 may output the target shooting game (eg, a shooting game) to the display 110 as a dictionary image. The user shoots toward the target through the user input unit 140, and the controller 130 may automatically adjust the focus to suit the user based on the target hit ratio.

Visual perception training images may vary in difficulty depending on the degree of visual impairment of the patient. The controller 130 may include at least one of the shape, size, position, color, saturation, and contrast of the figure / character included in the visual perceptual training image based on the correct answer rate of the answer input through the user input unit 140 during visual perception training. By adjusting the difficulty level, you can determine the level of visual perception training.

According to an embodiment of the present invention, during the visual perception training, the user's answer is received in real time through the user input unit 140, and the difficulty of the training is flexibly adapted to the level of the user according to the user's state based on the correct answer rate. There is an adjustable effect.

When the computing device providing visual perception training according to an embodiment of the present invention describes an initial setup process for a specific patient to perform visual perception training is as follows. The computing device 100 providing visual perception training provides a first object on a first screen of the head mounted display device, and provides a second object on a second screen of the head mounted display device. Thereafter, the computing device receives an operation for adjusting a positional relationship between the first object and the second object from a user. Thereafter, the computing device changes the positions of the first screen and the second screen according to the manipulation. Thereafter, the computing device outputs visual perceptual content to the field of view of the patient including the visual damage region with respect to the visual center direction.

The above-described method for providing visual perception training based on the head mounted display device according to an embodiment of the present invention may be implemented as a program (or an application) and stored in a medium to be executed in combination with a computer which is hardware.

The above-described program includes C, C ++, JAVA, machine language, etc. which can be read by the computer's processor (CPU) through the computer's device interface so that the computer reads the program and executes the methods implemented as the program. Code may be coded in the computer language of. Such code may include functional code associated with a function or the like that defines the necessary functions for executing the methods, and includes control procedures related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure. can do. In addition, the code may further include memory reference code for additional information or media required for the computer's processor to execute the functions at which location (address address) of the computer's internal or external memory should be referenced. have. Also, if the processor of the computer needs to communicate with any other computer or server remotely in order to execute the functions, the code may be used to communicate with any other computer or server remotely using the communication module of the computer. It may further include a communication related code for whether to communicate, what information or media should be transmitted and received during communication.

The stored medium is not a medium for storing data for a short time such as a register, a cache, a memory, but semi-permanently, and means a medium that can be read by the device. Specifically, examples of the storage medium include, but are not limited to, a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. That is, the program may be stored in various recording media on various servers to which the computer can access or various recording media on the computer of the user. The media may also be distributed over network coupled computer systems so that the computer readable code is stored in a distributed fashion.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

Claims (10)

In the method for performing visual perception training using a head mounted display device,
Adjusting, by the computer, convergence of the binocular gaze to match the eyes of the user; And
When the adjustment of the convergence is completed, the computer providing visual perception training to the patient by outputting visual perception content in the visual field of the patient including the visual damage region based on the visual center direction;
Adjusting the convergence (Convergence),
The computer provides a first object on a first screen of the headmount display device and a second object on a second screen of the headmount display device, wherein the first object is fixed at a specific position of the first screen. The second object is fixed to a specific position of the second screen, the object providing step;
A user operation receiving step of receiving, by a computer, a user operation for adjusting the positional relationship between the first object and the second object;
A screen position changing step of changing, by a computer, positions of the first screen and the second screen according to the user operation; And
The user operation receiving step and the screen position changing step are repeatedly performed until convergence matching is achieved, wherein the convergence matching is achieved such that the first point in the first object and the second point in the second object correspond to each other at the target point. Comprising a convergence matching step, wherein the first screen and the second screen is disposed,
The first screen is any one of a left eye screen and a right eye screen of the head mounted display device.
And wherein the second screen is not the first screen among a left eye screen and a right eye screen of the head mounted display device. The method of claim 1,
The user operation receiving step,
By the user to match the position of the first object and the second object on the target point,
And at least one of the first object and the second object is an exercise object. The method of claim 1,
And storing, by a computer, output conditions of the first screen and the second screen for a specific user. The method of claim 1,
Providing the visual perception training,
To provide one of the plurality of images in at least one point in a two-dimensional or three-dimensional space for each trial,
And comparing, by the computer, the response input from the user for each trial with the provided image type. The method of claim 4, wherein
Providing the visual perception training,
Convergence matching the first screen and the second screen at a center point; And
And displaying the object corresponding to the visual perceptual content at a specific depth in a specific two-dimensional space based on the depth point of the central point. The method of claim 4, wherein
Comprising the step of calculating the correct response rate for each point on the two-dimensional or three-dimensional space; setting the visual perception damage area of the user; further comprising, a head-mounted display device based visual perception training providing method. The method of claim 6,
And determining, by the computer, at least one of a task providing frequency, a task difficulty level, and a size of an output image of the task based on a periodically calculated visual perception damage region. How to provide training. The method of claim 1,
A method for providing visual perception training based on a head mounted display device, characterized in that a user detects a head movement or a pupil movement to search a field of view direction and provides a task based on a relative position based on the field of view direction. . A head mounted display apparatus-based visual perception training providing program coupled with a computer which is hardware and stored in a medium for executing the method of any one of claims 1 to 8. Adjust the convergence of binocular gaze according to the user's eyes,
When the adjustment of the convergence is completed, the computer provides visual perception training by outputting the visual perception content in the visual field of the patient including the visual impairment region based on the visual center direction,
Adjusting the convergence (Convergence),
Providing a first object on a first screen of the head mounted display device;
Providing a second object on a second screen of the head mounted display device;
Receive a user operation for adjusting the positional relationship between the first object and the second object from the user,
Changing the position of the first screen and the second screen to match the convergence according to the user operation, and changing the position of the first screen and the second screen so that the convergence matches the first object. And arranging the second objects so as to correspond to each other, and changing the positions of the first screen and the second screen such that the corresponding points meet at the target location.
The first screen is any one of a left eye screen and a right eye screen of the head mounted display device.
And the second screen is not the first screen among a left eye screen and a right eye screen of the head mounted display device.

Images