JP7218978B2 - Rehabilitation support system, rehabilitation support method and rehabilitation support program - Google Patents

Description

The present invention relates to a rehabilitation support system, a rehabilitation support method, and a rehabilitation support program.

In the above technical field, Patent Literature 1 discloses a system that assists rehabilitation performed for a hemiplegic patient due to a stroke or the like.

JP 2015-228957 A

However, with the technique described in the above document, the target cannot be actively updated according to the user's actions, and the same load has to be repeated for any user.
An object of the present invention is to provide a technique for solving the above problems.

In order to achieve the above object, the system according to the present invention comprises:
a motion detector that detects a user's rehabilitation motion;
a display control unit that displays an avatar image that moves according to the detected rehabilitation action and a target image that indicates a goal of the rehabilitation action;
an evaluation unit that evaluates that the target image has been recovered by the user's rehabilitation action when the position of the avatar image and the target position represented by the target image moving downward overlap ;
with
The display control unit assists in recovering the target image moving from top to bottom so as to soften the user's shock when the target image cannot be recovered by the user's rehabilitation action. This is a rehabilitation support system that displays images .

In order to achieve the above object, the method according to the present invention comprises:
a motion detection step of detecting a user's rehabilitation motion;
a display control step of displaying an avatar image that moves according to the detected rehabilitation action and a target image that indicates a goal of the rehabilitation action;
an evaluation step of evaluating that the target image has been recovered by the user's rehabilitation action when the position of the avatar image and the target position represented by the target image moving downward overlap ;
including
In the display control step, for the target image moving from top to bottom, if the target image cannot be retrieved by the user's rehabilitation movement, the user is assisted in retrieving the target image so as to soften the user's shock. This is a rehabilitation support method for displaying images .

In order to achieve the above object, the program according to the present invention
a motion detection step of detecting a user's rehabilitation motion;
a display control step of displaying an avatar image that moves according to the detected rehabilitation action and a target image that indicates a goal of the rehabilitation action;
an evaluation step of evaluating that the target image has been recovered by the user's rehabilitation action when the position of the avatar image and the target position represented by the target image moving downward overlap ;
A rehabilitation support program that causes a computer to execute
In the display control step, for the target image that moves from top to bottom, if the target image cannot be retrieved by the user's rehabilitation action, the user is assisted in retrieving the target image so as to soften the user's shock. It is a rehabilitation support program that displays images .

ADVANTAGE OF THE INVENTION According to this invention, updating of a target can be performed actively according to a user's rehabilitation operation|movement.

It is a block diagram showing composition of a rehabilitation support system concerning a 1st embodiment of the present invention. It is a block diagram which shows the structure of the rehabilitation support system which concerns on 2nd Embodiment of this invention. It is a figure which shows the example of a display screen of the rehabilitation assistance system which concerns on 2nd Embodiment of this invention. It is a figure which shows the example of a display screen of the rehabilitation assistance system which concerns on 2nd Embodiment of this invention. It is a flow chart which shows a flow of processing of a rehabilitation support system concerning a 2nd embodiment of the present invention. It is a figure which shows the example of a display screen of the rehabilitation assistance system which concerns on 2nd Embodiment of this invention. It is a figure which shows the example of a display screen of the rehabilitation assistance system which concerns on 2nd Embodiment of this invention. It is a figure which shows the example of a display screen of the rehabilitation assistance system which concerns on 2nd Embodiment of this invention. It is a figure which shows the other program example of the rehabilitation support system which concerns on 2nd Embodiment of this invention. It is a figure which shows the structure of the database of the rehabilitation support system which concerns on 2nd Embodiment of this invention. It is a figure which shows the example of a display screen of the rehabilitation assistance system which concerns on 2nd Embodiment of this invention. It is a figure which shows the example of a display screen of the rehabilitation assistance system which concerns on 2nd Embodiment of this invention. It is a figure explaining the outline of operation|movement of the rehabilitation support system which concerns on 2nd Embodiment of this invention. It is a figure explaining the outline of operation|movement of the rehabilitation support system which concerns on 2nd Embodiment of this invention. It is a figure explaining the outline of operation|movement of the rehabilitation support system which concerns on 3rd Embodiment of this invention. FIG. 11 is a diagram for explaining an arrangement position of a visual recognition aid image in a rehabilitation support system according to a third embodiment of the present invention; FIG. 11 is a diagram illustrating another example of a visual recognition aid image in the rehabilitation support system according to the third embodiment of the present invention; FIG. 11 is a diagram illustrating still another example of a visual recognition aid image in the rehabilitation support system according to the third embodiment of the present invention; FIG. 11 is a diagram illustrating still another example of a visual recognition aid image in the rehabilitation support system according to the third embodiment of the present invention; FIG. 11 is a diagram illustrating still another example of a visual recognition aid image in the rehabilitation support system according to the third embodiment of the present invention; FIG. 11 is a diagram illustrating still another example of a visual recognition aid image in the rehabilitation support system according to the third embodiment of the present invention; FIG. 11 is a diagram illustrating still another example of a visual recognition auxiliary image in the rehabilitation support system according to the third embodiment of the present invention; It is a block diagram explaining the structure of the rehabilitation support system which concerns on 3rd Embodiment of this invention. It is a figure explaining an example of the patient table which the rehabilitation assistance server contained in the rehabilitation assistance system which concerns on 3rd Embodiment of this invention has. It is a figure explaining an example of the display parameter table which the rehabilitation assistance server contained in the rehabilitation assistance system which concerns on 3rd Embodiment of this invention has. It is a figure explaining an example of the image table which the rehabilitation assistance server contained in the rehabilitation assistance system which concerns on 3rd Embodiment of this invention has. It is a block diagram explaining the hardware constitutions of the rehabilitation support server contained in the rehabilitation support system which concerns on 3rd Embodiment of this invention. It is a flow chart explaining a processing procedure of a rehabilitation support server contained in a rehabilitation support system concerning a 3rd embodiment of the present invention. FIG. 11 is a flow chart for explaining a processing procedure for visual recognition auxiliary image display of a rehabilitation support server included in a rehabilitation support system according to a third embodiment of the present invention; FIG. It is a block diagram explaining the structure of the rehabilitation assistance system which concerns on 4th Embodiment of this invention. It is a figure explaining an example of the sound table which the rehabilitation assistance server contained in the rehabilitation assistance system which concerns on 4th Embodiment of this invention has. It is a figure explaining the hardware constitutions of the rehabilitation support server contained in the rehabilitation support system which concerns on 4th Embodiment of this invention. It is a flow chart explaining a processing procedure of a rehabilitation support server contained in a rehabilitation support system concerning a 4th embodiment of the present invention. It is a flowchart explaining the processing procedure of sound output control of the rehabilitation assistance server contained in the rehabilitation assistance system which concerns on 4th Embodiment of this invention. It is a figure for demonstrating the control method of the rehabilitation support system which concerns on 5th Embodiment of this invention. It is a figure for demonstrating the control method of the rehabilitation support system which concerns on 5th Embodiment of this invention. It is a figure which shows the example of a display screen of the rehabilitation assistance system which concerns on 5th Embodiment of this invention. It is a figure which shows the example of a display screen of the rehabilitation assistance system which concerns on 5th Embodiment of this invention. It is a figure which shows the example of a display screen of the rehabilitation assistance system which concerns on 5th Embodiment of this invention.

BEST MODE FOR CARRYING OUT THE INVENTION Exemplary embodiments of the present invention will be described in detail below with reference to the drawings. However, the components described in the following embodiments are merely examples, and the technical scope of the present invention is not limited to them.

[First embodiment]
A rehabilitation support system 100 as a first embodiment of the present invention will be described with reference to FIG.

As shown in FIG. 1 , rehabilitation support system 100 includes motion detection unit 101 , display control unit 102 , evaluation unit 103 and update unit 104 .

The motion detection unit 101 detects a rehabilitation motion of the user 110 . The display control unit 102 displays an avatar image that moves according to the detected rehabilitation action and a target image that indicates the goal of the rehabilitation action.

The evaluation unit 103 evaluates the user's rehabilitation ability according to the difference between the rehabilitation motion and the target position represented by the target image. The updating unit 104 updates the target position according to the evaluation result by the evaluating unit 103 .

The motion detection unit 101 further detects a second rehabilitation motion of the user during the first rehabilitation motion, and the evaluation unit 103 detects the first rehabilitation motion when only the first rehabilitation motion is evaluated to a predetermined value or higher. Rehabilitation performance is assessed based on both the movement and the secondary rehabilitation movement. As a result, it is possible to actively and accurately update the target in accordance with the user's rehabilitation action.

[Second embodiment]
Next, a rehabilitation support system 200 according to a second embodiment of the invention will be described using FIG. FIG. 2 is a diagram for explaining the configuration of the rehabilitation support system according to this embodiment.

As shown in FIG. 2, the rehabilitation support system 200 includes a rehabilitation support server 210, two base stations 231 and 232, a head mounted display 233, and two controllers 234 and 235. The head-mounted display 233 may be of non-transmissive type, video see-through type, or optical see-through type.

The rehabilitation support server 210 also includes a motion detection unit 211 , a display control unit 212 , an evaluation unit 213 , an update unit 214 , a voice input/output unit 215 , a target database 216 and a background image+question/answer database 217 .

The motion detection unit 211 acquires the positions of the controllers 234 and 235 held by the user 220 and the position of the head mounted display 233 via the base stations 231 and 232, and detects the rehabilitation motion of the user 220 from changes in the positions. .

The display control unit 212 causes the head-mounted display 233 to display an avatar image that moves according to the detected rehabilitation action and a target image that indicates the goal of the rehabilitation action. FIG. 3 is a diagram showing an example of avatar images 311 and 312 on screen 301 displayed on head-mounted display 233 . Avatar images 311 and 312 are superimposed on a background image 313 . In this example, the avatar images 311 and 312 have the same shape as the controllers 234 and 235 and move in the screen 301 according to the movements of the controllers 234 and 235 . Also, the background image 313 changes according to the position and orientation of the head mounted display 233 . As shown in the avatar images 311 and 312, the controllers 234 and 235 are provided with buttons to enable various setting operations. As the background image 313, here, a landscape image obtained by imaging an actual landscape (for example, a moving image of a cityscape of New York) is displayed. As the scenery image, an image of the road around the rehabilitation facility may be used, and it is possible to make the user feel as if he or she is taking a walk in a foreign country or a familiar place. By superimposing a landscape image, it is possible to entertain the patient and realize training in the presence of a large amount of information.

Further, the display control unit 212 displays an object 411 superimposed on the background image 313 on the screens 401 to 403 of the head mounted display 233 as shown in FIG. 4, for example. The object 411 is displayed while gradually changing its display position and size so that it appears to be falling downward from above the user's 220 head. The user 220 moves the controllers 234 and 235 to bring the avatar image 311 on the screen closer to the object 411 . When the avatar image 311 collides with the object 411, the object 411 disappears. In the screens 401 to 403, the character "left" near the sensor avatar image 311 means that the object 411 is touched with the left hand.

The evaluation unit 213 compares the rehabilitation motion detected by the motion detection unit 211 with the target position represented by the target image displayed by the display control unit 212 to evaluate the rehabilitation ability of the user. Specifically, whether or not the avatar image 311 moved corresponding to the rehabilitation motion detected by the motion detection unit 211 overlaps with the object 411 as the target image is determined by comparing the positions in the three-dimensional virtual space. . As a result, if these overlap, it will be evaluated as having cleared one rehabilitation movement, and points will be added. Various levels (for example, three levels) are prepared for the position of the object 411 in the depth direction, and different points (a high point for a distant object and a low point for a near object) are set for each.

The updating unit 214 updates the target assignment according to the accumulated points. For example, the target assignment may be updated using the assignment achievement rate (the number of goals achieved/the number of assignments).

FIG. 5 is a flow chart showing the flow of processing in the rehabilitation support server 210. As shown in FIG. In step S501, as a calibration process, the goal of rehabilitation motion is initialized according to the user. Specifically, each patient is first calibrated by performing work within the range of possible action, and after setting it as an initial value, the target is initialized according to the user.

Also, by referring to the goal database 216, goals are set according to the user's attribute information (eg, athlete, Parkinson's disease, etc.). For example, in the case of an injured athlete, an initial value that does not exacerbate the injury is set, and in the case of a user suffering from Parkinson's disease, an initial value of exercise that slows down the progress of the disease is set. do. Furthermore, each patient is first asked to perform work in the range of possible action, and after setting it as an initial value, the target is initialized according to the user.

Next, in step S503, avatar images 311 and 312 are displayed according to the positions of the controllers 234 and 235 detected by the motion detection unit 211. FIG. Furthermore, in step S505, the object 411 is displayed at a position and speed corresponding to the set task.

In step S507, the movement of the avatar images 311 and 312 and the movement of the object 411 are compared to determine whether the task has been completed. If the task has not been cleared, the process directly returns to step S505 to display the next object without changing the difficulty level of the task.

If the task is cleared, the process advances to step S509 to calculate accumulated points, task achievement probability, and the like. Further, in step S511, the accumulated points, task achievement probability, etc. are compared with a threshold T. If the accumulated points, task achievement probability, etc. exceed a predetermined threshold T, the process proceeds to step S513 to update the exercise intensity of the task. . If the accumulated points, task achievement probability, etc. do not reach the threshold value T, the process returns to step S505 to display the next object without changing the difficulty level of the task.

For example, when the degree of accomplishment of short distance exceeds 80% (or ten times or the like is also possible), the display frequency of medium distance objects is increased. When the degree of achievement of middle-distance objects exceeds 80% (or 10 times or the like is acceptable), the display frequency of long-distance objects is increased. Conversely, when the degree of achievement is low, the target value may be set to a short distance.

The updating of the tasks here is also changed according to the attributes of the user (whether the user is an injured athlete or a patient with Parkinson's disease). As a method of updating the assignment, a method of switching the background image is also conceivable.

After updating the task, the process advances to step S515 to calculate the fatigue level of the user and compare it with the threshold value N. FIG. If the degree of fatigue exceeds a predetermined threshold, the processing is terminated as a "stopping condition". For example, it can be calculated as (fatigue level=1−recovery rate of nearest object). Alternatively, it may be calculated as (fatigue level=1/eye movement). If the patient is obviously not concentrating (not looking for an object at all, does not shake his/her head, etc.), there is no point in continuing rehabilitation, so a break is taken. Furthermore, the degree of fatigue may be calculated by detecting a decrease in speed (acceleration) when extending the arm.

Also, for example, if the cumulative points exceed a predetermined threshold, as shown in a character image 601 in FIG. 6, which of the two controllers 234 and 235, left or right, should touch the object 411 (here, right)? to direct. As a result, the cognitive function of recognizing letters is required, and the difficulty of the movement increases, requiring a higher motor function. In other words, a dual task of cognitive function and motor function is required.

In FIG. 6, the instructions are given by characters, but the present invention is not limited to this, and the instructions may be given by arrows, colors, or voice. Thus, in this embodiment, the load is updated according to the evaluation of the rehabilitation motion.

(About dual task)
Healthy people do two or more things at the same time, such as "walking while talking" in their daily lives. This “ability to do two things at the same time” declines with age. For example, "If you talk to me while walking, I will stop walking." It is thought that the cause of falls in the elderly is not only "decreased motor function" but also "decreased ability to do two things at the same time". In fact, there are many elderly people who fall down after returning home even if they are judged to have sufficiently recovered their motor function through rehabilitation. One of the reasons for this is that the rehabilitation is being carried out in a state where the environment and conditions that allow the patient to concentrate on the rehabilitation movements are prepared. In other words, in the living environment, there are many factors that make it difficult to concentrate on the action, such as a poor view, the presence of obstacles, and the focus on conversation.

Therefore, it is considered important to perform rehabilitation that distributes attention, and it is desirable to specifically give dual tasks for training. This kind of dual-task training is an effective program not only for the elderly who fall but also for the prevention of dementia.

Dual-task training includes training that combines cognitive and motor tasks, as well as training that combines two types of motor tasks.

The cognitive task + motor task includes training such as walking while subtracting 1 from 100, and the motor task + motor task includes training such as walking so as not to overflow the water in the cup.

If the walking speed in the dual-task walking test is delayed by about 20% compared to simple walking, the evaluation unit 213 evaluates that the risk of falling is high, and notifies the display control unit 212 to repeat the dual-task. .

It should be noted that dual tasks are more likely to be effective for “people with relatively high mobility”. For example, for elderly people who cannot let go of a cane even when moving indoors, strengthening balance ability (muscular strength, sense of balance, etc.) has a higher priority than dual-tasking ability. As a rough judgment, it can be expressed that dual-tasking ability is important for people requiring support, and balance ability other than dual-tasking ability is important for people requiring care. View your calibration changes over time and visually see your range of motion improvement.

(Setting change by user attribute)
For patients in whom normal improvement can be expected (for patients with orthopedic diseases such as bone fractures and assuming complete improvement), the most difficult rehabilitation movements are set to accelerate improvement.

For patients with different degrees of improvement depending on the individual (cerebral infarction, etc., different forms of paralysis occur depending on the lesion site), the load of the task is increased to some extent, and the increase of the load is stopped at a certain level. do.

In patients with essentially declining function, such as Parkinson's disease, it is always useful to have regular assessments of their current mobility status.

(Other dual-task training examples)
FIG. 7 is a diagram showing another image example for dual-task training. Cognitive function is requested by setting a failure (bomb) in the object. Alternatively, as shown in FIG. 8, a question image (here, multiplication as an example) may be displayed superimposed on the background screen, and only acquisition of objects displaying answers may be evaluated. You can also display either goo, scissors, or par on the background screen and request collection of the object with the winning mark displayed on it.

Furthermore, it is also possible to simply display numbers on objects and evaluate only the acquisition of objects with large numbers. Alternatively, if a traffic light is displayed in the background image 313 and the object is picked up when the traffic light is red, the evaluation unit 213 may give a minus point.

According to this embodiment, since the task is updated according to the degree of achievement (probability of achievement, etc.) of the rehabilitation action, it is possible to give the user a load corresponding to the progress of the rehabilitation. Moreover, by displaying the background image 313, not only can the patient enjoy themselves, but rehabilitation can be performed in a situation where the patient is aware of the surroundings, and a safer life can be realized when returning to the real world.

FIG. 9 is a diagram showing another example of dual-task training. As shown in FIG. 9 , the voice input/output unit 215 outputs a voice question about the background image to the headphone 901 and acquires an answer to the question via the microphone 902 provided on the head mounted display 233 . The evaluation unit 213 performs speech recognition processing on the answer acquired as voice information, compares it with a prepared answer, and evaluates the user's rehabilitation ability according to the comparison result.

FIG. 10 is a diagram showing an example of the contents of the background image/question/answer database 217. As shown in FIG. Question sounds, answers, and points are associated with the background moving image and stored.

As a user reaction, it is expected that the collection rate of objects will decrease in dual task. As a goal, we expect that the achievement rate of collecting objects does not differ even if dual tasks are displayed. The collection rate of the object or the reach rate to the object is compared between single task and dual task, and training is repeated until it falls within a certain range.

Although the above is dual-task training that simultaneously requires motor function and cognitive function, the present invention is not limited to this, and dual-task training that simultaneously requires two motor functions may be used.

For example, as shown in FIG. 11, it may be requested to pick up an object 411 while avoiding a flying object 1111 . By detecting the position of the sensor provided in the head-mounted display 233, it can be determined whether the object 1111 has been successfully avoided. Based on the achievement points (for example, the achievement rate, etc.) of both of the two rehabilitation actions, the evaluation and the task may be updated.

Further, for example, as shown in an image 1201 in FIG. 12, water-filled glass images 1211 and 1212 are displayed as avatar images that move according to the operations of the controllers 234 and 235, and these glass images 1211 and 1212 are moved. and collect the object 411. However, when the cup image 1221 is tilted as shown in the image 1202 and water spills, even if the object 411 is recovered with the cup image 1222, points cannot be obtained. Points are added only when the object 411 is collected without spilling the water in the cup images 1231 and 1232 as in the image 1203 .

In addition, it is conceivable to request that the avatar image on the opposite side of the avatar image on the side collecting the object always touches the designated place. It is also possible to request collection of the object while pressing a designated button of the buttons provided on the controllers 234 and 235 a predetermined number of times. Further, when a sensor for acquiring the movement of the user's foot is provided, a specified movement of the foot may be requested.

[Third Embodiment]
Next, a rehabilitation support system according to a third embodiment of the present invention will be explained using FIGS. 13A to 17B. FIG. 13A is a diagram for explaining the outline of the operation of the second embodiment of the rehabilitation support system according to this embodiment. FIG. 13B is a diagram for explaining the outline of the operation of the rehabilitation support system according to this embodiment. The rehabilitation support system according to the present embodiment differs from the second embodiment in that it displays a visual recognition auxiliary image that improves the recognizability (for example, visibility) of the target image. Since other configurations and operations are similar to those of the second embodiment, the same configurations and operations are denoted by the same reference numerals, and detailed description thereof will be omitted.

In the second embodiment, the moving distance of the avatar image 1320, that is, the exercise distance 1312 of the user 220 is measured by the distance between the reference 1310 and the sensor of the avatar image 1320 (the head portion of the avatar image 1320). A target distance 1311, which is the distance at which the user 220 must move the arm or the like, is determined based on the distance between the reference 1310 and the reference line 1331 of the object 1330 as the target image. User 220 then moves avatar image 1320 to bring it closer to object 1330 as rehabilitation exercise.

However, as shown in FIG. 13B, when avatar image 1320 touches vertex 1332 of object 1330, the system determines that one of user's 220 rehabilitation movements is complete and targets new object 1330 as the next target. indicate.

As a rehabilitation exercise, the system provider wants the avatar image 1320 to come into contact with the object 1330 when the user 220 fully stretches his/her arm. However, when the size of the object 1330 is large (the distance between the vertex and the reference line 1331 is large), it is determined that the object 1330 has been touched just by the avatar image 1320 touching the edge of the object 1330 . Therefore, since the user 220 cannot move the arm by the initially assumed distance, it is difficult to obtain the expected rehabilitation effect.

In addition, since the user 220 can touch the object 1330 before fully extending his or her arm, the user 220 cannot obtain a sufficient sense of accomplishment or satisfaction from the rehabilitation exercise, which may reduce motivation for rehabilitation. .

In this case, the movement distance 1312, which is the distance that the avatar image 1320 actually moves, deviates from the target distance 1311, which is the distance that the user 220 should move. Therefore, the user 220 does not exercise for the exercise distance 1312 set before the start of rehabilitation, and the effect obtained from the rehabilitation is less than expected.

For example, assume that the length of one side of the object 1330 is 20.0 cm, and the diameter 1321 of the sensor portion of the avatar image 1320 (the head portion of the avatar image 1320) is 5.0 cm. In this case, when the user 220 touches the avatar image 1320 to the vertex 1332 instead of the reference line 1331 of the object 1330, an error of about 10.0 cm occurs between the target distance 1311 and the movement distance 1312. FIG.

Therefore, since the user 220 has not moved the avatar image 1320 by the exercise distance 1312 assumed before starting the rehabilitation, the effect of the rehabilitation that the user 220 should have originally enjoyed is reduced.

On the other hand, if the object 1330 is made smaller so that the user 220 can touch the object 1330 when the arm is fully extended, it becomes difficult for the user 220 to visually recognize the position of the object 1330 within the screen. If the object 1330 cannot be visually recognized, rehabilitation is not established.

Therefore, in this embodiment, the sensor portion (reaction portion) of the avatar image 1320 is set to be a smaller area than the head portion of the avatar image 1320 . This makes it possible to further reduce the deviation (error) between the target distance 1311 and the movement distance.

FIG. 13C is a diagram explaining the outline of the operation of the rehabilitation support system according to this embodiment. Therefore, in the present embodiment, the gradation of the central portion of the object 1330 is darkened so as not to cause a deviation between the assumed target distance 1311 and the movement distance 1312 of the avatar image 1320, and is used as the reaction portion. Then, the gradation of the portion around the reaction portion of the object 1330 is lightened. In other words, the size of the object 1330 shown in FIGS. 13A and 13B is reduced, and the object 1330 is surrounded by a visual assistance image 1333 larger than the object 1330 . That is, the object 1330 and the visual assistance image 1333 are superimposed and displayed.

From the viewpoint of the user 220, if the size of the object 1330 is reduced, it becomes difficult to see the object 1330 (visibility is reduced). 1333 is placed.

For example, assume that the length of one side of the object 1330 is 5.0 cm, the length of one side of the visual assistance image 1333 is 20.0 cm, and the diameter of the sensor portion 1322 of the avatar image 1320 is 2.0 cm. Then, the error (divergence) between the target distance 1311 and the movement distance 1312 is reduced to around 2.0 cm.

As a result, the deviation (error) between the target distance 1311 and the motion distance 1312 can be reduced while preventing the visibility of the object 1330 from deteriorating due to the difference in gradation and size of the object 1330 and the visual recognition auxiliary image 1333. can be done. Also, as a secondary effect, the degree of experience increases when the avatar image 1320 is brought into contact with the object 1330 . That is, for the user 220, the feeling when touching the object 1330 becomes clearer, and the joy of achieving the goal of rehabilitation increases.

FIG. 13D is a diagram for explaining the placement position of the visual recognition aid image in the rehabilitation support system according to this embodiment. In FIG. 13C , the object 1330 as the target image is displayed so as to be included in the visual assistance image 1333 and is arranged near the center of the visual assistance image 1333 .

However, as shown in FIG. 13D, the object 1330 may be placed near the lower side in front of the viewing aid image 1333 . That is, the object 1330 may be placed on the near side as viewed from the user 220 . In this way, the object 1330 may be placed anywhere in the visual aid image 1333 as long as it is displayed as if it were inside the visual aid image 1333 . As described above, when the size of the object 1330 is reduced to reduce the divergence between the target distance 1311 and the movement distance 1312, the visibility of the object 1330 is reduced. Therefore, in order to improve the visibility of the object 1330 , a visibility assistance image 1333 larger than the object 1330 is displayed around the object 1330 to compensate for the deterioration of the visibility of the object 1330 . Note that the visual recognition auxiliary image 1333 for improving the visibility of the object 1330 is not limited to a cube obtained by enlarging the magnification of the cubic object 1330 as shown here.

Next, another shape of the visual assistance image 1333 will be described with reference to FIGS. 13E to 13I. FIG. 13E is a diagram illustrating another example of the visual recognition aid image in the rehabilitation support system according to this embodiment. FIG. 13F is a diagram illustrating still another example of the visual recognition aid image in the rehabilitation support system according to this embodiment. FIG. 13G is a diagram illustrating still another example of the visual recognition aid image in the rehabilitation support system according to this embodiment. FIG. 13H is a diagram illustrating still another example of the visual recognition aid image in the rehabilitation support system according to this embodiment. FIG. 13I is a diagram illustrating still another example of the visual recognition aid image in the rehabilitation support system according to this embodiment.

As shown in FIG. 13E, the visual recognition aid image 1340 may be, for example, an arrow shape indicating the existing position of the object 1330 . The object 1330 is not included in the arrow-shaped visual assistance image 1340 . In other words, the object 1330 as the target image and the visual recognition aid image 1340 are not superimposed and displayed, but the visual recognition aid image 1340 is arranged outside the object 1330 . In this way, with the arrow-shaped visual recognition assistance image 1340, the user 220 can easily recognize that the object 1330 exists beyond the tip of the arrow.

As shown in FIG. 13F, viewing aid image 1350 may be shaped to attract the attention of user 220 . Note that the shape that attracts the attention of the user 220 is not limited to the shape shown in FIG. 13F, and may be, for example, a star shape, a cross shape, a polygonal shape, or the like. Also, the vertical line 1351 and the horizontal line 1352 may be displayed together to indicate that the object 1330 is placed in the center of the vertical line 1351 and the horizontal line 1352 .

As shown in FIG. 13G , visual aid image 1360 may be a dashed-dotted line extending from sensor portion 1322 of avatar image 1320 toward object 1330 . Note that the visual recognition auxiliary image 1360 is not limited to a one-dot chain line, and may be, for example, a straight line, a two-dot chain line, a dotted line, or the like.

The user 220 can recognize the position of the object 1330 by visually recognizing the object 1330 by moving the line of sight along the dashed-dotted line of the visual recognition assistance image 1360 as a guideline. Furthermore, by moving the avatar image 1320 along this dashed line, the avatar image 1320 can be caused to touch the object 1330 . By displaying the visual recognition auxiliary image 1333 together with the visual recognition auxiliary image 1370, the visibility of the object 1330 is further improved.

As shown in FIG. 13H , visual recognition aid image 1370 may be a plurality of arrows arranged in a straight line from sensor unit 1322 toward object 1330 . Using these arrows as guidelines, the user 220 can visually recognize the object 1330 and recognize the position of the object 1330 by moving his line of sight along the arrows. Furthermore, by moving the avatar image 1320 along the plurality of arrows, the avatar image 1320 can be caused to touch the object 1330 . The visibility of the object 1330 is further improved by displaying a cubic visual recognition auxiliary image 1333 together with the visual recognition auxiliary image 1370 .

As shown in FIG. 13I, a plurality of spherical viewing aid images 1380 are arranged at the top, bottom, left, and right of object 1330 . That is, in FIG. 13I, a plurality of spherical viewing aid images 1380 are arranged around the object 1330 such that the object 1330 is placed in the center of the four viewing aid images 1380 . Note that the shape of the visual recognition assistance image 1380 is not limited to a spherical shape, and may be, for example, a triangular shape, a rectangular shape, a polygonal shape, a star shape, or the like.

FIG. 13J is a diagram illustrating still another example of the visual recognition aid image in the rehabilitation support system according to this embodiment. The rehabilitation support server may change the size of the visual recognition assistance image 1333 displayed on the display unit 1402 according to the progress of rehabilitation of the user 220, for example. For example, the rehabilitation support server displays a large visual recognition auxiliary image 1333 at the beginning of rehabilitation. Then, when the rehabilitation of the user 220 has progressed, the size of the visual recognition auxiliary image 1333 may be reduced according to the progress of the rehabilitation.

Further, the rehabilitation support server may change the size of the visual recognition auxiliary image 1333 according to, for example, the visual acuity of the user 220 instead of according to the progress of rehabilitation of the user 220 . That is, the rehabilitation support server may display a large visual recognition auxiliary image 1333 for the user 220 with poor eyesight, and may display a small visual recognition auxiliary image 1333 for the user 220 with relatively good eyesight. In this way, the rehabilitation support server may display a visual recognition aid image having a size corresponding to the visual acuity of the user 220 .

Further, for example, when the user 220 has dementia, the rehabilitation support server may display the visual recognition auxiliary image 1333 having a size that matches the progress of dementia and cognitive function. Note that the size of the visual recognition auxiliary image 1333 may be automatically changed by the rehabilitation support server, or may be manually changed by an operator such as a doctor who operates the rehabilitation support system, and may be changed by the user 220 .

FIG. 14 is a block diagram illustrating the configuration of the rehabilitation support system according to this embodiment. Rehabilitation support system 1400 has rehabilitation support server 1401 and display unit 1402 . Elements included in the rehabilitation support system 1400 are not limited to these. The rehabilitation support server 1401 has a motion detection unit 1411 , a display control unit 1412 , an evaluation unit 1413 and an update unit 1414 .

The motion detection unit 1411 acquires the position of the controller held by the user 220 and the position of the head-mounted display worn by the user 220, and detects the movement of the user 220 ( rehabilitation movements).

The display control unit 1412 displays an avatar image 1320 that moves according to the detected rehabilitation action, a target image that indicates the goal of the rehabilitation action, and at least one visual recognition auxiliary image 1333 that improves the visibility of the target image. etc.

The display control unit 1412 causes the target image and the visual recognition auxiliary image 1333 to be superimposed and displayed. For example, the size of the target image is made smaller than the size of the visual assistance image 1333 so that the target image is displayed as if it is included in the visual assistance image 1333 .

The display control unit 1412 may display the target image near the center of the visual recognition auxiliary image 1333, for example. In addition, the display control unit 1412 displays the target image not in the vicinity of the center of the visual assistance image 1333 but in the visual assistance image 1333 and closer to the avatar image 1320 , that is, on the front side of the user 220 . You may let

The display control unit 1412 may display the object 1330 and the visual recognition auxiliary image 1333 so as to be identifiable. Specifically, for example, the gradation of the object 1330 is displayed darker than the gradation of the visual assistance image 1333 . As a result, the user 220 can reliably recognize the object 1330 because the object 1330 is displayed dark, and a difference in contrast with the visual recognition aid image 1333 displayed light is generated. Note that the method of applying gradation to the object 1330 and the visual recognition assistance image 1333 is not limited to the method shown here. For example, the gradation may be applied so that even the user 220 with weak eyesight can reliably distinguish between the object 1330 and the visual recognition auxiliary image 1333 .

In addition, the display control unit 1412 displays the object 1330 and the visual recognition aid image 1333 in different colors so that the object 1330 and the visual recognition aid image 1333 can be displayed in a distinguishable manner. For example, the display control unit 1412 colors the object 1330 in a dark color and colors the visual recognition auxiliary image 1333 in a light color, but the combination (pattern) of coloring is not limited to this. For example, even if the user 220 has color vision deficiency (color blindness), the object 1330 and the visual recognition aid image 1333 may be colored with a combination of colors that can be reliably identified. In addition, the display control unit 1412 may perform coloring that can correspond to various types of users 220 such as amblyopia, narrow vision, and color blindness. Note that the color to be used for the object 1330 and the visual recognition aid image 1333 may be selected by the user 220 or by an operator such as a doctor.

Although the gradation and color of the object 1330 and the visual assistance image 1333 have been described here, the gradation and color of the other visual assistance images 1340, 1350, 1360, 1370, and 1380 may be similarly changed. .

Furthermore, the display control unit 1412 controls the change of display of the visual recognition assistance image 1333 according to at least one of the visual acuity of the user 220 and the evaluation result by the evaluation unit 1413 . For example, the display control unit 1412 changes the size of the visual assistance image 1333 according to the visual acuity of the user 220, progress of rehabilitation of the user 220, progress of dementia of the user 220, and the like.

The evaluation unit 1413 compares the rehabilitation motion detected by the motion detection unit 1411 with the target position represented by the object 1330 as the target image displayed by the display control unit 1412 to evaluate the rehabilitation ability of the user 220 .

The updating unit 1414 updates the target position represented by the object 1330 according to the evaluation result by the evaluating unit 1413 .

The display unit 1402 displays the target image, the visibility auxiliary image, and the like under the control of the display control unit 1412 . The display unit 1402 is a head-mounted display, display, screen, or the like, but is not limited to these.

FIG. 15A is a diagram illustrating an example of a patient table held by a rehabilitation support server included in the rehabilitation support system according to this embodiment. The patient table 1501 stores attribute information 1512 , rehabilitation target 1513 , current level 1514 and rehabilitation menu 1515 in association with patient ID (Identifier) 1511 . A patient ID 1511 is an identifier for identifying a patient. The attribute information 1512 is information indicating patient attributes such as age and sex. The rehabilitation target 1513 is data indicating which part of the patient's body is to be rehabilitated, for example, data representing a body part such as an arm or a leg.

Current level 1514 is data indicating the patient's current level of rehabilitation. In other words, it is data indicating the progress of patient's rehabilitation. This data is divided into a plurality of ranks, such as A rank and B rank, from the initial stage to the final stage of rehabilitation. Note that the classification of rehabilitation levels is not limited to this. The rehabilitation menu 1515 is information on the rehabilitation menu that the patient should receive.

Next, FIG. 15B is a diagram illustrating an example of a display parameter table possessed by a rehabilitation support server included in the rehabilitation support system according to this embodiment. The display parameter table 1502 stores a target image ID 1521 , a visual assistance image ID 1522 and a display parameter 1523 in association with the rehabilitation menu 1515 .

The target image ID 1521 is an identifier that identifies the object 1330 displayed on the display unit 1402 . The visual recognition auxiliary image ID is an identifier for identifying the visual recognition auxiliary images 1333 , 1340 , 1350 , 1360 , 1370 , and 1380 displayed on the display unit 1402 . The display parameter 1523 is a parameter necessary for displaying the object 1330 and the visual assistance images 1333, 1340, 1350, 1360, 1370, and 1380 on the display unit 1402. FIG. The display parameters 1523 include, for example, information such as position and magnification, but the information included in the display parameters 1523 is not limited to these.

FIG. 15C is a diagram illustrating an example of an image table held by a rehabilitation support server included in the rehabilitation support system according to this embodiment. Image table 1503 stores image data 1532 , display position 1533 and magnification 1534 in association with image type 1531 . Items stored in the image table 1503 are not limited to these.

The image type 1531 is information for distinguishing whether an image to be displayed is a target image or a visual recognition auxiliary image. The image data 1532 is image data such as the object 1330 displayed on the display unit 1402 and the visual recognition auxiliary image 1333, and includes image data in various image file formats. The display position 1533 is data representing the position in the display unit 1402 to be displayed, and is, for example, a set of data of (X coordinate position, Y coordinate position, Z coordinate position). The magnification 1534 is data for determining in what size the object 1330, visual recognition auxiliary image 1333, and the like are to be displayed on the display unit 1402. FIG.

Then, the rehabilitation support server 1401 refers to these tables 1501 , 1502 and 1503 and causes the display unit 1402 to display visual recognition auxiliary images 1333 , 1340 , 1350 , 1360 , 1370 and 1380 .

FIG. 16 is a block diagram illustrating the hardware configuration of a rehabilitation support server included in the rehabilitation support system according to this embodiment. A CPU (Central Processing Unit) 1610 is a processor for arithmetic control, and implements a functional component of the rehabilitation support server 1401 in FIG. 14 by executing a program. A ROM (Read Only Memory) 1620 stores fixed data such as initial data and programs, and other programs. Network interface 1630 also communicates with other devices and the like via a network. Note that the number of CPUs 1610 is not limited to one, and may include a plurality of CPUs or a GPU (Graphics Processing Unit) for image processing. Moreover, the network interface 1630 preferably has a CPU independent of the CPU 1610 and writes or reads transmission/reception data in a RAM (Random Access Memory) 1640 area. It is also desirable to provide a DMAC (Direct Memory Access Controller) for transferring data between RAM 1640 and storage 1650 (not shown). Further, input/output interface 1660 preferably has a CPU independent of CPU 1610 to write or read input/output data to/from RAM 1640 area. Therefore, CPU 1610 recognizes that data has been received or transferred to RAM 1640 and processes the data. Also, the CPU 1610 prepares the processing result in the RAM 640 , and leaves the subsequent transmission or transfer to the network interface 1630 , DMAC, or input/output interface 1660 .

A RAM 1640 is a random access memory used by the CPU 1610 as a work area for temporary storage. The RAM 1640 has an area for storing data necessary for implementing this embodiment. The patient data 1641 is data related to patients undergoing rehabilitation using the rehabilitation support system. The image data 1642 is data of the object 1330 as the target image displayed on the display unit 1402, the visual recognition auxiliary image 1333, and the like. The display position 1643 is data indicating at which position on the display unit 1402 the object 1330 and the visual recognition auxiliary image 1333 are to be displayed. The magnification 1644 is data indicating how large an image such as the object 1330 and the visual recognition auxiliary image 1333 should be displayed on the display unit 1402 . These data are read from the patient table 1501, the display parameter table 1502 and the image table 1503, for example.

The input/output data 1645 is data input/output via the input/output interface 1660 . Transmitted/received data 1646 is data transmitted/received via network interface 1630 . The RAM 1640 also has an application execution area 1647 for executing various application modules.

The storage 1650 stores a database, various parameters, or the following data or programs necessary for realizing this embodiment. Storage 1650 stores patient table 1501 , display parameter table 1502 and image table 1503 . The patient table 1501 is a table for managing the relationship between the patient ID 1511 and the attribute information 1512 shown in FIG. 15A. The display parameter table 1502 is a table for managing the relationship between the rehabilitation menu 1515 and the display parameters 1523 shown in FIG. 15B. The image table 1503 is a table for managing the relationship between the image type 1531 and the image data 1532 shown in FIG. 15C.

Storage 1650 further stores motion detection module 1651 , display control module 1652 , evaluation module 1653 and update module 1654 .

The motion detection module 1651 is a module that detects a rehabilitation motion of the user 220 . The display control module 1652 is a module that causes the display unit 1402 to display an avatar image 1320 , an object 1330 as a target image, a visual assistance image 1333 that improves the visibility of the object 1330 , and the like. Evaluation module 1653 is a module that evaluates the rehabilitation ability of user 220 . The update module 1654 is a module that updates the target position represented by the target image according to the evaluation result. These modules 1651 to 1654 are read by the CPU 1610 into the application execution area 1647 of the RAM 1640 and executed. The control program 1655 is a program for controlling the rehabilitation support server 1401 as a whole.

The input/output interface 1660 interfaces input/output data with input/output devices. A display unit 1661 and an operation unit 1662 are connected to the input/output interface 1660 . A storage medium 1664 may also be connected to the input/output interface 1660 . Further, a speaker 1663 as an audio output unit, a microphone as an audio input unit, or a GPS (Global Positioning System) position determination unit may be connected. Note that the RAM 1640 and the storage 1650 shown in FIG. 16 do not show programs or data relating to general-purpose functions possessed by the rehabilitation support server 1401 or other realizable functions.

FIG. 17A is a flowchart for explaining the processing procedure of the rehabilitation support server included in the rehabilitation support system according to this embodiment. FIG. 17B is a flowchart for explaining the processing procedure for visual recognition auxiliary image display of the rehabilitation support server included in the rehabilitation support system according to the present embodiment. These flowcharts are executed by the CPU 1610 using the RAM 1640, and realize the functional configuration part of the rehabilitation support server 1401 in FIG.

In step S1701, the rehabilitation support server 1401 causes the display unit 1402 or the like to display a visual recognition assistance image.

In step S1721, the rehabilitation support server 1401 uses the rehabilitation support system 1400 to acquire patient information such as attributes of the patient undergoing rehabilitation and what kind of rehabilitation menu is received.

In step S<b>1723 , the rehabilitation support server 1401 acquires display parameters necessary for displaying the visual recognition assistance image 1333 to be displayed on the display unit 1402 on the display unit 1402 . The acquired display parameters are, for example, parameters related to the position and magnification of the visual recognition auxiliary image 1333 and the like.

In step S<b>1725 , the rehabilitation support server 1401 acquires image data of the visual recognition auxiliary image 1333 . In step S<b>1727 , the rehabilitation support server 1401 displays the visual recognition auxiliary image 1333 and the like on the display unit 1402 .

In step S1729, the rehabilitation support server 1401 determines whether it is necessary to change the display of the visual recognition auxiliary image 1333 or the like. If display change is not necessary (NO in step S1729), the rehabilitation support server 1401 terminates the process. If the display needs to be changed (YES in step S1729), the rehabilitation support server 1401 proceeds to the next step.

In step S1731, the rehabilitation support server 1401 changes the size of the visual recognition assistance image 1333 according to the user's 220 visual acuity and the evaluation result of the user's 220 rehabilitation ability.

According to the present embodiment, even if the size of the target image is reduced in order to reduce the divergence between the target distance and the exercise distance, the target distance and the exercise distance are brought closer together while maintaining the visibility of the target image. can increase the effectiveness of rehabilitation. Also, the user can feel a sense of satisfaction when the target image is touched.

[Fourth Embodiment]
Next, a rehabilitation support system according to a fourth embodiment of the present invention will be explained using FIGS. 18 to 21. FIG. FIG. 18 is a block diagram illustrating the configuration of a rehabilitation support system according to this embodiment. The rehabilitation support system according to this embodiment differs from that of the third embodiment in that it has a sound output unit. Since other configurations and operations are the same as those of the second and third embodiments, the same configurations and operations are denoted by the same reference numerals, and detailed description thereof will be omitted.

A rehabilitation support system 1800 includes a rehabilitation support server 1801 and a sound output unit 1802 . The rehabilitation support server 1801 has a sound output control section 1811 . The sound output control unit 1811 controls sound output according to the positional relationship between the object 1330 as the target image and the avatar image 1320 . The sound whose output is controlled by the sound output control section 1811 is output from the sound output section 1802 .

For example, when the object 1330 is falling downward, the sound output control unit 1811 outputs sound based on the distance between the object 1330 and the avatar image 1320, that is, the positional relationship.

As the distance between object 1330 and avatar image 1320 decreases, that is, as object 1330 approaches avatar image 1320, the output sound may be of higher frequency. Similarly, as the distance between object 1330 and avatar image 1320 increases, that is, as object 1330 moves away from avatar image 1320, the output sound may have a lower frequency. That is, like the Doppler effect, an acoustic effect is expressed in which different frequencies of sound (waves) are observed according to the distance between the object 1330 (sound source) and the avatar image 1320 (user 220 (observer)). You may Note that instead of changing the frequency of the output sound, the volume of the output sound may be increased or decreased according to the distance between the object 1330 and the avatar image 1320 .

Also, the position of the object 1330 may be taught to the user 220 by outputting a sound from the sound output control section 1811 . In other words, the position of the object 1330 is taught using the sense of hearing of the user 220 .

For example, consider a case where user 220 wears headphones when using rehabilitation support system 1800 . When the object 1330 as the target image is on the right side of the avatar image 1320 (user 220), the rehabilitation support server 1801 outputs sound from the right ear headphone. Similarly, when the object 1330 is to the left of the avatar image 1320 (user 220), sound is output from the left ear headphone. Accordingly, the user 220 can determine whether the object 1330 is on the right side or the left side of the user 220 from the direction of the sound. Additionally, when object 1330 is in front of avatar image 1320 (user 220), both headphones emit sound.

In the above description, the position of the object 1330 is taught using the user's 220 senses of sight and hearing. The location of 1330 may be taught.

For example, a sensor may be placed on the tongue of the user 220 so that the user 220 feels a taste according to the position of the object 1330 . Alternatively, a controller held by the user 220, headphones worn by the user 220, a head-mounted display, or the like may be vibrated. In other words, the position of the object 1330 may be taught using the tactile sense of the user 220 .

FIG. 19 is a diagram illustrating an example of a sound table possessed by a rehabilitation support server included in the rehabilitation support system according to this embodiment. The sound table 1901 stores sound data 1911 in association with the image type 1531 . Then, the rehabilitation support server 1801 refers to the sound table 1901 and controls sounds to be output.

FIG. 20 is a diagram illustrating the hardware configuration of a rehabilitation support server included in the rehabilitation support system according to this embodiment. A RAM 2040 is a random access memory used by the CPU 1610 as a work area for temporary storage. The RAM 2040 has an area for storing data necessary for implementing the present embodiment. The sound data 2041 is data relating to sounds to be output. This data is then read from the sound table 1901, for example.

The storage 2050 stores a database, various parameters, or the following data or programs necessary for realizing this embodiment. Storage 2050 stores sound table 1901 . The sound table 1901 is a table for managing the relationship between the image type 1531 and the sound data 1911 shown in FIG.

Storage 2050 further stores sound output control module 2051 . The sound output control module 2051 is a module that controls sound output according to the positional relationship between the object 1330 as the target image and the avatar image 1320 . This module 2051 is read into the application execution area 1647 of the RAM 2040 by the CPU 1610 and executed. It should be noted that the RAM 2040 and the storage 2050 shown in FIG. 20 do not show programs or data relating to general-purpose functions of the rehabilitation support server 1801 or other realizable functions.

FIG. 21A is a flowchart for explaining the processing procedure of the rehabilitation support server included in the rehabilitation support system according to this embodiment. FIG. 21B is a flowchart for explaining the sound output control processing procedure of the rehabilitation support server included in the rehabilitation support system according to the present embodiment. These flow charts are executed by the CPU 1610 using the RAM 2040 to implement the functional components of the rehabilitation support server 1801 in FIG.

In step S2101, the rehabilitation support server 1801 controls sound output. In step S<b>2121 , the rehabilitation support rehabilitation support server 1801 acquires the position of the avatar image 1320 . In step S<b>2123 , the rehabilitation support server 1801 acquires the position of the object 1330 . In step S<b>2125 , the rehabilitation support server 1801 determines the positional relationship between the avatar image 1320 and the object 1330 . In step S2127, the rehabilitation support server 1801 controls sound output according to the determined positional relationship.

According to this embodiment, rehabilitation is performed using the user's sense of hearing in addition to the user's sense of sight, so the user can more easily visually recognize the object, and the effect obtained by rehabilitation can be further enhanced. Also, the user can grasp the position of the object not only visually but also aurally. In addition, since sound is output, even a user with weak eyesight can receive rehabilitation according to the present embodiment.

[Fifth embodiment]
Next, a system according to a fifth embodiment of the present invention will be explained using FIGS. 22 to 24. FIG. The rehabilitation support system according to this embodiment differs from the third embodiment in that goals are clarified using a plurality of parameters. Since other configurations and operations are similar to those of the third embodiment, the same configurations and operations are denoted by the same reference numerals, and detailed description thereof will be omitted.

FIG. 22 is a diagram showing in detail the contents of the target DB 216 in this embodiment. As shown in FIG. 22, goals to be achieved in rehabilitation are set for each patient. First, the patient is assessed separately for motor and cognitive levels as attributes. If the exercise level or cognitive level is high, it will be evaluated as A, if it is low, it will be evaluated as C, and in the meantime it will be evaluated as B. For example, patient ID 001 has a high exercise level but a low cognitive level. In this case, the distance to the object, that is, the maximum reachable distance of the hand, is far (level 5 out of 5 in this example), the appearance range of the object is rather narrow (level 3 in this example), and the object moves The speed is slow (here level 2 as an example). Furthermore, the object appearance interval is long (level 1 as an example here), and both the object size and sensor size are large (level 1 as an example here).

On the other hand, in the case of patient ID002, the exercise level is low, but the cognitive level is high. In this case, the distance to the object, that is, the maximum reachable distance of the hand, is short (here, level 2 on a scale of 5), the appearance range of the object is wide (here, level 5, for example), and the movement speed of the object is is slow (level 1 here as an example). On the other hand, the object appearance interval is short (here, 5 out of 5 as an example), and both the object size and the sensor size are small (here, as an example, 5 out of 5).

Patient ID 003 has low motor and cognitive levels. In this case, the distance to the object, that is, the maximum reachable distance of the hand is short (here, level 1 on a scale of 5), the appearance range of the object is narrow (here, level 1), and the object's moving speed is Slow (level 1 here as an example). Furthermore, the object appearance interval is long (here, 1 out of 5 steps as an example), and both the object size and the sensor size are large (here, as an example, 1 out of 5 steps).

In this way, the parameters are variously changed according to the attributes of the patient.

Generally, as shown in FIG. 23, the relationship between motor function, cognitive function, and various parameters is expected, but the rehabilitation support system of the present invention does not set parameters limited to this relationship. Instead, depending on the condition and ability of each patient, various parameters (distance, range, speed, interval and size) can be changed to seek the rehabilitation intensity that is just right for that patient.

FIG. 24 shows an example screen 2400 displayed on the head mounted display 233 by the display control unit 212 in this embodiment. The display control unit 212 displays the object 2411 superimposed on the background image 2401 . The display control unit 212 gradually changes the display position and size so that the sweet potato-shaped object 2411 appears to be falling downward from above the head of the user 220 . Here, as a preliminary state for the object 2411 to appear, an image 2412 of a farmer crouching is displayed. When the user recognizes the peasant 2412 who is crouching, the user predicts that the object 2411 will appear next from that direction. In addition, since the farmer 2413 throws the object 2411 upward, the user naturally looks up at the object 2411 and looks up. In other words, it is possible to give an instruction that naturally causes the user to be aware of the upward direction, rather than a linguistic instruction.

Thereafter, the user 220 moves the basket-shaped avatar image 311 (not shown in FIG. 24) by moving the controllers 234 and 235 in accordance with the position of the fallen object 2411 . If the fallen object 2411 enters the basket, the mission is cleared, and the requested rehabilitation action is completed. A child may be displayed to help collect the sweet potatoes so as to ease the user's mental shock and stress when the object 2411 cannot be put in the basket. A supplementary indicator 2414 may be displayed to inform the user of the appearance position of the farmer.

In this example, various parameters (distance to the falling sweet potato, range in which the farmer appears, speed at which the sweet potato falls, interval at which the farmer throws the sweet potato, sweet potato and basket size) can be varied to set the rehabilitation intensity appropriate for the user.

FIG. 25 shows another screen example 2500 displayed on the head mounted display 233 by the display control unit 212 in this embodiment. Here, the display control unit 212 displays the object 2511 superimposed on the background image 2501 . The display control unit 212 gradually changes the display position and size so that an apple-shaped object 2511 in this example appears to be falling downward from above the user 220 . As a preliminary state in which an apple-shaped object 2511 falls, an image 2512 of a monkey shaking a tree is displayed. When the user recognizes the monkey, the user can predict that the object 2511 will fall next from that direction. Thereafter, the user 220 moves the basket-shaped avatar image 311 (not shown in FIG. 25) by moving the controllers 234 and 235 in accordance with the position of the fallen object 2511 . If the fallen object 2511 enters the basket, the mission is cleared, and the requested rehabilitation action is completed. Even when the object 2511 cannot be put in the basket, a child helping to collect apples may be displayed so as to relieve the user's mental shock and stress.

FIG. 26 shows another screen example 2600 displayed on the head mounted display 233 by the display control unit 212 in this embodiment. Here, the display control unit 212 displays the object 2611 superimposed on the background image 2601 . The display control unit 212 gradually changes the display position and size so that an object 2611 in the shape of Dracula in this example approaches the user 220 from the front. The user moves the cross-shaped avatar image 2613 by moving the controllers 234 and 235 in accordance with the position of the approaching Dracula. If the cross hits Dracula, the mission is cleared, and the required rehabilitation action has been completed. A child may be displayed to help relieve the user's mental shock or stress even if he or she is unable to place the cross on Dracula.

According to the above example, it is possible to give a task to both the user's motor function and cognitive function. For example, by displaying a preliminary state such as a farmer crouching or a monkey coming out, the user's cognitive function can be challenged. It can give challenges to functions. That is, more effective rehabilitation is performed by having the patient perform both the exercise rehabilitation action of stretching the arm and the cognitive rehabilitation action of predicting the appearance position of the next object and moving the line of sight. becomes possible.

24 to 26, the visual recognition aid image described in the third embodiment may be additionally displayed. You can change the size. In addition, the avatar image as a sensor touches only the visual assistance image (outline) and the case where it touches the object, and a stepwise evaluation is performed (e.g., good if only the outline is touched, very good if touched to the middle). may

[Other embodiments]
Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. For example, the display device is not limited to a head mount, and may be a large screen. The controller is also not limited to the grip type, and may be a wearable sensor.

Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. Also, any system or apparatus that combines separate features included in each embodiment is also included in the scope of the present invention.

Further, the present invention may be applied to a system composed of a plurality of devices, or may be applied to a single device. Furthermore, the present invention is also applicable when an information processing program that implements the functions of the embodiments is directly or remotely supplied to a system or apparatus. Therefore, in order to implement the functions of the present invention on a computer, a program installed in a computer, a medium storing the program, and a WWW (World Wide Web) server from which the program is downloaded are also included in the scope of the present invention. . In particular, non-transitory computer readable media containing programs that cause a computer to perform at least the processing steps included in the above-described embodiments are included within the scope of the present invention.

Claims (5)

a motion detector that detects a user's rehabilitation motion;
a display control unit that displays an avatar image that moves according to the detected rehabilitation action and a target image that indicates a goal of the rehabilitation action;
an evaluation unit that evaluates that the target image has been recovered by the user's rehabilitation action when the position of the avatar image and the target position represented by the target image moving downward overlap ;
with
The display control unit assists in recovering the target image moving from top to bottom so as to soften the user's shock when the target image cannot be recovered by the user's rehabilitation action. A rehabilitation support system that displays images . The display control unit has, as setting parameters, a distance to the target image, a direction of the target image, a speed at which the target image moves, an interval at which the target image appears, a size of the target image, and a size of the avatar image. 2. The rehabilitation support system according to claim 1, wherein the nature and intensity of said rehabilitation movement are changed by changing at least one of strength. The display control unit displays an avatar image that moves according to the detected rehabilitation action, a contour of a target image that indicates a target of the rehabilitation action, and a contour of a visual assistance image that improves visibility of the target image. The rehabilitation support system according to claim 1 or 2. a motion detection step of detecting a user's rehabilitation motion;
a display control step of displaying an avatar image that moves according to the detected rehabilitation action and a target image that indicates a goal of the rehabilitation action;
an evaluation step of evaluating that the target image has been recovered by the user's rehabilitation action when the position of the avatar image and the target position represented by the target image moving downward overlap ;
including
In the display control step, for the target image that moves from top to bottom, if the target image cannot be retrieved by the user's rehabilitation action, the user is assisted in retrieving the target image so as to soften the user's shock. A rehabilitation support method for displaying images . a motion detection step of detecting a user's rehabilitation motion;
a display control step of displaying an avatar image that moves according to the detected rehabilitation action and a target image that indicates a goal of the rehabilitation action;
an evaluation step of evaluating that the target image has been recovered by the user's rehabilitation action when the position of the avatar image and the target position represented by the target image moving downward overlap ;
A rehabilitation support program that causes a computer to execute
In the display control step, for the target image moving from top to bottom, if the target image cannot be retrieved by the user's rehabilitation movement, the user is assisted in retrieving the target image so as to soften the user's shock. Rehabilitation support program that displays images .

Images