JP7448943B2 - Rehabilitation support device, method and program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act 1. Publication date October 24, 2018 Publication Progress in Rehabilitation Medicine Volume 3 (2018) 2. Date of sale: March 26, 2019 Place of sale: mediVR Co., Ltd. Osaka Head Office

The present invention relates to a rehabilitation support device, its method, and program.

In the above technical field, Patent Document 1 discloses a system for supporting rehabilitation.

Japanese Patent Application Publication No. 2015-228957

However, with the technology described in the above-mentioned document, it was not possible to avoid danger to the user undergoing rehabilitation. When wearing a head-mounted display, users cannot see their surroundings, so if they collide with an object in real space, they will not be able to protect themselves and there is a high possibility that the user will suffer an injury such as a sprain.

An object of the present invention is to provide a technique for solving the above-mentioned problems.

In order to achieve the above object, the present invention includes:
A device that supports rehabilitation of a user,
a detection unit that detects the direction of the head of the user wearing the head-mounted display;
a first detection unit that detects the position of a floor surface or a wall surface in real space around the user;
a display control unit that generates a target object for rehabilitation that is visible to the user in a virtual space and displays it on the head-mounted display according to the orientation of the user's head detected by the detection unit;
Equipped with
The display control unit does not display the target object in a place where there is a risk of falling or disease aggravation, depending on the movement level of each user .

According to the present invention, danger to the user undergoing rehabilitation or the surroundings can be avoided.

1 is a diagram showing the configuration of a rehabilitation support device according to a first embodiment of the present invention. It is a figure showing the composition of the rehabilitation support device concerning a 2nd embodiment of the present invention. It is a figure showing an example of display control of a rehabilitation support device concerning a 2nd embodiment of the present invention. It is a flowchart which shows the flow of processing of the rehabilitation support device concerning a 2nd embodiment of the present invention. It is a figure showing the composition of the rehabilitation support device concerning a 3rd embodiment of the present invention. It is a figure showing the composition of the rehabilitation support device concerning a 4th embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described in detail below by way of example with reference to the drawings. However, the components described in the following embodiments are merely examples, and the technical scope of the present invention is not intended to be limited thereto.

[First embodiment]
A rehabilitation support device 100 as a first embodiment of the present invention will be described using FIG. 1. The rehabilitation support device 100 is a device that supports and evaluates the rehabilitation of the user 130.

As shown in FIG. 1, rehabilitation support device 100 includes a detection section 101 and a display control section 102.

The detection unit 101 detects the direction of the head of the user 130 wearing the head mounted display 120.

The display control unit 102 generates a rehabilitation target object 140 in the virtual space 150 that is visible to the user 130 and displays it on the head-mounted display 120 according to the direction of the user's 130 head detected by the detection unit 101.

The display control unit 102 further controls the display on the head-mounted display 120 in order to avoid the risk of the user 130 interfering with objects in the real space (for example, the floor 160 or the wall 170).

For example, consider a system in which the user 130 touches an object 140 falling in the virtual space 150 using the controllers 134 and 135, thereby realizing a rehabilitation movement of the user 130. In this case, for example, by creating a program that erases the object 140 at 20 cm above the floor, it is possible to prevent the user from hitting the controller or hand on the floor 160. Of course, the number 20 cm is just an example, and it may be 25 cm or 30 cm.

The display control for the user 130 and his/her surroundings to avoid danger is not limited to the control of "erasing objects in a predetermined range", and various other methods can be considered; therefore, the present invention is not limited to such control. It's not something you can do. Since the target object disappears 20 cm above the floor, it is possible to prevent the risk of injury from hitting the controller on the floor or from leaning forward and falling.

[Second embodiment]
A rehabilitation support system 200 according to a second embodiment of the present invention will be described using FIG. 2. FIG. 2 is a diagram for explaining the configuration of the rehabilitation support system 200 according to this embodiment.

As shown in FIG. 2, the rehabilitation support system 200 includes a rehabilitation support device 210, two base stations 231 and 232, a head mounted display 233, and two controllers 234 and 235. The user 220 performs rehabilitation motions while sitting on the chair 221 by tilting the upper body, twisting, and extending the arms in various directions in accordance with the display on the head-mounted display 233. Although this embodiment will be explained on the assumption that rehabilitation is performed while sitting on a chair, the present invention is not limited to this, and can be performed while standing, walking, running, etc. You can also do it while sleeping. Furthermore, even if you use some type of equipment to maintain a specific posture or condition, or use a treadmill or other walking support device to maintain a steady position at any angle or incline while walking, Or you can do it while running.

The two base stations 231 and 232 detect the position and movement of the head mounted display 233 and the positions and movements of the controllers 234 and 235 and send them to the rehabilitation support device 210. Rehabilitation support device 210 controls the display of head-mounted display 233 while evaluating the rehabilitation motion of user 220 based on the position and movement of head-mounted display 233 and the positions and movements of controllers 234 and 235. The head mounted display 233 may be of a non-transparent type, a video see-through type, or an optical see-through type. A headphone speaker 236 is attached to the head mounted display 233, and the rehabilitation support device 210 outputs audio from the headphone speaker 236 according to the evaluation result of the rehabilitation motion of the user 220. Note that the audio output method is not limited to headphone speakers, and may also use an external audio output device, bone conduction earphones, or the like.

In this embodiment, the controllers 234 and 235 held in the hands of the user 220 and the base stations 231 and 232 are shown as examples of sensors for detecting the motion of the user 220, but the present invention is not limited to this. It's not a thing. Cameras (including depth sensors) that detect the position of the user's hand through image processing, sensors that detect the position of the user's hand based on temperature, wristwatch-type wearable devices that are worn on the user's arm, and motion sensors. A capture or the like may be included in the motion detection section.

The rehabilitation support device 210 includes a motion detection section 211 , a display control section 212 , an obstacle detection section 213 , an evaluation section 214 , an update section 215 , a task set database 216 , and a feedback section 217 .

The motion detection unit 211 acquires the positions of the controllers 234 and 235 held in the hands of the user 220 via the base stations 231 and 232, and detects the rehabilitation motion of the user 220 based on a change in the position of the user's 220 hands.

The obstacle detection unit 213 detects an object that becomes an obstacle for the user 220 to perform rehabilitation operations. For example, the obstacle detection unit 213 detects the position of at least one of a floor surface 260 and a wall surface 270, which are obstacles in the real space around the user. The obstacle detected by the obstacle detection unit 213 may be any object, whether it is a static object or a dynamic object, as long as it is around the user 220. That is, the object is not limited to the floor surface 260 and the wall surface 270, but may also be a person, a pet, a desk, a door, other home appliances, an animal, a plant, an insect, a curtain, water vapor, water droplets, ice, flame, or the like.

The display control unit 212 generates in the virtual space an avatar object 241 that moves according to the detected rehabilitation action, and a goal object 242 that indicates the goal of the rehabilitation action. Other objects (background objects, etc.) may be included in the virtual space. Display control unit 212 displays images of avatar object 241 and target object 242 on display screen 240 according to the orientation and position of head-mounted display 233 detected by motion detection unit 211.

The images of the avatar object 241 and the target object 242 are displayed superimposed on the background image. Here, the avatar object 241 has the same shape as the controllers 234 and 235, but is not limited to this, and may be an image in the shape of a human hand or an image expressing another body part. , or an image in the shape of some kind of tool. The avatar object 241 moves within the display screen 240 in accordance with the movements of the controllers 234 and 235. The controllers 234 and 235 are provided with buttons, and are configured to allow various setting operations.

The display control unit 212 moves the target object 242 within the display screen 240 while gradually changing the display position and size. For example, the target object 242 may be displayed by moving as if it is descending from above the user 220, or the target object 242 may be displayed by moving so as to approach the user 220. The moving direction of the target object 242 is not limited to the vertical direction, but may be the horizontal direction or the diagonal direction. Note that although an example in which the target object 242 moves has been described here, the present invention is not limited to this, and a stationary target object 242 may be displayed.

The user 220 moves the controllers 234 and 235 to move the avatar object 241 on the screen closer to the target object 242. When the avatar object 241 collides with the target object 242, the display control unit 212 causes the target object 242 to disappear. At this time, the feedback unit 217 determines that the target movement has been achieved, displays a message, simultaneously outputs audio from the headphone speaker 236, and vibrates the controller.

The evaluation unit 214 evaluates the rehabilitation ability of the user 220 by comparing the rehabilitation motion detected by the motion detection unit 211 with the target position represented by the target object displayed by the display control unit 212. Specifically, the evaluation is performed by comparing the positions in the three-dimensional virtual space of the avatar object 241 and the target object 242, which move in response to the rehabilitation motion detected by the motion detection unit 211. In other words, the evaluation unit 214 changes the evaluation of the rehabilitation motion depending on how far the distance between the sensor object included in the avatar object 241 (for example, the center of the avatar object 241) and the target object 242 has decreased. For example, if the sensor object contacts the center of the target object 242, a high rating is given, and if the sensor object contacts only the area around the center of the target object 242, but does not contact the center, a low rating is given. If the sensor object does not touch any part of the target object 242, no evaluation is performed. These evaluations by the evaluation section 214 are performed in conjunction with feedback from the feedback section 217.

The feedback unit 217 changes feedback such as a message via the display control unit 212 according to the evaluation of the rehabilitation operation. For example, if the sensor object contacts the center of the target object 242, "Appare" is displayed, and if the sensor object contacts only the area around the center of the target object 242, "Great" is displayed. The feedback unit 217 provides feedback that stimulates at least two of the five senses to the user who has virtually touched the target object 242 with the avatar object 241 at approximately the same time as when the user touches the target object 242. .

The display control unit 212 can cause the target object 242 to appear at different positions in the depth direction (for example, three positions). The evaluation unit 214 gives different points to each object (high points to distant objects, low points to nearby objects).

The task set database 216 stores sets of multiple tasks. A task indicates one rehabilitation action that the user should perform. Specifically, information representing one task includes information such as where, at what speed, and what size the target object appeared, and what size the avatar object was at that time. It is stored.

Specifically, the contents of the task include the size of the left and right target objects 242, the size of the avatar object 241 (sensor object), the falling speed of the target object 242, the position of the target object, and the like. More specifically, the task also includes the radius of the visual recognition object (visual size) to make the position of the target object easier to see on the right and left sides, and the radius (evaluation size) of the target object that reacts with the avatar object 241. Can be set as . That is, while showing the user a ball with a radius of 20 cm, the user is evaluated as "appare" only after touching a ball with a radius of 10 cm located in the center of the ball. A small viewing size makes it difficult for the user to find the target object. Increasing the visible size makes it easier for the user to find the target object. If the evaluation size is increased, the tolerance for displacement of the avatar object 241 increases, and the tolerance for positional displacement increases. If the evaluation size is made smaller, the allowable amount of displacement of the avatar object 241 becomes smaller, and the rehabilitation motion can be evaluated more severely. It is also possible to match these visible sizes and evaluation sizes.

Parameters in the task set DB, including the sensor size (size of the sensor object) of the avatar object 241, can also be set separately for the left and right sides or for each task. If the sensor size is large, even if the hand position deviates significantly from the target object, the task will still be accomplished, and the difficulty of the rehabilitation action will decrease. Conversely, if the sensor size is small, the hand must be accurately moved to the central area of the target object (depending on the evaluation size), which increases the difficulty of the rehabilitation operation.

If there is a difference in motor ability between the left and right sides, such as motor impairment due to after-effects of cerebral infarction, more effective rehabilitation can be performed by changing the sensor size of the avatar object between the left and right sides.

The task set database 216 stores task sets that determine in what order the plurality of tasks as described above are to be provided to the user. For example, a task set may be stored as a template for each hospital, or a history of executed task sets may be stored for each user. Rehabilitation support device 210 may be configured to be able to communicate with other rehabilitation support devices via the Internet. In that case, a single task set can be executed by the same user in multiple locations, and various templates can be shared among multiple remote users.

The display control unit 212 displays the target object 242 on the display screen 240 according to the task set read from the task set database 216, and requests a virtual touching action with the avatar object 241.

The updating unit 215 updates the target task according to the accumulated points. For example, the target task may be updated using the task achievement rate (number of goals achieved/number of tasks).

(Danger avoidance processing)
The display control unit 212 controls the display on the head-mounted display 233 in order to avoid the risk of the user 220 interfering with the floor 260 and wall surface 270, which are objects in real space. Specifically, in order to avoid the risk of collision with the floor 260 or wall 270, the display control unit 212 displays the target object 242 at a position at least a predetermined distance from the floor 260 or wall 270, and displays the target object 242 at a predetermined distance or more. It will not be displayed in positions within this range. In the example of FIG. 2, when an apple, which is a target object 242, falls from a tree and reaches a predetermined distance (for example, 20 cm) from the floor 260, a raccoon 243 grabs it and prevents it from falling below its height. to stop. The raccoon 243 then takes the apple, which is the target object 242, back to its nest.

FIG. 3A shows various display examples for avoiding the danger of the user 220 interfering with objects in the real space, such as the floor 260 and wall surface 270. (a) shows an example in which the target object 242 disappears at a position 301 at a predetermined distance (for example, 20 cm) from the floor surface 260. (b) shows an example in which the target object 242 stops at a position 301 at a predetermined distance from the floor surface 260. (c) shows an example in which the target object 242 bounces upward at a position 301 that is a predetermined distance from the floor surface 260. (d) shows an example in which a water surface is displayed at a position 301 at a predetermined distance from the floor surface 260, and a leaf as a target object 242 floats on the water surface. (e) shows an example in which a child is displayed at a position 301 at a predetermined distance from the floor 260, and a soap bubble as a target object 242 moves while floating upward from the child. Alternatively, a configuration may be adopted in which the soap bubble serving as the target object 242 is reversed and raised 20 cm above the floor, and disappears once it reaches a certain height. Alternatively, the target object may be a mole sticking out of the ground like in a game of whack-a-mole. A fish jumping out of the sea may be used as the target object. A missile launched upward from a launch pad may be used as a target object. The target object may be a balloon that the child lets go of and goes up. In this case, the display may be controlled so that the number of failures is equal to the number of times the tree gets caught and does not go too far upwards. If there is a direction in which the rehabilitation action cannot be performed sufficiently due to walls in the real space surrounding the user, the target object may not be displayed in that direction, or the display may be changed depending on the distance to the wall.

FIG. 3B is a flowchart for explaining the flow of processing by the rehabilitation support device 210. First, in step S301, the direction of the head of the user 220 wearing the head mounted display 233 is detected. Next, in step S303, a rehabilitation target object 242 that is visible to the user 220 is generated in the virtual space and displayed on the head-mounted display 233 according to the direction of the user's 220 head. Furthermore, in step S305, the display control unit 212 controls the display in order to avoid the risk of the user 220 interfering with the floor surface 260, which is an object in real space.

As described above, according to the present embodiment, the target object 242 is not displayed at a position less than a predetermined distance from the floor surface 260, so the user 220 may hit the floor surface 260 with the controllers 234, 235 and get injured, or the user 220 may be injured by leaning forward. This prevents the risk of falling. That is, it is possible to avoid danger to the user undergoing rehabilitation or the surroundings.

[Third embodiment]
Next, a rehabilitation support device according to a third embodiment of the present invention will be described using FIG. 4. FIG. 4 is a diagram for explaining the configuration of the rehabilitation support device according to this embodiment. Compared to the second embodiment described above, the rehabilitation support device according to the present embodiment dynamically controls the display according to the type, size, and movement of the dynamic obstacle detected by the obstacle detection unit 213. They differ in some respects. That is, the difference is that the display is controlled when a person 401 or the like passes by the user 220. The other configurations and operations are the same as those in the second embodiment, so the same configurations and operations are given the same reference numerals and detailed explanation thereof will be omitted.

In order to avoid the person 401 in real space, a camera 402 provided on the head-mounted display 233 is used. It is not limited to the camera 402 as long as it can detect obstacles, and other configurations (radar, ultrasonic sensor, etc.) may be used. Although the person 401 is illustrated here as an example of a dynamic obstacle, other animals such as pets and insects may also be included. In addition, we range from free-standing obstacles such as home appliances, to curtains, doors, desks, plants, etc., which can be generally classified as static obstacles that can move due to external factors such as wind or external stress, or water vapor or water droplets. The concept of the present invention also includes the detection of obstacles such as ice, fire, etc.

The display control unit 212 does not display the target object 242 in the direction of the person 401 detected by the camera 402 and the obstacle detection unit 213. That is, the display control unit 212 displays the target object 242 in a direction other than the direction in which the obstacle 401 exists, in order to avoid the risk of colliding with the obstacle 401 detected by the obstacle detection unit 213.

Furthermore, the display control unit 212 displays a wall 403 as an obstacle object in the direction of the person 401 detected by the camera 402 and the obstacle detection unit 213, so that the user does not feel like reaching out. The obstacle object is not limited to a wall, but may also be an image of a tree or a person. Additionally, a prohibition notification image (message in words or symbols) prohibiting reaching out may be displayed, the color of the target object may be changed to red, for example, to notify of danger, or the target object may be marked as a bomb. You may change it to something else such as Alternatively, in addition to these display controls, a danger notification using tactile stimulation such as audio or vibration stimulation may be combined.

Furthermore, in order to avoid the risk of colliding with the obstacle 401 detected by the obstacle detection unit 213, the display control unit 212 may stop displaying the target object 242 when the obstacle 401 is detected. Alternatively, you can temporarily stop the screen displayed on the head-mounted display, stop the screen display itself like a blackout, or even stop the movement of the target object while it is displayed. good.

When the obstacle detection unit 213 detects an obstacle, the real space image captured by the camera 402 may be displayed as is on the head mounted display 233.

If the head-mounted display 233 is an optical see-through type, when the obstacle detection unit 213 detects an obstacle, the transparency of the head-mounted display 233 may be changed so that the actual obstacle 401 can be seen. good. In other words, switching from virtual reality technology VR to augmented reality technology AR or mixed reality technology MR may be performed.

In this embodiment, the obstacle detection unit 213 uses the camera 402 of the head-mounted display 233, but is not limited to this, and may use other external cameras or external sensors.

It is desirable to prepare special images such as wall images in advance in the task set database 216 as obstacle object data.

The obstacle detection unit 213 may perform object recognition using existing pattern matching technology, or may display an image depending on the type of obstacle. In that case, the task set database 216 stores obstacle objects (walls, trees, rocks, etc.) to be displayed, corresponding to the types of obstacles (adults, children, pets, etc.).

[Fourth embodiment]
Next, a rehabilitation support device according to a fourth embodiment of the present invention will be described using FIG. 5. FIG. 5 is a diagram for explaining the configuration of the rehabilitation support device 510 according to this embodiment. The rehabilitation support device 510 according to this embodiment differs from the second embodiment in that it includes a calibration section 501 and a level evaluation section 502. The other configurations and operations are the same as those in the second embodiment, so the same configurations and operations are given the same reference numerals and detailed explanation thereof will be omitted.

The calibration unit 501 pre-evaluates the user's movement level before rehabilitation. The display control unit 212 determines the generation position of the target object based on the evaluation result of the calibration unit 501. Specifically, the target motion is not made to appear in areas where there is a risk of falling, disease aggravation, etc., depending on the user's individual motion level.

The level evaluation unit 502 evaluates the user's movement level during rehabilitation. The display control unit 212 may similarly determine the generation position of the target object based on the evaluation result of the level evaluation unit 502.

[Other embodiments]
Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. The structure and details of the present invention can be modified in various ways within the technical scope of the present invention, which can be understood by those skilled in the art. Furthermore, systems or devices that combine the separate features included in each embodiment in any way are also included within the technical scope of the present invention.

Moreover, the present invention may be applied to a system composed of a plurality of devices, or may be applied to a single device. Furthermore, the present invention is also applicable when an information processing program that implements the functions of the embodiments is supplied to a system or device and executed by a built-in processor. Therefore, in order to realize the functions of the present invention on a computer, a program installed on the computer, a medium storing the program, a WWW (World Wide Web) server from which the program is downloaded, and a processor executing the program are also falls within the technical scope of the invention. In particular, a non-transitory computer readable medium storing at least a program that causes a computer to execute the processing steps included in the embodiments described above is within the technical scope of the present invention.

Claims (9)

A device that supports rehabilitation of a user,
a detection unit that detects the direction of the head of the user wearing the head-mounted display;
a first detection unit that detects the position of a floor surface or a wall surface in real space around the user;
a display control unit that generates a target object for rehabilitation that is visible to the user in a virtual space and displays it on the head-mounted display according to the orientation of the user's head detected by the detection unit;
Equipped with
The display control unit is a rehabilitation support device in which the display control unit does not display the target object in a place where there is a risk of falling or disease aggravation, depending on the movement level of each user . further comprising a second detection unit that detects the position of an obstacle in real space around the user,
The rehabilitation support device according to claim 1, wherein the display control unit displays the target object in a direction other than the direction of the obstacle in order to avoid the risk of colliding with the obstacle detected by the second detection unit. . further comprising a second detection unit that detects the position of an obstacle in real space around the user,
3. The display control section displays an obstacle object or a prohibition notification image in the direction of the obstacle in order to avoid the risk of collision with the obstacle detected by the second detection section. Rehabilitation support device. further comprising a second detection unit that detects the position of an obstacle in real space around the user,
In order to avoid the risk of colliding with the obstacle detected by the second detection unit, the display control unit stops displaying the target object or displays the target object on the head-mounted display when the obstacle is detected. The rehabilitation support device according to any one of claims 1 to 3, wherein the rehabilitation support device temporarily stops a screen displayed on the head-mounted display, or stops the screen display itself of the head-mounted display, or stops movement of the target object. The rehabilitation support device according to any one of claims 1 to 4, wherein the first detection section is provided in the head mounted display. It further includes a calibration section that pre-evaluates the user's movement level before rehabilitation,
The rehabilitation support device according to any one of claims 1 to 5, wherein the display control unit determines the generation position of the target object based on the evaluation result of the calibration unit. Further comprising a level evaluation unit that evaluates the user's movement level during rehabilitation,
The rehabilitation support device according to any one of claims 1 to 6, wherein the display control unit determines the generation position of the target object based on the evaluation result of the level evaluation unit. A method for supporting rehabilitation of a user, the method comprising:
a detection step of detecting the direction of the head of the user wearing the head-mounted display;
a detection step of detecting the position of a floor or wall surface in real space around the user;
a display control step of generating a rehabilitation target object in a virtual space to be visually recognized by the user, and displaying it on the head-mounted display according to the orientation of the user's head detected in the detection step;
an evaluation step of determining whether the user's rehabilitation action has a risk of causing disease exacerbation;
Equipped with
In the rehabilitation support method, the display control step does not display the target object in a place where there is a risk of falling or disease aggravation, depending on the movement level of each user . A program that supports rehabilitation of users,
a detection step of detecting the direction of the head of the user wearing the head-mounted display;
a detection step of detecting the position of a floor or wall surface in real space around the user;
a display control step of generating a rehabilitation target object in a virtual space to be visually recognized by the user and displaying it on the head-mounted display according to the orientation of the user's head detected in the detection step;
an evaluation step of determining whether the user's rehabilitation action has a risk of causing disease exacerbation;
Equipped with
The display control step is a rehabilitation support program in which the target object is not displayed in a location where there is a risk of falling or disease aggravation, depending on the user's individual movement level .

Images