CN113101612B - An immersive upper limb rehabilitation system - Google Patents

Abstract

The invention relates to an immersive upper limb rehabilitation system, which at least comprises: the robot body is used for assisting the arm of the patient to carry out rehabilitation training and collecting training data in the training process; the active and passive control platform is used for sending a control instruction to the robot body and recording training data; a virtual reality platform for displaying a virtual training environment providing rehabilitation training visual feedback for a patient, wherein the virtual reality platform is configured to: when the active training rehabilitation mode is entered, displaying the virtual training environment constructed by the patient to the patient; the patient is guided in an adjustment action by synchronously or asynchronously establishing a first virtual three-dimensional model corresponding to a part of the real objects and a second virtual three-dimensional model of the virtual objects corresponding to the current virtual training environment, and/or by replicating at least one virtual three-dimensional model in the current virtual training environment under non-contact interaction between the patient and the virtual training environment.

Description

An immersive upper limb rehabilitation system

technical field

The invention relates to the technical field of medical rehabilitation equipment, in particular to an immersive upper limb rehabilitation system.

Background technique

In the event of cerebrovascular disease, severe brain trauma or other serious neurological diseases, patients will experience limb movement disorders, which will cause a lot of inconvenience to patients' lives. Traditional rehabilitation methods relying solely on therapists for rehabilitation training will undoubtedly restrict the improvement of rehabilitation training efficiency and methods, and the training effect is affected by the level of therapists. With the development of the economy and the improvement of robot technology, the use of upper limb rehabilitation robots to assist patients in upper limb rehabilitation has become a mainstream development trend. Moreover, the robot for shoulder joint rehabilitation training is a new technology that has developed rapidly in recent years, and it is a new application of robot technology in the medical field. The upper limb rehabilitation robot can drive the patient's upper limbs to perform training actions based on the treatment task, which is conducive to the effective recovery of the patient's upper limb strength and the rehabilitation of upper limb motor function.

In the prior art, the patent document with the publication number CN104363982B proposes an upper limb rehabilitation robot system, which includes a computer and a rehabilitation robot; the computer is used to interact with the rehabilitation robot, record training information, and send The rehabilitation robot sends control instructions; and displays the virtual training environment, provides visual feedback for rehabilitation training, and displays the control interface and rehabilitation training information; the rehabilitation robot, as a system actuator, is connected to the computer for receiving the computer control instructions to complete motion control and terminal force output, and send sensor data to the computer at the same time. The interaction force between the patient and the handle is recorded by a multi-dimensional force/torque sensor. The system has an active training rehabilitation mode, a passive training rehabilitation mode and an active and passive training rehabilitation mode.

In the above technical solutions, especially in the active training mode, the patient actively moves the affected upper limb based on his own understanding of the virtual training environment. In fact, the patient’s movement in this mode often cannot meet the standard training requirements, and the rehabilitation effect cannot be guaranteed. Avoid sports injuries. In addition, the sensory data obtained in the above cases are obtained by the patient when the standard training action is not achieved, and cannot actually be used to characterize the patient's exercise capacity. Therefore, the above technical solution is based on this type of sensory The rehabilitation training evaluation made by the data has great deviation and cannot truly reflect the patient's rehabilitation situation.

In addition, on the one hand, due to differences in the understanding of those skilled in the art; The present invention does not possess the characteristics of these prior art, on the contrary, the present invention already possesses all the characteristics of the prior art, and the applicant reserves the right to add relevant prior art to the background technology.

Contents of the invention

Aiming at the problem that the upper limb rehabilitation robot currently used to drive the patient's upper limbs to make training actions cannot realize the interaction with the patient, resulting in poor rehabilitation effect and poor patient experience, as disclosed in the patent document with the publication number CN104363982B in the prior art A robot system for upper limb rehabilitation is proposed. In its technical solution, especially in the active training mode, the patient can actively move the affected upper limb based on his own understanding of the virtual training environment. In fact, the movement of the patient in this mode often cannot reach Standard training requirements cannot guarantee the effect of rehabilitation and avoid sports injuries. In addition, the sensory data obtained in the above cases are obtained by the patient when the standard training action is not achieved, and cannot actually be used to characterize the patient's exercise capacity. Therefore, the above technical solution is based on this type of sensory The rehabilitation training evaluation made by the data has great deviation and cannot truly reflect the patient's rehabilitation situation.

Aiming at the deficiencies of the prior art, the present invention provides an immersive upper limb rehabilitation system, which at least includes: a robot body for assisting the patient's arm in rehabilitation training and collecting training data during the training process; an active and passive control platform for providing The robot body sends control instructions and records training data; the virtual reality platform is used to display the virtual training environment and provide patients with visual feedback on rehabilitation training. It is characterized in that the virtual reality platform is configured to: when entering the active training rehabilitation mode, send Displaying the virtual training environment constructed by it; synchronously or asynchronously establishing the first virtual three-dimensional model corresponding to the part of the real object and the second virtual three-dimensional model of the virtual object corresponding to the current virtual training environment, and/or by Under the non-contact interactive operation between the patient and the virtual training environment, at least one virtual three-dimensional model is reproduced and projected in the current virtual training environment to guide the patient to adjust actions.

Synchronous or asynchronous establishment is relative to the moment when the virtual training environment is established, which means that the two virtual 3D models can be constructed simultaneously and appear in the virtual training environment, or they can be constructed and presented to the virtual training environment at different times. Copying and projection can refer to synchronously copying the virtual 3D model that has been established in the virtual training environment and projecting it on a relative spatial position different from the copied object in the virtual training environment. The copied virtual 3D model can be maintained with the copied object. Synchronize.

According to a preferred embodiment, during the patient's use of the rehabilitation system, the virtual reality platform monitors the movement of the patient and when it is detected that it triggers at least one action guiding condition, by introducing following anchor points and/or Transparency changes overlap, guiding patients to adjust movements.

According to a preferred embodiment, the virtual reality platform is further configured to: when it is detected that the movement deviation degree of the patient's movement reaches the first preset deviation degree threshold, trigger the first movement guidance condition; , to guide the patient to adjust the action.

According to a preferred embodiment, the virtual reality platform is further configured to trigger the second action guidance condition when it is detected that the number of times that the movement deviation degree of the patient's movement reaches the first preset deviation degree threshold reaches the first preset number of times threshold ; By superimposing two virtual three-dimensional models with transparency changes, the patient is guided to adjust the action.

According to a preferred embodiment, the two virtual three-dimensional models may refer to the second virtual three-dimensional model and the third virtual three-dimensional model, and the second virtual three-dimensional model may be a virtual object that performs standard actions corresponding to the current virtual training environment.

According to a preferred embodiment, the third virtual three-dimensional model may be obtained by copying and projecting the first virtual three-dimensional model, and the relative spatial positions of the third and first virtual three-dimensional models in the virtual training environment are different.

According to a preferred embodiment, when two virtual three-dimensional models are superimposed with transparency changes, the patient can simultaneously observe them as the motion mapping in the first viewing angle and as the motion mapping in the third viewing angle.

According to a preferred embodiment, the virtual reality platform is further configured to: when the first guidance mode is selected, when the second action guidance condition is triggered, at least construct the second and third virtual three-dimensional models according to the training data, And at the same time switch the relative spatial position of the first virtual three-dimensional model in the virtual training environment, so that the second and third virtual three-dimensional models can be in relative spatial positions observable by the patient in the virtual training environment.

According to a preferred embodiment, the virtual reality platform is further configured to: in the case of selecting the second guidance mode, when the second action guidance condition is triggered, maintain the relative spatial position of the first virtual three-dimensional model in the virtual training environment , and introduce a newly constructed third virtual three-dimensional model into the virtual training environment according to at least the training data.

According to a preferred embodiment, the virtual reality platform is further configured to: trigger the third action guidance condition when the number of times the action deviation of the patient's motion reaches the second preset deviation threshold exceeds the second preset number of times threshold, by setting The two virtual 3D models are superimposed with transparency changes and combined with the introduction of follow-up positioning points into the virtual training environment to guide patients to adjust their actions.

Description of drawings

Fig. 1 is a simplified structural block diagram of an active and passive control platform of a preferred embodiment provided by the present invention;

Fig. 2 is a simplified structural block diagram of a virtual reality platform of a preferred embodiment provided by the present invention;

Fig. 3 is a schematic diagram of a simplified overall structure of a robot body in a preferred embodiment provided by the present invention;

Fig. 4 is a schematic diagram of a simplified local structure of the robot body provided by the present invention;

Fig. 5 is a simplified assembly schematic diagram of the motor mounting block provided by the present invention;

Fig. 6 is a simplified top view structural schematic diagram of the bar set provided by the present invention;

Fig. 7 is a schematic diagram of a simplified overall structure of the arm rest provided by the present invention;

Fig. 8 is a simplified overall structural schematic diagram of the AB bar provided by the present invention;

Fig. 9 is a schematic diagram of a simplified overall structure of the BC bar provided by the present invention;

Fig. 10 is a schematic diagram of a simplified overall structure of the CD rod provided by the present invention;

Fig. 11 is a schematic diagram of a simplified overall structure of the motor pallet provided by the present invention;

Fig. 12 is a schematic diagram of a simplified overall structure of the motor connecting board provided by the present invention;

Fig. 13 is a schematic diagram of a simplified overall structure of the connecting plate of the rod group provided by the present invention.

List of reference signs

1: Support rod 2: Display group 3: Motor mounting block

4: Connecting rod group 5: Installation table 6: Robot body

7: Drive system 8: Sensor group 9: Single chip microcomputer

10: Host computer 11: Encoder 12: Medical interface display group

13: 3D scene display group 14: Multi-axis arm group 15: Axis arm

301: Motor 302: Motor connection plate 303: Motor support plate

401: Armrest 402: BC Rod 403: AB Rod

404: CD rod 405: Connecting plate of rod group 406: Bearing

detailed description

The application will be described in detail below in conjunction with the accompanying drawings.

This application proposes an immersive upper limb rehabilitation system, which mainly includes a robot body 6, an active and passive control platform and a virtual reality platform.

The virtual reality platform includes a display device, which may be a conventional display, a VR head-mounted virtual reality device, an AR head-mounted virtual reality device or an MR head-mounted virtual reality device. The monitor can be used to show the virtual training environment to the patient, as well as to provide the patient with visual feedback on the rehabilitation training.

When the rehabilitation system enters the active training rehabilitation mode, the virtual reality platform displays the constructed virtual training environment to the patient through the display device, and establishes the first virtual three-dimensional model corresponding to some real objects, and/or is consistent with the current virtual training environment. A second virtual three-dimensional model of the corresponding virtual object.

The real object refers to the real object, that is, the patient who uses the rehabilitation system. Part of the real object refers to a body part of the patient, which specifically may be the patient's upper limbs or upper body. The first virtual three-dimensional model corresponding to some real objects is established, that is, the virtual three-dimensional model simulated and constructed in the virtual training environment, which can follow the patient's upper limb movement in real time, that is, map the patient's upper limb movement on the display.

Virtual objects refer to objects constructed virtually to guide patients in rehabilitation training. The virtual object corresponds to the current virtual training environment, and can change to different types of objects correspondingly with the virtual training environment. Preferably, the virtual object can be a virtual character who walks with the patient and makes a rowing action in a virtual training environment. The virtual object is in a position observable by the patient in the virtual training environment. For example, the virtual object may be a virtual crew member who is on the same boat as the patient and sits in front of the first virtual three-dimensional model corresponding to the patient. The actions performed by the virtual object should be standard actions corresponding to the current virtual training environment.

Most of the upper limb rehabilitation robots that have been proposed have adopted a virtual environment, which can map the patient's movement to a virtual object to achieve the immersion of the patient during rehabilitation training. However, in such technical solutions, either the patient's movement can only be collected actively The data cannot provide real-time feedback on whether the patient’s movements are standardized, or the patient is reminded of whether the movements are standardized through text and voice. Text and voice are not enough to enable patients to understand how to achieve the required movements, and doctors and nurses are often required to accompany them and provide specific guidance. , leading to a heavy workload of medical staff and requiring medical staff to take care of the recovery process one-on-one. Based on this, this application proposes to use a second virtual 3D model other than the first virtual 3D model corresponding to the patient. The patient no longer only pays attention to the mapping of his movement in the virtual environment, but can also intuitively observe the correct The standard action, and then achieve the purpose of effective rehabilitation training guidance for patients. In addition, the two virtual 3D models are simultaneously displayed in the virtual environment, and patients can directly see the difference between their movements and standard movements, and patients can actively adjust their movements even without medical guidance , to better match the standard movements, so as to achieve the full immersion of patients during rehabilitation training and greatly improve the effect of rehabilitation training.

Preferably, the virtual training environment can be a virtual space where the scene changes as the patient uses his upper limbs to make a rowing action. The virtual training environment may be, for example, a ship on a river, a submarine on the seabed, an airship in the air, a ship hull on a track, and the like.

The virtual reality platform includes at least two guidance modes, including at least a first guidance mode and a second guidance mode. In the first guidance mode, when the patient enters the virtual training environment, the first virtual three-dimensional model corresponding to the patient sits at the bow of the boat, and there is no second virtual three-dimensional model in the virtual training environment at this time. In the second guidance mode, when the patient enters the virtual training environment, the first virtual three-dimensional model corresponding to the patient sits in a position other than the bow, that is, the midship or the stern, and the second virtual three-dimensional model is present in the virtual training environment at this time .

Preferably, when the virtual reality platform is started in the first guidance mode, the virtual reality platform displays the constructed virtual training environment to the patient through the display device, and establishes a first virtual three-dimensional model corresponding to a part of the real object, and the current virtual training There is no second virtual three-dimensional model in the environment.

Preferably, when the virtual reality platform is started in the second guidance mode, the virtual reality platform displays the constructed virtual training environment to the patient through the display device, and establishes a first virtual three-dimensional model corresponding to a part of the real object, and a first virtual three-dimensional model corresponding to the current virtual object. A second virtual three-dimensional model corresponding to the training environment.

According to the actual situation of the patient, the first guidance mode or the second guidance mode of the virtual reality platform can be arbitrarily selected. The main difference between the two modes is which position the patient is mainly located on the hull most of the time in the virtual training environment. Compared with the second guidance mode, the first guidance mode can enable patients to obtain a wider field of vision and better exercise experience, but at the same time, its rehabilitation training guidance effect is relatively weak. There is always a further second virtual 3D model, but precisely because of this a better rehabilitation guiding effect can also be obtained. Therefore, doctors and nurses can choose the first guidance mode for patients with strong learning ability or cognitive response in the rehabilitation training process, and choose the first guidance mode for patients with weak learning ability or cognitive response in the rehabilitation training process. Second boot mode.

During the process of patients using the rehabilitation system, the virtual reality platform monitors the patient's movement, and when the patient's movement does not match the set action, the first action guidance condition is triggered, and the patient is guided to adjust the action by introducing follow-up positioning points into the virtual training environment.

The discrepancy between the patient's movement and the set action mentioned in this application may mean that the deviation degree of the patient's movement reaches the first preset deviation degree threshold. The degree of movement deviation can be the quantitative data of the difference degree of the patient's movement relative to the standard movement, and the quantitative data can be calculated and evaluated from aspects such as the amplitude, angle, force or speed of the patient's movement.

When entering the active training rehabilitation mode, the virtual training environment constructed by it is displayed to the patient; the first virtual three-dimensional model corresponding to some real objects and the second virtual model of the virtual object corresponding to the current virtual training environment are established synchronously or asynchronously. 3D model, and/or by making at least one virtual 3D model reproduce and project in the current virtual training environment under the non-contact interactive operation between the patient and the virtual training environment, to guide the patient to adjust the action.

The anchor point in the following anchor point can refer to the position point that the upper limb should reach in the standard action relative to the current action of the first virtual three-dimensional model, and the position point can be the palm, the wrist joint, and the elbow joint in the upper limb. Wait for the corresponding position. Following the anchor point may mean that the anchor point can move correspondingly with the change of the patient's motion. A complete paddle-pulling cycle includes single-direction movements such as the stage of lifting the paddle into the water, the stage of pulling the paddle, the stage of pressing the paddle and the beginning of pushing the paddle, and the stage of pushing the paddle. For different single-directional movements, there are different following positioning points. One action enters the next action, and the following positioning points change accordingly, so that the following positioning points in the display screen are limited and clearly specified, which will not mislead the patient. In this way, the patient can be guided to adjust the action in a timely and effective manner when the patient's action does not match occasionally. The follow-up anchor point may have a meteor-shaped trail extending to the patient's upper limbs to display the task action path indicating the patient's movement. At the same time, the meteor-shaped trail is directional so that the patient can determine the following direction of his movement.

Converting the second virtual three-dimensional model into a frame structure and selecting at least one structure point on the frame structure as a tracking anchor point. When the virtual reality platform is in the first guidance mode and only the first virtual three-dimensional model is established in the current virtual training environment, the tracking anchor point is introduced into the virtual training environment by constructing the second virtual three-dimensional model. The second virtual 3D model constructed by the virtual reality platform with a large amount of data loading and delayed response will not be completely loaded into the virtual training environment, but it will be simplified to follow positioning with a smaller amount of data processing and faster response. Points, using the following positioning points to convert the standard range of motion or standard motion path into visual information that patients can intuitively observe and compare. If the method of directly introducing the virtual 3D model is adopted, the virtual 3D model will be imported once when the patient’s movements do not match. Regardless of whether the correct standard movements are displayed in the virtual 3D environment, it is difficult for the patient to completely keep up with the correct standard movements. The so-called irregular actions are easy to occur frequently, which leads to the system needing to repeatedly load and hide different prompt models, resulting in a large amount of data loading and increased response delay. Correspondingly, the amount of data processing required for loading and tracking positioning points and the impact on the patient's rehabilitation process are very small, avoiding the situation that the system needs to repeatedly load and hide different prompt models due to frequent irregular movements of patients.

The interaction between the second virtual three-dimensional model and the first virtual three-dimensional model may refer to a process in which the motion parameters of the first virtual three-dimensional model affect the motion parameters of the second virtual three-dimensional model accordingly.

The virtual scene realization unit can regulate the movement stage and movement speed of the second virtual three-dimensional model based on the training data related to the patient's upper limb movement. The motion stage can refer to each stage in a single paddle-pulling cycle. By regulating the first and second virtual three-dimensional models to maintain the same motion stage, and in combination with regulating the motion speed of the second virtual three-dimensional model, it is possible to avoid standard actions that are too large for the patient's adaptability. Fast or too slow, which greatly facilitates the patient to observe the difference between their actions and standard actions, and quickly and effectively adjust the posture of the upper limbs. Under the control of the virtual scene realization unit, in order to facilitate the patient to follow and enhance the patient's sense of immersion, the movement time difference between the second and first virtual 3D models does not exceed two movement stages, and the movement speed of the second virtual 3D model is faster than that of the first The moving speed of a virtual three-dimensional model is fast but does not exceed a preset speed threshold. When the movement time difference between the second and the first virtual three-dimensional model reaches two movement stages, instruct the second virtual three-dimensional model to repeat the standard actions corresponding to the two movement stages until the first virtual three-dimensional model completes the above two a movement phase. Repeating display may mean that the second virtual three-dimensional model only repeats standard actions corresponding to the two motion stages.

The tracking anchor point may be obtained by copying and projecting the existing second virtual three-dimensional model to the virtual training environment. In the case that the virtual reality platform is in the second guidance mode and the first and second virtual three-dimensional models are established in the current virtual training environment, copy and project the existing second virtual three-dimensional model to the virtual training environment to introduce the tracking anchor point . The copy projection of the second virtual 3D model is actually a copy projection of the frame structure of the second virtual 3D model, and at least one structure point on the frame structure is selected to obtain the tracking anchor point.

Copy projection may refer to copy projection in a manner of establishing a synchronous association relationship between the replicated and the replicator. At the same time of copying, a synchronous association relationship between the tracking anchor point and the second virtual three-dimensional model is established. Based on this, following the anchor point can maintain the dynamic corresponding relationship between it and the second virtual three-dimensional model.

Copying projections may refer to copying projections in a manner of establishing a synchronous association relationship between replicators and the virtual training environment. At the same time of projection, a synchronous relationship between the tracking point and the virtual training environment is established. Based on this, following the positioning point can maintain its relative position in the virtual training environment, and guide the patient to exercise along a path that meets the requirements of rehabilitation training.

During the process of the patient using the rehabilitation system, the virtual reality platform monitors the movement of the patient, and when the number of times that the movement of the patient does not match the set action reaches the first preset threshold, the second action guidance condition is triggered. The three-dimensional model is superimposed with transparency changes to guide the patient to adjust the action.

The two virtual 3D models may refer to the second virtual 3D model and the third virtual 3D model.

The second virtual three-dimensional model may always refer to a virtual object that performs standard actions corresponding to the current virtual training environment. The third virtual three-dimensional model may be obtained by copying and projecting the first virtual three-dimensional model. The third and first virtual three-dimensional models are at different positions in the display screen.

In the case of superimposing the two virtual three-dimensional models with transparency changes, the patient can simultaneously observe them as a motion map under the first viewing angle and as a motion mapping under the third viewing angle. Since patients can observe their own movement from the first perspective and the third perspective at the same time, patients can adjust their movements better and more effectively to achieve a more effective rehabilitation training effect.

Transparency changes overlap, wherein the transparency may refer to the degree of visibility corresponding to the two virtual three-dimensional models in the display interface. Transparency change may mean that the transparency corresponding to the two virtual three-dimensional models in the display interface is not fixed but dynamically variable. Overlapping may mean that the relative spatial positions of the two virtual three-dimensional models in the display interface in the virtual training environment are the same. For example, the main torsos of the two virtual 3D models are fused into one, and the two virtual 3D models can be displayed correspondingly through different upper limb movements.

Under the overlapping setting of transparency changes, the virtual reality platform can selectively highlight a specific action in the continuous motion that is inconsistent with the set action by adjusting the transparency changes of the second and third virtual three-dimensional models. Specifically, after triggering the second or third motion-guiding condition, the patient may be able to keep up with the virtual object making the standard rowing motion synchronously or almost synchronously, or may not be able to keep up, resulting in a gap between the patient and the virtual object's motion. There is a certain time difference. In any case, the patient's upper limb posture is wrong or the stretch is not in place. If the patient is only guided by showing the second virtual 3D model, it is difficult for the patient to pay attention to the upper limb posture and the upper limb amplitude at the same time. Whether it needs to be adjusted, it is easy to overstretch or enter the next stage without completing the current action.

In this regard, in the virtual reality platform proposed in this application, when the second or third action guidance condition is triggered, when it is detected that the patient training data matches the standard action, at least part of the second virtual three-dimensional model is increased. transparency. At this time, the patient's actions meet the requirements of rehabilitation training and are less dependent on the second virtual 3D model. Based on this, the visibility of at least part of the second virtual 3D model can be reduced to avoid unnecessary interference to the patient caused by the interlacing of the two 3D models.

Preferably, if the patient and the virtual object are in a synchronous state, the transparency of the second virtual three-dimensional model corresponding to the virtual object is increased. Preferably, if the patient is not fully synchronized with the virtual object, the transparency of the second virtual three-dimensional model is gradually increased in a direction backward from the movement of the upper limbs. The two virtual 3D models in the synchronous state coincide or almost coincide, and the two virtual 3D models in the incomplete synchronous state have a certain time difference and do not completely coincide with each other. The non-complete synchronization state is different from the action mismatch. There are two situations: the action coincidence and the action non-conformity in the synchronous state or in the non-complete synchronization state.

When it is detected that the patient's training data does not match the standard action, the first area is delineated based on the upper limb movement deviation between the second and third virtual three-dimensional models, and the second area is delimited based on the first area and the upper limb movement direction, based on The third area is defined by the first and second areas, and the sharpness corresponding to the first to third areas decreases in turn.

At least one of the first to third regions may be defined in an irregular shape.

The first area is delineated based on the upper limb movement deviation between the second and third virtual three-dimensional models. According to the upper limb movement deviation between the second and third virtual 3D models, the patient's corresponding movement path/task movement path that needs to be adjusted can be obtained, and the movement path can be obtained by instructing the patient to move at least one upper limb joint point After completion, the area where at least one upper limb joint point corresponding to the motion path is located is defined as the first area. The corresponding upper limb joint points of the second and third virtual three-dimensional models are preserved in the first area, so that the patient can know the upper limb joint points that need to be adjusted directly by observing the first area. The joints of the upper limbs may be wrist joints or elbow joints, for example.

When the movement deviation of the upper limbs corresponds to at least two movement paths, for example, the wrist joint needs to be adjusted to meet the standard movement of the forearm and the elbow joint needs to be adjusted to meet the standard movement of the upper arm. For the first area, redefine the first area according to the wrist joint. It can better adapt to the operating habits of the human body.

The first predetermined shape corresponds to the area where the movement path is located, and the outer edge of the first predetermined shape is expanded outward or inward so that the first area also contains the corresponding upper limb joint points of the two virtual three-dimensional models. The outer edge of the first area is determined, so as to ensure that all required content is included, and reduce unnecessary other picture content.

The first predetermined shape can be a preset shape, such as a regular circle, and the first predetermined shape can be adjusted to an unconventional circle by counting and analyzing the shape of the first area defined each time, and the first predetermined shape The division of regions is more consistent. The first predetermined shape may be a shape determined from several preset shapes corresponding to the deviation degree of upper limb movement.

The second area is defined based on the first area and the movement direction of the upper body. In order to ensure that the patient can continue to perform rehabilitation actions without interruption while adjusting the posture, the outer edge of the first area is used as the second predetermined shape of the second area, and the outer edge of the second predetermined shape is expanded outward so that the second area also contains The upper limb moves toward the corresponding upper arm or forearm region, and the second region is determined based on the expanded outer edge.

The outer edge of the second area serves as the third predetermined shape of the third area, and the outer edge of the third predetermined shape is expanded outward to define the third area. The third area does not limit its required enclosing content.

The sharpness corresponding to the first to third areas decreases successively. The reduction in sharpness can be achieved by increasing the blurriness of the corresponding area.

The discrepancy between the training data and the standard action may refer to the fact that there is a certain deviation between the current patient's training data and the standard action corresponding to the position. The discrepancy between the training data and the standard action may also refer to the fact that there is a deviation between the patient's training data and the execution action of the second virtual three-dimensional model at a certain moment in the virtual training environment. Based on this, the system avoids the situation that the system is frequently prompted to make mistakes due to the slow movement speed of the patient and fails to keep up with the virtual object. The patient can independently adjust the movement speed according to his physical fitness and feelings, which is conducive to improving the experience.

The second virtual three-dimensional model is executed at a preset speed in such a way that it is always kept in the same motion stage as the first virtual three-dimensional model. The movement speed of the second virtual 3D model is often faster than that of the patient. Based on this, it can effectively show the user the standard movement that needs to be performed. At the same time, the two always maintain the same movement stage, which limits the movement time difference between the two. The patient can better follow through on a phase-by-phase basis.

In the prior art, the patient is often reminded of the entire continuous movement. For example, after the patient completes a paddle cycle, the patient is prompted that the action completed does not meet the standard or the range of motion is not enough. For the patient, it cannot Knowing specifically which action in the paddle-pulling cycle has a problem, one can only proceed to the next paddle-pulling cycle according to one's own understanding of the prompt, which is not conducive to the recovery of the patient. In this regard, the rehabilitation system proposed in this application adopts the overlapping setting of transparency changes, and the motion prompts can be embodied in the stage of lifting the oar into the water, the stage of pulling the oar, the stage of pressing the oar and pushing the oar, and the stage of pushing the oar. In a single action such as this, the patient can clearly know that there is a specific gap in the action, and through the transparency change overlap setting, the gap between the patient's action and the standard action is presented to the patient in a visual way, so that the patient can use it. Adjust its own actions in a quantized way.

The virtual reality platform can regulate the transparency change overlap based on the non-contact interactive operation between the patient and the virtual training environment. Non-contact interactive operation refers to the process in which the patient does not touch the display screen, but uses the sensor carried on the robot body 6 to virtually map the corresponding movement made according to the picture on the display screen on the display screen. The non-contact interactive operation may refer to a movement process rather than a single action, and the movement process may refer to a single paddle pulling cycle.

If the virtual reality platform is started in the first guidance mode, when the second action guidance condition is triggered, the second and third virtual three-dimensional models are constructed at least according to the training data, and the first virtual three-dimensional model is switched in the virtual training environment at the same time The relative spatial positions of the second and third virtual three-dimensional models can be in relative spatial positions observable by the patient in the virtual training environment.

Switching the relative spatial position of the first virtual three-dimensional model in the virtual training environment may refer to: in the first guidance mode, the first virtual three-dimensional model corresponding to the patient is at the bow position; after switching its relative spatial position, the first virtual three-dimensional model The 3D model is turned to a position other than the bow, that is, the midship or the stern. The relative spatial positions in the virtual training environment originally corresponding to the first virtual three-dimensional model are vacated, so as to be used to create the second and third virtual three-dimensional models. In the upper limb rehabilitation system proposed in the prior art, some of the standard action demonstration videos are directly inserted into the current virtual training environment, and the patient has to pause the exercise to watch the video, which not only interrupts the patient's rehabilitation but directly affects the patient's use experience, and the patient Standard movements can only be imitated from the senses, but it is impossible to determine whether the movements meet the requirements, which will lead to multiple interruptions of training and insertion of standard movement demonstration videos, which will seriously affect rehabilitation training.

If the virtual reality platform is started in the second guidance mode, when the second action guidance condition is triggered, the relative spatial position of the first virtual three-dimensional model in the virtual training environment is maintained, and new information is introduced into the virtual training environment at least according to the training data. A third virtual 3D model is constructed.

Maintaining the relative spatial position of the first virtual three-dimensional model in the virtual training environment may refer to: in the second guidance mode, the first virtual three-dimensional model corresponding to the patient is in the midship or stern of the ship, without changing the first virtual three-dimensional model The current position of , and directly import the newly-created third virtual 3D model. Since the third virtual 3D model is formed by copying and projecting the first virtual 3D model, the model itself can be obtained without data processing, which will not cause excessive burden on the system operation and cause the screen to be stuck and not smooth problems, and at the same time enable patients to observe their own sports from the first perspective and the third perspective at the same time, which is conducive to improving the effect of rehabilitation training.

During the process of the patient using the rehabilitation system, the virtual reality platform monitors the movement of the patient, and triggers the third action guidance when the movement deviation of the patient reaches the second preset deviation threshold and exceeds the second preset threshold. Conditions, by superimposing the transparency changes of two virtual 3D models and introducing follow-up anchor points into the virtual training environment to guide patients to adjust their actions. The two virtual 3D models are superimposed with transparency changes while introducing tracking positioning points. For patients, it is possible to better clarify the actions that need to be adjusted and how to adjust them, which is conducive to further improving the effect of rehabilitation training.

If the virtual reality platform is started in the first guidance mode, when the third action guidance condition is triggered, at least construct the second virtual three-dimensional model according to the training data, and simultaneously switch the relative space of the first virtual three-dimensional model in the virtual training environment Angle of view, by superimposing the first and second virtual three-dimensional models with transparency changes and combining with introducing tracking positioning points into the virtual training environment, the patient is guided to adjust actions. The relative spatial viewing angle may refer to the first viewing angle or the third viewing angle observed by the patient. Here, switching the relative spatial viewing angle may refer to rotating the first virtual three-dimensional model in the virtual training environment to realize the patient's first viewing angle. The perspective switches to a third perspective. For example, the ship hull in the virtual training environment and the three-dimensional model personnel on the ship are turned to the side, preferably to the side corresponding to the affected upper limb of the patient, so as to better observe the movement of the affected upper limb. When the third action guidance condition is triggered, the virtual training environment cancels the first viewing angle and only retains the third viewing angle, and the patient can intuitively observe the difference between the affected upper limb and the standard action. And combined with following the positioning point, it can enhance the standardization of the patient's movement trajectory. Follow the anchor points, including the highest point and the lowest point that need to be reached in a single-oriented movement, and also include several anchor points in the movement trajectory of a single-oriented movement, so that patients can control the effective bending angle or effective extension of the upper limbs during exercise Angle, to achieve better rehabilitation training effect.

If the virtual reality platform is started in the second guidance mode, then when the third action guidance condition is triggered, switch the relative spatial perspective of the first virtual three-dimensional model in the virtual training environment, by making the first and second virtual three-dimensional models transparent The changes overlap and combine to introduce follow-up anchors to the virtual training environment to guide the patient in adjusting movements. In the second guidance mode, the first and second virtual three-dimensional models have already been constructed. Based on this, when the third action guidance condition is triggered, the transparency of the first and second virtual three-dimensional models that have been constructed can be changed and overlapped.

The robot body 6 is used to assist the patient's arm in rehabilitation training and collect training data during the training process. The training data mainly refers to the data corresponding to the motor 301 contained in the robot body 6 and several sensors during the rehabilitation training process. Preferably, the robot body 6 can be an existing upper limb rehabilitation device. The upper limb rehabilitation device may be, for example, an indoor rowing machine.

As a preferred embodiment, the robot body 6 can be a robot based on motion mapping and virtual reality proposed according to the trajectory curve fitted by the upper limb movement. Through different motor 301 control strategies and the connecting rod structure proposed in this application, the robot body 6 can help the patient's upper limbs to move, and complete active and passive rehabilitation training.

The robot body 6 mainly includes a motor mounting block 3 and a connecting rod group 4 .

The connecting rod group 4 is fixedly arranged on the installation table 5 in an adjustable manner. The connecting rod group 4 is fixedly arranged on the installation table 5 through at least one supporting rod 1 . Each support rod 1 is vertically assembled on the installation table 5 , and the connecting rod group 4 is slidably connected to the rod body of the support rod 1 . The operator can adjust the height of the connecting rod group 4 on the installation table 5 up and down, so as to better adapt to different patients.

The connecting rod set 4 includes an AB rod 403 , a BC rod 402 and a CD rod 404 which are rotationally connected to each other in turn. The BC bar 402 is fixedly equipped with an arm rest 401, and the arm rest 401 is used to place the patient's upper limb forearm. The shape of the arm support 401 is adapted to the shape of the forearm of the human body, which is similar to a long U-shaped structure.

The axis centerline of the AB rod 403 rotating around the BC rod 402 is coplanar with the axis centerline of the CD rod 404 rotating around the BC rod 402 . In this way the robot can support multi-pose movements of the patient's forearm in the plane of the BC bar 402 .

A motor mounting block 3 is arranged on the AB rod 403 at the end corresponding to the A fulcrum. The output end of the motor 301 in the motor mounting block 3 is connected to the end corresponding to the A fulcrum on the AB rod 403 . That is, the control of the rotation of the AB rod 403 can be realized by regulating the motor 301. Due to the linkage relationship between the AB rod 403, the BC rod 402 and the CD rod 404, the movement of the patient's forearm located on the BC rod 402 can be driven synchronously. sports.

The motor mounting block 3 is stably assembled above the mounting table 5 through the multi-axis arm group 14 . One end of the multi-axis arm group 14 is slidably connected with the support rod 1 to realize the adjustable height of the connecting rod group 4 on the installation table 5 . The other end of the multi-axis arm group 14 is connected to the motor support plate 303 in the motor mounting block 3 to achieve stable support for the motor mounting block 3 .

The multi-axis arm group 14 includes at least one shaft and at least one shaft arm 15 that is sequentially connected to each other in rotation. Two adjacent shaft arms 15 are rotatably connected to each other through a rotating shaft arranged at the end of the rod body. A rotating shaft connecting block is slidably arranged on the supporting rod 1 , and a rotating shaft connecting block is arranged below the motor support plate 303 . The axis centerlines of the shaft arms 15 rotating around each other are parallel to each other.

Two supporting rods 1 are arranged on the installation table 5, and the motor mounting block 3 or the AB rod 403 and the CD rod 404 are respectively connected to a supporting rod 1 through a multi-axis arm group 14, so as to realize a pair of connecting rods of the supporting rod Adjustable supports for group 4.

The weight of the motor 301 in the motor mounting block 3 is mainly transferred to the support rod 1 through the multi-axis arm group 14, so that the movement of the patient's upper limbs will not be affected by the weight of the motor 301, so that the movement of the patient's upper limbs can be more accurately evaluated to ensure rehabilitation training effect.

The D fulcrum of the CD rod 404 adopts a bearing 406 with a flange and a locking screw to connect the inner and outer shafts, and the transmission is simple and the structure is stable.

Only one motor mounting block 3 is provided in the robot body 6, and the fitting of the movement trajectory of the upper limb can be accurately completed by relying on the operation of one motor 301. The operation is simple, the cost is low, and it is suitable for popularization.

The specific installation method at the motor installation block 3 is described as follows. Install the motor support plate 303 on the positioning seat through at least one positioning hole, for example, 8 positioning holes. The positioning seat is arranged on one end of the multi-axis arm group 14 . The motor 301 is positioned and installed through at least one through hole on the motor connecting plate 302 , for example, 4 through holes. Then the motor 301 is placed on the motor pallet 303 . Finally, the motor support plate 303 is connected with at least one elongated hole on the motor connecting plate 302, for example two elongated holes, to complete the installation and positioning of the motor 301.

The specific installation method at the connecting rod group 4 will be described as follows. The protruding shaft of the B end of the AB rod 403 extends into the inner ring of the bearing 406 with a flange and locking screws and is fastened by two locking screws. The outer ring flange of the same bearing 406 is connected to the B end of the BC rod 402 through at least one set of bolts and nuts, for example, three sets of bolts and nuts. The C end of the BC rod 402 is connected to the outer ring flange of another bearing 406 in the same way. The inner ring of the bearing 406 is fastened with the protruding shaft at the end of the CD rod 404C through a locking screw. The D end of the CD rod 404 is flanged to the outer ring of the bearing 406 with a flange and locking screws. The inner ring of the bearing 406 is fastened with the protruding shaft on the connecting plate 405 of the rod group. The arm rest 401 is mounted on the BC rod 402 through at least one set of bolts and nuts, for example, two sets of bolts and nuts. The two sets of square holes on the arm support 401 can be sewed with two sections of elastic bands for auxiliary fixation.

The connection between the connecting rod group 4 and the motor mounting block 3 is the key connection between the output shaft of the motor 301 and the end hole of the AB rod 403A. The connecting rod group 4 is connected to the supporting rod 1 through at least one installation and positioning hole, for example, eight installation and positioning holes on the connecting plate 405 of the rod group.

When in use, the motor 301 is driven to rotate by a 48V power supply, a pulse controller and a driver. Through the key connection, the output shaft of the motor 301 transmits the motion to the AB rod 403, and the AB rod 403 acts as a crank to complete a complete circular motion. Synchronously drive the BC rod 402 and the CD rod 404 to move according to a certain track. The trajectory at the armrest 401 is the fitting trajectory of the required upper limb movement.

As a preferred embodiment, the main structural dimensions and main installation dimensions of the robot body 6 proposed in this application are described as follows. The center distance between the two ends of the AB rod 403 is 129.60 mm. The center distance between the two ends of the BC rod 402 is 187.00 mm. The center distance between the two ends of the CD rod 404 is 313.80 mm. The arm rest 401 has a length of 150 mm, a major diameter of 90 mm, and a minor diameter of 70 mm. Bearing 406 with flange and locking screw has an inner ring diameter of 22mm and a flange connection diameter of the outer ring of 60mm. The vertical distance between the midpoint of the AB rod 403A end and the center of the CD rod 404D end is 120.32 mm, and the horizontal distance is 290.85 mm. The horizontal inclination angle at which the arm support 401 is connected to the BC rod 402 is 45.5°. The vertical height between the midpoint of the CD rod 404D end and the desktop of the installation table 5 is 165mm.

The active and passive control platform is used to send control instructions to the robot body 6 and record training data. The active and passive control platform mainly includes a drive system 7 , a single chip microcomputer 9 , a sensor group 8 , an encoder 11 , a host computer 10 and a display group 2 .

The active and passive control platform is provided with at least two modes, which at least include a passive recovery mode and an active recovery mode. In the passive rehabilitation mode, according to the preset passive rehabilitation scheme, according to the predetermined angular velocity value in the passive rehabilitation scheme, the controller sends the motor 301 pulse command to the driver, and the driver outputs a given pulse to drive the stepping motor 301 to rotate at a certain angular velocity And drive the movement of the connecting rod group 4, the patient's arm stretches into the arm rest 401 and follows the movement of the connecting rod group 4 to complete the rehabilitation process. The passive rehabilitation program may refer to the preset correspondence between angular velocity and time.

The sensor group 8 may include at least one of an angle sensor, a force sensor and a speed sensor disposed on the arm support 401 of the robot body 6 . The sensor group 8 can detect the pressure between the patient's upper limb and the arm rest 401 , the movement trajectory of the patient's upper limb, the movement speed of the patient's upper limb, etc., and transmit the data to the single-chip microcomputer 9 . The pressure value converted by the single-chip microcomputer 9 is transmitted to the upper computer 10 through the USB serial port, and the upper computer 10 displays data such as the pressure value and the rotational speed of the upper limbs in real time on its medical interface.

The display group 2 may include a three-dimensional scene display group 13 and a medical interface display group 12 . The virtual reality platform is connected with the three-dimensional scene display group 13 and the medical interface display group 12 respectively. The display group 2 can be stably and adjustably assembled on the installation table 5 through the support bar 1 and at least one multi-axis arm group 14 arranged on the installation table 5 . The three-dimensional scene display group 13 is used to display the selected virtual scene for the patient using the arm rest 401 to watch when performing upper limb rehabilitation, and guide the patient to carry out standardized and effective rehabilitation training. The medical interface display group 12 is used to display the collected data and calculation and analysis data of multiple sensors arranged in the linkage group 4, so that the medical personnel can more clearly define the patient's rehabilitation training situation.

As a preferred embodiment, the 3D scene display group 13 can be a display that is set facing the side of the patient and can display a 3D scene picture, and the virtual reality platform can show a 3D scene picture that can change with the patient's actions on the display. Preferably, the three-dimensional scene display group 13 may be a head-mounted display device using VR technology, AR technology or MR technology. VR technology (Virtual Reality), generally refers to virtual reality technology, using VR head-mounted display devices to close people's vision and hearing to the outside world, and guide patients to feel like they are in a virtual environment. VR head-mounted display devices are more immersive than conventional displays, and the scene simulation effect is better. In addition to pure virtual digital picture technology such as VR, AR (Augmented Reality) virtual digital picture + naked-eye reality technology, or MR (Mediated Reality) digital reality + virtual digital picture technology can also be used.

In the active rehabilitation mode, the arm of the patient mainly moves actively, and the link group 4 associated with the arm support 401 is driven by the upper limb of the patient. The pressure sensor and the encoder 11 can detect training data such as the pressure value of the upper limbs and the angular velocity value of the upper limbs driving the movement of the connecting rod group 4 . The angular velocity of the upper limb rotation is obtained after the counting pulse obtained by the encoder 11 per unit time is converted by the single-chip microcomputer 9, and displayed on the medical interface. The involvement of patients' willingness to exercise can strengthen the central nervous system and effectively improve the rehabilitation effect of patients.

It should be noted that the above specific embodiments are exemplary, and those skilled in the art can come up with various solutions inspired by the disclosure of the present invention, and these solutions also belong to the scope of the disclosure of the present invention and fall within the scope of this disclosure. within the scope of protection of the invention. Those skilled in the art should understand that the description and drawings of the present invention are illustrative rather than limiting to the claims. The protection scope of the present invention is defined by the claims and their equivalents. The description of the present invention contains a number of inventive concepts, such as "preferably", "according to a preferred embodiment" or "optionally" all indicate that the corresponding paragraph discloses an independent concept, and the applicant reserves the right to propose a division based on each inventive concept right to apply.

Claims (4)

1. An immersive upper limb rehabilitation system comprising at least:

the robot body is used for assisting the arm of the patient to carry out rehabilitation training and collecting training data in the training process;

the active and passive control platform is used for sending a control instruction to the robot body and recording training data;

a virtual reality platform for displaying a virtual training environment and providing visual feedback for rehabilitation training for a patient,

wherein the virtual reality platform is configured to:

when the active training rehabilitation mode is entered, displaying the virtual training environment constructed by the patient to the patient;

by synchronously or asynchronously building a first virtual three-dimensional model corresponding to a part of the real object and a second virtual three-dimensional model of the virtual object corresponding to the current virtual training environment,

and/or

By means of the at least one virtual three-dimensional model replicating projections in the current virtual training environment under non-contact interaction between the patient and the virtual training environment,

guiding the patient to adjust the motion;

when the patient uses the rehabilitation system, the virtual reality platform monitors the movement of the patient and when the virtual reality platform monitors that the patient triggers at least one action guide condition, the patient is guided to adjust the action by introducing the change overlap of the following positioning points and/or the transparency into the virtual training environment,

wherein the virtual reality platform is capable of regulating transparency change overlap based on non-contact interaction between the patient and the virtual training environment;

the virtual reality platform is further configured to:

triggering a first action guiding condition when the action deviation degree of the movement of the patient is monitored to reach a first preset deviation degree threshold value;

guiding the patient to adjust the action by introducing a following positioning point into the virtual training environment;

the virtual reality platform is further configured to:

triggering a second action guiding condition when the number of times that the action deviation degree of the movement of the patient reaches a first preset deviation degree threshold value reaches a first preset number threshold value is monitored;

guiding the patient to adjust the action by overlapping transparency changes of the two virtual three-dimensional models;

the two virtual three-dimensional models refer to a second virtual three-dimensional model and a third virtual three-dimensional model, and the second virtual three-dimensional model is a virtual object for executing standard actions corresponding to the current virtual training environment;

the third virtual three-dimensional model is obtained by copying and projecting the first virtual three-dimensional model, and the relative spatial positions of the third virtual three-dimensional model and the first virtual three-dimensional model in the virtual training environment are different;

under the condition that transparency change overlapping is carried out on the two virtual three-dimensional models, the patient simultaneously observes the motion mapping under the first visual angle and the motion mapping under the third visual angle;

converting the second virtual three-dimensional model into a frame structure, selecting at least one structure point on the frame structure as a following positioning point, and introducing the following positioning point into the virtual training environment in a mode of constructing the second virtual three-dimensional model under the condition that the virtual reality platform is in the first guide mode and only the first virtual three-dimensional model is established in the current virtual training environment;

the virtual scene implementation unit regulates and controls the motion stage and the motion speed executed by the second virtual three-dimensional model based on the training data related to the upper limb actions of the patient;

if the patient and the virtual object are in a synchronous state, increasing the transparency of a second virtual three-dimensional model corresponding to the virtual object; and if the patient and the virtual object are in a non-complete synchronous state, gradually increasing the transparency of the second virtual three-dimensional model in a direction back to the upper limb movement.

2. The system of claim 1, wherein the virtual reality platform is further configured to:

in the case where the first guidance mode is selected, when a second action guidance condition is triggered, the second and third virtual three-dimensional models are constructed from at least the training data, and the relative spatial positions of the first virtual three-dimensional model in the virtual training environment are simultaneously switched so that the second and third virtual three-dimensional models can be in the relative spatial positions observed by the patient in the virtual training environment.

3. The system of claim 2, wherein the virtual reality platform is further configured to:

in the case that the second guidance mode is selected, when a second action guidance condition is triggered, the relative spatial position of the first virtual three-dimensional model in the virtual training environment is maintained, and a newly constructed third virtual three-dimensional model is introduced to the virtual training environment at least in accordance with the training data.

4. The system of any one of claims 1-3, wherein the virtual reality platform is further configured to:

and when the times that the motion deviation degree of the patient exceeds a second preset time threshold value when the motion deviation degree of the patient exceeds the second preset time threshold value, triggering a third motion guide condition, and guiding the patient to adjust the motion by performing transparency change overlapping on the two virtual three-dimensional models and introducing a following positioning point into the virtual training environment in a combined manner.

Images