CN114010184A - Motion data acquisition and mirror image method for planar rehabilitation robot - Google Patents
Motion data acquisition and mirror image method for planar rehabilitation robot Download PDFInfo
- Publication number
- CN114010184A CN114010184A CN202111239596.7A CN202111239596A CN114010184A CN 114010184 A CN114010184 A CN 114010184A CN 202111239596 A CN202111239596 A CN 202111239596A CN 114010184 A CN114010184 A CN 114010184A
- Authority
- CN
- China
- Prior art keywords
- robot
- mirror image
- data
- motion
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000012549 training Methods 0.000 claims abstract description 48
- 238000012545 processing Methods 0.000 claims abstract description 34
- 238000006243 chemical reaction Methods 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000009471 action Effects 0.000 claims description 5
- 230000003287 optical effect Effects 0.000 claims description 4
- 230000002146 bilateral effect Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 238000002560 therapeutic procedure Methods 0.000 abstract description 10
- 230000000694 effects Effects 0.000 abstract description 6
- 230000000638 stimulation Effects 0.000 abstract description 6
- 230000000007 visual effect Effects 0.000 abstract description 5
- 210000001364 upper extremity Anatomy 0.000 description 9
- 208000006011 Stroke Diseases 0.000 description 4
- 238000013480 data collection Methods 0.000 description 3
- 201000010099 disease Diseases 0.000 description 3
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 3
- 210000002310 elbow joint Anatomy 0.000 description 3
- 238000003672 processing method Methods 0.000 description 3
- 206010021403 Illusion Diseases 0.000 description 2
- 210000003414 extremity Anatomy 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 206010008190 Cerebrovascular accident Diseases 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003925 brain function Effects 0.000 description 1
- 230000002490 cerebral effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 210000000653 nervous system Anatomy 0.000 description 1
- 230000002980 postoperative effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000007634 remodeling Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1121—Determining geometric values, e.g. centre of rotation or angular range of movement
- A61B5/1122—Determining geometric values, e.g. centre of rotation or angular range of movement of movement trajectories
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
- A61B5/1128—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique using image analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H1/00—Apparatus for passive exercising; Vibrating apparatus ; Chiropractic devices, e.g. body impacting devices, external devices for briefly extending or aligning unbroken bones
- A61H1/02—Stretching or bending or torsioning apparatus for exercising
- A61H1/0274—Stretching or bending or torsioning apparatus for exercising for the upper limbs
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/12—Driving means
- A61H2201/1207—Driving means with electric or magnetic drive
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/50—Control means thereof
- A61H2201/5007—Control means thereof computer controlled
Abstract
The invention relates to a motion data acquisition and mirror image method of a planar rehabilitation robot, which is characterized in that motion data acquisition and mirror image processing are carried out by any one of motion capture and mirror image methods, depth camera acquisition and mirror image methods and video mirror image methods, and then a robot mechanical arm is combined, so that a patient is driven to move by the traction of the planar robot while carrying out mirror image therapy on the patient. The data acquired by the method can realize real-time mirror plane rehabilitation training (namely, video and motion track are simultaneously mirrored in real time) and non-real-time mirror plane rehabilitation training (namely, video and motion track are simultaneously mirrored in non-real time), and force sense stimulation and visual stimulation are combined, so that the rehabilitation efficiency and effect of a patient are effectively improved.
Description
Technical Field
The invention relates to a data processing technology, in particular to a motion data acquisition and mirroring method for a planar rehabilitation robot.
Background
At present, stroke is a common disease and frequently encountered disease of a nervous system, and is also a disease with high disability rate. The rehabilitation of stroke patients requires one-to-one manual accompanying training of rehabilitation therapists, and at present, the rehabilitation therapists are insufficient in hands along with the aggravation of aging. And the plane robot has advantages such as simple structure, small, with low costs, stability is stronger, so the plane rehabilitation robot is just progressively using in the motion rehabilitation training of cerebral apoplexy patient postoperative, when promoting recovered efficiency, combines earlier relevant recovered recreation, increases the interest of rehabilitation training, has greatly promoted recovered effect.
The mirror image therapy in rehabilitation is a very effective treatment method for stroke patients who do not have distal movement of upper limbs in the early stage. The mirror image therapy has wide and obvious curative effect in clinical rehabilitation in recent years, and is a rehabilitation training means which is based on visual stimulation, utilizes the plane mirror imaging principle, copies the moving picture of a healthy side to an affected side, enables a patient to imagine the motion of the affected side, and combines visual illusion, visual feedback and virtual reality. Particularly, when the device is applied to exercise training and sensory training, the patient can visually feel the movement or touch of the healthy side limb by means of visual illusion, so that the affected side is affected, and the function recovery of the affected side is promoted.
Therefore, rehabilitation combining mirror image therapy and a robot is gradually becoming a trend, but no product combining mirror image therapy and a planar robot is available at present. Therefore, the upper limb rehabilitation system based on the combination of the mirror image therapy and the planar robot can increase visual stimulation while exercise rehabilitation is carried out, and then the rehabilitation effect is improved.
In addition, the mirror image therapy usually adopts a real-time mirror image mode to show the movement action of the healthy side, and the healthy side patient needs to participate simultaneously, however, in the movement training, a training track needs to be made in advance sometimes, and then the arm of the affected side executes the training action, and the mirror image therapy and the planar rehabilitation robot can collect the healthy side movement data under the real-time mirror image and the non-real-time mirror image, and feed back the data to the affected side after the mirror image processing, so that the mirror image movement of the affected side is realized.
The specification with publication number CN106618957B discloses an upper limb rehabilitation robot, the motion control method of which is as follows: the Kinect is adopted to collect upper limb movement data, the movement data are processed through the upper computer, a control signal is obtained, and the lower computer transmits the control signal to the servo driver to drive the exoskeleton wearable mechanical arm to move. The user can control the robot to carry out the coordinated synchronous mirror motion of the two upper limbs through the motion of the healthy side limbs of the user. The invention is primarily directed to a motion control method.
Publication No. CN 112245224 a discloses an elbow joint mirror image rehabilitation training device. The invention discloses an elbow joint mirror image rehabilitation training device. The elbow joint mirror image rehabilitation training device is low in load, simple in structure, convenient to use, easy to wear and convenient to operate. But only aiming at the elbow, and can not effectively aim at the multi-joint movement of the upper limb.
Publication number CN109330825A proposes a hand rehabilitation mirror image training robot, which is used for training by driving an affected hand with the rehabilitation hand robot under the condition of mirror image training. But only for fine rehabilitation training of the hands. The mirror image mode of the hand mirror image robot system is to record mirror image video of hand movement by reflection of mirror image of a camera directly facing the back of a screen. The disadvantage is that the mirror is of limited size, limited range of recording, inconvenient to carry, increases the cost of the equipment and is more fragile.
The invention discloses an upper limb rehabilitation training robot and an upper limb rehabilitation training method, which can enable a patient to perform active training, passive training, coordination training or mirror image training, perform upper limb rehabilitation training in the whole period of the morning, the noon and the evening and promote brain function and nerve remodeling. The mirroring method of the three-dimensional mirroring robot system comprises the following steps: and recording a handle with mark points through a depth camera to identify the motion track. The disadvantages are that: because the space moving range of the six-degree-of-freedom mechanical arm is too large, the placement position of the camera is too high, and the equipment cost and the space are increased.
Disclosure of Invention
In order to further improve the effectiveness of the mirror image therapy in rehabilitation medical treatment, a motion data acquisition and mirror image method of a plane rehabilitation robot is provided, and effective data is provided for a system based on the combination of the plane rehabilitation robot and a mirror image system.
The technical scheme of the invention is as follows: a plane rehabilitation robot motion data acquisition and mirror image method comprises any one of motion capture and mirror image, depth camera acquisition and mirror image and video mirror image methods;
1) the motion capture and mirror method comprises the following steps: data acquisition by optical or inertial motion capture systems, 1.1) hand-eye calibration: performing hand-eye calibration on the motion capture system and the mechanical arm to obtain a conversion relation T between a global coordinate system of the motion capture system and a robot base coordinate system;
1.2) real-time motion capture, namely, a handheld capture device and a system capture the motion track of the device in real time;
1.3) trajectory conversion: the acquired data are coordinates under a motion capture system, and the expression of the track under a robot base coordinate system is obtained by utilizing the conversion relation T in the step 1.1);
1.4) track mirroring: when real-time training is carried out, mirror image calculation is carried out immediately every time one frame of motion data is obtained, namely mirror image processing is carried out on the track under the base coordinate system of the robot, and then the track is sent to the robot end to be executed; when non-real-time training is carried out, after track recording is finished, mirror image processing is carried out on the data to obtain mirror image track data, then a data file is stored in a text, and when necessary, the robot executes the mirrored track;
2) the depth camera acquisition and mirroring method comprises the following steps: the human hand is directly identified by the depth camera,
2.1) human hand recognition: identifying a human hand in real time by using an identification algorithm, and taking a characteristic point as a hand motion track acquisition point;
2.2) calibrating the hand and the eye: obtaining a conversion relation T1 between the camera coordinate system and the robot base coordinate system;
2.3) hand tracking: tracking hand movement in real time by the camera and returning the representation of the characteristic points in a camera coordinate system;
2.4) trajectory conversion: the acquired data are coordinates in a camera coordinate system, and the representation of the track in a robot base coordinate system is obtained by utilizing a conversion relation T1;
2.5) track mirroring: carrying out mirror image processing on the track under the robot base coordinate system, and carrying out real-time mirror image training: performing mirror image calculation immediately after each frame of motion data is acquired, and then sending the data to a robot end for execution; when non-real-time mirror image training is carried out: obtaining mirror image track data in a mirror image after the track is finished;
3) the video mirroring method comprises the following steps: the video that is directly taken is mirrored,
3.1) the camera is arranged at the front upper part of the robot, so that the view of the camera covers a motion area;
3.2) after the robot system is ready, opening the camera through software;
3.3) the camera takes a video at a fixed frequency;
3.4) real-time video mirroring: when a camera shoots a frame of video, carrying out mirror image processing on pixels, and then displaying the frame of image on a computer; non-real-time video mirroring: the method comprises the steps that a camera shoots a video, a computer displays a current shot image in real time, the shot image is stored in a file after shooting is completed, then mirror image processing is carried out on each frame of image of the video, the processed video is a mirror image video, and the current mirror image video can be played after the video is opened at any time and subjected to mirror image processing subsequently.
The motion data acquisition and mirroring method under the double-mechanical-arm mode comprises the following steps: symmetrically installing the plane mechanical arms on two sides; when real-time mirror image training is carried out, the data of the healthy side driving robot motion are obtained in real time through the bilateral robot system as follows: firstly, establishing real-time data communication between a healthy side and an affected side mechanical arm, then driving the healthy side mechanical arm to move in real time by a patient, returning the current joint angle of each joint by the healthy side mechanical arm, negating the joint angle by the affected side mechanical arm to obtain mirrored movement data, and moving to the angle; when non-real-time mirror image training is carried out, a patient drives the robot to move through the healthy side, joint angle data or track points are collected and stored in a text, a mechanical arm on the affected side needs to read the data and perform negation on the data, and mirror image action is executed.
The non-real-time motion data acquisition and mirror image method under the single mechanical arm mode comprises the following steps:
firstly, driving a robot to move freely by a side-exercising arm, and acquiring the angle position of a joint space and the position data of a Cartesian space by the robot and storing the angle position and the position data in real time;
then, mirror image processing is carried out according to the stored data, all the position data of the Cartesian space are symmetrically processed about the X axis of the robot base coordinate system to obtain a group of pose data, and then proper inverse solution data are selected according to the kinematic inverse solution of the robot and the configuration of the robot and stored;
and finally, point data obtained by mirroring is planned out in an off-line mode to obtain a track sequence of a joint space according to a B spline interpolation method, the robot automatically and slowly runs from a first point position of the track to a first point position of a mirror image point of mirroring processing, the robot stops, and a non-real-time mirroring training starting instruction is waited.
The single mechanical arm mode real-time motion data acquisition and mirror image method comprises the following steps:
firstly, enabling a side-exercising arm to move freely, and acquiring Cartesian space pose data of the tail end of the arm by the robot according to external equipment and storing the data in real time;
then, mirror image processing is carried out on the stored data, all pose data of a Cartesian space are symmetrically processed about an X axis of a robot base coordinate system to obtain a group of pose data, and then proper inverse solution data are selected according to kinematic inverse solution of the robot and the configuration of the robot;
and finally, point position data obtained by mirroring is subjected to on-line planning to obtain a track sequence of a Cartesian space according to a B-spline interpolation method, joint space position data is obtained according to a robot kinematics inverse solution algorithm and is sent to a motor, and the robot drives the affected arm to perform mirroring motion relative to the healthy arm in real time.
Further, the camera in the step 2.1) tracks the feature points by collecting the handle with the two-dimensional code or mark points through the camera.
Further, the camera tracking feature points in the step 2.1) are tracked and positioned through a handle formed by the IMU and the Bluetooth, and the IMU acquires the coordinates of the motion position in real time and transmits the coordinates to the computer system through the Bluetooth.
The invention has the beneficial effects that: the invention relates to a motion data acquisition and mirror image method of a planar rehabilitation robot, which drives a patient to move by traction of the planar robot while the patient performs mirror image therapy. Combines the force sense stimulation and the visual stimulation, and effectively improves the rehabilitation efficiency and effect of the patient.
Drawings
FIG. 1 is a flow chart of a data acquisition and mirror image processing method of a planar rehabilitation robot according to the present invention;
FIG. 2 is a flow chart of a video mirroring method according to the present invention.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation manner and a specific operation process are given, but the scope of the present invention is not limited to the following embodiments.
The mirror image processing method is based on a mirror image mode of a planar rehabilitation mirror image robot system, namely a method for performing mirror image training and processing on video images and motion trail images in two different training modes of real-time robot image data and non-real-time image data.
1. Mirror image data acquisition and mirror image processing method
1.1 motion capture mode:
as shown in fig. 1, the acquisition method by the optical or inertial motion capture system includes the following specific steps:
step1 hand-eye calibration: performing hand-eye calibration on the motion capture system and the mechanical arm to obtain a conversion relation T between a global coordinate system of the motion capture system and a robot base coordinate system;
step2, real-time motion capture, namely, handheld capture equipment (an optical marker and an inertial receiver) and a system capture the motion track of the equipment in real time;
and (4) converting a Step3 track: the acquired data are coordinates under a motion capture system, and the representation of the track under the robot base coordinate system is obtained by utilizing the conversion relation T;
step4 trace mirroring: when real-time training is carried out, mirror image calculation is carried out immediately every time one frame of motion data is obtained, namely mirror image processing is carried out on the track under the robot base coordinate system: and (x1 is-x, y1 is y), the data are sent to a robot end to be executed, when non-real-time training is carried out, mirror image processing is carried out on the data after track recording is finished to obtain mirror image track data, then the data file is stored in a text, and the robot executes the mirrored track if necessary.
1.2 depth camera mode
The method for directly identifying the human hand through the depth camera comprises the following specific steps:
step1 human hand recognition: identifying human hands in real time by using an identification algorithm (such as YOLOv3), and taking a certain characteristic point as a hand motion track acquisition point;
step2 hand-eye calibration: obtaining a conversion relation T1 between the camera coordinate system and the robot base coordinate system;
step3 hand tracking: tracking hand movement in real time by the camera and returning the representation of the characteristic points in a camera coordinate system;
and (4) converting a Step4 track: the acquired data are coordinates in a camera coordinate system, and the representation of the track in a robot base coordinate system is obtained by utilizing a conversion relation T1;
step5 trace mirroring: carrying out mirror image processing on the track under the robot base coordinate system: (x1 ═ -x, y1 ═ y); when real-time mirror image training is carried out: performing mirror image calculation immediately after each frame of motion data is acquired, and then sending the data to a robot end for execution; when non-real-time mirror image training is carried out: obtaining mirror image track data in a mirror image after the track is finished;
1.3 double-mechanical arm mode
Step1 symmetrically installing the two side plane mechanical arms
Step2, when real-time mirror image training is carried out, the data of the healthy side driving robot motion are obtained in real time through the bilateral robot system as follows: firstly, establishing real-time data communication between a healthy side and an affected side mechanical arm, then driving the healthy side mechanical arm to move in real time by a patient, returning the current joint angle of each joint by the healthy side mechanical arm, negating the joint angle by the affected side mechanical arm to obtain mirrored movement data, and moving to the angle; when non-real-time mirror image training is carried out, a patient drives the robot to move through the healthy side, joint angle data (or track points) are collected and stored in a text, a mechanical arm on the affected side needs to read the data and perform negation on the data, and mirror image action is executed.
1.4 Single robot arm mode
A. Non-real time mirrored data collection
Step1: the side-exercising arm drives the robot to move freely, and the robot acquires the angle position of the joint space and the position data of the Cartesian space and stores the angle position and the position data in real time.
Step2: and carrying out mirror image processing according to the stored data, symmetrically processing all the position data of the Cartesian space about the X axis of the robot base coordinate system to obtain a group of pose data, selecting proper inverse solution data according to the kinematic inverse solution of the robot and the configuration of the robot, and storing the data.
Step3: and (4) mirroring the obtained sequence point data, and planning a track sequence of the joint space in an off-line manner according to a B spline interpolation method.
Step 4: and (3) the robot automatically and slowly runs from the first point position of the track in the step (1) to the first point position of the mirror image point in the mirror image processing in the step (2) and stops, and a non-real-time mirror image training starting instruction is waited.
B. Real-time mirrored data collection
The method comprises the following key steps of real-time mirror image motion data processing:
step1: the side-exercising arm moves freely, and the robot acquires Cartesian space pose data of the tail end of the arm according to external equipment (a real-time data acquisition mode of a dynamic capture system and a depth camera) and stores the data in real time.
Step2: and performing mirror image processing on the stored data, symmetrically processing all pose data of a Cartesian space about an X axis of a robot base coordinate system to obtain a group of pose data, and selecting proper inverse solution data according to the kinematic inverse solution of the robot and the configuration of the robot.
Step3: point position data obtained by mirroring is subjected to on-line planning to obtain a Cartesian space track sequence according to a B-spline interpolation method, joint space position data are obtained according to a robot kinematics inverse solution algorithm and are sent to a motor, and the robot drives the affected arm to perform mirroring motion relative to the healthy arm in real time.
2. Video mirroring method
As shown in fig. 2, the video mirroring process for the camera direct shooting includes the following specific steps:
step1: the camera is arranged at the front upper part of the robot, so that the view of the camera covers a motion area;
step2, after the robot system is ready, opening a camera through software;
step3, the camera shoots the video at a fixed frequency;
step 4: real-time video mirroring: when a camera shoots a frame of video, carrying out mirror image processing on pixels, and then displaying the frame of image on a computer;
non-real-time video mirroring: the camera shoots a video, and the computer displays a current shot image in real time. After shooting is finished, the video is stored in a file, then mirror image processing is carried out on each frame of image of the video, the processed video is the mirror image video, and the current mirror image video can be played after the video can be opened at any time and mirror image processing is carried out subsequently.
For non-real-time mirror image, real-time mirror image is more obvious to patient's recovered effect, and in the real-time mirror image, the motion data collection station that the patient was good for the side and is held can also follow tracks of the location through the handle that IMU and bluetooth are constituteed except that the handle that has the two-dimensional code is gathered through the camera. The IMU collects the coordinates of the motion position in real time and transmits the coordinates to the computer system through Bluetooth. Or a handle with mark points is provided, and the depth camera identifies and tracks the mark points to realize the tracking of the motion trail.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (6)
1. A plane rehabilitation robot motion data acquisition and mirror image method is characterized by comprising any one of motion capture and mirror image, depth camera acquisition and mirror image and video mirror image methods;
1) the motion capture and mirror method comprises the following steps: data acquisition is performed by optical or inertial motion capture systems,
1.1) calibrating the hand and the eye: performing hand-eye calibration on the motion capture system and the mechanical arm to obtain a conversion relation T between a global coordinate system of the motion capture system and a robot base coordinate system;
1.2) real-time motion capture, namely, a handheld capture device and a system capture the motion track of the device in real time;
1.3) trajectory conversion: the acquired data are coordinates under a motion capture system, and the expression of the track under a robot base coordinate system is obtained by utilizing the conversion relation T in the step 1.1);
1.4) track mirroring: when real-time training is carried out, mirror image calculation is carried out immediately every time one frame of motion data is obtained, namely mirror image processing is carried out on the track under the base coordinate system of the robot, and then the track is sent to the robot end to be executed; when non-real-time training is carried out, after track recording is finished, mirror image processing is carried out on the data to obtain mirror image track data, then a data file is stored in a text, and when necessary, the robot executes the mirrored track;
2) the depth camera acquisition and mirroring method comprises the following steps: the human hand is directly identified by the depth camera,
2.1) human hand recognition: identifying a human hand in real time by using an identification algorithm, and taking a characteristic point as a hand motion track acquisition point;
2.2) calibrating the hand and the eye: obtaining a conversion relation T1 between the camera coordinate system and the robot base coordinate system;
2.3) hand tracking: tracking hand movement in real time by the camera and returning the representation of the characteristic points in a camera coordinate system;
2.4) trajectory conversion: the acquired data are coordinates in a camera coordinate system, and the representation of the track in a robot base coordinate system is obtained by utilizing a conversion relation T1;
2.5) track mirroring: carrying out mirror image processing on the track under the robot base coordinate system, and carrying out real-time mirror image training: performing mirror image calculation immediately after each frame of motion data is acquired, and then sending the data to a robot end for execution; when non-real-time mirror image training is carried out: obtaining mirror image track data in a mirror image after the track is finished;
3) the video mirroring method comprises the following steps: the video that is directly taken is mirrored,
3.1) the camera is arranged at the front upper part of the robot, so that the view of the camera covers a motion area;
3.2) after the robot system is ready, opening the camera through software;
3.3) the camera takes a video at a fixed frequency;
3.4) real-time video mirroring: when a camera shoots a frame of video, carrying out mirror image processing on pixels, and then displaying the frame of image on a computer; non-real-time video mirroring: the method comprises the steps that a camera shoots a video, a computer displays a current shot image in real time, the shot image is stored in a file after shooting is completed, then mirror image processing is carried out on each frame of image of the video, the processed video is a mirror image video, and the current mirror image video can be played after the video is opened at any time and subjected to mirror image processing subsequently.
2. The planar rehabilitation robot motion data acquisition and mirroring method according to claim 1, wherein the motion data acquisition and mirroring method in a double-robot mode comprises: symmetrically installing the plane mechanical arms on two sides; when real-time mirror image training is carried out, the data of the healthy side driving robot motion are obtained in real time through the bilateral robot system as follows: firstly, establishing real-time data communication between a healthy side and an affected side mechanical arm, then driving the healthy side mechanical arm to move in real time by a patient, returning the current joint angle of each joint by the healthy side mechanical arm, negating the joint angle by the affected side mechanical arm to obtain mirrored movement data, and moving to the angle; when non-real-time mirror image training is carried out, a patient drives the robot to move through the healthy side, joint angle data or track points are collected and stored in a text, a mechanical arm on the affected side needs to read the data and perform negation on the data, and mirror image action is executed.
3. The planar rehabilitation robot motion data acquisition and mirroring method according to claim 1, wherein the non-real-time motion data acquisition and mirroring method in a single mechanical arm mode:
firstly, driving a robot to move freely by a side-exercising arm, and acquiring the angle position of a joint space and the position data of a Cartesian space by the robot and storing the angle position and the position data in real time;
then, mirror image processing is carried out according to the stored data, all the position data of the Cartesian space are symmetrically processed about the X axis of the robot base coordinate system to obtain a group of pose data, and then proper inverse solution data are selected according to the kinematic inverse solution of the robot and the configuration of the robot and stored;
and finally, point data obtained by mirroring is planned out in an off-line mode to obtain a track sequence of a joint space according to a B spline interpolation method, the robot automatically and slowly runs from a first point position of the track to a first point position of a mirror image point of mirroring processing, the robot stops, and a non-real-time mirroring training starting instruction is waited.
4. The planar rehabilitation robot motion data acquisition and mirroring method according to claim 1, wherein the real-time motion data acquisition and mirroring method in a single robot mode:
firstly, enabling a side-exercising arm to move freely, and acquiring Cartesian space pose data of the tail end of the arm by the robot according to external equipment and storing the data in real time;
then, mirror image processing is carried out on the stored data, all pose data of a Cartesian space are symmetrically processed about an X axis of a robot base coordinate system to obtain a group of pose data, and then proper inverse solution data are selected according to kinematic inverse solution of the robot and the configuration of the robot;
and finally, point position data obtained by mirroring is subjected to on-line planning to obtain a track sequence of a Cartesian space according to a B-spline interpolation method, joint space position data is obtained according to a robot kinematics inverse solution algorithm and is sent to a motor, and the robot drives the affected arm to perform mirroring motion relative to the healthy arm in real time.
5. The planar rehabilitation robot motion data acquisition and mirroring method according to claim 1, wherein the camera tracking feature points in the step 2.1) are achieved by acquiring a handle with a two-dimensional code or mark points through a camera.
6. The planar rehabilitation robot motion data acquisition and mirroring method according to claim 1, wherein the camera tracking feature points in the step 2.1) are tracked and positioned through a handle formed by an IMU and Bluetooth, and the IMU acquires motion position coordinates in real time and transmits the motion position coordinates to a computer system through Bluetooth.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111239596.7A CN114010184A (en) | 2021-10-25 | 2021-10-25 | Motion data acquisition and mirror image method for planar rehabilitation robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111239596.7A CN114010184A (en) | 2021-10-25 | 2021-10-25 | Motion data acquisition and mirror image method for planar rehabilitation robot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114010184A true CN114010184A (en) | 2022-02-08 |
Family
ID=80057438
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111239596.7A Pending CN114010184A (en) | 2021-10-25 | 2021-10-25 | Motion data acquisition and mirror image method for planar rehabilitation robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114010184A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115762709A (en) * | 2022-11-09 | 2023-03-07 | 苏州智康机器人有限公司 | Desktop mirror image rehabilitation robot system and application method thereof |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108392816A (en) * | 2018-04-19 | 2018-08-14 | 温州医科大学 | A kind of lower limb digitlization mirror image therapy rehabilitation training system |
| US20180228430A1 (en) * | 2017-02-10 | 2018-08-16 | Mindmaze Holding Sa | System, method and apparatus for rehabilitation with tracking |
| CN110013419A (en) * | 2019-05-14 | 2019-07-16 | 上海大学 | A kind of both arms healing robot and training method |
| CN112089427A (en) * | 2020-08-31 | 2020-12-18 | 南京麦澜德医疗科技有限公司 | Finger joint rehabilitation training evaluation method and system |
| CN112132894A (en) * | 2020-09-08 | 2020-12-25 | 大连理工大学 | Mechanical arm real-time tracking method based on binocular vision guidance |
| KR20210151360A (en) * | 2020-06-05 | 2021-12-14 | 계명대학교 산학협력단 | A neurofeedback system and method based on virtual reality mirror therapy for brain activation in stroke patients |
-
2021
- 2021-10-25 CN CN202111239596.7A patent/CN114010184A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180228430A1 (en) * | 2017-02-10 | 2018-08-16 | Mindmaze Holding Sa | System, method and apparatus for rehabilitation with tracking |
| CN108392816A (en) * | 2018-04-19 | 2018-08-14 | 温州医科大学 | A kind of lower limb digitlization mirror image therapy rehabilitation training system |
| CN110013419A (en) * | 2019-05-14 | 2019-07-16 | 上海大学 | A kind of both arms healing robot and training method |
| KR20210151360A (en) * | 2020-06-05 | 2021-12-14 | 계명대학교 산학협력단 | A neurofeedback system and method based on virtual reality mirror therapy for brain activation in stroke patients |
| CN112089427A (en) * | 2020-08-31 | 2020-12-18 | 南京麦澜德医疗科技有限公司 | Finger joint rehabilitation training evaluation method and system |
| CN112132894A (en) * | 2020-09-08 | 2020-12-25 | 大连理工大学 | Mechanical arm real-time tracking method based on binocular vision guidance |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115762709A (en) * | 2022-11-09 | 2023-03-07 | 苏州智康机器人有限公司 | Desktop mirror image rehabilitation robot system and application method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5982542B2 (en) | Method and system for detecting the presence of a hand in a minimally invasive surgical system | |
| JP6000387B2 (en) | Master finger tracking system for use in minimally invasive surgical systems | |
| US9727779B2 (en) | Motion information processing apparatus | |
| CN109875501B (en) | Physiological parameter measurement and feedback system | |
| JP5702797B2 (en) | Method and system for manual control of remotely operated minimally invasive slave surgical instruments | |
| JP5702798B2 (en) | Method and apparatus for hand gesture control in a minimally invasive surgical system | |
| WO2021068542A1 (en) | Force feedback technology-based robot system for active and passive rehabilitation training of upper limbs | |
| CN109605385B (en) | Rehabilitation assisting robot driven by hybrid brain-computer interface | |
| US20130222565A1 (en) | System and Method for Sensor Fusion of Single Range Camera Data and Inertial Measurement for Motion Capture | |
| Wang et al. | Free-view, 3d gaze-guided, assistive robotic system for activities of daily living | |
| Loconsole et al. | A new Kinect-based guidance mode for upper limb robot-aided neurorehabilitation | |
| Tao et al. | Trajectory planning of upper limb rehabilitation robot based on human pose estimation | |
| Wang et al. | Feature evaluation of upper limb exercise rehabilitation interactive system based on kinect | |
| CN114010184A (en) | Motion data acquisition and mirror image method for planar rehabilitation robot | |
| Wei et al. | Real-time limb motion tracking with a single imu sensor for physical therapy exercises | |
| Wang et al. | Humanoid robot control system based on AR-SSVEP | |
| CN112451306B (en) | Arm muscle rehabilitation training system based on VR | |
| WO2022073467A1 (en) | Dual-arm multitask parallel processing robot device for caregiving massages | |
| Prataksita et al. | Brain-robot control interface: Development and application | |
| CN108062102A (en) | A kind of gesture control has the function of the Mobile Robot Teleoperation System Based of obstacle avoidance aiding | |
| CN105549733B (en) | Brain-computer interface system and method based on stable state vision inducting under a kind of intelligent space | |
| WO2021190762A1 (en) | Joint virtual reality and neurostimulation methods for visuomotor rehabilitation | |
| CN113925742B (en) | Control method and control system of target-driven upper limb exoskeleton rehabilitation robot | |
| Yang et al. | Head-free, human gaze-driven assistive robotic system for reaching and grasping | |
| CN111968723A (en) | Kinect-based upper limb active rehabilitation training method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |