CN114005511A

CN114005511A - Rehabilitation training method and system, training self-service equipment and storage medium

Info

Publication number: CN114005511A
Application number: CN202111304075.5A
Authority: CN
Inventors: 陈伟; 李小平; 李继才; 何佩东; 简卓; 曾浩; 王道雨
Original assignee: Shanghai Zhuodao Medical Technology Co ltd
Current assignee: Shanghai Zhuodao Medical Technology Co ltd
Priority date: 2021-11-05
Filing date: 2021-11-05
Publication date: 2022-02-01

Abstract

The invention provides a rehabilitation training method, a rehabilitation training system, training self-service equipment and a storage medium, wherein the method comprises the following steps: acquiring identity information of patients participating in rehabilitation training, and matching according to the identity information to obtain a corresponding rehabilitation training plan; acquiring coding information of a target training accessory selected according to a rehabilitation training plan from a scanning recognizer; acquiring a guide training picture and a guide training audio matched with a patient according to the coding information and the rehabilitation training plan; controlling a display screen to display a matched guide training picture and controlling a sound sensor to synchronously play a guide training audio; the posture data of the patient during rehabilitation training by using the target training accessory is obtained, and the posture data is sent to the display screen to synchronously display the role model and the motion state of the patient. The invention can guide the patient timely and effectively, improve the efficiency of rehabilitation training, and visually feed back the training condition of the patient to improve the willingness of the patient to participate actively.

Description

Rehabilitation training method and system, training self-service equipment and storage medium

Technical Field

The invention relates to the technical field of exercise rehabilitation, in particular to a rehabilitation training method, a rehabilitation training system, training self-service equipment and a storage medium.

Background

At present, the hospital rehabilitation department mostly uses the traditional tools to manually evaluate and train the limb functions of a patient, and the evaluation mainly comprises joint mobility, balance capability, posture control, motion control capability and the like; the training mainly comprises the exercise capacity, the muscle strength, the joint mobility, the balance function, the heart and lung function, the exercise tolerance intensity, the cognitive function and the like. The traditional training mode is mainly to perform active nerve rehabilitation training under the guidance of a therapist.

However, in the evaluation and training process, the attribute information of the training accessories of the patient, the standard degree of the action, whether compensation training exists or not, the active participation degree of the patient and the like cannot obtain real-time visual data, the training is relatively boring and tedious, the willingness of the active participation of the patient is reduced, the effect of the whole rehabilitation process is reduced, in the training process, a therapist needs to remind and guide the patient, and for the original lacking resources of the rehabilitation therapist, the patient cannot be guided timely and effectively, the professional degree of the training is reduced, and the efficiency of the rehabilitation training is reduced.

Disclosure of Invention

The invention aims to provide a rehabilitation training method, a rehabilitation training system, a training self-service device and a storage medium, and solves the technical problems that in the prior art, the rehabilitation training condition of a patient cannot be visualized, the training condition of the patient cannot be monitored in real time, so that real-time professional rehabilitation training guidance cannot be given, and the rehabilitation training cost is extremely high under the condition of relative shortage of rehabilitation medical resources.

The technical scheme provided by the invention is as follows:

the invention provides a rehabilitation training method, which comprises training self-service equipment, training accessories, an image sensor, a display screen, a sound sensor and a scanning recognizer, wherein the training accessories are provided with identity labels; the method is applied to the training self-service equipment and comprises the following steps:

acquiring identity information of patients participating in rehabilitation training, and matching according to the identity information to obtain a corresponding rehabilitation training plan;

acquiring coding information of a target training accessory selected according to the rehabilitation training plan from the scanning recognizer;

acquiring a guide training picture and a guide training audio matched with a patient according to the coding information and a rehabilitation training plan;

controlling the display screen to display and match the guide training picture, and controlling the sound sensor to synchronously play the guide training audio;

and acquiring posture data of the patient during training from the image sensor and/or the training accessory, and sending the posture data to the display screen to synchronously display the motion state of the patient role model.

Further, the acquiring the identity information of the patient participating in the rehabilitation training comprises the steps of:

if the display screen is a single-screen device, acquiring account information input by the patient from a first display area of the display screen, and matching the account information with a preset patient information base to obtain identity information of the patient; or the like, or, alternatively,

if the display screen is a multi-screen device and comprises an operation screen, acquiring account information input by the patient from the operation screen, and matching the account information with a preset patient information base to obtain the identity information of the patient; or the like, or, alternatively,

acquiring a face image of the patient from the image sensor, and matching the face image with a preset patient information base to obtain identity information of the patient; or the like, or, alternatively,

and acquiring the identification code of the identification signboard carried by the scanned patient from the scanning identifier, and matching the identification code with a preset patient information base to obtain the identity information of the patient.

Further, the display screen includes operation screen and interactive screen, training self-service equipment includes the equipment principal, training accessory, image sensor, display screen, sound sensor and scanning recognizer are located in the equipment principal, the operation screen rotate install in the equipment principal, still include the step:

adjusting the orientation of the operation screen to avoid the operation screen from blocking the interactive screen, acquiring input information of a therapist from the operation screen, creating a new patient record according to the input information, and updating the preset patient information base according to the patient record;

the preset patient information base comprises corresponding relations among account information, identification codes, face images and identity information of patients.

Further, after the guiding training picture and the guiding training audio matched with the patient are obtained, the step of controlling the display screen to display and match the guiding training picture comprises the following steps:

adjusting the orientation of the operation screen to face away from the visual field of the patient, or turning off the display content of the operation screen.

Further, the acquiring of the identity information of the patient participating in the rehabilitation training comprises the following steps:

identifying location coordinates of the patient relative to an image sensor from image data acquired from the image sensor;

generating a guide image comprising a guide navigation icon according to the position information of the target identification area, the role model of the patient and the position coordinates;

and controlling the interactive screen or a second display area of the display screen to display the guide graph so as to instruct the patient to perform rehabilitation training in the target recognition area.

Further, the acquiring the posture data of the patient during training from the image sensor and/or the training accessory, and sending the posture data to the display screen to synchronously display the motion state of the patient character model includes the steps of:

acquiring posture data of the patient during rehabilitation training from the image sensor and/or the training accessory;

acquiring training accessory attribute information from the target training accessory;

and synchronously displaying the motion state of the character model on an interactive screen or a second display area of the display screen according to the attitude data.

The invention also provides a rehabilitation training system, which comprises training self-service equipment, training accessories, an image sensor, a display screen, a sound sensor and a scanning recognizer, wherein the training accessories are provided with identity labels; the training self-service device comprises:

the processing module is used for acquiring the identity information of the patients participating in the rehabilitation training and obtaining a corresponding rehabilitation training plan according to the identity information in a matching mode;

the wireless communication module is used for acquiring the coding information of the target training accessory selected according to the rehabilitation training plan from the scanning recognizer;

the processing module is further used for acquiring a guide training picture and a guide training audio matched with the patient according to the coding information and the rehabilitation training plan;

the control module is used for controlling the display screen to display and match the guide training picture and controlling the sound sensor to synchronously play the guide training audio;

the wireless communication module is further used for acquiring posture data of a patient during rehabilitation training by using the target training accessory from the image sensor and/or the training accessory;

the control module is further used for sending the posture data to the display screen so as to synchronously display the role model and the motion state of the patient.

The invention also provides a training self-service device, which comprises a processor, a memory and a computer program which is stored in the memory and can run on the processor, wherein the processor is used for executing the computer program stored in the memory to realize the operation executed by the rehabilitation training method.

The invention also provides a storage medium, wherein at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to realize the operation executed by the rehabilitation training method.

By the aid of the rehabilitation training method and system, the training self-service equipment and the storage medium, the patient can be effectively guided in time, rehabilitation training efficiency is improved, and the training condition of the patient is visually fed back to improve the willingness of the patient to participate actively.

Drawings

The above features, technical features, advantages and implementations of a rehabilitation training method, system, training self-service device and storage medium will be further described in the following detailed description of preferred embodiments in a clearly understandable manner in connection with the accompanying drawings.

FIG. 1 is a flow chart of one embodiment of a rehabilitation training method of the present invention;

FIG. 2 is a schematic diagram of a single-screen configuration of one embodiment of the training kiosk of the present invention;

FIG. 3 is a schematic diagram of a dual-screen configuration of another embodiment of the training kiosk of the present invention;

FIG. 4 is a schematic view of a scene of an image selection patient of a rehabilitation training method of the present invention;

FIG. 5 is a diagram illustrating a scenario of image-synchronized display of patient movements for a rehabilitation training method according to the present invention;

fig. 6 is a schematic structural diagram of an embodiment of a rehabilitation training system of the present invention.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

For the sake of simplicity, the drawings only schematically show the parts relevant to the present invention, and they do not represent the actual structure as a product. In addition, in order to make the drawings concise and understandable, components having the same structure or function in some of the drawings are only schematically illustrated or only labeled. In this document, "one" means not only "only one" but also a case of "more than one".

It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

In addition, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not intended to indicate or imply relative importance.

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will be made with reference to the accompanying drawings. It is obvious that the drawings in the following description are only some examples of the invention, and that for a person skilled in the art, other drawings and embodiments can be derived from them without inventive effort.

In an embodiment of the present invention, as shown in fig. 1, fig. 2 and fig. 6, a rehabilitation training method includes a training self-service device 30, a training accessory 9, an image sensor 2, a display screen 10, a sound sensor 8 and a scanning recognizer 7, wherein an identity tag is arranged on the training accessory 9;

specifically, the image sensor 2 includes, but is not limited to, an RGB camera, an infrared camera, a depth camera, and the like having an image data acquisition function. The sound sensor 8 includes, but is not limited to, a speaker, a microphone array, and the like having a sound data collecting or playing function. The training self-service device 30 may comprise a device body, and the host, the image sensor 2, the display screen 10, the sound sensor 8 and the scanning identifier 7 are arranged on the device body, that is, the training self-service device 30 may be a robot integrating the host, the image sensor 2, the display screen 10, the sound sensor 8 and the scanning identifier 7 as shown in fig. 2. Certainly, the training self-service device 30 may also be only a terminal device such as a mobile phone, a tablet, a computer, a television, and the like, and the other terminal devices provided with the image sensor 2, the display screen 10, the sound sensor 8, and the scanning identifier 7 acquire the required data, and interact with the training self-service device 30 to implement the rehabilitation training method of the present invention.

The method is applied to the training self-service device 30 and comprises the following steps:

s100, acquiring identity information of a patient 3 participating in rehabilitation training, and matching according to the identity information to obtain a corresponding rehabilitation training plan;

specifically, the host computer can query basic information (including but not limited to age, sex, etc.) and disease information (dyskinesia type, dyskinesia level, electronic medical record, etc.) of the patient 3 according to the identity information of the patient 3, wherein the dyskinesia type includes upper limb dyskinesia, lower limb dyskinesia, etc. And inquiring a corresponding training database according to the basic information and the disease information, wherein the training database is preset with corresponding relations between training contents and age and sex, the type of the dyskinesia and the grade of the dyskinesia, and the training contents comprise the type of movement, the range of movement, the speed of movement, the training time and the times, so that a rehabilitation training plan suitable for the patient 3 can be searched according to the basic information and the disease information.

S200, acquiring coding information of a target training accessory 9 selected according to the rehabilitation training plan from the scanning recognizer 7;

s300, acquiring a guide training picture and a guide training audio matched with the patient 3 according to the coding information and the rehabilitation training plan;

s400, controlling the display screen 10 to display and match the guide training picture, and controlling the sound sensor 8 to synchronously play the guide training audio;

s500, posture data of the patient 3 during training is acquired from the image sensor 2 and/or the training accessory 9, and the posture data is sent to the display screen 10 to synchronously display the motion state of the character model 4 of the patient 3.

Specifically, the display screen 10 does not display in the standby state, and only displays the advertisement, Logo, and the like. The patient 3 or the therapist logs in (inputs account login, face recognition login, fingerprint login and the like) to enter an operation interface, the display screen 10 displays a menu page of the operation interface to select the patient 3, the display screen 10 simultaneously displays the character model 4 of the patient 3 to guide (maps the position coordinates of the patient 3 in the area relative to the image sensor 2 into the display screen 10) the patient 3 to go to the optimal recognition training area, namely the target recognition area 6, and the image sensor 2 is guided to walk to the optimal area so that the image sensor 2 can completely capture the human body action in the training process due to the fact that the image sensor 2 has the size of the field of view. The rehabilitation training plan is formulated and generated according to the identity information of the patient 3, the training accessory 9 is taken to perform scanning and recognition according to the rehabilitation training plan, then the patient 3 starts to perform rehabilitation training, the display screen 10 displays a guide training picture to perform picture guide rehabilitation training in the training process of the patient 3, and meanwhile, the voice playing of the sound sensor 8 guides the training audio to perform voice guide rehabilitation training. And synchronously driving the character model 4 corresponding to the patient 3 according to the posture data of the patient 3 during rehabilitation training by using the target training accessory 9 so as to synchronously display the motion state corresponding to the character model 4 of the patient 3 on the display screen 10, so that the character model 4 on the display screen 10 synchronously moves while the patient 3 moves.

The invention creates a brand-new interactive rehabilitation training mode of the patient 3 and the character model 4, and the patient 3 and the dynamic character model 4 describing the state of the patient 3 are synchronously interacted, so that the patient 3 is prevented from having bored emotion, the interactive reality sense is realized in real time, the interaction efficiency is improved, the rehabilitation training of the patient 3 is vivid and interesting, the willingness of the patient 3 to actively participate is improved, and the effect of the whole rehabilitation process is greatly improved. In addition, the rehabilitation training plan and the target training accessory 9 suitable for the patient 3 are matched according to the identity information of the patient 3, a guide training picture corresponding to the matched rehabilitation training plan is displayed on the display screen 10 in the training process, the guide training audio is synchronously played by the sound sensor 8, a therapist does not need to remind and guide the patient 3 personally, the patient 3 is assisted by the training self-service equipment 30 to accompany training, and the patient 3 is effectively guided in time to improve the efficiency of rehabilitation training.

In one embodiment of the invention, the rehabilitation training method comprises a training self-service device 30, a training accessory 9, an image sensor 2, a display screen 10, a sound sensor 8 and a scanning recognizer 7, wherein an identity label is arranged on the training accessory 9; the method is applied to the training self-service device 30 and comprises the following steps:

s110, if the display screen 10 is a single-screen device, acquiring account information input by the patient 3 from a first display area of the display screen 10, and matching the account information with a preset patient 3 information base to obtain identity information of the patient 3; or the like, or, alternatively,

specifically, as shown in fig. 2, the display screen 10 is a single-screen device, and the display screen 10 may include a first display area, where the first display area is used to receive touch operations of the patient 3 or the therapist, so as to realize interaction with the patient 3 or the therapist. For example, the first display area displays an input box, and the patient 3 can input account information (which may be an account ID of the patient 3, a mobile phone number or an identification number of the patient 3, or the like) in the input box.

S120, if the display screen 10 is a multi-screen device and the display screen 10 comprises an operation screen 5, acquiring account information input by the patient 3 from the operation screen 5, and matching the account information with a preset patient 3 information base to obtain identity information of the patient 3; or the like, or, alternatively,

specifically, as shown in fig. 3, the display screen 10 is a dual-screen device, the multi-screen display screen 10 includes an operation screen 5, and the first operation screen 5 is used for receiving a touch operation of the patient 3 or the therapist, so as to implement an interactive interaction with the patient 3 or the therapist. Of course, the display screen 10 may be a three-screen or other number of screens 10, and still be within the scope of the present invention. The double display screens 10 of the invention are convenient for the therapist to operate when the patient 3 carries out rehabilitation training, and can not influence the training of the patient 3. The operation display screen 10 is rotationally installed in the equipment main part, makes things convenient for patient 3 or therapist to use, and when patient 3 trained, the therapist can rotate operation display screen 10, keeps off main display screen 10 when preventing the therapist operation and influences patient 3's rehabilitation training. It is also possible to facilitate the operation for a small patient 3.

S130, acquiring a face image of the patient 3 from the image sensor 2, and matching the face image with a preset patient 3 information base to obtain identity information of the patient 3; or the like, or, alternatively,

s140, the identification code of the identification signboard carried by the scanned patient 3 is obtained from the scanning identifier 7, and the identification information of the patient 3 is obtained by matching the identification code with a preset patient 3 information base. Wherein, the identification signboard includes bracelet, identification card etc. that is equipped with the RFID chip.

S150, matching according to the identity information to obtain a corresponding rehabilitation training plan;

According to the invention, the identities of the patients 3 participating in the rehabilitation training are identified in the above manner, so that the rehabilitation training plans adapted to different patients 3 are matched and searched for different patients 3, the targeted rehabilitation training guidance is performed on different patients 3, the treatment pertinence is enhanced, and the individual rehabilitation training requirements of the patients 3 are well met. In addition, the invention is mainly based on an image recognition technology, the posture, the action, the position, the limb movement speed and the like of the human body are recognized and analyzed through the real-time acquisition of the image sensor 2 and the image processing technology, and the posture information of the patient 3 is mapped in a virtual environment with rich scenes to carry out the human-computer interaction rehabilitation training system and the method thereof. The invention provides a rehabilitation training device which is efficient to use, professional in training, safe in performance, interesting in content and diversified in functions, and a task-oriented training method with multi-dimensional sensory feedback is formed by combining image and voice interactive guidance, vibration feedback and the like, so that the active participation of a patient 3 is improved, the training effect is improved by combining a professional training method, the training efficiency is improved, and the patient can return to the family, the society and the work in 3 days.

In one embodiment of the invention, the rehabilitation training method comprises a training self-service device 30, a training accessory 9, an image sensor 2, a display screen 10, a sound sensor 8 and a scanning recognizer 7, wherein an identity label is arranged on the training accessory 9; the display screen 10 comprises an operation screen 5 and an interaction screen 1, the training self-service device 30 comprises a device body, the training accessories 9, the image sensor 2, the display screen 10, the sound sensor 8 and the scanning recognizer 7 are arranged on the device body, the operation screen 5 is rotatably installed on the device body, and the method is applied to the training self-service device 30 and comprises the following steps:

s010 identifies the position coordinates of the patient 3 with respect to the image sensor 2 from the image data acquired from the image sensor 2;

s020 generating a guide image comprising a guide navigation icon according to the position information of the target identification area 6, the role model 4 of the patient 3 and the position coordinate;

s030 controls the interactive screen 1 or a second display area of the display screen 10 to display the guide graphics so as to instruct the patient 3 to perform rehabilitation training in the target recognition area 6;

specifically, as shown in fig. 2, the display screen 10 is a single-screen device, and the display screen 10 may include a second display area, where the second display area is generally used for displaying the graphic data. For example, the second display area displays a publicity advertisement, Logo, etc., and may also display the character model 4 of the patient 3, a guide training screen, etc. As shown in fig. 3, the display screen 10 is a multi-screen device and the display screen 10 further includes an interactive screen 1, where the interactive screen 1 has the same function as the second display area of the single-screen display screen 10, and can display advertisement, Logo, and the like, and can also display the character model 4 of the patient 3, a guidance training picture, and the like.

s350, adjusting the orientation of the operation screen 5 to face away from the visual field of the patient 3, or turning off the display content of the operation screen 5;

s510, acquiring posture data of the patient 3 during rehabilitation training from the image sensor 2 and/or the training accessory 9;

s520, synchronously displaying the motion state of the character model 4 on a second display area of the interactive screen 1 or the display screen 10 according to the attitude data;

specifically, the patient 3 is bound with a role model 4 in a resource library, the role model 4 comprises skeleton information, the posture data is analyzed into real-time action parameters, the real-time action parameters comprise action part information, displacement information and angle information, the real-time action parameters are bound with the skeleton information to obtain corresponding model action data, and the role model 4 is controlled to generate virtual animation consistent with the behavior of the patient 3 according to the model action data. Illustratively, as shown in fig. 4 and 5, the posture data of the patient 3 is arm lifting, the arm lifting motion is analyzed into the information of the arm motion part, the arm displacement information and the arm rotation angle information, the arm of the person is bound with the arm skeleton information of the patient 3 character model 4, meanwhile, the arm lifting displacement information and the angle information are matched with the arm lifting animation information of the patient 3 character model 4 to obtain model motion data of arm lifting synchronous motion, and then the arm skeleton motion of the character model 4 is controlled according to the model motion data corresponding to the arm lifting of the patient 3 to generate a synchronous picture consistent with the arm lifting motion of the patient 3.

S600, acquiring the attribute information of the training accessory 9 from the target training accessory 9, and storing and recording the attribute information and the posture data of the training accessory 9 to obtain training data;

s700, the rehabilitation condition of the patient 3 is evaluated according to the training data, and the rehabilitation training plan is adjusted according to the evaluation result.

Specifically, the invention comprises an equipment body and a training accessory 9, wherein the equipment body comprises an image sensor 2, a host, a display screen 10 (an interactive screen 1 and an operation screen 5), a wireless communication module 32 (used for communication and data transmission with the training accessory 9) and an NFC identification module; almost all of the existing training kiosks 30 are one display screen 10.

The image sensor 2 is used for acquiring a human body image, wherein the image can be an RGB image or a depth image, but is not limited to one or a combination form thereof, and is mainly determined by an image processing algorithm; the image sensor 2 is fixed on a 2-degree-of-freedom holder, the angle of the image sensor 2 can be adjusted, and the adjusted object is beneficial to adapting to different use scenes and different body types of users, because the FOV (field of view) of the image sensor 2 is fixed, only image information in a certain range can be acquired within a certain distance, if the FOV (field of view) is fixed, the image range in a unit distance is fixed, but image information in a larger range can be acquired by adjusting the angle of the image sensor 2, so that more scenes can be adapted;

meanwhile, the image sensor 2 has a human face identification function, and can identify the face of the patient 3 in training through human face identification, so that the training termination caused by the fact that other people mistakenly enter an identification range in the training process is prevented; in addition, the human body tracking function of the image sensor 2 can be subjected to auxiliary identification and judgment through face identification, whether the target patient 3 tracked by the human body is the originally tracked patient 3 or not is judged, and an auxiliary quick judgment result is helped; in addition, when the human body of the image sensor 2 tracks a lost target (the loss reason can be that other people or objects block the characteristic region of the target object), the self-service device 30 can be trained to quickly retrieve the tracked target.

Meanwhile, the face recognition can be used for recognizing the identity information of the user, for recognizing the user information, and for the patient 3 to log in the training self-service device 30 quickly, and manage the information of the patient 3 (including the training information for training the patient 3) so that the training self-service device 30 can train the patient 3 in a customized manner. Meanwhile, the face recognition technology can also recognize other logged-in user information, for example, a therapist can log in quickly through the face recognition function to manage the patient 3 and the training self-service device 30.

Preferably, the method further comprises the following steps: s800, adjusting the orientation of the operation screen 5 to prevent the operation screen 5 from blocking the interactive screen 1, acquiring input information of a therapist from the operation screen 5, creating a new patient 3 record according to the input information, and updating the preset patient 3 information base according to the patient 3 record;

the preset patient 3 information base comprises corresponding relations among account information, identification codes, face images and identity information of the patient 3. S800 may occur between any one of the steps in the flow of steps S100 to S700, that is, in the process of the rehabilitation training of the patient 3, when the therapist may adjust the orientation of the operation screen 5 at any time to use the operation screen 5, the therapist may adjust the position convenient for the operation to operate conveniently, and when the therapist may adjust the orientation of the operation screen 5 at any time to use the operation screen 5, the screen of the operation screen 5 may be prevented from interfering with the patient 3 in the training process of the patient 3. Of course, in order to avoid the image interference of the operation panel 5, it may also be possible to directly and automatically stop sending the image-text data to the operation panel 5, or stop supplying power to the operation panel 5 so that the operation panel 5 is powered off and is blank, so as to turn off the display content of the operation panel 5.

The host is a processing center of the whole equipment main body and is used for collecting and processing various data. The display screen 10 mainly displays interactive information, including training pictures and the like, and for the convenience of user operation, the display screen 10 is divided into an interactive screen 1 and an operation screen 5, wherein the interactive screen 1 mainly displays the pictures of training and evaluation, and guides, feeds back and the like to a training patient 3; wherein, operation screen 5 has touch function, and user information's input is mutual, can revise training plan, parameter etc. of system, and for convenient use, this display screen 10's angle can be controlled, angle modulation from top to bottom, adjusts comfortable angle and position, and the person of facilitating the use uses, and simultaneously, operation screen 5 when operating, can not disturb and influence the interactive content of training screen. Meanwhile, the adjusting function of the operation screen 5 can be adapted to more people for operation without other tools. The wireless communication module 32 can be bluetooth, wifi, etc., and is mainly used for communication and data transmission between the host and the training accessory 9; the important of NFC identification module is the identification training accessory 9 attribute information, can be used for with training accessory 9 between quick pair, can be used to discernment NFC card, cell-phone NFC, can be used for therapist, patient 3's quick login, user information input and discernment. Wherein training accessory 9 mainly comprises intelligent wearing equipment, the accessory that increases weight, VR glasses, and training accessory 9 kind is not limited to this.

The intelligent wearable equipment is internally provided with a gesture recognition module, a vibration unit and a pressure recognition module, wherein the gesture recognition module can detect the gesture of the hand 3 of the patient, the pressure recognition module can recognize the real-time gripping strength of the patient 3, can complement the current bone recognition technology, supplements fine actions, and has the advantages of forearm pronation and supination, wrist joint flexion and extension, finger gripping state recognition and the like which are more than those of the current common bone recognition technology; wherein the vibration unit may provide vibrations of different vibration amplitudes, in combination with the training scenario, providing different vibration feedback to the patient 3. The intelligent wearable device is connected with the training self-service device 30, and transmits data to the training self-service device 30. Of course, the weight-increasing accessory may be provided with a motion sensor (e.g., a gyroscope, an acceleration sensor, etc.) and a wireless communication module 32, so that the training self-service device 30 can obtain posture data of the patient 3 during rehabilitation training from the weight-increasing accessory.

Wherein, the accessory that increases weight can be dumbbell, sand bag etc. for use with rehabilitation training plan cooperation, can enrich the training function. The training accessory 9 can be quickly identified by the training self-service device 30, and the identification mode is an NFC label which can be quickly identified and marked by an NFC identification module of the training self-service device 30; furthermore, the training accessory 9 can also be labeled with a two-dimensional code label, wherein the two-dimensional code label can be recognized by the image sensor 2 of the training self-service device 30. After recognition, the training self-service device 30 adapts to the proper training difficulty and training action according to the type of the patient 3 using the training accessories 9, meanwhile, the training self-service device 30 matches the attribute information of the training accessories 9 to the training, records the attribute information of the training accessories 9 into the posture data of the patient 3, and really grasps the actual recovery condition and the training performance of the patient 3.

Wherein VR glasses provide a immersive interactive mode of emulation, among the VR virtual scene that the scene of will training fuses for the scene of training accords with daily life more, increases the taste and the practicality of training, makes patient 3 fuse into real life more easily, fast after the training.

The cloud platform of the image sensor 2 is provided with an attitude recognition module, the attitude of the cloud platform is compensated, the current view field offset angle of the image sensor 2 is accurately calculated, and the image sensor is used for image compensation to obtain accurate human body attitude information. The image sensor 2 is arranged on a cloud platform, the cloud platform is provided with two groups of motors for controlling the pitch angle (pitch) and yaw angle (yaw) of the image sensor 2 to form left-right and up-down adjustment of the FOV of the image sensor 2, the left-right adjustment can be realized, and the FOV (field of view) range can be expanded on software. The cloud platform is provided with the attitude recognition module for calculating the attitude of the cloud platform in a compensation mode, wherein the attitude of the cloud platform can calculate the current attitude angle of the cloud platform according to the rotation angle of a cloud platform motor, but the attitude of the cloud platform is influenced due to the existence of possible errors, and the cloud platform is provided with the attitude recognition module which can perform compensation analysis on the calculated attitude angle and can accurately calculate the current view field offset angle of the image sensor 2.

The view field offset angle can perform offset compensation on the image, and the final result of the bone identification is compensated and corrected, so that the accuracy of the obtained bone information is ensured. Since the range of the field of view of the image sensor 2 is fixed, but when the user uses the training self-service device 30, the user needs to walk or move according to the rehabilitation action displayed on the display screen 10, and the user may go out of the field of view, so that the training self-service device 30 cannot accurately identify the user. After the visual field is changed, the user can be tracked, no matter how high or low the user is, whether the user stands close to the training self-service device 30 shown in fig. 2 or 3 or is far away from the training self-service device 30 shown in fig. 2 or 3, the user can be tracked by the visual field, but the user needs to move in the target recognition area 6 to be tracked and recognized. It is worth mentioning that the user may be not only the patient 3 but also a therapist.

The therapist firstly inputs the basic information of the patient 3, selects the training of the patient 3, the patient 3 follows the training track displayed on the interactive screen 1 or the second display area of the display screen 10, the image sensor 2 on the device can capture the motion of the patient 3, the training motion of the patient 3 is compared with the training track generated by the device, and finally, after the training is finished, the training report of the patient 3 is generated. Next, when the patient 3 is to be trained again, the therapist can perform the training by selecting the information of the patient 3 from the system without inputting the information of the patient 3 again. The image sensor 2 and/or the training accessories 9 of the invention are used for acquiring the posture data of the patient 3 in real time in the training process, and recording the posture data of the patient 3 in real time, matching with a corresponding algorithm to judge whether the action is correct or not, so as to analyze and count the accuracy of the rehabilitation training action of the patient 3, carrying out intelligent analysis according to the accuracy and accessory attribute information (including accessory weight, accessory type and the like), evaluating the rehabilitation condition of the patient 3, and automatically adjusting the training content, the training difficulty and the training progress according to the evaluation result, so that the training process is self-adaptive to the patient 3, intelligent and personalized rehabilitation training is realized, the efficiency of the rehabilitation training is improved, and the effect of the rehabilitation training is improved.

The invention is mainly based on an image recognition technology, and is a rehabilitation training system and a rehabilitation training method which are used for recognizing and analyzing the posture, the action, the position, the limb movement speed and the like of a human body through the real-time acquisition of an image sensor 2 and the combination of an image processing technology, and mapping the posture information of a patient 3 in a virtual environment with rich scenes to carry out man-machine interaction. The invention provides a rehabilitation training device which is efficient to use, professional in training, safe in performance, interesting in content and diversified in functions, a training method based on task guidance and provided with multi-dimensional sensory feedback is formed by combining a human body posture recognition system, voice, interactive guidance, vibration feedback and the like of the device, the active participation of a patient 3 is improved, the training effect is improved by combining a professional training method, the training efficiency is improved, and the patient 3 can return to the family, the society and work early.

According to the invention, the training accessory 9 is used for assisting the patient 3 to perform rehabilitation training, so that the training is finer, the existing equipment can only perform large-scale rehabilitation training, and after the training accessory 9 is worn, more training is performed, so that the application range of the equipment is wider. The change of the projection mode increases the interest, so that the participation of the patient 3 is higher. The equipment can not only perform orthopedic rehabilitation, but also perform nerve rehabilitation such as cerebral apoplexy. The device can be used for training some joints and also can be used for training some brains, and the training is carried out according to the animation output by the device. After the device automatically identifies the patient 3, the device automatically gives a treatment scheme and the like, so that the work of a therapist is reduced, and the labor is saved. The effect of rehabilitation training of the patient 3 can be automatically evaluated through the image sensor 2, and the interaction with the patient 3 is tighter. The image sensor 2 assists the patient 3 in training in the target recognition area 6, and can perform rehabilitation training not only at a predetermined place but also at home or the like.

The joint mobility, balance ability evaluation and the like of the existing patient 3 are carried out by using a paper mode, the operation is not simple, and the efficiency is low; meanwhile, the paper evaluation record can not facilitate the rehabilitation of the ability of the patient 3, can not generate visual data display, and can not visually display the recovery condition of the patient 3, the invention identifies and judges the common evaluation method of the hospital through an image processing technology, automatically evaluates the joint mobility and the balance ability of the patient 3, stores the evaluation result in the local and cloud sides, forms a record, and can check and analyze the training record of the patient 3 at any time. In addition, the invention stores the evaluation result in the equipment through the digitization of the hospital common evaluation and evaluation scale, is convenient and quick, and the equipment outputs the evaluation and evaluation result to form the evaluation record data of the patient 3 and visually displays the recovery condition of the patient 3.

In addition, the wearable and handheld training accessory 9 can be used in a combined mode, so that the equipment can cover more applicable people, meanwhile, due to the fact that fine movements of hands are added, the training interaction dimensionality is increased, and training is more interesting. Meanwhile, the vibration feedback module of the training accessory 9 is worn, so that the sensory feedback function can be provided for the patient 3, and the training can effectively attract the attention of the patient 3 by the aid of the guidance of spatial audio, so that one cycle of hands, eyes and brains is realized. The invention can automatically identify the patient 3 and automatically provide a treatment scheme by combining the evaluation and training results of the patient 3, thereby reducing the work of a rehabilitation therapist and saving the labor.

The invention also provides a rehabilitation training system, as shown in fig. 6, comprising a training self-service device 30, a training accessory 9, an image sensor 2, a display screen 10, a sound sensor 8 and a scanning recognizer 7, wherein the training accessory 9 is provided with an identity tag; the training self-service device 30 comprises:

the processing module 31 is configured to acquire identity information of the patient 3 participating in the rehabilitation training, and obtain a corresponding rehabilitation training plan according to the identity information;

the wireless communication module 32 is used for acquiring the coding information of the target training accessory 9 selected according to the rehabilitation training plan from the scanning recognizer 7;

the processing module 31 is further configured to obtain a guidance training picture and a guidance training audio matched with the patient 3 according to the encoded information and the rehabilitation training plan;

the control module 33 is configured to control the display screen 10 to display and match the guidance training picture, and control the sound sensor 8 to synchronously play the guidance training audio;

the wireless communication module 32 is further configured to acquire posture data of the patient 3 during rehabilitation training using the target training accessory 9 from the image sensor 2 and/or the training accessory 9;

the control module 33 is further configured to send the posture data to the interactive screen 1 to synchronously display the character model 4 and the motion state of the patient 3.

Specifically, this embodiment is a device embodiment corresponding to the method embodiment, and specific effects refer to the method embodiment, which is not described in detail herein.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of program modules is illustrated, and in practical applications, the above-described distribution of functions may be performed by different program modules, that is, the internal structure of the apparatus may be divided into different program units or modules to perform all or part of the above-described functions. Each program module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one processing unit, and the integrated unit may be implemented in a form of hardware, or may be implemented in a form of software program unit. In addition, the specific names of the program modules are only used for distinguishing the program modules from one another, and are not used for limiting the protection scope of the application.

In one embodiment of the invention, a training self-service device 30 comprises a host and a memory, wherein the memory is used for storing a computer program; and the host is used for executing the computer program stored on the memory to realize the rehabilitation training method in the corresponding method embodiment.

The training self-service device 30 may be a desktop computer, a notebook, a palm computer, a tablet computer, a mobile phone, a human-computer interaction screen, or the like. The training kiosk 30 may include, but is not limited to, a host, a memory. Those skilled in the art will appreciate that the above is merely an example of a training self-service device 30, does not constitute a limitation of training self-service device 30, and may include more or fewer components than shown, or some components in combination, or different components, such as: training kiosk 30 may also include input/output interfaces, display devices, network access devices, communication buses, communication interfaces, and the like. The communication interface and the communication bus can also comprise an input/output interface, wherein the host, the memory, the input/output interface and the communication interface are communicated with each other through the communication bus. The memory stores a computer program, and the host is used for executing the computer program stored on the memory to realize the rehabilitation training method in the corresponding method embodiment.

The host may be a Central Processing Unit (CPU), other general purpose host, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, or the like. The general purpose host may be a micro-host or the host may be any conventional host or the like.

The memory may be an internal storage unit of the training self-service device 30, such as: training the hard disk or memory of the self-service device 30. The memory may also be an external storage device of the training self-service device 30, such as: the training self-service device 30 is equipped with a plug-in hard disk, an intelligent memory Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like. Further, the memory may also include both an internal storage unit and an external storage device of the training self-service device 30. The memory is used to store the computer program and other programs and data required by the training kiosk 30. The memory may also be used to temporarily store data that has been output or is to be output.

A communication bus is a circuit that connects the described elements and enables transmission between the elements. For example, the host receives commands from other elements through the communication bus, decrypts the received commands, and performs calculations or data processing according to the decrypted commands. The memory may include program modules such as a kernel (kernel), middleware (middleware), an Application Programming Interface (API), and applications. The program modules may be comprised of software, firmware or hardware, or at least two of the same. The input/output interface forwards commands or data entered by a user via the input/output interface (e.g., sensor, keyboard, touch screen). The communication interface connects the training self-service device 30 with other network devices, user devices, networks. For example, the communication interface may be connected to a network by wire or wirelessly to connect to external other network devices or user devices. The wireless communication may include at least one of: wireless fidelity (WiFi), Bluetooth (BT), Near Field Communication (NFC), Global Positioning Satellite (GPS) and cellular communications, among others. The wired communication may include at least one of: universal Serial Bus (USB), high-definition multimedia interface (HDMI), asynchronous transfer standard interface (RS-232), and the like. The network may be a telecommunications network and a communications network. The communication network may be a computer network, the internet of things, a telephone network. The training kiosks may connect to the network through a communication interface, and a protocol used to train the kiosks to communicate with other network devices may be supported by at least one of an application, an Application Programming Interface (API), middleware, a kernel, and a communication interface.

In an embodiment of the present invention, a storage medium stores at least one instruction, and the instruction is loaded and executed by a host to implement the operation performed by the rehabilitation training method according to the embodiment. For example, the storage medium may be a read-only memory (ROM), a Random Access Memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

They may be implemented in program code that is executable by a computing device such that it is executed by the computing device, or separately, or as individual integrated circuit modules, or as a plurality or steps of individual integrated circuit modules. Thus, the present invention is not limited to any specific combination of hardware and software.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or recited in detail in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/training self-service device and method may be implemented in other ways. For example, the above-described apparatus/training kiosk embodiments are merely illustrative, and for example, the division of the modules or units is merely a logical division, and other divisions may be implemented in practice, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units may be stored in a storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the processes in the method according to the embodiments of the present invention may also be implemented by sending instructions to relevant hardware through a computer program, where the computer program may be stored in a storage medium, and when the computer program is executed by a host, the steps of the method embodiments may be implemented. Wherein the computer program may be in source code form, object code form, an executable file or some intermediate form, etc. The storage medium may include: any entity or device capable of carrying the computer program, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signal, telecommunication signal, software distribution medium, etc. It should be noted that the content of the storage medium may be increased or decreased as appropriate according to the requirements of legislation and patent practice in the jurisdiction, for example: in certain jurisdictions, in accordance with legislation and patent practice, computer-readable storage media do not include electrical carrier signals and telecommunications signals.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

It should be noted that the above embodiments can be freely combined as necessary. The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A rehabilitation training method is characterized by comprising training self-service equipment, training accessories, an image sensor, a display screen, a sound sensor and a scanning recognizer, wherein the training accessories are provided with identity labels; the method is applied to the training self-service equipment and comprises the following steps:

2. The rehabilitation training method according to claim 1, wherein the acquiring identity information of the patient participating in the rehabilitation training comprises the steps of:

3. The rehabilitation training method of claim 1, wherein the display screen comprises an operation screen and an interaction screen, the training self-service device comprises a device body, the training accessory, the image sensor, the display screen, the sound sensor and the scanning recognizer are arranged on the device body, the operation screen is rotatably mounted on the device body, and the method further comprises the following steps:

4. The rehabilitation training method according to claim 3, wherein after the acquisition of the guidance training picture and the guidance training audio matched with the patient, before the control of the display screen to display the matched guidance training picture, the method comprises the steps of:

5. The rehabilitation training method according to claim 2, wherein said obtaining identity information of the patient participating in the rehabilitation training comprises the steps of:

6. The rehabilitation training method according to any one of claims 1-5, wherein the step of acquiring posture data of the patient during training from the image sensor and/or the training accessory, and sending the posture data to the display screen to synchronously display the motion state of the patient character model comprises the steps of:

7. The rehabilitation training method according to claim 6, wherein the step of synchronously displaying the motion state of the character model on the interactive screen or the second display area of the display screen according to the posture data comprises the following steps:

acquiring training accessory attribute information from the target training accessory, and storing and recording the training accessory attribute information and the posture data to obtain training data;

and evaluating the rehabilitation condition of the patient according to the training data, and adjusting the rehabilitation training plan according to the evaluation result.

8. A rehabilitation training system is characterized by comprising training self-service equipment, training accessories, an image sensor, a display screen, a sound sensor and a scanning recognizer, wherein the training accessories are provided with identity labels; the training self-service device comprises:

9. A training kiosk comprising a processor, a memory, and a computer program stored in and executable on the memory, the processor configured to execute the computer program stored on the memory to perform the operations performed by the rehabilitation training method of any one of claims 1-7.

10. A storage medium having stored therein at least one instruction, which is loaded and executed by a processor to perform the operations performed by the rehabilitation training method according to any one of claims 1 to 7.