CN113681541B - Exoskeleton control system and method based on Internet of things - Google Patents

Abstract

The present invention relates to the technical field of exoskeleton cooperative control, in particular to an exoskeleton control system and method based on the Internet of Things, including a controller, at least one camera, and at least one exoskeleton, and the controller is connected with the camera and the exoskeleton respectively Communication connection, the camera is installed in the activity space of the exoskeleton user, and is used to collect images in the activity space of the exoskeleton user, and the controller is used to obtain and identify the images collected by the camera, so as to identify the movement intention of the exoskeleton user and/or The gait is used to optimize the control parameters of the exoskeleton in a targeted manner, to realize the customized rehabilitation plan and the adjustment of rehabilitation strategies at different stages. The present invention utilizes existing cameras and controllers, as well as mature image recognition technology, to control the motion intention and/or gait recognition of the exoskeleton used in the activity space.

Description

A kind of exoskeleton control system and method based on internet of things

technical field

The present invention relates to the technical field of exoskeleton cooperative control, in particular to an exoskeleton control system and method based on the Internet of Things.

Background technique

In the gait synchronization control of the existing exoskeleton, the plantar pressure sensor, inclination sensor, etc. are usually used to sense the human's movement intention or gait, so as to realize the human-machine cooperative movement of the exoskeleton and the human. However, physical sensors have a lag in time, which is prone to misjudgment when the ground is uneven or when going up and down stairs. Bioelectric sensing information can directly reflect the wearer's motion intention. Compared with physical sensors, its response is faster, but the price is more expensive and the technology is not mature enough.

Now the Internet of Things technology has been widely used, and it is very common for smart devices, including surveillance cameras, to communicate with each other in homes, rehabilitation centers or other public places. For the exoskeleton used in a specific space, the present invention aims to use the existing equipment and combine mature image recognition technology to carry out cooperative control of the exoskeleton, so as to overcome the defects in the prior art.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies of the prior art, and provide an exoskeleton control system and method based on the Internet of Things, so as to overcome the problems of misjudgment and hysteresis in the existing exoskeleton collaborative control, and to reduce the exoskeleton the cost of.

An exoskeleton control system based on the Internet of Things, including a controller, at least one camera, at least one exoskeleton, and multiple adsorption detection units;

The adsorption detection unit includes a photosensitive point, an acceleration sensor, a gyroscope, a magnet, and a casing; the photosensitive point is fixed on the top of the casing, and an acceleration sensor and a gyroscope are arranged inside the casing; the magnet is arranged on the casing The bottom of the body; the adsorption detection unit is adsorbed on each joint of the exoskeleton; the camera detects photosensitive points in real time, and the acceleration sensor and gyroscope are wirelessly connected to the controller; the outermost layer of the exoskeleton is made of magnetic materials;

The controller is respectively connected in communication with the camera and the exoskeleton;

The camera is installed in the activity space of the exoskeleton user, and is used to collect image information of photosensitive points in the activity space of the exoskeleton user;

The controller receives the photosensitive point image information, and combines the detection data of the acceleration sensor and gyroscope in the detection of the adsorption detection unit to obtain the gait characteristics of the rehabilitation personnel. The gait characteristics include stride length, frequency, basic gait curve, active rehabilitation The resistance coefficient of different stages in a walking cycle under the mode;

The gait characteristics are compared with the historical gait characteristic data of the rehabilitation personnel stored in the controller, and the controller compares the rehabilitation recovery model or recovery curve to set the operating parameters of the exoskeleton robot for the next rehabilitation.

Further, no separate movement intention and/or gait recognition device is set on the exoskeleton.

Further, the exoskeleton is provided with an exoskeleton ID that can be identified by the controller through an image.

Further, the camera includes a driving device for driving the camera to rotate in all directions.

Further, the controller is connected with the camera and the exoskeleton by means of short distance communication.

Further, the short-distance communication method is Bluetooth, WIFI or ZigBee.

A kind of exoskeleton control method based on Internet of things, control method comprises the steps:

Step 1: The camera set in the activity space of the exoskeleton collects the images in the activity space of the exoskeleton user in real time, and transmits the images to the controller;

Step 2: The controller recognizes the image collected by the camera, and judges whether there is an exoskeleton device in the image; if it exists, execute step 3, and if not, return to step 1;

Step 3: According to the above image, further judge whether the exoskeleton is worn on the user's body, if so, identify the exoskeleton ID set on the exoskeleton, and make the controller establish a short-distance communication connection with the exoskeleton corresponding to the exoskeleton ID ;

Step 4: The controller recognizes the user's motion intention and/or gait according to the image collected by the camera, and controls the exoskeleton according to the recognition result, so that it can cooperate with the user.

Further, it also includes: Step 5: The controller controls the camera driving device to move in real time according to the position of the exoskeleton user in the image, so as to adjust the angle of the camera and follow the exoskeleton user so that the exoskeleton user is always in the The center position of the image.

Further, it also includes: Step 6: The controller controls the camera driving device to perform actions in real time according to the position of the exoskeleton user in the image, so as to adjust the angle of the camera so as to capture the front or side of the exoskeleton user to the maximum extent, This allows for better recognition of motion intentions or gait.

Further, in step 4, the controller not only recognizes the user's motion intention and/or gait according to the images collected by the camera, but also recognizes the user's current terrain, and according to the motion intention and/or The gait is used to recognize the structure and terrain recognition results, and the exoskeleton is controlled to make it cooperate with the user.

Compared with prior art, the beneficial effect of the present invention is:

The cameras and controllers installed in the activity space of the exoskeleton user used in the present invention are all original Internet of Things devices in the activity space, such as cameras and controllers that play a monitoring role. That is, the present invention utilizes existing external cameras and controllers, as well as mature image recognition technology, to control the motion intention and/or gait recognition of the exoskeleton used in the activity space. Based on this method, the exoskeleton A separate movement intention and/or gait recognition device does not need to be set on the skeleton, which can reduce costs, and the camera and the controller are all original IoT devices, and no additional installation is required.

In addition, due to the use of mature image recognition technology for motion intention and/or gait recognition, compared with physical sensors, its recognition accuracy is better, and it is faster in time, because it can pass through The relevant movements of the user's other limbs are used to judge the user's movement intention.

Description of drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The schematic embodiments and descriptions of the application are used to explain the application and do not constitute an improper limitation to the application. In the attached picture:

Fig. 1 is a schematic diagram of the exoskeleton control system of the present invention.

Fig. 2 is a schematic diagram of the communication connection between the controller and the exoskeleton of the present invention.

Fig. 3 is a flow chart of the exoskeleton control method of the present invention.

Fig. 4 is a schematic structural diagram of the adsorption detection unit of the present invention.

Fig. 5 is a schematic diagram of the installation of the adsorption detection unit of the present invention.

Detailed ways

The implementation of the present application will be described in detail below with reference to the accompanying drawings and examples, so as to fully understand and implement the implementation process of how the present application uses technical means to solve technical problems and achieve technical effects.

Embodiment one

As shown in Figures 1 and 2, the exoskeleton control system based on the Internet of Things described in this embodiment includes a controller 2, at least one camera 1, and at least one exoskeleton 3; the controller is connected to the camera and the exoskeleton respectively Skeleton communication connection; the camera is installed in the exoskeleton user’s activity space, such as on the wall or on the column, and is used to collect images in the exoskeleton user’s activity space in real time; the exoskeleton user’s activity space refers to the exoskeleton user’s activity space The space where the user of the exoskeleton resides, or the preset space in which the user of the exoskeleton can perform activities, such as home, hospital, rehabilitation center, etc. The activity space can be closed or open. The controller is used to obtain and recognize the images collected by the camera, so as to recognize the motion intention and/or gait of the user of the exoskeleton, and control the action of the exoskeleton according to the recognition result. Use images to identify the user's movement intention and/or gait. Specifically, the user's body posture can be judged in real time through image recognition to identify the user's movement intention. When the user is in a sitting position but leans forward, it is determined that the user intends to get up and move forward. Preset body movements can also be used to express specific motion intentions. When the user makes specific body movements, the controller can recognize the corresponding motion intentions. For example, the user waves his palm forward to indicate that he wants to For the intention of walking, if the user turns his head left or right twice, it means he wants to turn left or right. When the controller recognizes that the user is waving his palm forward through the image collected by the camera, it will identify the motion intention of the user. Walking forward, when it is detected that the user turns his head left or right twice in a row, the motion intention will be recognized that the user wants to turn left or right. For gait recognition, image recognition can be performed through the controller to obtain real-time human motion posture, such as joint movement angle, stride and other information, to recognize gait in real time. Of course, multiple gaits can also be pre-stored in the controller. Each gait can preset stride length and stride frequency, and express a specific gait through preset body movements. For example, the user stretches out a finger, which means that the user will use the preset first gait Walking, stretching out two fingers means that the user will walk with the preset second gait, and the controller acquires the user's gait by recognizing the preset body movements through the image. Of course, when performing movement intention and/or gait recognition, if the controller detects a preset body movement, the exoskeleton will be controlled according to the movement intention and/or gait corresponding to the body movement; otherwise, the image will be used to Recognition detects the user's body posture and/or motion posture in real time to recognize motion intention and/or gait and control the exoskeleton.

That is to say, the motion intention and/or gait recognition is performed by using an external camera and controller, instead of the motion intention and/or gait recognition device installed on the traditional exoskeleton equipment. Of course, the controller can also recognize the current terrain of the user at the same time, and recognize the structure and terrain recognition results according to the motion intention and/or gait, and control the action of the exoskeleton according to the recognition results. To control the action of the exoskeleton according to the recognition result, the controller may provide the user's motion intention and/or gait as an input signal to the main control system of the exoskeleton, and the main control system of the exoskeleton controls the movement of the exoskeleton accordingly. The controller can also directly control the power device of the exoskeleton according to the recognition result, and at this time, the controller also functions as the main control system of the exoskeleton. Controlling the movement of the exoskeleton is mainly to control the power device of the exoskeleton, at least including the control of the size of the exoskeleton, the speed of the movement, and the swing angle.

As shown in Figures 4 and 5, the adsorption detection unit 6 includes a photosensitive point 1, an acceleration sensor 3, a gyroscope 4, a magnet 5, and a housing 2;

The photosensitive point 1 is fixed on the top of the casing 2, and the interior of the casing 2 is provided with an acceleration sensor 3 and a gyroscope 4; the magnet 5 is arranged at the bottom of the casing 2; the adsorption detection unit 6 is adsorbed on the exoskeleton 7 Each joint position; the camera detects photosensitive points in real time, and the acceleration sensor and gyroscope are wirelessly connected to the controller;

The present invention is aimed at an exoskeleton used in a specific activity space (including but not limited to the user's home, hospital, rehabilitation center, etc.), and it is generally believed that the user will not leave the above-mentioned space wearing the exoskeleton device. When the exoskeleton is in the user's activity space, image recognition can be performed through the camera and the controller in the activity space to obtain the user's movement intention and/or gait. Therefore, there is no need to set a separate movement intention and/or gait recognition device on the exoskeleton, which reduces the manufacturing cost of the exoskeleton.

The exoskeleton is provided with an exoskeleton ID, and the controller can obtain the exoskeleton ID of the exoskeleton in the working state in the image, and establish a communication connection with the exoskeleton corresponding to the exoskeleton ID. The exoskeleton ID can be a serial number set outside the exoskeleton, or a specific pattern, etc., as long as it can be used to determine the identity of the exoskeleton.

Further, if there are multiple exoskeletons in the image captured by the camera, the controller can identify whether each exoskeleton is worn on the user, the user's movement intention and/or gait, and the ID of the exoskeleton .

The camera includes a driving device for driving the camera to rotate in all directions. Since the user is moving, in order to keep the best angle for shooting, the camera driving device can be controlled to make the camera follow and shoot the user. When there are multiple exoskeleton devices working at the same time, the angle of the camera can be controlled so that all the exoskeletons in the working state can be photographed by the camera at the same time.

The controller communicates with the camera and the exoskeleton by means of short-distance communication. The short-range communication method can use Bluetooth, WIFI or ZigBee. The short-range communication method can ensure that the distance between the controller, the camera and the exoskeleton will not be too far, and also limit the range of activities of the exoskeleton user to prevent them from leaving the preset The event space is in danger.

A reminder device can also be provided on the exoskeleton equipment. When the user of the exoskeleton moves out of the short-distance communication distance, that is, when the communication range with the controller is out of the range, the reminder device on the exoskeleton uses sound, light and/or vibration way to remind the user or accompanying personnel.

Embodiment two

As shown in Figure 3, the exoskeleton control method based on the Internet of Things described in this embodiment can use the exoskeleton control system of the first embodiment, and the control method includes:

Step 1: at least one camera set in the exoskeleton activity space collects images in the exoskeleton user activity space in real time, and transmits the images to the controller;

Step 2: The controller recognizes the image collected by the camera, and judges whether there is an exoskeleton device in the image; if it exists, execute step 3, and if not, return to step 1;

Step 3: According to the above image, further judge whether the exoskeleton is worn on the user's body, if so, identify the exoskeleton ID set on the exoskeleton, and make the controller establish a short-distance communication connection with the exoskeleton corresponding to the exoskeleton ID ;

Step 4: The controller recognizes the user's motion intention and/or gait according to the image collected by the camera, and controls the exoskeleton according to the recognition result, so that it can cooperate with the user.

Further, it also includes: Step 5: The controller controls the camera driving device to move in real time according to the position of the exoskeleton user in the image, so as to adjust the angle of the camera and follow the exoskeleton user.

The following of the exoskeleton user refers to making the exoskeleton user always in the center of the image, or photographing the front or side of the exoskeleton user to the maximum extent. The exoskeleton user is always in the center of the image, and the situation around the user can be photographed to the maximum extent, so that the controller can better perceive the information around the user, and the exoskeleton user can be photographed to the maximum extent. The front or the side is because the movement intention or gait can be judged more accurately from the front or the side, therefore, the movement intention or gait can be better recognized by adopting this following method.

Of course, if there are multiple exoskeleton devices in working state at the same time, you can also choose to enable all the exoskeleton devices in working state to be captured by the camera at the same time to control the camera.

Further, in step 4, the controller not only recognizes the user's motion intention and/or gait according to the images collected by the camera, but also recognizes the user's current terrain, and according to the motion intention and/or The gait is used to recognize the structure and terrain recognition results, and the exoskeleton is controlled to make it cooperate with the user.

When controlling, combined with the terrain recognition results, on the one hand, it can better eliminate misjudgments, and on the other hand, it can also predict the user's movement intention and/or gait according to the current terrain results, so as to better assist the user walk. Recognition of terrain, including but not limited to the recognition of the flatness of the ground, obstacles, and stairs. When in a special terrain, the user may adopt a specific walking style. For example, when the ground is found to be uneven (such as a pit) or there are obstacles, it is reasonable to expect that the user may choose to increase the stride or choose to go around Uneven ground or obstacles, through which the user's movement intention and/or gait can be predicted.

Based on this control method, there is no need to set up a separate movement intention and/or gait recognition device on the exoskeleton, which can reduce costs, and the camera and controller are all original IoT devices, and no additional installation is required. In addition, due to the use of mature image recognition technology for motion intention and/or gait recognition, compared with physical sensors, its recognition accuracy is better, and it is faster in time.

Certain terms are used, for example, in the description and claims to refer to particular components. Those skilled in the art should understand that hardware manufacturers may use different terms to refer to the same component. The specification and claims do not use the difference in name as a way to distinguish components, but use the difference in function of components as a criterion for distinguishing. As mentioned throughout the specification and claims, "comprising" is an open term, so it should be interpreted as "including but not limited to". "Approximately" means that within an acceptable error range, those skilled in the art can solve the technical problem within a certain error range and basically achieve the technical effect. In addition, the term "coupled" herein includes any direct and indirect electrical coupling means. Therefore, if it is described that a first device is coupled to a second device, it means that the first device may be directly electrically coupled to the second device, or indirectly electrically coupled through other devices or coupling means. connected to the second device. The subsequent description of the specification is a preferred implementation mode for implementing the application, but the description is for the purpose of illustrating the general principle of the application, and is not intended to limit the scope of the application. The scope of protection of the present application should be defined by the appended claims.

It should be noted that the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a good or system comprising a set of elements includes not only those elements but also items not expressly listed other elements, or also include elements inherent in the commodity or system. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the article or system comprising said element.

The above description shows and describes several preferred embodiments of the present invention, but as mentioned above, it should be understood that the present invention is not limited to the forms disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various Various other combinations, modifications, and environments can be made within the scope of the inventive concept described herein, by the above teachings or by skill or knowledge in the relevant field. However, changes and changes made by those skilled in the art do not depart from the spirit and scope of the present invention, and should all be within the protection scope of the appended claims of the present invention.

Claims (7)

1. An exoskeleton control system based on the Internet of things is characterized by comprising a controller, at least one camera, at least one exoskeleton and a plurality of adsorption type detection units; the adsorption type detection unit comprises a light sensing point, an acceleration sensor, a gyroscope, a magnet and a shell; the light sensing point is fixed at the top of the shell, and an acceleration sensor and a gyroscope are arranged in the shell; the magnet is arranged at the bottom of the shell; the absorption type detection units are respectively absorbed on the head, the shoulder, the chest, the abdomen, the hand joints, the hands, the hip joints, the knee joints and the feet of the exoskeleton; the camera detects the photosites in real time, and the acceleration sensor and the gyroscope are in wireless connection with the controller; the outermost layer of the exoskeleton is made of magnetic materials; a controller in communication with the camera and the exoskeleton, respectively; the camera is arranged in the motion space of the exoskeleton user and is used for acquiring the image information of the photosites in the motion space of the exoskeleton user; the controller receives image information of the photosensitive points, and combines detection data of the acceleration sensor and the gyroscope in detection of the adsorption detection unit to obtain gait characteristics of rehabilitation personnel, wherein the gait characteristics comprise stride, frequency, a basic gait curve and resistance coefficients of different stages in a walking cycle in an active rehabilitation mode; the gait characteristics are compared with the historical gait characteristic data of the rehabilitation personnel prestored in the controller, the controller compares a rehabilitation recovery model or a recovery curve, and the running parameters of the exoskeleton robot during the next rehabilitation are set; the camera comprises a driving device, a driving device and a control device, wherein the driving device is used for driving the camera to rotate in all directions; when a plurality of exoskeleton devices are in working states at the same time, controlling the angle of the camera to enable all the exoskeletons in the working states to be shot by the camera at the same time; the controller can also simultaneously identify the terrain where the user is located, identify the structure and the terrain identification result according to the movement intention and/or the gait, and control the action of the exoskeleton according to the identification result.

2. An internet of things based exoskeleton control system as claimed in claim 1 wherein the system can determine patient or health care personnel ID from face recognition.

3. The internet of things-based exoskeleton control system of claim 1 wherein the controller is coupled to the camera and the exoskeleton in a close range communication.

4. The internet of things-based exoskeleton control system of claim 3 wherein the close range communication means is Bluetooth, WIFI or ZigBee.

5. An internet of things-based exoskeleton control method as claimed in claim 1, wherein the control method of the internet of things-based exoskeleton control system comprises the steps of:

step 1: the camera arranged in the exoskeleton activity space acquires images in the exoskeleton user activity space in real time and transmits the images to the controller;

step 2: the controller identifies the image acquired by the camera and judges whether a patient exists in the image; if yes, executing step 3, and if no, returning to step 1;

and step 3: further judging whether the exoskeleton is worn on the user according to the image, if so, identifying the exoskeleton ID arranged on the exoskeleton, and enabling the controller to establish close-range communication connection with the exoskeleton corresponding to the exoskeleton ID;

and 4, step 4: and issuing the running parameters of the exoskeleton robot before rehabilitation, wherein the running parameters comprise stride, frequency, a basic gait curve and resistance coefficients of different stages in a walking cycle in an active rehabilitation mode.

6. The method for exoskeleton control based on the internet of things of claim 5, further comprising: and 5: the controller controls the camera driving device to act in real time according to the position of the exoskeleton user in the image so as to adjust the angle of the camera and follow the patient.

7. The method for exoskeleton control based on the internet of things as claimed in claim 6 wherein said following the patient means keeping the exoskeleton user in the center position of the image or shooting the front or side of the exoskeleton user to the maximum extent so as to better identify the movement intention or gait.

Images