CN112214109A - Compound control method, device and system based on myoelectricity and attitude data - Google Patents
Compound control method, device and system based on myoelectricity and attitude data Download PDFInfo
- Publication number
- CN112214109A CN112214109A CN202011063852.7A CN202011063852A CN112214109A CN 112214109 A CN112214109 A CN 112214109A CN 202011063852 A CN202011063852 A CN 202011063852A CN 112214109 A CN112214109 A CN 112214109A
- Authority
- CN
- China
- Prior art keywords
- signal
- target
- myoelectricity
- determining
- control mode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Dermatology (AREA)
- General Health & Medical Sciences (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the invention provides a compound control method, a device and a system based on myoelectricity and posture data, which are applied to a compound control system based on the myoelectricity and posture data, wherein the compound control system based on the myoelectricity and posture data comprises at least one myoelectricity collector and an inertial sensor, and myoelectricity signals collected by the at least one myoelectricity collector are obtained; determining a target control mode corresponding to the electromyographic signal; acquiring attitude data acquired by an inertial sensor; and executing target control operation in the target control mode according to the posture data, so that the operation control can be cooperatively performed based on myoelectricity and posture recognition, and the flexibility of the compound control system based on the myoelectricity and the posture data is improved.
Description
Technical Field
The invention relates to the field of operation control, in particular to a compound control method, a device and a system based on myoelectricity and posture data.
Background
In a single myoelectricity control system, myoelectricity generated by movement intentions is utilized, a pattern recognition algorithm is applied, a control pattern is recognized according to myoelectricity signals, and then the operation of the system is controlled, but a recognition result can only represent scalar quantity (only size without direction), so that one recognition result can only control one fixed operation of the system at the same time, and a complex system with directionality is difficult to realize system cooperative operation, and each recognition result has large delay and is not beneficial to continuous operation.
Disclosure of Invention
The embodiment of the invention provides a compound control method, device and system based on myoelectricity and posture data, which can be used for operating and controlling based on myoelectricity and posture recognition cooperation, so that the flexibility of the compound control system based on the myoelectricity and posture data is improved.
The embodiment of the invention provides a compound control method based on myoelectricity and posture data, which is applied to a compound control system based on the myoelectricity and posture data, wherein the compound control system based on the myoelectricity and posture data comprises at least one myoelectricity collector and an inertial sensor, and the method comprises the following steps:
acquiring myoelectric signals collected by the at least one myoelectric collector;
determining a target control mode corresponding to the electromyographic signal;
acquiring attitude data acquired by the inertial sensor;
and executing target control operation in the target control mode according to the attitude data.
Optionally, the determining a target control mode corresponding to the electromyographic signal includes:
processing the electromyographic signals to obtain target signal processing data;
and determining a target control mode corresponding to the target signal processing data according to a mapping relation between preset signal processing data and control modes.
Optionally, the processing the electromyographic signal to obtain target signal processing data includes:
the electromyographic signals are processed to obtain an electromyographic signal curve,
determining a target segment with a signal peak value in the electromyographic signal curve, wherein the starting position of the target segment corresponds to a first signal valley value, and the ending position of the target segment corresponds to a second signal valley value;
determining a target signal peak value, a first signal point with a rising speed larger than a first value and a second signal point with a falling speed larger than a second value in the target segment, wherein the first signal point corresponds to a first time point; the second signal point corresponds to a second time point;
determining a first signal strength value corresponding to the first signal point and a second signal strength value corresponding to the second signal point, the first signal strength value being greater than or equal to the first signal valley, the second signal strength value being greater than or equal to the second signal valley;
determining a first duration between the first point in time and the second point in time;
determining a first signal strength difference between the first signal strength value and the second signal strength value;
determining a first offset value corresponding to the target signal peak value;
and determining a first signal range according to the first offset value and the first signal strength difference value, and taking the first time length and the first signal range as the target signal processing data.
Optionally, the determining, according to a mapping relationship between preset signal processing data and a control mode, a target control mode corresponding to the target signal processing data includes:
matching the first time length with the time length in the mapping relation between the preset signal processing data and the control mode; matching the first signal range with a signal range in a mapping relation between preset signal processing data and a control mode to obtain a preset time length successfully matched with the first time length and a preset signal range successfully matched with the first signal range;
and determining a target control mode corresponding to the preset duration and the preset signal range in the mapping relation.
Optionally, the method further comprises:
if the electromyographic signal curve comprises a plurality of segments, each segment comprises a signal peak value, and the maximum peak value in a plurality of signal peak values corresponding to the segments is determined;
determining a target segment of the plurality of segments corresponding to the maximum peak.
Optionally, the myoelectricity and posture data-based compound control system further comprises a controlled device, and the controlled device comprises at least two of the following components: the device comprises a chassis, a holder, a mechanical arm and a mechanical arm;
the target control mode includes any one of: an emergency stop mode, a chassis translation mode, a chassis rotation and pan-tilt rotation mode, a robotic arm mode, and a manipulator mode.
Optionally, the attitude data comprises acceleration and angular velocity; the executing of the target control operation in the target control mode according to the attitude data includes:
determining a target component for performing the target control operation in the target control mode;
calculating control parameters of the target component according to the attitude data;
and controlling the target component to execute the target control operation according to the control parameter.
A second aspect of an embodiment of the present invention provides a compound control device based on myoelectricity and posture data, which is applied to a compound control system based on myoelectricity and posture data, wherein the compound control system based on myoelectricity and posture data includes at least one myoelectricity collector and an inertial sensor, and the device includes:
the acquisition unit is used for acquiring the electromyographic signals acquired by the at least one electromyographic acquisition unit;
the determining unit is used for determining a target control mode corresponding to the electromyographic signal;
the acquisition unit is also used for acquiring attitude data acquired by the inertial sensor;
and the execution unit is used for executing target control operation in the target control mode according to the attitude data.
A third aspect of an embodiment of the present invention provides a compound control system based on myoelectricity and posture data, which includes at least one myoelectricity collector, an inertial sensor, a controlled device and a controller, where the at least one myoelectricity collector and the inertial sensor are both connected to the controller, where,
the myoelectricity collector is used for collecting myoelectricity signals of a human body;
the controller is used for determining a target control mode corresponding to the electromyographic signal;
the inertial sensor is used for acquiring posture data of a human body;
the controller is further used for controlling the controlled equipment to execute target control operation in the target control mode according to the attitude data.
A fourth aspect of embodiments of the present invention provides a computer-readable storage medium for storing a computer program, the computer program being executed by a processor to implement some or all of the steps described in the method according to the first aspect of embodiments of the present invention.
A fifth aspect of embodiments of the present invention provides a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the method according to the first aspect of embodiments of the present invention.
The embodiment of the invention has at least the following beneficial effects:
the myoelectricity and posture data-based compound control method, the device and the system are applied to a myoelectricity and posture data-based compound control system, the myoelectricity and posture data-based compound control system comprises at least one myoelectricity collector and an inertial sensor, and myoelectricity signals collected by the at least one myoelectricity collector are obtained; determining a target control mode corresponding to the electromyographic signal; acquiring attitude data acquired by an inertial sensor; and executing target control operation in the target control mode according to the posture data, so that the operation control can be cooperatively performed based on myoelectricity and posture recognition, and the flexibility of the compound control system based on the myoelectricity and the posture data is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic structural diagram of a compound control system based on electromyography and posture data according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a compound control method based on electromyography and posture data according to an embodiment of the present invention;
FIG. 3 is a flow chart of another compound control method based on electromyography and posture data according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a compound control device based on myoelectric and posture data according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," and the like in the description and claims of the present invention and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by the person skilled in the art that the described embodiments of the invention can be combined with other embodiments.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a compound control system based on myoelectricity and posture data according to an embodiment of the present invention, where the compound control system based on myoelectricity and posture data includes at least one myoelectricity collector, an inertial sensor, a controlled device and a controller, and the at least one myoelectricity collector and the inertial sensor are both connected to the controller, where,
the myoelectricity collector is used for collecting myoelectricity signals of a human body;
the controller is used for determining a target control mode corresponding to the electromyographic signal;
the inertial sensor is used for acquiring posture data of a human body;
the controller is further used for controlling the controlled equipment to execute target control operation in the target control mode according to the attitude data.
The myoelectric collector comprises an electrode array, at least 2 paths of differential signal collecting circuits, a control module, a transmission module and a power supply module, wherein the electrode array is used for contacting with a human body to collect myoelectric signals, the at least 2 paths of differential signal collecting circuits are used for preprocessing the collected myoelectric signals to obtain processed myoelectric signals, and the preprocessing can comprise at least one of the following steps: signal amplification, analog-to-digital conversion, low-pass filtering, electromagnetic interference filtering and the like; the transmission module is used for transmitting the preprocessed electromyographic signals to the controller.
The inertial sensor can be used for collecting posture data such as acceleration, angular velocity and the like generated when limbs of a human body move.
The controller controls the controlled device to execute the target control operation in the target control mode according to the attitude data, so that after the target control mode is identified according to the electromyographic signals, the target control operation in the target control mode is directionally executed according to the attitude data control, and the operation control intelligence is improved.
Optionally, in the aspect of determining the target control mode corresponding to the electromyographic signal, the controller is specifically configured to:
processing the electromyographic signals to obtain target signal processing data;
and determining a target control mode corresponding to the target signal processing data according to a mapping relation between preset signal processing data and control modes.
Optionally, in the aspect of processing the electromyographic signal to obtain target signal processing data, the controller is specifically configured to:
the electromyographic signals are processed to obtain an electromyographic signal curve,
determining a target segment with a signal peak value in the electromyographic signal curve, wherein the starting position of the target segment corresponds to a first signal valley value, and the ending position of the target segment corresponds to a second signal valley value;
determining a target signal peak value, a first signal point with a rising speed larger than a first value and a second signal point with a falling speed larger than a second value in the target segment, wherein the first signal point corresponds to a first time point; the second signal point corresponds to a second time point;
determining a first signal strength value corresponding to the first signal point and a second signal strength value corresponding to the second signal point, the first signal strength value being greater than or equal to the first signal valley, the second signal strength value being greater than or equal to the second signal valley;
determining a first duration between the first point in time and the second point in time;
determining a first signal strength difference between the first signal strength value and the second signal strength value;
determining a first offset value corresponding to the target signal peak value;
and determining a first signal range according to the first offset value and the first signal strength difference value, and taking the first time length and the first signal range as the target signal processing data.
Optionally, in the aspect that the target control mode corresponding to the target signal processing data is determined according to a preset mapping relationship between the signal processing data and the control mode, the controller is specifically configured to:
matching the first time length with the time length in the mapping relation between the preset signal processing data and the control mode; matching the first signal range with a signal range in a mapping relation between preset signal processing data and a control mode to obtain a preset time length successfully matched with the first time length and a preset signal range successfully matched with the first signal range;
and determining a target control mode corresponding to the preset duration and the preset signal range in the mapping relation.
Optionally, the controlled device comprises at least two of the following components: the device comprises a chassis, a holder, a mechanical arm and a mechanical arm;
the target control mode includes any one of: an emergency stop mode, a chassis translation mode, a chassis rotation and pan-tilt rotation mode, a robotic arm mode, and a manipulator mode.
Alternatively, the controller may be disposed on the controlled device, or may be independently disposed outside the controlled device.
Optionally, the controller is further configured to:
if the electromyographic signal curve comprises a plurality of segments, each segment comprises a signal peak value, and the maximum peak value in a plurality of signal peak values corresponding to the segments is determined;
determining a target segment of the plurality of segments corresponding to the maximum peak.
Optionally, the attitude data comprises acceleration and angular velocity; in the aspect of performing the target control operation in the target control mode according to the attitude data, the controller is specifically configured to:
determining a target component for performing the target control operation in the target control mode;
calculating control parameters of the target component according to the attitude data;
and controlling the target component to execute the target control operation according to the control parameter.
Wherein the controlled device may comprise the following components: the system comprises a chassis, a cloud deck, a mechanical arm and a mechanical arm, wherein specifically, in an emergency stop mode, controlled equipment can be controlled to stop immediately; in the chassis translation mode, the controlled equipment can be controlled to translate by using the control parameters; under the modes of chassis rotation and holder rotation, the controlled equipment can be controlled to rotate and the holder can be controlled to rotate respectively by using control parameters; in the mechanical arm mode, the control parameters can be used for respectively controlling the mechanical arm to move up and down and move left and right; in the manipulator mode, the manipulator can be controlled to grip using the control parameters.
The myoelectricity and posture data based composite control system comprises at least one myoelectricity collector, an inertial sensor, controlled equipment and a controller, wherein the at least one myoelectricity collector and the inertial sensor are connected with the controller, and the myoelectricity collector is used for collecting myoelectricity signals of a human body; the controller is used for determining a target control mode corresponding to the electromyographic signal; the inertial sensor is used for acquiring posture data of a human body; the controller is also used for controlling the controlled device to execute target control operation in a target control mode according to the posture data, so that the operation control can be cooperatively carried out based on myoelectricity and posture recognition, and the flexibility of the compound control system based on the myoelectricity and the posture data is improved.
Referring to fig. 2, fig. 2 is a flowchart illustrating a compound control method based on electromyography and posture data according to an embodiment of the present invention. As shown in fig. 2, the compound control method based on the myoelectricity and posture data according to the embodiment of the present invention is applied to the compound control system based on the myoelectricity and posture data shown in fig. 1, wherein the compound control system based on the myoelectricity and posture data comprises at least one myoelectricity collector and an inertial sensor, and the method may include the following steps:
201. acquiring myoelectric signals collected by the at least one myoelectric collector;
wherein, at least one myoelectricity collector can collect the myoelectricity signal of human body.
Optionally, the number of the myoelectric collectors is multiple, the myoelectric collectors are distributed at different positions of the human body, and the myoelectric signals at different positions can be collected by the myoelectric collectors at different positions, so that the movement intention of the user can be more accurately analyzed according to the myoelectric signals reflected by different positions of the human body.
202. Determining a target control mode corresponding to the electromyographic signal;
in a specific implementation, a plurality of control modes for realizing various functions by different parts of the controlled device can be set for different controlled devices, and corresponding myoelectric signals are different when a user wants to select different control modes, so that a mapping relation between the myoelectric signals and a target control mode can be preset, and the target control mode corresponding to the myoelectric signals can be determined according to the mapping relation.
Wherein the target control mode includes any one of: an emergency stop mode, a chassis translation mode, a chassis rotation and pan-tilt rotation mode, a robotic arm mode, and a manipulator mode.
Optionally, in step 202, the determining a target control mode corresponding to the electromyographic signal includes:
21. processing the electromyographic signals to obtain target signal processing data;
22. and determining a target control mode corresponding to the target signal processing data according to a mapping relation between preset signal processing data and control modes.
The method includes the steps of obtaining a plurality of preset signal processing data in advance, obtaining a control mode corresponding to each signal processing data in the plurality of signal processing data to obtain a plurality of control modes, and then establishing a mapping relation between the signal processing data and the control modes. As shown in table 1 below, an example of a mapping relationship between signal processing data and control modes is provided for the embodiment of the present invention:
| signal processing data | Control mode |
| Signal processing data 1 | Control mode 1 |
| Signal processing data 2 | Control mode 2 |
| Signal processing data 3 | Control mode 3 |
| ... | ... |
| Signal processing data n | Control mode n |
TABLE 1
Alternatively, the electromyographic signal may be subjected to signal processing to obtain an electromyographic signal curve, and then, a target control mode corresponding to the electromyographic signal curve is determined according to a mapping relationship between a preset signal curve and the control mode.
The method includes the steps of obtaining a plurality of preset signal curves in advance, obtaining a control mode corresponding to each signal curve in the signal curves to obtain a plurality of control modes, and then establishing a mapping relation between the signal curves and the control modes.
Optionally, in step 21, the processing the electromyographic signal to obtain target signal processing data includes:
2101. the electromyographic signals are processed to obtain an electromyographic signal curve,
2102. determining a target segment with a signal peak value in the electromyographic signal curve, wherein the starting position of the target segment corresponds to a first signal valley value, and the ending position of the target segment corresponds to a second signal valley value;
2103. determining a target signal peak value, a first signal point with a rising speed larger than a first value and a second signal point with a falling speed larger than a second value in the target segment, wherein the first signal point corresponds to a first time point; the second signal point corresponds to a second time point;
2104. determining a first signal strength value corresponding to the first signal point and a second signal strength value corresponding to the second signal point, the first signal strength value being greater than or equal to the first signal valley, the second signal strength value being greater than or equal to the second signal valley;
2105. determining a first duration between the first point in time and the second point in time;
2106. determining a first signal strength difference between the first signal strength value and the second signal strength value;
2107. determining a first offset value corresponding to the target signal peak value;
2108. and determining a first signal range according to the first offset value and the first signal strength value, and taking the first time length and the first signal range as the target signal processing data.
In a specific implementation, considering that when a user generates different movement intentions, the reaction time of the electromyographic signals and the signal intensity of the electromyographic signals present different electromyographic signal curves, the electromyographic signal curves can be analyzed to identify the movement intentions for the user, specifically, a target segment with a signal peak value in the electromyographic signal curve can be determined, then a target signal peak value in the target segment can be determined, a first signal point with a rising speed larger than a first value and a second signal point with a falling speed larger than a second value in the target segment can also be determined, generally, on the side of the target signal peak value, the signal intensity of the electromyographic signals presents a rising trend, and a first signal point with the steepest change of the target signal peak value in the electromyographic signal curve on the side can be determined; on the other side of the target signal peak value, the signal intensity of the electromyographic signal is in a descending trend, a second signal point of the target signal peak value with the steepest change in the side electromyographic signal curve can be determined, and the first signal point corresponds to the first time point; the second signal point corresponds to a second time point; furthermore, a first time length can be determined according to the first time point and the second time point, the first time length can be used for representing the time length of the myoelectric signal of the human body in the process of generating the movement consciousness, and a first signal strength difference value between a first signal strength value and a second signal strength value can also be determined; determining a first offset value corresponding to a target signal peak value; the first signal range is determined according to the first offset value and the first signal strength value, the first signal range can represent the range of the change of the electromyographic signals of the human body in the process of generating the sports consciousness, the change of the duration and the signal strength of the change of the electromyographic signals of the human body in the process of generating the sports consciousness are different under different sports intentions, therefore, the target control mode of the user can be determined according to the first duration and the first signal range, the first offset value can be used for representing the error amount corresponding to the peak value of the target signal, the first signal range can be controlled within the error allowable range by determining the first signal range according to the first offset value and the first signal strength difference value, and therefore, the target control mode can be determined more accurately according to the first duration and the first signal range.
Optionally, in step 22, the determining, according to a preset mapping relationship between the signal processing data and the control mode, the target control mode corresponding to the target signal processing data includes:
matching the first time length with the time length in the mapping relation between the preset signal processing data and the control mode; matching the first signal range with a signal range in a mapping relation between preset signal processing data and a control mode to obtain a preset time length successfully matched with the first time length and a preset signal range successfully matched with the first signal range;
and determining a target control mode corresponding to the preset duration and the preset signal range in the mapping relation.
In specific implementation, a mapping relationship between the signal processing data and the control mode may be preset, as shown in table 2 below, which is another example of a mapping relationship between the signal processing data and the control mode provided in the embodiment of the present invention:
TABLE 2
The target control mode corresponding to the preset time length and the preset signal range is determined, and therefore the target control mode can be determined more accurately based on the first time length and the first signal range.
203. Acquiring attitude data acquired by the inertial sensor;
the posture data may include acceleration, angular velocity, and the like of the human body entity.
In an implementation, the user may wear the inertial sensor on a limb of the body for controlling the controlled device, such as an arm, a wrist, a finger, a waist, and the like, without limitation.
204. And executing target control operation in the target control mode according to the attitude data.
Specifically, the euler angle may be calculated from the attitude data such as the acceleration, the angular velocity, and the like, and then the controlled device may be controlled to perform the target control operation in the target control mode according to the euler angle.
The euler angles may include a pitch angle, a roll angle, and a yaw angle, among others.
Optionally, the controlled device of the compound control system based on electromyography and posture data comprises at least two components: the device comprises a chassis, a holder, a mechanical arm and a mechanical arm; the target control mode includes any one of: an emergency stop mode, a chassis translation mode, a chassis rotation and pan-tilt rotation mode, a robotic arm mode, and a manipulator mode.
Optionally, the executing the target control operation in the target control mode according to the attitude data includes:
determining a target component for performing the target control operation in the target control mode;
calculating control parameters of the target component according to the attitude data;
and controlling the target component to execute the target control operation according to the control parameter.
Specifically, an emergency stop mode, a chassis translation mode, a chassis rotation and pan-tilt rotation mode, a robot arm mode, and a manipulator mode may be set in advance for the above-described controlled apparatus. Therefore, in the emergency stop mode, the controlled equipment can be controlled to stop immediately; in the chassis translation mode, the controlled equipment can be controlled to translate by using the control parameters; under the modes of chassis rotation and holder rotation, the controlled equipment can be controlled to rotate and the holder can be controlled to rotate respectively by using control parameters; in the mechanical arm mode, the control parameters can be used for respectively controlling the mechanical arm to move up and down and move left and right; in the manipulator mode, the manipulator can be controlled to grip using the control parameters.
Wherein, the control parameter may include at least one of the following: the distance of chassis translation, the direction of chassis rotation, the angle of chassis rotation, the direction of pan-tilt rotation, the angle of pan-tilt rotation, the direction of arm movement, the amplitude of arm movement, and the like, without limitation.
Alternatively, in order to ensure control stability and safety of the composite control system based on myoelectric and posture data, the state of the control mode may be indicated using a light, vibration, or the like.
It can be seen that in the embodiment of the invention, the myoelectric signals collected by at least one myoelectric collector are obtained; determining a target control mode corresponding to the electromyographic signal; acquiring attitude data acquired by an inertial sensor; and executing target control operation in the target control mode according to the posture data, so that the operation control can be cooperatively performed based on myoelectricity and posture recognition, and the flexibility of the compound control system based on the myoelectricity and the posture data is improved.
For example, as shown in fig. 3, fig. 3 is a flowchart of another compound control method based on electromyography and posture data according to the present invention. The method comprises the steps of acquiring myoelectric signals acquired by at least one myoelectric acquisition unit, processing the myoelectric signals to obtain target signal processing data, and determining a target control mode corresponding to the target signal processing data from a mapping relation between preset signal processing data and control modes. Acquiring attitude data such as acceleration, angular velocity and the like acquired by an inertial sensor; determining a target component for executing the target control operation in the target control mode, performing attitude calculation according to acceleration and angular velocity to obtain control parameters of the target component, wherein the control parameters can comprise pitch angle and roll angle, controlling the target component to execute the target control operation according to the pitch angle and the roll angle, and controlling the controlled equipment to stop immediately in the emergency stop mode; in the chassis translation mode, the controlled equipment can be controlled to translate by using the control parameters; under the modes of chassis rotation and holder rotation, the controlled equipment can be controlled to rotate and the holder can be controlled to rotate respectively by using control parameters; in the mechanical arm mode, the control parameters can be used for respectively controlling the mechanical arm to move up and down and move left and right; in the manipulator mode, the manipulator can be controlled to hold by using the control parameters, so that the controlled equipment can be directionally controlled, and the flexibility of the compound control system based on myoelectricity and posture data is improved.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a combined control apparatus based on myoelectric and posture data 400 according to the present embodiment, which is applied to the combined control system based on myoelectric and posture data shown in fig. 1, the combined control system based on myoelectric and posture data includes at least one myoelectric collector and an inertial sensor, the apparatus 400 includes an acquisition unit 401, a determination unit 402 and an execution unit 403, wherein,
the acquiring unit 401 is configured to acquire the electromyographic signals acquired by the at least one electromyographic acquisition unit;
the determining unit 402 is configured to determine a target control mode corresponding to the electromyographic signal;
the obtaining unit 401 is further configured to obtain attitude data collected by the inertial sensor;
the execution unit 403 is configured to execute a target control operation in the target control mode according to the attitude data.
Optionally, in terms of determining the target control mode corresponding to the electromyographic signal, the determining unit 402 is specifically configured to:
processing the electromyographic signals to obtain target signal processing data;
and determining a target control mode corresponding to the target signal processing data according to a mapping relation between preset signal processing data and control modes.
Optionally, in terms of processing the electromyographic signal to obtain target signal processing data, the determining unit 402 is specifically configured to:
the electromyographic signals are processed to obtain an electromyographic signal curve,
determining a target segment with a signal peak value in the electromyographic signal curve, wherein the starting position of the target segment corresponds to a first signal valley value, and the ending position of the target segment corresponds to a second signal valley value;
determining a target signal peak value, a first signal point with a rising speed larger than a first value and a second signal point with a falling speed larger than a second value in the target segment, wherein the first signal point corresponds to a first time point; the second signal point corresponds to a second time point;
determining a first signal strength value corresponding to the first signal point and a second signal strength value corresponding to the second signal point, the first signal strength value being greater than or equal to the first signal valley, the second signal strength value being greater than or equal to the second signal valley;
determining a first duration between the first point in time and the second point in time;
determining a first signal strength difference between the first signal strength value and the second signal strength value;
determining a first offset value corresponding to the target signal peak value;
and determining a first signal range according to the first offset value and the first signal strength difference value, and taking the first time length and the first signal range as the target signal processing data.
Optionally, in the aspect that the target control mode corresponding to the target signal processing data is determined according to a mapping relationship between preset signal processing data and a control mode, the determining unit 402 is specifically configured to:
matching the first time length with the time length in the mapping relation between the preset signal processing data and the control mode; matching the first signal range with a signal range in a mapping relation between preset signal processing data and a control mode to obtain a preset time length successfully matched with the first time length and a preset signal range successfully matched with the first signal range;
and determining a target control mode corresponding to the preset duration and the preset signal range in the mapping relation.
Optionally, the controlled device comprises at least two of the following components: the device comprises a chassis, a holder, a mechanical arm and a mechanical arm;
the target control mode includes any one of: an emergency stop mode, a chassis translation mode, a chassis rotation and pan-tilt rotation mode, a robotic arm mode, and a manipulator mode.
Optionally, the determining unit 402 is further configured to:
if the electromyographic signal curve comprises a plurality of segments, each segment comprises a signal peak value, and the maximum peak value in a plurality of signal peak values corresponding to the segments is determined;
determining a target segment of the plurality of segments corresponding to the maximum peak.
Optionally, the attitude data comprises acceleration and angular velocity; in terms of performing the target control operation in the target control mode according to the posture data, the execution unit 403 is specifically configured to:
determining a target component for performing the target control operation in the target control mode;
calculating control parameters of the target component according to the attitude data;
and controlling the target component to execute the target control operation according to the control parameter.
It can be seen that the compound control device based on myoelectricity and posture data described in the embodiment of the present invention obtains myoelectricity signals collected by at least one myoelectricity collector; determining a target control mode corresponding to the electromyographic signal; acquiring attitude data acquired by an inertial sensor; and executing target control operation in the target control mode according to the posture data, so that the operation control can be cooperatively performed based on myoelectricity and posture recognition, and the flexibility of the compound control system based on the myoelectricity and the posture data is improved.
An embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any one of the electromyography acquisition methods described in the above method embodiments.
Embodiments of the present invention also provide a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program enables a computer to execute part or all of the steps of any one of the electromyography acquisition methods described in the above method embodiments.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus can be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some interfaces, devices or units, and may be an electric or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may be implemented in the form of a software program module.
The integrated units, if implemented in the form of software program modules and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a memory and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned memory comprises: various media capable of storing program codes, such as a usb disk, a read-only memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and the like.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, which may include: flash memory disks, read-only memory, random access memory, magnetic or optical disks, and the like.
The above embodiments of the present invention are described in detail, and the principle and the implementation of the present invention are explained by applying specific embodiments, and the above description of the embodiments is only used to help understanding the method of the present invention and the core idea thereof; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A compound control method based on myoelectricity and posture data is characterized by being applied to a compound control system based on myoelectricity and posture data, wherein the compound control system based on myoelectricity and posture data comprises at least one myoelectricity collector and an inertial sensor, and the method comprises the following steps:
acquiring myoelectric signals collected by the at least one myoelectric collector;
determining a target control mode corresponding to the electromyographic signal;
acquiring attitude data acquired by the inertial sensor;
and executing target control operation in the target control mode according to the attitude data.
2. The method according to claim 1, wherein the determining the target control pattern corresponding to the electromyographic signal comprises:
processing the electromyographic signals to obtain target signal processing data;
and determining a target control mode corresponding to the target signal processing data according to a mapping relation between preset signal processing data and control modes.
3. The method according to claim 2, wherein the processing the electromyographic signals to obtain target signal processing data comprises:
the electromyographic signals are processed to obtain an electromyographic signal curve,
determining a target segment with a signal peak value in the electromyographic signal curve, wherein the starting position of the target segment corresponds to a first signal valley value, and the ending position of the target segment corresponds to a second signal valley value;
determining a target signal peak value, a first signal point with a rising speed larger than a first value and a second signal point with a falling speed larger than a second value in the target segment, wherein the first signal point corresponds to a first time point; the second signal point corresponds to a second time point;
determining a first signal strength value corresponding to the first signal point and a second signal strength value corresponding to the second signal point, the first signal strength value being greater than or equal to the first signal valley, the second signal strength value being greater than or equal to the second signal valley;
determining a first duration between the first point in time and the second point in time;
determining a first signal strength difference between the first signal strength value and the second signal strength value;
determining a first offset value corresponding to the target signal peak value;
and determining a first signal range according to the first offset value and the first signal strength difference value, and taking the first time length and the first signal range as the target signal processing data.
4. The method of claim 3, wherein the determining the target control mode corresponding to the target signal processing data according to the preset mapping relationship between the signal processing data and the control mode comprises:
matching the first time length with the time length in the mapping relation between the preset signal processing data and the control mode; matching the first signal range with a signal range in a mapping relation between preset signal processing data and a control mode to obtain a preset time length successfully matched with the first time length and a preset signal range successfully matched with the first signal range;
and determining a target control mode corresponding to the preset duration and the preset signal range in the mapping relation.
5. The method of claim 3 or 4, further comprising:
if the electromyographic signal curve comprises a plurality of segments, each segment comprises a signal peak value, and the maximum peak value in a plurality of signal peak values corresponding to the segments is determined;
determining a target segment of the plurality of segments corresponding to the maximum peak.
6. The method according to any one of claims 1-5, wherein the combined electromyography and posture data-based control system further comprises a controlled device comprising at least two of: the device comprises a chassis, a holder, a mechanical arm and a mechanical arm;
the target control mode includes any one of: an emergency stop mode, a chassis translation mode, a chassis rotation and pan-tilt rotation mode, a robotic arm mode, and a manipulator mode.
7. The method of claim 6, wherein the attitude data comprises acceleration and angular velocity; the executing of the target control operation in the target control mode according to the attitude data includes:
determining a target component for performing the target control operation in the target control mode;
calculating control parameters of the target component according to the attitude data;
and controlling the target component to execute the target control operation according to the control parameter.
8. A compound control device based on myoelectricity and posture data is characterized by being applied to a compound control system based on myoelectricity and posture data, wherein the compound control system based on myoelectricity and posture data comprises at least one myoelectricity collector and an inertial sensor, and the device comprises:
the acquisition unit is used for acquiring the electromyographic signals acquired by the at least one electromyographic acquisition unit;
the determining unit is used for determining a target control mode corresponding to the electromyographic signal;
the acquisition unit is also used for acquiring attitude data acquired by the inertial sensor;
and the execution unit is used for executing target control operation in the target control mode according to the attitude data.
9. A composite control system based on myoelectricity and posture data is characterized by comprising at least one myoelectricity collector, an inertial sensor, controlled equipment and a controller, wherein the at least one myoelectricity collector and the inertial sensor are connected with the controller,
the myoelectricity collector is used for collecting myoelectricity signals of a human body;
the controller is used for determining a target control mode corresponding to the electromyographic signal;
the inertial sensor is used for acquiring posture data of a human body;
the controller is further used for controlling the controlled equipment to execute target control operation in the target control mode according to the attitude data.
10. A computer-readable storage medium, characterized in that a computer program for electronic data exchange is stored, wherein the computer program causes a computer to perform the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011063852.7A CN112214109B (en) | 2020-09-30 | 2020-09-30 | Myoelectricity and gesture data-based composite control method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011063852.7A CN112214109B (en) | 2020-09-30 | 2020-09-30 | Myoelectricity and gesture data-based composite control method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112214109A true CN112214109A (en) | 2021-01-12 |
| CN112214109B CN112214109B (en) | 2023-06-23 |
Family
ID=74051687
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011063852.7A Active CN112214109B (en) | 2020-09-30 | 2020-09-30 | Myoelectricity and gesture data-based composite control method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112214109B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113021349A (en) * | 2021-03-24 | 2021-06-25 | 季华实验室 | Remote operation control method, device, system, equipment and storage medium |
| WO2022193330A1 (en) * | 2021-03-19 | 2022-09-22 | 深圳市韶音科技有限公司 | Exercise monitoring method and system |
| TWI837620B (en) | 2021-03-19 | 2024-04-01 | 大陸商深圳市韶音科技有限公司 | Method and system for motion monitoring |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001054507A (en) * | 1999-08-17 | 2001-02-27 | Sony Corp | Motion capture device using myoelectric information, its controlling method, electric stimulator using this, force tactile presenting device and controlling method of these |
| CN107553499A (en) * | 2017-10-23 | 2018-01-09 | 上海交通大学 | Natural the gesture motion control system and method for a kind of Multi-shaft mechanical arm |
| CN109901539A (en) * | 2019-03-27 | 2019-06-18 | 辽东学院 | A kind of man-machine interactive system and its control method applied to smart home |
| CN110169851A (en) * | 2019-05-28 | 2019-08-27 | 南京航空航天大学 | The artificial hand control system of electromyography signal automatic adjusument |
-
2020
- 2020-09-30 CN CN202011063852.7A patent/CN112214109B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001054507A (en) * | 1999-08-17 | 2001-02-27 | Sony Corp | Motion capture device using myoelectric information, its controlling method, electric stimulator using this, force tactile presenting device and controlling method of these |
| CN107553499A (en) * | 2017-10-23 | 2018-01-09 | 上海交通大学 | Natural the gesture motion control system and method for a kind of Multi-shaft mechanical arm |
| CN109901539A (en) * | 2019-03-27 | 2019-06-18 | 辽东学院 | A kind of man-machine interactive system and its control method applied to smart home |
| CN110169851A (en) * | 2019-05-28 | 2019-08-27 | 南京航空航天大学 | The artificial hand control system of electromyography signal automatic adjusument |
Non-Patent Citations (2)
| Title |
|---|
| 胡家垒等: "基于肢体运动信息的机器人示教操作系统", 《机电一体化》 * |
| 胡家垒等: "基于肢体运动信息的机器人示教操作系统", 《机电一体化》, 28 February 2019 (2019-02-28), pages 36 - 41 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022193330A1 (en) * | 2021-03-19 | 2022-09-22 | 深圳市韶音科技有限公司 | Exercise monitoring method and system |
| TWI837620B (en) | 2021-03-19 | 2024-04-01 | 大陸商深圳市韶音科技有限公司 | Method and system for motion monitoring |
| CN113021349A (en) * | 2021-03-24 | 2021-06-25 | 季华实验室 | Remote operation control method, device, system, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112214109B (en) | 2023-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wilson et al. | Formulation of a new gradient descent MARG orientation algorithm: Case study on robot teleoperation | |
| CN107378944B (en) | Multidimensional surface electromyographic signal artificial hand control method based on principal component analysis method | |
| Simao et al. | A review on electromyography decoding and pattern recognition for human-machine interaction | |
| CN112214109A (en) | Compound control method, device and system based on myoelectricity and attitude data | |
| CN110076772B (en) | Grabbing method and device for mechanical arm | |
| Xu et al. | Development of a hybrid motion capture method using MYO armband with application to teleoperation | |
| Bouteraa et al. | A gesture-based telemanipulation control for a robotic arm with biofeedback-based grasp | |
| CN110125909B (en) | Multi-information fusion human body exoskeleton robot control protection system | |
| CN109521880B (en) | Teleoperation robot system and method based on mixed bioelectricity signal driving | |
| Liang et al. | Writing skills transfer from human to robot using stiffness extracted from sEMG | |
| CN113558771B (en) | Robot plane limit control method and device and surgical robot | |
| Dwivedi et al. | A shared control framework for robotic telemanipulation combining electromyography based motion estimation and compliance control | |
| Anvaripour et al. | FMG-and RNN-based estimation of motor intention of upper-limb motion in human-robot collaboration | |
| Boru et al. | Novel technique for control of industrial robots with wearable and contactless technologies | |
| Patel et al. | EMG-based human machine interface control | |
| CN111358659B (en) | Robot power-assisted control method and system and lower limb rehabilitation robot | |
| CN113561172B (en) | Dexterous hand control method and device based on binocular vision acquisition | |
| Lee et al. | Verification of a fast training algorithm for multi-channel semg classification systems to decode hand configuration | |
| CN115276831B (en) | Signal interference elimination method, system, storage medium and terminal equipment | |
| Winges et al. | Effects of object compliance on three-digit grasping | |
| Wei et al. | Research on robotic arm movement grasping system based on MYO | |
| CN115376156A (en) | Gesture recognition method and device and electronic equipment | |
| Ciocarlie et al. | Biomimetic grasp planning for cortical control of a robotic hand | |
| Suryanarayanan et al. | EMG-based interface for position tracking and control in VR environments and teleoperation | |
| Zhao et al. | A multimodal-signals-based gesture recognition method for human machine interaction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |