CN114310903A - Manipulator control method and system based on bilateral teleoperation - Google Patents
Manipulator control method and system based on bilateral teleoperation Download PDFInfo
- Publication number
- CN114310903A CN114310903A CN202210062107.3A CN202210062107A CN114310903A CN 114310903 A CN114310903 A CN 114310903A CN 202210062107 A CN202210062107 A CN 202210062107A CN 114310903 A CN114310903 A CN 114310903A
- Authority
- CN
- China
- Prior art keywords
- operation instruction
- manipulator
- controller
- control method
- bilateral teleoperation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000002146 bilateral effect Effects 0.000 title claims abstract description 25
- 230000005540 biological transmission Effects 0.000 claims abstract description 11
- 238000004891 communication Methods 0.000 claims description 25
- 230000005484 gravity Effects 0.000 claims description 12
- 230000003993 interaction Effects 0.000 claims description 6
- 238000013461 design Methods 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 230000009471 action Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 239000012636 effector Substances 0.000 description 2
- 210000001145 finger joint Anatomy 0.000 description 2
- 239000002253 acid Substances 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 239000003513 alkali Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000009916 joint effect Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 231100000331 toxic Toxicity 0.000 description 1
- 230000002588 toxic effect Effects 0.000 description 1
Images
Landscapes
- Manipulator (AREA)
Abstract
The invention discloses a manipulator control method and system based on bilateral teleoperation, and relates to the field of automatic control. The invention comprises the following steps: acquiring an operation instruction of a main terminal, wherein the operation instruction comprises sound and gestures; transmitting an operation instruction through a network; correcting errors existing in the transmission process of the operation instruction; and the slave end acquires the operation instruction, executes the operation instruction and transmits an execution result to the master end. The invention can be applied to dangerous places to replace human hands to work, thereby greatly improving the safety; the main control is completed by people, and the flexibility is greatly improved.
Description
Technical Field
The invention relates to the field of automatic control, in particular to a manipulator control method and system based on bilateral teleoperation.
Background
The advent of robots has changed our world greatly, and more robots have been developed in succession. Among them, the multi-joint and multi-degree-of-freedom industrial robot technology applied to the industrial field is an important embodiment of national high-tech strength and development level. The robot hand is one of the most important execution tools, and is an important branch of the robot field, and the traditional robot hand performs grabbing through an end effector, but the mode is relatively single, and the flexibility is lacked, and grabbing facing complex environments and tasks cannot be met, so that the multi-finger machine with multiple degrees of freedom and multiple functions is widely concerned by countries in the world, wherein the humanoid robot hand is one of the concerned hotspots.
The mechanical arm is undoubtedly the most important execution structure for the robot to complete various tasks, and the traditional mechanical arm performs grabbing through an end effector, but the mode is relatively single and lacks flexibility. For extreme working environments, such as high temperature, toxic, acid and alkali, etc., remote operation is required when complex operations are performed in these environments.
Therefore, how to realize the remote operation of the manipulator needs to be solved by those skilled in the art.
Disclosure of Invention
In view of the above, the invention provides a manipulator control method and system based on bilateral teleoperation, so that the manipulator replaces human hands to perform some operations, can be applied to some dangerous places, and replaces human hands to work, thereby greatly improving the safety; the main control is completed by people, and the flexibility is greatly improved.
In order to achieve the purpose, the invention adopts the following technical scheme:
a manipulator control method based on bilateral teleoperation comprises the following steps:
acquiring an operation instruction of a main terminal, wherein the operation instruction comprises sound and gestures;
transmitting an operation instruction through a network;
correcting errors existing in the transmission process of the operation instruction;
and the slave end acquires the operation instruction, executes the operation instruction and transmits an execution result to the master end.
Optionally, the obtaining of the main-end operation gesture instruction is specifically as follows:
acquiring hand position information in real time by using a visual sensor on equipment; tracking and recording a hand motion track by utilizing a gesture recognition tracking algorithm; and converting the hand motion track into a digital signal to generate a gesture command.
Optionally, the obtaining of the main-end operation sound instruction specifically includes: detecting voice activity to obtain voice information; and enhancing the voice information, extracting effective voice, and comparing the effective voice information with a preset voice information database to obtain a voice command.
Optionally, the error existing in the transmission process of the operation instruction is corrected, specifically as follows:
acquiring the structure of a bilateral teleoperation system based on network communication and asymmetric time-varying delay existing in the network communication;
and selecting a Lyapunov function according to the generated asymmetric time-varying delay, and processing an integral cross term in a derivative of the Lyapunov function by different integral inequality methods to reduce network communication errors.
Optionally, the method further comprises selecting a gravity item from the end as a parameter uncertainty item, and adding a gravity estimation item in the design of the controller to eliminate the uncertainty of the system caused by the gravity item of the system.
Optionally, the method further includes decoupling the high-order nonlinear system with the pure feedback structure into a strict feedback form by using a median theorem, and controlling the slave end by using a backstepping control method.
A manipulator control system based on bilateral teleoperation comprises a master controller, a slave manipulator and a communication system;
the master controller gives an instruction to the slave manipulator through the communication system; the slave manipulator feeds back an instruction to the master controller through a communication system;
the main end controller is provided with main end force feedback man-machine interaction equipment; the main end force feedback man-machine interaction equipment is used for sensing a main end control signal.
Optionally, the system further comprises a position error controller, a slave end force feedback controller and a speed position error controller, wherein the position error controller, the slave end force feedback controller and the speed position error controller are all connected to the communication system.
Optionally, the system further comprises a real-time display module, and the real-time display module is used for displaying real-time dynamics of the manipulator.
Optionally, the communication system is provided with an error correction module for correcting errors.
According to the technical scheme, compared with the prior art, the manipulator control method and system based on bilateral teleoperation are disclosed and provided, instructions are accurately and flexibly sensed through a main controller, finger joint action information is transmitted to the manipulator through a communication system, various operations of the manipulator are controlled remotely, the adaptability of the manipulator to a complex environment is improved on the basis of flexible control, and the flexible grabbing function of the manipulator in multiple degrees of freedom, multiple functions and multiple application environments can be realized; in addition, the invention adopts the control of time-varying delay to reduce the transmission energy consumption, and ensures the transparency of the system and enables the bilateral teleoperation system to be synchronous by using a slave torque feedback method.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a schematic diagram of the bilateral control of the present invention;
FIG. 2 is a schematic flow chart of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention discloses a manipulator control method based on bilateral teleoperation, which comprises the following steps of:
acquiring an operation instruction of a main terminal, wherein the operation instruction comprises sound and gestures;
transmitting an operation instruction through a network;
correcting errors existing in the transmission process of the operation instruction;
and the slave end acquires the operation instruction, executes the operation instruction and transmits an execution result to the master end.
The main end operation gesture instruction is obtained as follows:
acquiring hand position information in real time by using a visual sensor on equipment; tracking and recording a hand motion track by utilizing a gesture recognition tracking algorithm; and converting the hand motion track into a digital signal to generate a gesture command.
In addition, the step of obtaining the main-end operation gesture command may further be: the catching motion sensor is worn on the hand of an operator in a glove shape, the motion information instruction of the finger joint of the operator is remotely transmitted to the catching mechanical arm through the wireless radio frequency transmission module, and the catching mechanical arm drives the corresponding motor to operate after receiving the corresponding motion instruction, so that the catching motion same as that of the finger of the operator is realized; meanwhile, the capturing manipulator sends the state information of the system to the mobile phone through a Bluetooth communication protocol, the state information of the manipulator is remotely mastered through the mobile phone, and the action of the manipulator can be directly controlled through the mobile phone when necessary.
The main end operation sound instruction is obtained as follows: detecting voice activity to obtain voice information; and enhancing the voice information, extracting effective voice, and comparing the effective voice information with a preset voice information database to obtain a voice command.
Optionally, the error existing in the transmission process of the operation instruction is corrected, specifically as follows:
acquiring the structure of a bilateral teleoperation system based on network communication and asymmetric time-varying delay existing in the network communication;
and selecting a Lyapunov function according to the generated asymmetric time-varying delay, and processing an integral cross term in a derivative of the Lyapunov function by different integral inequality methods to reduce network communication errors.
Specifically, in the present embodiment, a teleoperation system in a case where uncertain dynamic parameters exist in the system is considered. As far as the system is concerned, uncertainty caused by the gravity item has a large influence on the teleoperation system, so the gravity item is taken as a parameter uncertainty item. In order to eliminate the uncertainty of the system caused by the gravity term of the system, a gravity estimation term is added in the design of the controller. Then, the adaptation law of the system is designed. And constructing a proper Lyapunov functional on the basis that the lower bound of the selected system delay is zero. Likewise, the derivative is scaled by a new integral inequality method, and finally the system is obtained to be bounded and stable. And the stability criterion is presented in the form of a linear matrix inequality.
In the embodiment, a gravity item is selected from the end as a parameter uncertainty item, and a gravity estimation item is added in the design of the controller to eliminate the uncertainty of the system caused by the gravity item of the system.
The method comprises the steps of correcting errors existing in the transmission process of an operation instruction, decoupling a high-order nonlinear system of a pure feedback structure into a strict feedback form by utilizing a median theorem, and controlling a slave end by utilizing a backstepping control method.
Specifically, the present embodiment researches a tracking control problem of a high-order nonlinear system with a pure feedback structure. On the premise of assuming that the nonlinear function of the system is a continuous function, decoupling the high-order nonlinear system of a pure feedback structure into a strict feedback form by using a median theorem, and designing the system by using a backstepping control method on the basis. Under the framework of a backstepping control method, an unknown item in the system is approximated by a radial basis function neural network, and the problem of uncertainty nonlinearity existing in the system is converted into the problem of parameter updating of the radial basis function neural network. And a finite time stability theory is used as a performance index for designing a controller of the system, and a control scheme capable of enabling the controlled system to be stable in finite time is provided. Under the action of the control scheme, the controlled system can quickly reach a stable state, and the output signal of the system can track the reference signal with more ideal precision. In addition, due to the consideration of the influence of the input dead zone phenomenon on the system performance, compared with the traditional control scheme, the control scheme of the embodiment can better meet the requirement of an actual control task.
A manipulator control system based on bilateral teleoperation, as shown in fig. 1, includes a master controller, a slave manipulator and a communication system; the communication system is provided with an error correction module for correcting errors.
The master controller gives instructions to the slave manipulator through the communication system; the slave manipulator feeds back an instruction to the master controller through a communication system;
the main end controller is provided with main end force feedback man-machine interaction equipment; the main end force feedback man-machine interaction equipment is used for sensing the main end control signal.
In addition, the system also comprises a position error controller, a slave end force feedback controller and a speed position error controller, wherein the position error controller, the slave end force feedback controller and the speed position error controller are all accessed into the communication system; the manipulator real-time display system further comprises a real-time display module, and the real-time display module is used for displaying the real-time dynamics of the manipulator.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A manipulator control method based on bilateral teleoperation is characterized by comprising the following steps:
acquiring an operation instruction of a main terminal, wherein the operation instruction comprises sound and gestures;
transmitting an operation instruction through a network;
correcting errors existing in the transmission process of the operation instruction;
and the slave end acquires the operation instruction, executes the operation instruction and transmits an execution result to the master end.
2. The manipulator control method based on bilateral teleoperation according to claim 1, wherein the obtaining of the master-end operation gesture command is specifically as follows:
acquiring hand position information in real time by using a visual sensor on equipment; tracking and recording a hand motion track by utilizing a gesture recognition tracking algorithm; and converting the hand motion track into a digital signal to generate a gesture command.
3. The manipulator control method based on bilateral teleoperation according to claim 1, wherein the obtaining of the master end operation sound command is specifically as follows: detecting voice activity to obtain voice information; and enhancing the voice information, extracting effective voice, and comparing the effective voice information with a preset voice information database to obtain a voice command.
4. The manipulator control method based on bilateral teleoperation according to claim 1, wherein errors existing in the transmission process of the operation instruction are corrected, specifically as follows:
acquiring the structure of a bilateral teleoperation system based on network communication and asymmetric time-varying delay existing in the network communication;
and selecting a Lyapunov function according to the generated asymmetric time-varying delay, and processing an integral cross term in a derivative of the Lyapunov function by different integral inequality methods.
5. The manipulator control method based on bilateral teleoperation as claimed in claim 1, further comprising selecting a gravity term from the slave end as a parameter uncertainty term, and adding a gravity estimation term in the design of the controller.
6. The manipulator control method based on bilateral teleoperation according to claim 1, characterized in that errors existing in the transmission process of the operation instruction are corrected, and further comprising decoupling a high-order nonlinear system of a pure feedback structure into a strict feedback form by using a median theorem, and controlling a slave end by using a backstepping control method.
7. A manipulator control system based on bilateral teleoperation is characterized by comprising a master controller, a slave manipulator and a communication system;
the master controller gives an instruction to the slave manipulator through the communication system; the slave manipulator feeds back an instruction to the master controller through a communication system;
the main end controller is provided with main end force feedback man-machine interaction equipment; the main end force feedback man-machine interaction equipment is used for sensing a main end control signal.
8. The manipulator control system based on bilateral teleoperation as claimed in claim 1, further comprising a position error controller, a slave end force feedback controller and a velocity position error controller, wherein the position error controller, the slave end force feedback controller and the velocity position error controller are all connected to a communication system.
9. The manipulator control system based on bilateral teleoperation as claimed in claim 1, further comprising a real-time display module, wherein the real-time display module is configured to display real-time dynamics of the manipulator.
10. The manipulator control system based on bilateral teleoperation as claimed in claim 1, wherein the communication system is provided with an error correction module for correcting errors.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210062107.3A CN114310903A (en) | 2022-01-19 | 2022-01-19 | Manipulator control method and system based on bilateral teleoperation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210062107.3A CN114310903A (en) | 2022-01-19 | 2022-01-19 | Manipulator control method and system based on bilateral teleoperation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114310903A true CN114310903A (en) | 2022-04-12 |
Family
ID=81028759
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210062107.3A Pending CN114310903A (en) | 2022-01-19 | 2022-01-19 | Manipulator control method and system based on bilateral teleoperation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114310903A (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107632699A (en) * | 2017-08-01 | 2018-01-26 | 东南大学 | Natural human-machine interaction system based on the fusion of more perception datas |
| CN108646566A (en) * | 2018-06-12 | 2018-10-12 | 南京理工大学 | A kind of slave torque feedback control method of the bilateral teleoperation system based on sampling |
| CN109085749A (en) * | 2018-08-07 | 2018-12-25 | 浙江大学 | A kind of non-linear remote operating bilateral control method based on adaptive fuzzy inverting |
| CN109108931A (en) * | 2018-07-24 | 2019-01-01 | 南昌大学 | The design method of remote operating controller based on homogeneous theoretical finite time convergence control |
| CN109240086A (en) * | 2018-10-16 | 2019-01-18 | 浙江大学 | A kind of adaptive robust control method of non-linear bilateral teleoperation system |
| CN110794678A (en) * | 2019-11-05 | 2020-02-14 | 燕山大学 | Four-channel teleoperation force feedback control method under hysteresis nonlinearity limitation |
| CN110815258A (en) * | 2019-10-30 | 2020-02-21 | 华南理工大学 | Robot teleoperation system and method based on electromagnetic force feedback and augmented reality |
| CN111438673A (en) * | 2020-03-24 | 2020-07-24 | 西安交通大学 | High-altitude operation teleoperation method and system based on stereoscopic vision and gesture control |
| CN112099526A (en) * | 2020-09-09 | 2020-12-18 | 北京航空航天大学 | Unmanned aerial vehicle control system and control method based on voice and gesture recognition |
| CN112346570A (en) * | 2020-11-06 | 2021-02-09 | 戴姆勒股份公司 | Method and equipment for man-machine interaction based on voice and gestures |
| CN112405530A (en) * | 2020-11-06 | 2021-02-26 | 齐鲁工业大学 | Robot vision tracking control system and control method based on wearable vision |
-
2022
- 2022-01-19 CN CN202210062107.3A patent/CN114310903A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107632699A (en) * | 2017-08-01 | 2018-01-26 | 东南大学 | Natural human-machine interaction system based on the fusion of more perception datas |
| CN108646566A (en) * | 2018-06-12 | 2018-10-12 | 南京理工大学 | A kind of slave torque feedback control method of the bilateral teleoperation system based on sampling |
| CN109108931A (en) * | 2018-07-24 | 2019-01-01 | 南昌大学 | The design method of remote operating controller based on homogeneous theoretical finite time convergence control |
| CN109085749A (en) * | 2018-08-07 | 2018-12-25 | 浙江大学 | A kind of non-linear remote operating bilateral control method based on adaptive fuzzy inverting |
| CN109240086A (en) * | 2018-10-16 | 2019-01-18 | 浙江大学 | A kind of adaptive robust control method of non-linear bilateral teleoperation system |
| CN110815258A (en) * | 2019-10-30 | 2020-02-21 | 华南理工大学 | Robot teleoperation system and method based on electromagnetic force feedback and augmented reality |
| CN110794678A (en) * | 2019-11-05 | 2020-02-14 | 燕山大学 | Four-channel teleoperation force feedback control method under hysteresis nonlinearity limitation |
| CN111438673A (en) * | 2020-03-24 | 2020-07-24 | 西安交通大学 | High-altitude operation teleoperation method and system based on stereoscopic vision and gesture control |
| CN112099526A (en) * | 2020-09-09 | 2020-12-18 | 北京航空航天大学 | Unmanned aerial vehicle control system and control method based on voice and gesture recognition |
| CN112346570A (en) * | 2020-11-06 | 2021-02-09 | 戴姆勒股份公司 | Method and equipment for man-machine interaction based on voice and gestures |
| CN112405530A (en) * | 2020-11-06 | 2021-02-26 | 齐鲁工业大学 | Robot vision tracking control system and control method based on wearable vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104440864B (en) | A kind of master-slave mode remote operating industrial robot system and its control method | |
| CN106945043B (en) | Multi-arm cooperative control system of master-slave teleoperation surgical robot | |
| JP5686775B2 (en) | Method for dynamic optimization of robot control interface | |
| CN115454240B (en) | Meta universe virtual reality interaction experience system and method | |
| CN203765607U (en) | Combined double-arm multifunctional robot | |
| CN105500345A (en) | Separating type robot dragging and demonstrating handle and demonstrating method thereof | |
| CN106774176A (en) | A kind of digital control processing control system and method for work based on wireless blue tooth communication | |
| CN112052688B (en) | Behavior generation method based on semantics | |
| CN112207835A (en) | Method for realizing double-arm cooperative work task based on teaching learning | |
| Raessa et al. | Teaching a robot to use electric tools with regrasp planning | |
| Afzal et al. | Gesture control robotic arm using flex sensor | |
| Yun et al. | Immersive and interactive cyber-physical system (I2CPS) and virtual reality interface for human involved robotic manufacturing | |
| CN209175754U (en) | A kind of infrared gesture identification manipulator | |
| TW201220009A (en) | Program converting module for use with machines with multi-axis simultaneously coordinated motion and method of converting the programs | |
| CN114310903A (en) | Manipulator control method and system based on bilateral teleoperation | |
| CN106774178B (en) | Automatic control system and method and mechanical equipment | |
| Reis et al. | Modeling and control of a multifingered robot hand for object grasping and manipulation tasks | |
| JP5004108B2 (en) | Master / slave device and control information determination method for determining control information of robot and hand | |
| Parvin et al. | Human-Machine Interface (HMI) Robotic Arm Controlled by Gyroscopically Acceleration | |
| KR101263487B1 (en) | Teaching device for robot | |
| CN203045733U (en) | Network embedded type robot control device | |
| Yuan | An interactive approach of assembly planning | |
| Barbagallo et al. | Human-robot interaction through kinect and graphics tablet sensing devices | |
| JP7168859B2 (en) | ROBOT CONTROL DEVICE, GRIP SYSTEM, AND ROBOT HAN CONTROL METHOD | |
| CN114603551A (en) | Control method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220412 |
|
| RJ01 | Rejection of invention patent application after publication |