CN112060088A - Non-cooperative target accurate capturing teleoperation method under variable time delay condition - Google Patents
Non-cooperative target accurate capturing teleoperation method under variable time delay condition Download PDFInfo
- Publication number
- CN112060088A CN112060088A CN202010888700.4A CN202010888700A CN112060088A CN 112060088 A CN112060088 A CN 112060088A CN 202010888700 A CN202010888700 A CN 202010888700A CN 112060088 A CN112060088 A CN 112060088A
- Authority
- CN
- China
- Prior art keywords
- slave
- mechanical arm
- cooperative target
- slave end
- movement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1689—Teleoperation
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J15/00—Gripping heads and other end effectors
- B25J15/08—Gripping heads and other end effectors having finger members
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J18/00—Arms
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1661—Programme controls characterised by programming, planning systems for manipulators characterised by task planning, object-oriented languages
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a non-cooperative target accurate capture teleoperation method under a time delay variable condition, which belongs to the technical field of teleoperation of space robots and comprises the following steps: a step of predicting movement: at the current moment, generating an instruction at the current moment at the master end according to the moving signal and sending the instruction to the slave end; predicting the state of the slave mechanical arm after the slave mechanical arm executes the instruction sent by the master at the current moment based on the instruction sequence sent from the master to the slave and the state sequence of the slave; the instruction is used for indicating the movement angle of each joint of the slave end mechanical arm; a movement signal is generated based on a prediction result at a previous time, and is used for indicating position information moved from the end of the end mechanical arm; repeatedly executing the prediction moving step until the distance d between the tail end of the slave end mechanical arm and the non-cooperative target is within a preset range R; and when the distance d is within the preset range R, the slave end mechanical arm is used for autonomously identifying and capturing the non-cooperative target. The invention can realize the accurate capture of the non-cooperative target.
Description
Technical Field
The invention belongs to the technical field of teleoperation of space robots, and particularly relates to a method for accurately capturing teleoperation of a non-cooperative target under a variable time delay condition.
Background
Most of the in-orbit serviced spacecraft and space debris have 3 features: 1) no gripping mechanism (handle) for mechanical arm capture and cooperative markers and feature blocks for auxiliary measurements, etc. are installed; 2) the motion state of the target star is unknown, and the target star can be stable in three axes, stable in spinning, even rolling in an out-of-control state and the like; 3) there is no direct information exchange between the target star and the tracking star. Such targets are referred to as non-cooperative targets. In order to better use space robots for on-orbit services, the key problem of accurate capture of non-cooperative targets must be solved.
A visual camera is arranged on the space manipulator, the non-cooperative target is automatically identified by shooting the scene of the space manipulator, and then a control instruction is sent to the space manipulator to capture the non-cooperative target. Aiming at the accurate capture of a space non-cooperative target, the current method mainly utilizes the automatic identification and capture of a mechanical arm, but because the space mechanical arm has the characteristics of unstructured working environment and variable working tasks and is limited by the development of key supporting technologies such as computers, control, artificial intelligence, mechanisms and the like, the current method can only utilize the mechanical arm to perform the automatic identification and capture under the space environment, particularly when the space mechanical arm is far away from the non-cooperative target, and the accurate capture of the non-cooperative target is difficult to realize.
Disclosure of Invention
Aiming at the defects and improvement requirements of the prior art, the invention provides a non-cooperative target accurate capture teleoperation method under the condition of variable time delay, and aims to realize accurate capture of the non-cooperative target in a space environment.
To achieve the above object, according to an aspect of the present invention, there is provided a non-cooperative target capture teleoperation method under a variable delay condition, including:
a step of predicting movement: at the current moment, generating an instruction at the current moment at the master end according to the moving signal and sending the instruction to the slave end; predicting the state of the slave mechanical arm after the slave mechanical arm executes the instruction sent by the master at the current moment based on the instruction sequence sent from the master to the slave and the state sequence of the slave; the instruction is used for indicating the movement angle of each joint of the slave end mechanical arm; the movement signal is a signal generated based on a prediction result of the artificial neural network at a previous moment and is used for indicating position information of the movement from the tail end of the end mechanical arm; the main end is positioned on the ground and comprises a main end controller; the slave end is positioned in the space and comprises a slave end controller and a slave end mechanical arm; the master end and the slave end communicate through a world link;
ground direct operation step: repeatedly executing the predicting and moving step until the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within a preset range;
a local autonomous operation step: and when the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within a preset range, the slave end mechanical arm is used for automatically identifying the non-cooperative target and capturing the non-cooperative target.
The non-cooperative target capture teleoperation method under the variable time delay condition comprises two working modes, namely a ground direct operation mode and a local autonomous operation mode, wherein in the ground direct operation mode, a slave-end mechanical arm can rapidly approach a non-cooperative target in a larger range in a teleoperation mode, and in the local autonomous operation mode, the slave-end mechanical arm autonomously finishes capture of the non-cooperative target; the invention can effectively solve the problem that the slave end mechanical arm can not accurately capture the non-cooperative target by autonomous integrity when the distance is long through teleoperation. In the process of teleoperation, a time delay exists in a heaven-earth link of communication between a master end and a slave end, so that the working environment is time-varying, the time-varying condition is considered, the motion state of a mechanical arm at the slave end is predicted by utilizing an artificial neural network at the master end, and a corresponding instruction is sent to the slave end based on the prediction result, so that the mechanical arm at the slave end approaches to a non-cooperative target according to an expected moving mode, and the dynamic-varying environment and task in the capturing process of the non-cooperative target can be adapted; when the slave end mechanical arm is close to the non-cooperative target, the slave end mechanical arm is only used for autonomously identifying and capturing the non-cooperative target, so that time delay caused by a world link can be avoided, and the local autonomous operation of the slave end mechanical arm is fully utilized to finish accurate capturing of the non-cooperative target. In general, the method combines a ground direct operation mode and a local autonomous operation mode, in the ground direct operation mode, an artificial neural network is used for predicting the state of a slave-end mechanical arm and generating a corresponding instruction based on a prediction result, so that the slave end can rapidly approach to a non-cooperative target in an expected mode, and in the local autonomous operation mode, the slave-end mechanical arm is used for autonomously identifying and capturing the non-cooperative target, so that the non-cooperative target can be accurately captured.
Further, the predicting movement step further comprises:
receiving the real state of the mechanical arm execution instruction fed back by the slave end at the master end;
and updating the network weight of the artificial neural network by using a back propagation algorithm by taking the real state of the slave end mechanical arm after executing an instruction sent by the master end at a certain moment as a teacher signal and taking the sum of square errors between the output of the artificial neural network and the teacher signal at a corresponding moment as a loss function.
The method updates the network weight of the artificial neural network based on the real state information fed back from the slave end, so that the artificial neural network can better adapt to the dynamically changing working environment, and the prediction precision of the artificial neural network is improved.
Further, the predicting movement step further comprises:
at the master end, updating the state sequence of the slave end by using the received teacher signal;
synchronously scrolling on the instruction sequence sent from the master end to the slave end and the state sequence of the slave end by using the first rolling window and the second rolling window so as to enable the second rolling window to contain the latest state;
and utilizing the instruction subsequence in the first rolling window and the state subsequence in the second rolling window as the input of the artificial neural network at the next moment of the moment corresponding to the latest state.
The invention uses the rolling window to intercept the latest subsequence in the state sequence and the corresponding instruction subsequence in real time as the input of the artificial neural network model, realizes a rolling prediction mode, thereby continuously correcting the model, tracking the reference value in real time, improving the prediction precision, and realizing the repeated iteration and the rapid training of the artificial neural network based on the small sample.
In some embodiments, the movement signal is generated by the master-end controller according to a preset movement strategy.
In some embodiments, the master further comprises a hand controller with force sense information, and the movement signal is from the hand controller.
The hand controller with force sense information is important ground direct interaction control mechanical arm equipment and has the functions of motion output and force feedback: on one hand, the hand controller transmits the motion information of the hand of the operator to a remote robot controller to control the motion of the robot; on the other hand, the interaction force/moment or 'virtual' force/moment information of the remote robot and the environment is fed back to the main end, and the force feedback of the hand controller is used for acting on the hand of an operator, so that the force telepresence is provided for the operator. The invention utilizes the hand controller to output the movement signal generated according to the prediction result of the artificial neural network, thereby improving the man-machine interaction.
Further, the hand controller works in an MS mode, and a scale factor adjusting function is added in the hand controller;
the scale factor adjusting function is used for outputting the motion and force feedback of the hand controller according to a preset scale factor so as to generate a moving signal.
Because the working space of the hand controller is far smaller than the motion space of the slave mechanical arm, the invention outputs the motion and force feedback of the hand controller according to the preset scale factor by adding the scale factor adjusting function to the hand controller, and can amplify the motion output in the limited working space of the hand controller into the motion of the slave mechanical arm in the motion space of the slave mechanical arm, thereby ensuring the accuracy of capturing the non-cooperative target.
Further, the predicting movement step of generating a command for the next time based on the received movement signal includes:
and an incremental control mode is adopted, the movement signal output by the hand controller is mapped into the terminal pose of the slave end mechanical arm, and a corresponding instruction is generated.
According to the invention, the incremental control mode is adopted to map the movement signal output by the hand controller into the tail end pose of the slave-end mechanical arm, so that the tail end pose of the slave-end mechanical arm can be prevented from sudden change in the initial stage of movement, and the safety of the slave-end mechanical arm can be ensured.
Further, when the variation of the tail end pose of the slave end mechanical arm after mapping exceeds the working space of the manual controller before mapping, a mapping space drifting strategy is adopted to complete the mapping from the mobile signal to the tail end pose of the slave end mechanical arm;
the mapping space drift strategy is executed according to the following steps:
(S1) calculating a total movement distance of the hand controller as a target distance according to a preset scale factor;
(S2) continuing the movement of the hand controller until the total distance the hand controller has moved reaches the target distance, or the hand controller moves to the limit position;
(S3) if the total distance the hand controller has moved reaches the target distance, proceeding to step (S4); if the hand controller moves to the limit position, the hand controller is reset to the initial position, and the step (S2) is carried out;
(S4) the mapping is ended.
Because the working space of the hand controller is far smaller than the movement space of the slave mechanical arm, and the scale factor for adjusting the scale factor by the hand controller is limited in order to ensure the control precision, in this case, the movement of the slave mechanical arm may exceed the working space of the hand controller; according to the invention, when the variation of the tail end pose of the slave-end mechanical arm is large after mapping, the mapping of the tail end pose of the slave-end mechanical arm is completed by adopting a mapping space drifting strategy, and the effective coverage of a mapping space can be ensured while the control precision is ensured.
Further, the predicting movement step further comprises:
and displaying the motion of the slave end mechanical arm at the master end in a virtual reality mode according to the prediction result of the artificial neural network.
According to the invention, the motion of the mechanical arm at the slave end is presented at the master end in a virtual reality mode according to the prediction result of the artificial neural network, so that the motion condition of the mechanical arm at the slave end can be conveniently monitored in real time.
According to another aspect of the present invention, there is provided a non-cooperative target precise-capture teleoperation system under a variable-delay condition, comprising: a master and a slave communicating over a world link; the main end is positioned on the ground and comprises a main end controller; the slave end is positioned in the space and comprises a slave end controller and a slave end mechanical arm;
a main-end controller for executing the predicted movement step; the movement prediction step includes: at the current moment, generating an instruction at the current moment at the master end according to the moving signal and sending the instruction to the slave end; predicting the state of the slave mechanical arm after the slave mechanical arm executes the instruction sent by the master at the current moment based on the instruction sequence sent from the master to the slave and the state sequence of the slave; the instruction is used for indicating the movement angle of each joint of the slave end mechanical arm; the movement signal is a signal generated based on a prediction result of the artificial neural network at a previous moment and is used for indicating position information of the movement from the tail end of the end mechanical arm; the main end is positioned on the ground and comprises a main end controller; the slave end is positioned in the space and comprises a slave end controller and a slave end mechanical arm; the master end and the slave end communicate through a world link;
the main end controller is also used for executing the ground direct operation step; the ground direct operation step comprises: repeatedly executing the predicting and moving step until the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within a preset range;
a slave-side controller for performing the local autonomous operation step; the local autonomous operation step includes: and when the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within a preset range, controlling the slave end mechanical arm to autonomously identify the non-cooperative target and capture the non-cooperative target.
Generally, by the above technical solution conceived by the present invention, the following beneficial effects can be obtained:
(1) according to the method, a ground direct operation mode and a local autonomous operation mode are combined, in the ground direct operation mode, the state of the slave-end mechanical arm is predicted by using the artificial neural network, and a corresponding instruction is generated based on a prediction result, so that the slave-end is enabled to rapidly approach to a non-cooperative target in an expected mode, and in the local autonomous operation mode, the slave-end mechanical arm is used for autonomously identifying and capturing the non-cooperative target, so that the non-cooperative target can be accurately captured.
(2) The method updates the network weight of the artificial neural network based on the real state information fed back from the slave end, so that the artificial neural network can better adapt to the dynamically changing working environment, and the prediction precision of the artificial neural network is improved.
(3) The invention uses the rolling window to intercept the latest subsequence in the state sequence and the corresponding instruction subsequence in real time as the input of the artificial neural network model, can continuously correct the model, track the reference value in real time, improve the prediction precision, and can realize the repeated iteration and the rapid training of the artificial neural network based on the small sample.
Drawings
Fig. 1 is a flowchart of a non-cooperative target capture teleoperation method under a variable delay condition according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a non-cooperative target capture teleoperation system under a variable delay condition according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an artificial neural network model according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of scrolling prediction according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the hand controller according to the embodiment of the present invention;
fig. 6 is a diagram of a network operation structure in the MS mode of the hand controller according to the embodiment of the present invention;
FIG. 7 is a schematic diagram of end pose mapping of a robotic arm according to an embodiment of the present invention;
fig. 8 is a schematic diagram of a mapping space shift strategy according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
In the present application, the terms "first," "second," and the like (if any) in the description and the drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
When the tail end of the slave end mechanical arm is far away from the non-cooperative target, the posture of the non-cooperative target relative to the tail end of the slave end mechanical arm changes rapidly, so that the slave end mechanical arm cannot automatically complete accurate capture of the non-cooperative target; in order to solve the technical problem that the conventional slave-end mechanical arm cannot accurately capture a non-cooperative target in a complex and changeable space environment, the invention provides a non-cooperative target accurate capture teleoperation method under a variable time delay condition, which has the overall thought that: through the combination of two modes, namely a ground direct operation mode and a local autonomous operation mode, a corresponding instruction is sent based on the prediction of the state of the slave-end mechanical arm in the ground direct operation mode, the slave-end mechanical arm is enabled to approach a non-cooperative target in a large range, so that the motion state of the non-cooperative target relative to the slave-end mechanical arm is stabilized, and in the local autonomous operation mode, the slave-end mechanical arm is utilized to autonomously identify and capture the non-cooperative target, so that the non-cooperative target is finally accurately captured.
The following are examples:
example 1:
a non-cooperative target capture teleoperation method under a variable delay condition, as shown in fig. 1, includes:
a step of predicting movement: at the current moment, generating an instruction at the current moment at the master end according to the moving signal and sending the instruction to the slave end; predicting the state of the slave mechanical arm after the slave mechanical arm executes the instruction sent by the master at the current moment based on the instruction sequence sent from the master to the slave and the state sequence of the slave; the instruction is used for indicating the movement angle of each joint of the slave end mechanical arm; the movement signal is a signal generated based on a prediction result of the artificial neural network at a previous moment and is used for indicating position information of the movement from the tail end of the end mechanical arm; the teleoperation system related to the embodiment is composed of a main end and a slave end, wherein the main end is positioned on the ground and comprises a main end controller; the slave end is positioned in the space and comprises a slave end controller and a slave end mechanical arm; the master end and the slave end communicate through a world link;
ground direct operation step: repeatedly executing the predicting and moving step until the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within a preset range;
a local autonomous operation step: and when the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within a preset range, the slave end mechanical arm is used for automatically identifying the non-cooperative target and capturing the non-cooperative target.
In this embodiment, the artificial neural network is as shown in fig. 3, and there is no connection between neurons in the same layer; input x of artificial neural network1～xIThe slave end is used for transmitting a command sequence and a state sequence of the slave end to the master end, and the state sequence of the slave end is collected by a slave end sensor and then fed back to the master end; output o of artificial neural network1～oKIs the predicted state after the slave end mechanical arm executes the instruction; i and K are the neuron number of the input layer and the output layer of the artificial neural network respectively;
in this embodiment, the step of predicting movement further includes:
receiving the real state of the mechanical arm execution instruction fed back by the slave end at the master end;
the real state d of the slave end mechanical arm after the slave end mechanical arm executes the instruction sent by the master end at a certain time1～dKFor teacher signal, output o of artificial neural network at corresponding time1～oKAnd teacher signal d1～dKThe sum of squared errors between the two is a loss function, and a Back Propagation (BP) algorithm is used for updating the network weight of the artificial neural network;
wherein an input signal x is inputted1～xIInputting the artificial neural network, and transmitting to the output layer via the intermediate layer to obtain the neuron output o of the output layer1～oK(ii) a Neuron output o of output layer1～oKAnd teacher signal d1～dKThe sum of squared errors between:
in the formula ok(K-1, 2, …, K) represents the output of output layer neuron K, netkThe input sums representing output layer neuron k are:
ok＝f(netk) (2)
in the BP algorithm, the least square average principle is applied, and the update quantity of the weight between the middle layer and the output layer is firstly solved:
in the formula, eta is a positive learning constant,okis the value of output layer neuron k;
when y isjIs the output of intermediate layer neurons j, netjWhen the sum is input to the intermediate layer neuron j, the weight value between the input layer and the intermediate layer is updated by
yj＝f(netj) (output of intermediate layer neuron j) (5)
Similarly, can obtain
All netskDirectly dependent on yj(ii) a Eta is a learning constant; w is akjAnd vjiAll represent weights, the condition that the neuron input-output function in the BP algorithm should satisfy is a monotonically increasing function, the most common being the following doubly-curved function:
the embodiment updates the network weight of the artificial neural network based on the real state information fed back from the end, so that the artificial neural network can better adapt to the dynamically changing working environment, and the prediction precision of the artificial neural network is improved.
In order to further improve the prediction accuracy of the artificial neural network and reduce the calculation amount, in this embodiment, the predicting and moving step further includes:
at the master end, updating the state sequence of the slave end by using the received teacher signal;
synchronously scrolling on the instruction sequence sent from the master end to the slave end and the state sequence of the slave end by using the first rolling window and the second rolling window so as to enable the second rolling window to contain the latest state;
and utilizing the instruction subsequence in the first rolling window and the state subsequence in the second rolling window as the input of the artificial neural network at the next moment of the moment corresponding to the latest state.
The invention intercepts the latest subsequence and the corresponding instruction subsequence in the state sequence in real time by utilizing a rolling window to be used as the input of an artificial neural network Model, thereby realizing a rolling prediction mode, wherein the rolling prediction is specifically shown in figure 4, wherein a Model _ p and a Model _ t respectively represent the instruction sequence and the state sequence which are input into the artificial neural network, and a First _ p and a First _ t respectively represent the instruction sequence and the state sequence which are input at the initial moment; forcast _ out represents a prediction result output by the neural network model, and Forcast _ in is an instruction generated based on the prediction result; as shown in fig. 4, in this embodiment, based on the rolling prediction, the last segment of data in the input data is replaced with new data every time, so that the model can be continuously corrected, the reference value can be tracked in real time, the prediction accuracy can be improved, and meanwhile, the iterative iteration and the fast training of the artificial neural network can be realized based on a small sample.
In this embodiment, the movement signal is generated by the master controller according to a preset movement policy based on the prediction result of the artificial neural network, so as to instruct the slave robot arm to perform a corresponding movement after executing the instruction.
The embodiment comprises two working modes, namely a ground direct operation mode and a local autonomous operation mode, wherein the ground direct operation mode is completed by a prediction movement step and a ground direct operation step, and the local autonomous operation mode is completed by a local autonomous operation step; in the embodiment, in a ground direct operation mode, the slave-end mechanical arm can rapidly approach a non-cooperative target in a large range in a teleoperation mode, and in a local autonomous operation mode, the slave-end mechanical arm autonomously finishes capturing the non-cooperative target; in addition, aiming at the problem that the time delay exists in the heaven-earth link communication, which causes the time variation of the working environment, the motion state of the mechanical arm at the slave end is predicted at the master end by using the artificial neural network in the ground direct operation mode, and a corresponding instruction is sent to the slave end based on the prediction result, so that the mechanical arm at the slave end approaches to a non-cooperative target according to an expected moving mode, and the dynamic changing environment and task in the capturing process of the non-cooperative target can be adapted; when the slave end mechanical arm is close to the non-cooperative target, the attitude of the non-cooperative target relative to the tail end of the slave end mechanical arm is stable, the slave end mechanical arm is used for automatically identifying and capturing the non-cooperative target, time delay caused by a world link can be avoided, and the local automatic operation of the slave end mechanical arm is fully utilized to complete accurate capturing of the non-cooperative target.
Accordingly, in this embodiment, the preset range of the distance between the end of the slave end robot arm and the non-cooperative target is actually a switching condition between two modes, and the preset range may be set according to the actual spatial environment and the autonomous capturing energy of the slave end robot arm, so as to ensure that when the distance between the end of the slave end robot arm and the non-cooperative target is within the preset range, accurate capturing of the non-cooperative target can be achieved by using the autonomous capturing capability of the slave end robot arm.
Example 2:
a non-cooperative target capture teleoperation method under a variable time delay condition, which is similar to embodiment 1, except that in this embodiment, as shown in fig. 2 and 5, a master further includes a hand controller with force sense information, and a moving signal is from the hand controller;
in this embodiment, the hand controller operates in an MS (Master-slave control mode), and in this mode, the hand controller and the slave end mechanical arm are completely in a dynamic coupling, which is suitable for completing a complex and precise control task; the operation in MS mode is shown in fig. 6;
because the working space of the hand controller is far smaller than the motion space of the slave mechanical arm, in order to ensure the accuracy of capturing the non-cooperative target, a scale factor adjusting function is added in the hand controller in the embodiment;
the scale factor adjusting function is used for outputting the motion and force feedback of the hand controller according to a preset scale factor so as to generate a moving signal;
according to the embodiment, the scale factor adjusting function is added to the hand controller, the motion and force feedback of the hand controller are output according to the preset scale factor, the motion output in the limited working space of the hand controller can be amplified to the motion of the slave end mechanical arm in the motion space of the slave end mechanical arm, and therefore the capturing accuracy of the non-cooperative target is guaranteed.
Although the hand controller and the slave end mechanical arm used in the embodiment have the same degree of freedom, the structures are different, and the hand controller and the slave end mechanical arm belong to heterogeneous human-computer interaction control, so that the hand controller is output and mapped to the terminal pose of the slave end mechanical arm in a point-to-point mode, the mapping control flow is shown in fig. 7, the point-to-point mapping mode depends on the kinematic analysis of the slave end mechanical arm, and the kinematic inverse solution of the six-degree-of-freedom robot can be obtained through a D-H parameter method. The mapping of the end pose of the slave end mechanical arm can be realized by two modes of guide control and increment control: the guiding control is follow-up control, and the base coordinate system of the robot and the coordinate system of the hand controller are always kept to be coincident, so that the tail end of the robot moves along with the movement of the hand controller; incremental control is realized by mapping the relative increment of the motion of the hand controller to the current terminal pose of the robot.
As a preferred embodiment, in the step of predicting movement, the step of generating a command of the next time based on the received movement signal specifically includes:
mapping a moving signal output by the hand controller into the tail end pose of the slave end mechanical arm by adopting an incremental control mode, and generating a corresponding instruction;
in the embodiment, the incremental control mode is adopted to map the movement signal output by the hand controller into the tail end pose of the slave-end mechanical arm, so that the tail end pose of the slave-end mechanical arm can be prevented from sudden change in the initial stage of movement, and the safety of the slave-end mechanical arm can be ensured.
Because the working space of the hand controller is far smaller than the movement space of the slave mechanical arm, and the scale factor for adjusting the scale factor by the hand controller is limited in order to ensure the control precision, in this case, the movement of the slave mechanical arm may exceed the working space of the hand controller; as a preferred implementation manner, in this embodiment, when the amount of change in the end pose of the slave-end mechanical arm after mapping exceeds the working space of the hand controller before mapping, a mapping space drifting strategy is used to complete mapping of the movement signal to the end pose of the slave-end mechanical arm;
the mapping space drift strategy is shown in fig. 8, and is specifically executed according to the following steps:
(S1) calculating a total movement distance of the hand controller as a target distance according to a preset scale factor;
(S2) continuing the movement of the hand controller until the total distance the hand controller has moved reaches the target distance, or the hand controller moves to the limit position;
(S3) if the total distance the hand controller has moved reaches the target distance, proceeding to step (S4); if the hand controller moves to the limit position, the hand controller is reset to the initial position, and the step (S2) is carried out;
(S4) the mapping is ended.
According to the embodiment, when the variation of the terminal pose of the slave-end mechanical arm is large after mapping, the mapping of the terminal pose of the slave-end mechanical arm is completed by adopting a mapping space drifting strategy, and the effective coverage of a mapping space can be ensured while the control precision is ensured.
As an optional implementation manner, in this embodiment, the predicting movement step further includes:
according to the prediction result of the artificial neural network, the motion of the slave-end mechanical arm is displayed at the master end in a virtual reality mode, so that the motion condition of the slave-end mechanical arm can be monitored conveniently in real time.
The hand controller with force sense information is important ground direct interaction control mechanical arm equipment and has the functions of motion output and force feedback: on one hand, the hand controller transmits the motion information of the hand of the operator to a remote robot controller to control the motion of the robot; on the other hand, the interaction force/moment or 'virtual' force/moment information of the remote robot and the environment is fed back to the main end, and the force feedback of the hand controller is used for acting on the hand of an operator, so that the force telepresence is provided for the operator. In the embodiment, the hand controller is used for outputting the movement signal generated according to the prediction result of the artificial neural network, so that the man-machine interaction is improved.
It should be noted that, the generation of the movement signal by using the hand controller based on the prediction result of the artificial neural network in the embodiment is only a preferred embodiment of the present invention, and should not be construed as the only limitation to the present invention; in other embodiments of the present invention, a human-computer interaction device such as a space mouse, an eye tracker, a motion sensing input device, etc. may be used to generate the movement signal, which will not be described herein.
Example 3:
a non-cooperative target capture teleoperation system under variable time delay condition comprises: a master and a slave communicating over a world link; the main end is positioned on the ground and comprises a main end controller; the slave end is positioned in the space and comprises a slave end controller and a slave end mechanical arm;
a main-end controller for executing the predicted movement step; the movement prediction step includes: at the current moment, generating an instruction at the current moment at the master end according to the moving signal and sending the instruction to the slave end; predicting the state of the slave mechanical arm after the slave mechanical arm executes the instruction sent by the master at the current moment based on the instruction sequence sent from the master to the slave and the state sequence of the slave; the instruction is used for indicating the movement angle of each joint of the slave end mechanical arm; the movement signal is a signal generated based on a prediction result of the artificial neural network at a previous moment and is used for indicating position information of the movement from the tail end of the end mechanical arm; the main end is positioned on the ground and comprises a main end controller; the slave end is positioned in the space and comprises a slave end controller and a slave end mechanical arm; the master end and the slave end communicate through a world link;
the main end controller is also used for executing the ground direct operation step; the ground direct operation step comprises: repeatedly executing the predicting and moving step until the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within a preset range;
a slave-side controller for performing the local autonomous operation step; the local autonomous operation step includes: when the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within a preset range, controlling the slave end mechanical arm to autonomously identify the non-cooperative target and capture the non-cooperative target;
in this embodiment, the detailed implementation of each controller can refer to the description of the method embodiment described above, and will not be repeated here.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A non-cooperative target capture teleoperation method under the condition of variable time delay is characterized by comprising the following steps:
a step of predicting movement: at the current moment, generating an instruction at the current moment at the master end according to the moving signal and sending the instruction to the slave end; predicting the state of a slave end mechanical arm after the slave end executes the instruction sent by the master end at the current moment by using an artificial neural network based on the instruction sequence sent by the master end to the slave end and the state sequence of the slave end; the instructions are used for indicating the movement angle of each joint of the slave end mechanical arm; the movement signal is a signal generated based on a prediction result of the artificial neural network at a previous moment and is used for indicating position information of the movement of the tail end of the slave end mechanical arm; the main end is positioned on the ground and comprises a main end controller; the slave end is positioned in the space and comprises a slave end controller and a slave end mechanical arm; the master end and the slave end communicate through a world link;
ground direct operation step: repeatedly executing the predicting movement step until the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within a preset range;
a local autonomous operation step: and when the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within the preset range, the slave end mechanical arm is used for autonomously identifying the non-cooperative target and capturing the non-cooperative target.
2. The non-cooperative target grabbing and teleoperation method under the variable delay condition of claim 1, wherein the step of predicting movement further comprises:
receiving the real state of the mechanical arm execution instruction fed back by the slave end at the master end;
and updating the network weight of the artificial neural network by using a back propagation algorithm by taking the real state of the slave-end mechanical arm after executing the instruction sent by the master end at a certain moment as a teacher signal and taking the sum of square errors between the output of the artificial neural network and the teacher signal at a corresponding moment as a loss function.
3. The method of non-cooperative target capture teleoperation under variable time delay conditions as claimed in claim 2, wherein said predicting movement step further comprises:
at the master end, updating the state sequence of the slave end by using the received teacher signal;
synchronously scrolling on the instruction sequence sent by the master terminal to the slave terminal and the state sequence of the slave terminal by utilizing a first scrolling window and a second scrolling window so as to enable the second scrolling window to contain the latest state;
and utilizing the instruction subsequence in the first rolling window and the state subsequence in the second rolling window as the input of the artificial neural network at the next moment of the moment corresponding to the latest state.
4. The non-cooperative target capturing teleoperation method under the variable time delay condition according to any one of claims 1 to 3, wherein the moving signal is generated by the master controller according to a preset moving strategy.
5. A non-cooperative target grabbing and teleoperation method under the variable time delay condition according to any one of claims 1-3, wherein the master terminal further comprises a hand controller with force sense information, and the movement signal is from the hand controller.
6. The method for capturing and teleoperating a non-cooperative target under the variable-delay condition as claimed in claim 5, wherein the hand controller operates in an MS mode, and a scale factor adjusting function is added to the hand controller;
and the scale factor adjusting function is used for outputting the motion and force feedback of the hand controller according to a preset scale factor so as to generate the movement signal.
7. The method of non-cooperative target grabbing and teleoperation under variable delay conditions according to claim 6, wherein the step of predicting movement, the step of generating the instruction of the next time according to the received movement signal, comprises:
and mapping the movement signal output by the hand controller into the tail end pose of the slave end mechanical arm by adopting an increment control mode, and generating a corresponding instruction.
8. The non-cooperative target capture teleoperation method under the variable time delay condition according to claim 7, wherein when the change amount of the terminal pose of the slave-end mechanical arm after mapping exceeds the working space of the hand controller before mapping, a mapping space drifting strategy is adopted to complete mapping of a moving signal to the terminal pose of the slave-end mechanical arm;
the mapping space drift strategy is executed according to the following steps:
(S1) calculating a total movement distance of the hand controller as a target distance according to the preset scale factor;
(S2) continuing the movement of the hand controller until the total distance the hand controller has moved reaches the target distance, or the hand controller moves to an extreme position;
(S3) if the total distance the hand controller has moved reaches the target distance, proceeding to step (S4); if the hand controller moves to the limit position, resetting the hand controller to the initial position, and turning to the step (S2);
(S4) the mapping is ended.
9. The non-cooperative target precise-capture teleoperation method under the variable-delay condition according to any one of claims 6 to 8, wherein the predicting movement step further comprises:
and displaying the motion of the slave end mechanical arm at the master end in a virtual reality mode according to the prediction result of the artificial neural network.
10. A non-cooperative target capture teleoperation system under a variable time delay condition is characterized by comprising: a master and a slave communicating over a world link; the main end is positioned on the ground and comprises a main end controller; the slave end is positioned in the space and comprises a slave end controller and a slave end mechanical arm;
the main end controller is used for executing the step of predicting movement; the movement prediction step includes: at the current moment, generating an instruction at the current moment at the master end according to the moving signal and sending the instruction to the slave end; predicting the state of a slave end mechanical arm after the slave end executes the instruction sent by the master end at the current moment by using an artificial neural network based on the instruction sequence sent by the master end to the slave end and the state sequence of the slave end; the instructions are used for indicating the movement angle of each joint of the slave end mechanical arm; the movement signal is a signal generated based on a prediction result of the artificial neural network at a previous moment and is used for indicating position information of the movement of the tail end of the slave end mechanical arm; the main end is positioned on the ground and comprises a main end controller; the slave end is positioned in the space and comprises a slave end controller and a slave end mechanical arm; the master end and the slave end communicate through a world link;
the main end controller is also used for executing the ground direct operation step; the ground direct operation step comprises: repeatedly executing the predicting movement step until the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within a preset range;
the slave controller is used for executing a local autonomous operation step; the locally autonomous operating step comprises: and when the distance between the tail end of the slave end mechanical arm and the non-cooperative target is within the preset range, controlling the slave end mechanical arm to autonomously identify the non-cooperative target and capture the non-cooperative target.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010888700.4A CN112060088A (en) | 2020-08-28 | 2020-08-28 | Non-cooperative target accurate capturing teleoperation method under variable time delay condition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010888700.4A CN112060088A (en) | 2020-08-28 | 2020-08-28 | Non-cooperative target accurate capturing teleoperation method under variable time delay condition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112060088A true CN112060088A (en) | 2020-12-11 |
Family
ID=73660186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010888700.4A Pending CN112060088A (en) | 2020-08-28 | 2020-08-28 | Non-cooperative target accurate capturing teleoperation method under variable time delay condition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112060088A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113927596A (en) * | 2021-10-07 | 2022-01-14 | 西北工业大学 | Time-varying output constraint robot teleoperation finite time control method based on width neural learning |
| CN114905478A (en) * | 2021-02-08 | 2022-08-16 | 腾讯科技（深圳）有限公司 | Bilateral teleoperation system and control method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101733746A (en) * | 2009-12-22 | 2010-06-16 | 哈尔滨工业大学 | Autonomously identifying and capturing method of non-cooperative target of space robot |
| US20120325972A1 (en) * | 2007-03-09 | 2012-12-27 | Macdonald Dettwiler & Associates Inc. | Robotic satellite refuelling tool |
| CN106864776A (en) * | 2017-01-18 | 2017-06-20 | 哈尔滨工业大学深圳研究生院 | A kind of method and system of the capture target satellite based on butt joint ring |
| CN107717994A (en) * | 2017-11-08 | 2018-02-23 | 西安交通大学 | Principal and subordinate's heterogeneous robot universal control method and system based on principal and subordinate's space reflection |
| CN107877517A (en) * | 2017-11-16 | 2018-04-06 | 哈尔滨工业大学 | Motion mapping method based on CyberForce remote operating mechanical arms |
| CN109131956A (en) * | 2018-10-18 | 2019-01-04 | 哈尔滨工业大学 | A kind of noncooperative target satellite-rocket docking ring capture mechanism and its catching method |
-
2020
- 2020-08-28 CN CN202010888700.4A patent/CN112060088A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120325972A1 (en) * | 2007-03-09 | 2012-12-27 | Macdonald Dettwiler & Associates Inc. | Robotic satellite refuelling tool |
| CN101733746A (en) * | 2009-12-22 | 2010-06-16 | 哈尔滨工业大学 | Autonomously identifying and capturing method of non-cooperative target of space robot |
| CN106864776A (en) * | 2017-01-18 | 2017-06-20 | 哈尔滨工业大学深圳研究生院 | A kind of method and system of the capture target satellite based on butt joint ring |
| CN107717994A (en) * | 2017-11-08 | 2018-02-23 | 西安交通大学 | Principal and subordinate's heterogeneous robot universal control method and system based on principal and subordinate's space reflection |
| CN107877517A (en) * | 2017-11-16 | 2018-04-06 | 哈尔滨工业大学 | Motion mapping method based on CyberForce remote operating mechanical arms |
| CN109131956A (en) * | 2018-10-18 | 2019-01-04 | 哈尔滨工业大学 | A kind of noncooperative target satellite-rocket docking ring capture mechanism and its catching method |
Non-Patent Citations (3)
| Title |
|---|
| 王宇; 宋爱国; 徐效农: "基于任务空间双模式结合的遥操作机器人系统", 《电子测量技术》 * |
| 甘凯: "基于虚拟夹具的空间机器人遥操作与示教技术研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 赵迪: "变时延环境下的任务级遥操作关键技术研究", 《中国博士学位论文全文数据库信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114905478A (en) * | 2021-02-08 | 2022-08-16 | 腾讯科技（深圳）有限公司 | Bilateral teleoperation system and control method |
| CN113927596A (en) * | 2021-10-07 | 2022-01-14 | 西北工业大学 | Time-varying output constraint robot teleoperation finite time control method based on width neural learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022088593A1 (en) | Robotic arm control method and device, and human-machine cooperation model training method | |
| US9862090B2 (en) | Surrogate: a body-dexterous mobile manipulation robot with a tracked base | |
| JPH06314103A (en) | Controller and passive sensing device | |
| US11975451B2 (en) | Simulation-in-the-loop tuning of robot parameters for system modeling and control | |
| CN112207835B (en) | Method for realizing double-arm cooperative work task based on teaching learning | |
| CN115469576B (en) | Teleoperation system based on human-mechanical arm heterogeneous motion space hybrid mapping | |
| CN113829343B (en) | Real-time multitasking and multi-man-machine interaction system based on environment perception | |
| CN110154024B (en) | Assembly control method based on long-term and short-term memory neural network incremental model | |
| CN112060088A (en) | Non-cooperative target accurate capturing teleoperation method under variable time delay condition | |
| CN114274129A (en) | Mechanical arm motion planning and control method, system and medium based on visual guidance | |
| KR20240052808A (en) | Multi-robot coordination using graph neural networks | |
| Brecher et al. | Towards anthropomorphic movements for industrial robots | |
| Santiago et al. | Human-inspired stable bilateral teleoperation of mobile manipulators | |
| Liang et al. | Fast dataset collection approach for articulated equipment pose estimation | |
| Das et al. | GeroSim: A simulation framework for gesture driven robotic arm control using Intel RealSense | |
| CN113103262A (en) | Robot control device and method for controlling robot | |
| Zhou et al. | RETRACTED: uncalibrated dynamic visual servoing via multivariate adaptive regression splines and improved incremental extreme learning machine | |
| EP4175795B1 (en) | Transfer between tasks in different domains | |
| Sugimoto et al. | Trajectory-model-based reinforcement learning: Application to bimanual humanoid motor learning with a closed-chain constraint | |
| Zorić et al. | Performance Comparison of Teleoperation Interfaces for Ultra-Lightweight Anthropomorphic Arms | |
| Akbulut et al. | Bimanual rope manipulation skill synthesis through context dependent correction policy learning from human demonstration | |
| Li et al. | Manipulator Motion Planning based on Actor-Critic Reinforcement Learning | |
| CN114083545B (en) | Moving object robot grabbing method and device based on visual perception | |
| Lee et al. | A probabilistic motion control approach for teleoperated construction machinery | |
| Heyu et al. | Impedance control method with reinforcement learning for dual-arm robot installing slabstone |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201211 |
|
| RJ01 | Rejection of invention patent application after publication |