CN112060088A - Non-cooperative target accurate capturing teleoperation method under variable time delay condition - Google Patents
Non-cooperative target accurate capturing teleoperation method under variable time delay condition Download PDFInfo
- Publication number
- CN112060088A CN112060088A CN202010888700.4A CN202010888700A CN112060088A CN 112060088 A CN112060088 A CN 112060088A CN 202010888700 A CN202010888700 A CN 202010888700A CN 112060088 A CN112060088 A CN 112060088A
- Authority
- CN
- China
- Prior art keywords
- slave
- movement
- master
- cooperative target
- mechanical arm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000013528 artificial neural network Methods 0.000 claims abstract description 57
- 238000013507 mapping Methods 0.000 claims description 34
- 238000005096 rolling process Methods 0.000 claims description 13
- 230000008859 change Effects 0.000 claims description 4
- 230000006870 function Effects 0.000 description 13
- 210000002569 neuron Anatomy 0.000 description 11
- 238000010586 diagram Methods 0.000 description 7
- 238000013459 approach Methods 0.000 description 6
- 230000003993 interaction Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000012549 training Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000006641 stabilisation Effects 0.000 description 2
- 238000011105 stabilization Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003238 somatosensory effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1689—Teleoperation
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J15/00—Gripping heads and other end effectors
- B25J15/08—Gripping heads and other end effectors having finger members
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J18/00—Arms
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1661—Programme controls characterised by programming, planning systems for manipulators characterised by task planning, object-oriented languages
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
本发明公开了一种变时延条件下非合作目标精确抓捕遥操作方法，属于空间机器人遥操作技术领域，包括：预测移动步骤：在当前时刻，在主端根据移动信号生成当前时刻的指令并发送至从端；基于主端向从端发送的指令序列和从端的状态序列，利用人工神经网络预测从端机械臂执行主端在当前时刻发送的指令后的状态；指令用于指示从端机械臂各关节的移动角度；移动信号基于前一时刻的预测结果生成，用于指示从端机械臂末端移动的位置信息；重复执行预测移动步骤，直至从端机械臂的末端与非合作目标之间的距离d位于预设范围R内；在距离d位于预设范围R内时，利用从端机械臂自主识别并抓捕非合作目标。本发明能够实现对非合作目标的精确抓捕。
The invention discloses a remote operation method for precise capture of non-cooperative targets under the condition of variable time delay, which belongs to the technical field of remote operation of space robots. and send it to the slave; based on the sequence of instructions sent by the master to the slave and the state sequence of the slave, the artificial neural network is used to predict the state of the slave arm after executing the instruction sent by the master at the current moment; the instruction is used to indicate the slave The movement angle of each joint of the manipulator; the movement signal is generated based on the prediction result of the previous moment, and is used to indicate the position information of the movement of the end of the slave manipulator; repeat the predicted movement steps until the end of the slave manipulator and the non-cooperative target are The distance d is within the preset range R; when the distance d is within the preset range R, the slave robot arm is used to autonomously identify and capture non-cooperative targets. The present invention can realize precise capture of non-cooperative targets.
Description
技术领域technical field
本发明属于空间机器人遥操作技术领域，更具体地，涉及一种变时延条件下非合作目标精确抓捕遥操作方法。The invention belongs to the technical field of teleoperation of space robots, and more particularly relates to a teleoperation method for precise capture of non-cooperative targets under the condition of variable time delay.
背景技术Background technique
大多数已在轨服务的航天器和空间碎片有3个特点：1)没有安装用于机械臂捕获的抓持机构(手柄)以及用于辅助测量的合作标志器和特征块等；2)目标星运动状态未知，可能为三轴稳定、自旋稳定甚至是失控状态下的翻滚等；3)目标星与追踪星之间没有直接的信息交流。此类目标称为非合作目标。为了将空间机器人更好地用于在轨服务，必须解决非合作目标精确抓捕的关键问题。Most spacecraft and space debris that have been in orbital service have 3 characteristics: 1) There is no gripping mechanism (handle) for robotic arm capture and cooperative markers and feature blocks for auxiliary measurements, etc.; 2) Targets The motion state of the star is unknown, and it may be three-axis stabilization, spin stabilization, or even tumbling in a runaway state; 3) There is no direct information exchange between the target star and the tracking star. Such goals are called non-cooperative goals. In order to better utilize space robots for on-orbit services, the key issue of precise capture of non-cooperative targets must be addressed.
空间机械臂上面有视觉摄像头，通过拍摄空间机械臂场景，自动识别非合作目标，然后发送控制指令给空间机械臂去抓捕非合作目标。针对空间非合作目标的精确捕获，目前的方法主要是利用机械臂的自动识别与抓捕，但由于空间机械臂工作环境非结构化及工作任务变化的特点，并且受计算机、控制、人工智能和机构等关键支撑技术发展的制约，目前能在空间环境下，尤其是当空间机械臂距离非合作目标较远时，仅仅利用机械臂进行自主识别和抓捕，难以实现对非合作目标的精确抓捕。There is a visual camera on the space manipulator. By shooting the scene of the space manipulator, it can automatically identify the non-cooperative target, and then send control commands to the space manipulator to capture the non-cooperative target. For the precise capture of non-cooperative targets in space, the current method mainly uses the automatic identification and capture of the manipulator. Due to the constraints of the development of key supporting technologies such as mechanisms, currently in the space environment, especially when the space robotic arm is far away from the non-cooperative target, it is difficult to achieve accurate grasping of the non-cooperative target by only using the robotic arm for autonomous identification and capture. catch.
发明内容SUMMARY OF THE INVENTION
针对现有技术的缺陷和改进需求，本发明提供了一种变时延条件下非合作目标精确抓捕遥操作方法，其目的在于，实现在空间环境下对非合作目标的精确抓捕。Aiming at the defects and improvement needs of the prior art, the present invention provides a teleoperation method for precise capture of non-cooperative targets under the condition of variable time delay, the purpose of which is to achieve precise capture of non-cooperative targets in a space environment.
为实现上述目的，按照本发明的一个方面，提供了一种变时延条件下非合作目标抓捕遥操作方法，包括：In order to achieve the above object, according to one aspect of the present invention, a non-cooperative target capture teleoperation method under variable time delay conditions is provided, comprising:
预测移动步骤：在当前时刻，在主端根据移动信号生成当前时刻的指令并发送至从端；基于主端向从端发送的指令序列和从端的状态序列，利用人工神经网络预测从端机械臂执行主端在当前时刻发送的指令后的状态；指令用于指示从端机械臂各关节的移动角度；移动信号为基于前一时刻人工神经网络的预测结果生成的信号，用于指示从端机械臂末端移动的位置信息；主端位于地面，包括主端控制器；从端位于空间，包括从端控制器和从端机械臂；主端和从端通过天地链路通信；Prediction of movement steps: at the current moment, the master generates the command at the current moment according to the movement signal and sends it to the slave; based on the sequence of instructions sent from the master to the slave and the state sequence of the slave, the artificial neural network is used to predict the robot arm of the slave The state after executing the command sent by the master at the current moment; the command is used to indicate the movement angle of each joint of the slave robot arm; the movement signal is a signal generated based on the prediction result of the artificial neural network at the previous moment, used to indicate the slave robot The position information of the movement of the end of the arm; the master end is located on the ground, including the master end controller; the slave end is located in space, including the slave end controller and the slave end robotic arm; the master end and the slave end communicate through the sky-earth link;
地面直接操作步骤：重复执行预测移动步骤，直至从端机械臂的末端与非合作目标之间的距离位于预设范围内；Ground direct operation step: Repeat the predicted movement step until the distance between the end of the slave robot arm and the non-cooperative target is within the preset range;
局部自主操作步骤：在从端机械臂的末端与非合作目标之间的距离位于预设范围内时，利用从端机械臂自主识别非合作目标并抓捕非合作目标。Local autonomous operation steps: when the distance between the end of the slave manipulator and the non-cooperative target is within a preset range, use the slave manipulator to autonomously identify the non-cooperative target and capture the non-cooperative target.
本发明所提供的变时延条件下非合作目标抓捕遥操作方法，包含两个工作模式，即地面直接操作模式和局部自主操作模式，在地面直接操作模式下，通过遥操作的方式使得从端机械臂能够在较大范围内快速接近非合作目标，在局部自主操作模式下，则由从端机械臂自主完成对非合作目标的抓捕；本发明通过遥操作能够有效解决距离较远时从端机械臂无法通过自主完整对非合作目标进行精确抓捕的问题。由于在遥操作的过程中，主端与从端之间通信的天地链路存在时延，导致工作环境是时变的，本发明考虑变时延条件，在主端利用人工神经网络对从端机械臂的运动状态进行预测，并基于预测结果向从端发送相应的指令，使从端机械臂按照预期的移动方式接近非合作目标，由此能够适应非合作目标抓捕过程中动态变化的环境和任务；在从端机械臂距非合作目标较近时，仅仅依靠从端机械臂自主识别和抓捕非合作目标，能够避免天地链路带来的时延，充分利用从端机械臂的局部自主操作完成对非合作目标的精确抓捕。总的来说，本发明利用地面直接操作模式和局部自主操作模式相结合，在地面直接操作模式中，利用人工神经网络对从端机械臂的状态进行预测并基于预测结果生成相应的指令，使从端按照期望的方式快速接近非合作目标，在局部自主操作模式中，利用从端机械臂自主识别和抓捕非合作目标，能够实现对非合作目标的精确抓捕。The remote operation method for capturing non-cooperative targets under the condition of variable time delay provided by the present invention includes two working modes, namely, the direct operation mode on the ground and the local autonomous operation mode. The end robot arm can quickly approach the non-cooperative target in a large range, and in the local autonomous operation mode, the slave end robot arm autonomously completes the capture of the non-cooperative target; the present invention can effectively solve the problem of long distance through remote operation. The problem that the slave robot arm cannot accurately capture non-cooperative targets through autonomous completeness. In the process of remote operation, there is a time delay in the communication between the master and the slave, resulting in a time-varying working environment. The present invention considers the time-varying condition, and uses an artificial neural network at the master to connect to the slave. The motion state of the robotic arm is predicted, and based on the prediction result, the corresponding command is sent to the slave side, so that the slave side robotic arm can approach the non-cooperative target according to the expected movement mode, so that it can adapt to the dynamically changing environment during the capture process of the non-cooperative target. and tasks; when the slave manipulator is close to the non-cooperative target, only relying on the slave manipulator to identify and capture the non-cooperative target autonomously can avoid the delay caused by the sky-earth link and make full use of the local part of the slave manipulator. The autonomous operation completes the precise capture of non-cooperative targets. In general, the present invention combines the ground direct operation mode and the local autonomous operation mode. In the ground direct operation mode, the artificial neural network is used to predict the state of the slave robot arm and generate corresponding instructions based on the prediction results, so that the The slave quickly approaches the non-cooperative target in the desired way. In the local autonomous operation mode, the slave robot can autonomously identify and capture the non-cooperative target, which can achieve precise capture of the non-cooperative target.
进一步地，预测移动步骤还包括：Further, the predicting movement step also includes:
在主端接收从端反馈的机械臂执行指令后的真实状态；The real state of the robot arm after the master receives the feedback from the slave and executes the command;
以从端机械臂执行主端在某一时刻发送的指令后的真实状态为教师信号，以对应时刻人工神经网络的输出与教师信号之间的平方误差和为损失函数，利用反向传播算法更新人工神经网络的网络权值。Take the real state of the slave robot arm after executing the instruction sent by the master at a certain moment as the teacher signal, take the sum of squared errors between the output of the artificial neural network and the teacher signal at the corresponding moment as the loss function, and use the back propagation algorithm to update The network weights of the artificial neural network.
本发明基于从端反馈的真实状态信息，对人工神经网络的网络权值进行更新，能够使人工神经网络能够更好地适应动态变化的工作环境，提高人工神经网络的预测精度。The invention updates the network weights of the artificial neural network based on the real state information fed back from the end, which can make the artificial neural network better adapt to the dynamically changing working environment and improve the prediction accuracy of the artificial neural network.
进一步地，预测移动步骤还包括：Further, the predicting movement step also includes:
在主端，利用接收到的教师信号更新从端的状态序列；On the master side, use the received teacher signal to update the state sequence of the slave side;
利用第一滚动窗口和第二滚动窗口在主端向从端发送的指令序列和从端的状态序列上同步滚动，以使得第二滚动窗口内包含最新的状态；Utilize the first scroll window and the second scroll window to scroll synchronously on the instruction sequence sent by the master to the slave and the state sequence of the slave, so that the second scroll window contains the latest state;
利用第一滚动窗口内的指令子序列和第二滚动窗口内的状态子序列作为在最新的状态所对应时刻的下一时刻人工神经网络的输入。The instruction subsequence in the first rolling window and the state subsequence in the second rolling window are used as the input of the artificial neural network at the next moment corresponding to the latest state.
本发明利用滚动窗口实时截取状态序列中最新的子序列及相对应的指令子序列，作为人工神经网络模型的输入，实现了一种滚动预测的方式，由此能够不断修正模型，实时跟踪参考值，提高预测精度，同时能够基于小样本实现对人工神经网络的反复迭代及快速训练。The invention uses the rolling window to intercept the latest subsequence and the corresponding instruction subsequence in the state sequence in real time, as the input of the artificial neural network model, and realizes a rolling prediction method, so that the model can be continuously revised and the reference value can be tracked in real time. , to improve the prediction accuracy, and at the same time, it can realize repeated iteration and fast training of artificial neural network based on small samples.
在一些实施例中，移动信号由主端控制器根据预设的移动策略生成。In some embodiments, the movement signal is generated by the master controller according to a preset movement strategy.
在一些实施例中，主端还包括带力觉信息的手控器，且移动信号来自于手控器。In some embodiments, the main terminal further includes a hand controller with force sense information, and the movement signal comes from the hand controller.
带力觉信息的手控器是一种重要的地面直接交互控制机械臂设备，同时具有运动输出和力反馈功能：一方面，手控器将操作者手部的运动信息传给远端的机器人控制器，以控制机器人的运动；另一方面，将远端机器人与环境的相互作用力/力矩或“虚拟”力/力矩信息反馈给主端，通过手控器的力反馈实现作用于操作者手部，为操作者提供力觉临场感。本发明利用手控器输出根据人工神经网络的预测结果生成的移动信号，提高了人机交互性。The hand controller with force sense information is an important ground-based direct interactive control robotic arm device, and has both motion output and force feedback functions: on the one hand, the hand controller transmits the motion information of the operator's hand to the remote robot. The controller is used to control the movement of the robot; on the other hand, the force/torque or "virtual" force/torque information of the interaction between the remote robot and the environment is fed back to the main end, and the force feedback of the hand controller is used to act on the operator. The hand provides the operator with a sense of force presence. The invention utilizes the hand controller to output the moving signal generated according to the prediction result of the artificial neural network, thereby improving the man-machine interaction.
进一步地，手控器工作在MS模式，且手控器中增加了比例因子调整功能；Further, the hand controller works in MS mode, and the scale factor adjustment function is added to the hand controller;
比例因子调整功能用于将手控器的运动及力反馈按照预设的比例因子进行输出，从而生成移动信号。The scale factor adjustment function is used to output the motion and force feedback of the hand controller according to the preset scale factor, thereby generating a movement signal.
由于手控器的工作空间远小于从端机械臂的运动空间，本发明通过在手控器增加比例因子调整功能，将手控器的运动及力反馈按照预设的比例因子进行输出，能够将手控器有限的工作空间内的运动输出放大为从端机械臂在其运动空间中的运动，从而保证了对非合作目标抓捕的准确性。Since the working space of the hand controller is much smaller than the motion space of the slave robot arm, the present invention can output the motion and force feedback of the hand controller according to the preset scale factor by adding a scale factor adjustment function to the hand controller, so that the The motion output in the limited working space of the hand controller is amplified into the motion of the slave-end robotic arm in its motion space, thereby ensuring the accuracy of capturing non-cooperative targets.
进一步地，预测移动步骤中，根据接收到的移动信号生成下一时刻的指令，包括：Further, in the step of predicting the movement, the instruction of the next moment is generated according to the received movement signal, including:
采用增量控制方式，将手控器输出的移动信号映射为从端机械臂的末端位姿，并生成相应的指令。Using the incremental control method, the movement signal output by the hand controller is mapped to the end pose of the slave robot arm, and the corresponding command is generated.
本发明采用增量控制方式将手控器输出的移动信号映射为从端机械臂的末端位姿，能够防止从端机械臂在运动的起始阶段，其末端位姿出现突变，由此能够保证从端机械臂的安全。The present invention uses the incremental control method to map the movement signal output by the hand controller to the terminal posture of the slave end robot arm, which can prevent the terminal end posture of the slave end robot arm from changing abruptly in the initial stage of movement, thereby ensuring that Safety of the slave arm.
进一步地，当映射后从端机械臂的末端位姿的变化量超过映射前手控器的工作空间时，采用映射空间漂移策略完成移动信号到从端机械臂末端位姿的映射；Further, when the change of the end pose of the slave manipulator after the mapping exceeds the workspace of the hand controller before the mapping, the mapping space drift strategy is used to complete the mapping of the movement signal to the end pose of the slave manipulator;
映射空间漂移策略按照如下步骤执行：The mapping space drift strategy is implemented as follows:
(S1)根据预设的比例因子计算手控器的总运动距离，作为目标距离；(S1) calculate the total movement distance of the hand controller according to the preset scale factor, as the target distance;
(S2)使手控器连续运动，直至手控器已运动的总距离达到目标距离，或者手控器运动到极限位置；(S2) make the hand controller move continuously, until the total distance that the hand controller has moved reaches the target distance, or the hand controller moves to the limit position;
(S3)若手控器已运动的总距离达到目标距离，则转入步骤(S4)；若手控器运动到极限位置，则将手控器复位到初始位置，并转入步骤(S2)；(S3) if the total distance that the hand controller has moved reaches the target distance, then go to step (S4); If the hand controller moves to the limit position, then reset the hand controller to the initial position, and go to step (S2);
(S4)映射结束。(S4) Mapping ends.
由于手控器的工作空间远小于从端机械臂的运动空间，而为了保证控制精度，手控器进行比例因子调整的比例因子的大小受限，在此情况下，从端机械臂的运动也可能超出手控器的工作空间；本发明在映射后从端机械臂的末端位姿的变化量较大时，采用映射空间漂移策略完成从端机械臂末端位姿的映射，能够在保证控制精度的同时，保证映射空间的有效覆盖。Since the working space of the hand controller is much smaller than the movement space of the slave manipulator, and in order to ensure the control accuracy, the size of the scale factor for the hand controller to adjust the scale factor is limited. In this case, the movement of the slave manipulator is also limited. It may exceed the working space of the hand controller; the present invention adopts the mapping space drift strategy to complete the mapping of the end pose of the slave manipulator when the change of the position and posture of the slave end robot arm after the mapping is large, which can ensure the control accuracy. At the same time, the effective coverage of the mapping space is guaranteed.
进一步地，预测移动步骤还包括：Further, the predicting movement step also includes:
根据人工神经网络的预测结果，将从端机械臂的运动以虚拟现实的方式在主端进行呈现。According to the prediction result of the artificial neural network, the motion of the robot arm from the slave end is presented on the master end in the form of virtual reality.
本发明通根据人工神经网络的预测结果，将从端机械臂的运动以虚拟现实的方式在主端进行呈现，能够便于实时监测从端机械臂的运动情况。According to the prediction result of the artificial neural network, the present invention presents the motion of the slave manipulator on the master side in the form of virtual reality, which can facilitate real-time monitoring of the motion of the slave manipulator.
按照本发明的另一个方面，提供了一种变时延条件下非合作目标精确抓捕遥操作系统，包括：通过天地链路通信的主端和从端；主端位于地面，包括主端控制器；从端位于空间，包括从端控制器和从端机械臂；According to another aspect of the present invention, a teleoperating system for precise capture of non-cooperative targets under variable delay conditions is provided, including: a master terminal and a slave terminal communicating through a sky-earth link; the master terminal is located on the ground, and the master terminal controls The slave end is located in the space, including the slave end controller and the slave end robot arm;
主端控制器，用于执行预测移动步骤；移动预测步骤包括：在当前时刻，在主端根据移动信号生成当前时刻的指令并发送至从端；基于主端向从端发送的指令序列和从端的状态序列，利用人工神经网络预测从端机械臂执行主端在当前时刻发送的指令后的状态；指令用于指示从端机械臂各关节的移动角度；移动信号为基于前一时刻人工神经网络的预测结果生成的信号，用于指示从端机械臂末端移动的位置信息；主端位于地面，包括主端控制器；从端位于空间，包括从端控制器和从端机械臂；主端和从端通过天地链路通信；The master-end controller is used to execute the predicted movement step; the movement prediction step includes: at the current moment, the master-end generates an instruction at the current moment according to the movement signal and sends it to the slave-end; based on the instruction sequence sent from the master-end to the slave-end and the slave-end The state sequence of the end, using artificial neural network to predict the state of the slave arm after executing the command sent by the master at the current moment; the command is used to indicate the movement angle of each joint of the slave arm; the movement signal is based on the artificial neural network at the previous moment. The signal generated by the prediction result is used to indicate the position information of the movement of the end of the slave robot arm; the master end is located on the ground, including the master end controller; the slave end is located in space, including the slave end controller and the slave end robot arm; the master end and The slave terminal communicates through the sky-earth link;
主端控制器，还用于执行地面直接操作步骤；地面直接操作步骤包括：重复执行预测移动步骤，直至从端机械臂的末端与非合作目标之间的距离位于预设范围内；The master-end controller is also used to perform the ground direct operation step; the ground direct operation step includes: repeating the predicted movement step until the distance between the end of the slave-end robotic arm and the non-cooperative target is within a preset range;
从端控制器，用于执行局部自主操作步骤；局部自主操作步骤包括：在从端机械臂的末端与非合作目标之间的距离位于预设范围内时，控制从端机械臂自主识别非合作目标并抓捕非合作目标。The slave-end controller is used to execute the local autonomous operation steps; the local autonomous operation steps include: when the distance between the end of the slave-end robotic arm and the non-cooperative target is within a preset range, controlling the slave-end robotic arm to autonomously identify the non-cooperative target target and capture non-cooperative targets.
总体而言，通过本发明所构思的以上技术方案，能够取得以下有益效果：In general, through the above technical solutions conceived by the present invention, the following beneficial effects can be achieved:
(1)本发明利用地面直接操作模式和局部自主操作模式相结合，在地面直接操作模式中，利用人工神经网络对从端机械臂的状态进行预测并基于预测结果生成相应的指令，使从端按照期望的方式快速接近非合作目标，在局部自主操作模式中，利用从端机械臂自主识别和抓捕非合作目标，能够实现对非合作目标的精确抓捕。(1) The present invention combines the ground direct operation mode and the local autonomous operation mode. In the ground direct operation mode, the artificial neural network is used to predict the state of the slave manipulator and generate corresponding instructions based on the prediction result, so that the slave end The non-cooperative target is quickly approached in the desired way. In the local autonomous operation mode, the non-cooperative target can be accurately captured by using the slave robot arm to autonomously identify and capture the non-cooperative target.
(2)本发明基于从端反馈的真实状态信息，对人工神经网络的网络权值进行更新，能够使人工神经网络能够更好地适应动态变化的工作环境，提高人工神经网络的预测精度。(2) The present invention updates the network weights of the artificial neural network based on the real state information fed back from the end, so that the artificial neural network can better adapt to the dynamically changing working environment and improve the prediction accuracy of the artificial neural network.
(3)本发明利用滚动窗口实时截取状态序列中最新的子序列及相对应的指令子序列，作为人工神经网络模型的输入，能不断修正模型，实时跟踪参考值，提高预测精度，同时能够基于小样本实现对人工神经网络的反复迭代及快速训练。(3) The present invention uses the rolling window to intercept the latest subsequence and the corresponding instruction subsequence in the state sequence in real time. As the input of the artificial neural network model, the model can be continuously revised, the reference value can be tracked in real time, and the prediction accuracy can be improved. Small samples realize repeated iteration and fast training of artificial neural network.
附图说明Description of drawings
图1为本发明实施例提供的变时延条件下非合作目标抓捕遥操作方法流程图；FIG. 1 is a flowchart of a non-cooperative target capture teleoperation method under variable delay conditions provided by an embodiment of the present invention;
图2为本发明实施例提供的变时延条件下非合作目标抓捕遥操作系统示意图；2 is a schematic diagram of a teleoperating system for capturing a non-cooperative target under a variable delay condition provided by an embodiment of the present invention;
图3为本发明实施例提供的人工神经网络模型示意图；3 is a schematic diagram of an artificial neural network model provided by an embodiment of the present invention;
图4为本发明实施例提供的滚动预测示意图；4 is a schematic diagram of rolling prediction provided by an embodiment of the present invention;
图5为本发明实施例提供的手控器操作示意图；FIG. 5 is a schematic diagram of an operation of a hand controller provided by an embodiment of the present invention;
图6为本发明实施例提供的手控器MS模式下的网络操作结构图；6 is a structural diagram of a network operation under the MS mode of the hand controller provided by an embodiment of the present invention;
图7为本发明实施例提供的机械臂末端位姿映射示意图；FIG. 7 is a schematic diagram of a robot arm end pose mapping provided by an embodiment of the present invention;
图8为本发明实施例提供的映射空间漂移策略示意图。FIG. 8 is a schematic diagram of a mapping space drift strategy provided by an embodiment of the present invention.
具体实施方式Detailed ways
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。此外，下面所描述的本发明各个实施方式中所涉及到的技术特征只要彼此之间未构成冲突就可以相互组合。In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as there is no conflict with each other.
在本发明中，本发明及附图中的术语“第一”、“第二”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。In the present invention, the terms "first", "second" and the like (if present) in the present invention and the accompanying drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
由于当从端机械臂末端距离非合作目标较远时，非合作目标相对于从端机械臂末端的姿态变化迅速，从端机械臂无法自主完成对非合作目标的精确抓捕；为了解决现有的从端机械臂在复杂多变的空间环境下无法精确抓捕非合作目标的技术问题，本发明提供了一种变时延条件下非合作目标精确抓捕遥操作方法，其整体思路在于：通过两种模式，即地面直接操作模式和局部自主操作模式相结合，在地面直接操作模式基于对从端机械臂状态的预测发送相应的指令，使从端机械臂在大范围内靠近非合作目标使得非合作目标相对于从端机械臂的运动状态达到稳定，在局部自主操作模式下，则利用从端机械臂自主识别和抓捕非合作目标，最终实现对非合作目标的精确抓捕。When the end of the slave manipulator is far away from the non-cooperative target, the attitude of the non-cooperative target relative to the end of the slave manipulator changes rapidly, and the slave manipulator cannot autonomously complete the precise capture of the non-cooperative target; in order to solve the problem of existing The technical problem that the slave robot arm cannot accurately capture the non-cooperative target in the complex and changeable space environment, the present invention provides a non-cooperative target precise capture teleoperation method under the condition of variable time delay, and the overall idea is as follows: Through the combination of two modes, namely the direct operation mode on the ground and the local autonomous operation mode, the direct operation mode on the ground sends corresponding commands based on the prediction of the state of the slave manipulator, so that the slave manipulator can approach the non-cooperative target in a large range The motion state of the non-cooperative target relative to the slave manipulator is stabilized. In the local autonomous operation mode, the slave manipulator is used to autonomously identify and capture the non-cooperative target, and finally achieve the precise capture of the non-cooperative target.
以下为实施例：The following are examples:
实施例1：Example 1:
一种变时延条件下非合作目标抓捕遥操作方法，如图1所示，包括：A non-cooperative target capture teleoperation method under the condition of variable delay, as shown in Figure 1, includes:
预测移动步骤：在当前时刻，在主端根据移动信号生成当前时刻的指令并发送至从端；基于主端向从端发送的指令序列和从端的状态序列，利用人工神经网络预测从端机械臂执行主端在当前时刻发送的指令后的状态；指令用于指示从端机械臂各关节的移动角度；移动信号为基于前一时刻人工神经网络的预测结果生成的信号，用于指示从端机械臂末端移动的位置信息；本实施例涉及的遥操作系统由主端和从端构成，主端位于地面，包括主端控制器；从端位于空间，包括从端控制器和从端机械臂；主端和从端通过天地链路通信；Prediction of movement steps: at the current moment, the master generates the command at the current moment according to the movement signal and sends it to the slave; based on the sequence of instructions sent from the master to the slave and the state sequence of the slave, the artificial neural network is used to predict the robot arm of the slave The state after executing the command sent by the master at the current moment; the command is used to indicate the movement angle of each joint of the slave robot arm; the movement signal is a signal generated based on the prediction result of the artificial neural network at the previous moment, used to indicate the slave robot The position information of the movement of the end of the arm; the teleoperation system involved in this embodiment is composed of a master end and a slave end, the master end is located on the ground, including the master end controller; the slave end is located in the space, including the slave end controller and the slave end robotic arm; The master and slave communicate through the sky-earth link;
地面直接操作步骤：重复执行预测移动步骤，直至从端机械臂的末端与非合作目标之间的距离位于预设范围内；Ground direct operation step: Repeat the predicted movement step until the distance between the end of the slave robot arm and the non-cooperative target is within the preset range;
局部自主操作步骤：在从端机械臂的末端与非合作目标之间的距离位于预设范围内时，利用从端机械臂自主识别非合作目标并抓捕非合作目标。Local autonomous operation steps: when the distance between the end of the slave manipulator and the non-cooperative target is within a preset range, use the slave manipulator to autonomously identify the non-cooperative target and capture the non-cooperative target.
本实施例中，人工神经网络如图3所示，相同层的神经元之间没有连接；人工神经网络的输入x1～xI为主端向从端发送的指令序列和从端的状态序列，从端的状态序列由从端传感器采集后反馈回主端；人工神经网络的输出o1～oK为预测的从端机械臂执行指令后的状态；I和K分别为人工神经网络输入层和输出层的神经元数量；In this embodiment, the artificial neural network is shown in Figure 3, and there is no connection between neurons in the same layer; the inputs x 1 to x I of the artificial neural network are the instruction sequence sent by the master to the slave and the state sequence of the slave, The state sequence of the slave end is collected by the slave end sensor and fed back to the master end; the outputs o 1 ~ o K of the artificial neural network are the predicted states of the slave end manipulator after executing the command; I and K are the input layer and output of the artificial neural network, respectively the number of neurons in the layer;
本实施例中，预测移动步骤还包括：In this embodiment, the predicting movement step further includes:
在主端接收从端反馈的机械臂执行指令后的真实状态；The real state of the robot arm after the master receives the feedback from the slave and executes the command;
以从端机械臂执行主端在某一时刻发送的指令后的真实状态d1～dK为教师信号，以对应时刻人工神经网络的输出o1～oK与教师信号d1～dK之间的平方误差和为损失函数，利用反向传播(Back Propagation，BP)算法更新人工神经网络的网络权值；The real state d 1 ～d K after the slave robot arm executes the instruction sent by the master at a certain moment is used as the teacher signal, and the output o 1 ～ o K of the artificial neural network at the corresponding moment and the teacher signal d 1 ～ d K are used as the difference between The sum of the squared errors between the two is the loss function, and the back propagation (BP) algorithm is used to update the network weights of the artificial neural network;
其中，将输入信号x1～xI输入人工神经网络，经过中间层传输向输出层，得到输出层的神经元输出o1～oK；输出层的神经元输出o1～oK与教师信号d1～dK之间的平方误差和为：Among them, the input signals x 1 ~ x I are input into the artificial neural network, and transmitted to the output layer through the intermediate layer, and the neuron output o 1 ~ o K of the output layer is obtained; the neuron output of the output layer o 1 ~ o K and the teacher signal The sum of squared errors between d 1 and d K is:
式中，ok(k＝1，2，…，K)表示输出层神经元k的输出，netk表示输出层神经元k的输入和，分别为：In the formula, ok ( k =1, 2, ..., K) represents the output of the output layer neuron k, and net k represents the input sum of the output layer neuron k, respectively:
ok＝f(netk) (2)o k = f(net k ) (2)
在BP算法中，应用最小二乘平均原理，先求中间层和输出层间的权值的更新量：In the BP algorithm, the least squares average principle is applied, and the update amount of the weights between the intermediate layer and the output layer is first calculated:
式中，η为正的学习常数，δok为输出层神经元k的δ值；In the formula, η is a positive learning constant, δ ok is the δ value of the output layer neuron k;
当yj为中间层神经元j的输出，netj为中间层神经元j的输入和时，输入层与中间层间的权值更新量有When y j is the output of the intermediate layer neuron j, and net j is the input sum of the intermediate layer neuron j, the weight update amount between the input layer and the intermediate layer has
yj＝f(netj)(中间层神经元j的输出) (5)y j = f(net j ) (output of neuron j in the middle layer) (5)
同样，可求得Likewise, it can be obtained
所有的netk直接依存于yj；η为学习常数；wkj和vji均表示权重，BP算法中的神经元输入输出函数应该满足的条件为单调增函数，最常用的是下面的双弯曲函数：All net k directly depend on y j ; η is a learning constant; w kj and v ji both represent weights, the condition that the neuron input and output functions in the BP algorithm should satisfy is a monotonically increasing function, the most commonly used is the following double curvature function:
本实施例基于从端反馈的真实状态信息，对人工神经网络的网络权值进行更新，能够使人工神经网络能够更好地适应动态变化的工作环境，提高人工神经网络的预测精度。In this embodiment, the network weights of the artificial neural network are updated based on the real state information fed back from the end, so that the artificial neural network can better adapt to the dynamically changing working environment and improve the prediction accuracy of the artificial neural network.
为了进一步提高人工神经网络的预测精度并减小计算量，本实施例中，预测移动步骤还包括：In order to further improve the prediction accuracy of the artificial neural network and reduce the amount of calculation, in this embodiment, the step of predicting the movement further includes:
在主端，利用接收到的教师信号更新从端的状态序列；On the master side, use the received teacher signal to update the state sequence of the slave side;
利用第一滚动窗口和第二滚动窗口在主端向从端发送的指令序列和从端的状态序列上同步滚动，以使得第二滚动窗口内包含最新的状态；Utilize the first scroll window and the second scroll window to scroll synchronously on the instruction sequence sent by the master to the slave and the state sequence of the slave, so that the second scroll window contains the latest state;
利用第一滚动窗口内的指令子序列和第二滚动窗口内的状态子序列作为在最新的状态所对应时刻的下一时刻人工神经网络的输入。The instruction subsequence in the first rolling window and the state subsequence in the second rolling window are used as the input of the artificial neural network at the next moment corresponding to the latest state.
本发明利用滚动窗口实时截取状态序列中最新的子序列及相对应的指令子序列，作为人工神经网络模型的输入，实现了一种滚动预测的方式，滚动预测具体如图4所示，其中的Model_p和Model_t分别表示输入人工神经网络的指令序列和状态序列，First_p和First_t分别表示初始时刻输入的指令序列和状态序列；Forcast_out表示神经网络模型输出的预测结果，Forcast_in为基于预测结果生成的指令；如图4所示，本实施例基于这种滚动预测的方式，每次都会用新数据替换掉输入数据中的最后一段数据，由此能够不断修正模型，实时跟踪参考值，提高预测精度，同时能够基于小样本实现对人工神经网络的反复迭代及快速训练。The present invention uses the rolling window to intercept the latest subsequence and the corresponding instruction subsequence in the state sequence in real time, as the input of the artificial neural network model, and realizes a rolling prediction method. Model_p and Model_t represent the instruction sequence and state sequence input to the artificial neural network respectively, First_p and First_t represent the instruction sequence and state sequence input at the initial moment, respectively; Forcast_out represents the prediction result output by the neural network model, and Forcast_in is the instruction generated based on the prediction result; As shown in Fig. 4, based on this rolling prediction method, this embodiment replaces the last piece of data in the input data with new data each time, so that the model can be continuously revised, the reference value can be tracked in real time, and the prediction accuracy can be improved. It can realize repeated iteration and fast training of artificial neural network based on small samples.
在本实施例中，移动信号由主端控制器在人工神经网路的预测结果的基础上，根据预设的移动策略生成，以指示从端机械臂执行指令后发生相应的移动。In this embodiment, the movement signal is generated by the master controller based on the prediction result of the artificial neural network and according to the preset movement strategy, so as to instruct the slave arm to move accordingly after executing the instruction.
本实施例，包含两个工作模式，即地面直接操作模式和局部自主操作模式，地面直接操作模式由预测移动步骤和地面直接操作步骤完成，局部自主操作模式由局部自主操作步骤完成；本实施例在地面直接操作模式下，通过遥操作的方式使得从端机械臂能够在较大范围内快速接近非合作目标，在局部自主操作模式下，则由从端机械臂自主完成对非合作目标的抓捕；并且，针对天地链路通信存在时延，导致工作环境时变的问题，本实施例在地面直接操作模式下在主端利用人工神经网络对从端机械臂的运动状态进行预测，并基于预测结果向从端发送相应的指令，使从端机械臂按照预期的移动方式接近非合作目标，由此能够适应非合作目标抓捕过程中动态变化的环境和任务；在从端机械臂距非合作目标较近时，非合作目标相对于从端机械臂末端的姿态较为稳定，此时利用从端机械臂自主识别和抓捕非合作目标，能够避免天地链路带来的时延，充分利用从端机械臂的局部自主操作完成对非合作目标的精确抓捕。This embodiment includes two working modes, namely the ground direct operation mode and the local autonomous operation mode, the ground direct operation mode is completed by the predicted movement step and the ground direct operation step, and the local autonomous operation mode is completed by the local autonomous operation step; this embodiment In the ground direct operation mode, the slave manipulator can quickly approach the non-cooperative target in a large range by means of teleoperation. In the local autonomous operation mode, the slave manipulator can autonomously complete the grasp of the non-cooperative target. In addition, in view of the problem of time-varying working environment due to the delay in the communication between the sky and the ground, this embodiment uses an artificial neural network on the master side to predict the motion state of the slave-side robotic arm in the ground direct operation mode. The prediction result sends corresponding instructions to the slave side, so that the slave side manipulator can approach the non-cooperative target according to the expected movement mode, so that it can adapt to the dynamically changing environment and tasks in the process of capturing the non-cooperative target; When the cooperative target is close, the attitude of the non-cooperative target relative to the end of the slave manipulator is relatively stable. At this time, the use of the slave manipulator to autonomously identify and capture the non-cooperative target can avoid the delay caused by the sky-earth link and make full use of the The local autonomous operation of the slave manipulator completes the precise capture of non-cooperative targets.
相应地，本实施例中，从端机械臂末端与非合作目标间距离的预设范围实际上是两种模式之间切换条件，该预设范围可根据实际的空间环境和从端机械臂的自主抓捕能量相应设定，以确保当端机械臂末端与非合作目标间距离位于该预设范围内时，利用从端机械臂的自主抓捕能力即可实现对非合作目标的精确抓捕。Correspondingly, in this embodiment, the preset range of the distance between the end of the slave manipulator and the non-cooperative target is actually a switching condition between the two modes, and the preset range can be based on the actual space environment and the slave manipulator. The autonomous capture energy is set accordingly to ensure that when the distance between the end of the robot arm and the non-cooperative target is within the preset range, the autonomous capture capability of the slave robot arm can be used to accurately capture the non-cooperative target. .
实施例2：Example 2:
一种变时延条件下非合作目标抓捕遥操作方法，本实施例与上述实施例1类似，所不同之处在于，本实施例中，如图2和图5所示，主端还包括带力觉信息的手控器，且移动信号来自于手控器；A remote operation method for capturing a non-cooperative target under the condition of variable delay, this embodiment is similar to the above-mentioned
本实施例中，手控器工作在MS(Master-slave control mode)模式，在该模式下，手控器与从端机械臂之间完全处于一种动态耦合之中，适合完成复杂、精密的控制任务；在MS模式下的工作过程如图6所示；In this embodiment, the hand controller works in the MS (Master-slave control mode) mode. In this mode, the hand controller and the slave robot arm are completely in a dynamic coupling, which is suitable for completing complex and precise Control task; the working process in MS mode is shown in Figure 6;
由于手控器的工作空间远小于从端机械臂的运动空间，为了保证对非合作目标抓捕的准确性，本实施例中，手控器中还增加了比例因子调整功能；Since the working space of the hand controller is much smaller than the motion space of the slave robot arm, in order to ensure the accuracy of capturing the non-cooperative target, in this embodiment, the scale factor adjustment function is also added to the hand controller;
比例因子调整功能用于将手控器的运动及力反馈按照预设的比例因子进行输出，从而生成移动信号；The scale factor adjustment function is used to output the motion and force feedback of the hand controller according to the preset scale factor, thereby generating a moving signal;
本实施例通过在手控器增加比例因子调整功能，将手控器的运动及力反馈按照预设的比例因子进行输出，能够将手控器有限的工作空间内的运动输出放大为从端机械臂在其运动空间中的运动，从而保证了对非合作目标抓捕的准确性。In this embodiment, the scale factor adjustment function is added to the hand controller, and the motion and force feedback of the hand controller are output according to the preset scale factor, so that the motion output in the limited working space of the hand controller can be amplified into a slave machine. The movement of the arm in its motion space ensures the accuracy of capturing non-cooperative targets.
本实施例使用的手控器与从端机械臂虽然具有相同的自由度，但结构却不相同，属于异构式人机交互控制，因此采用点对点的方式将手控器输出映射给从端机械臂末端位姿，映射控制流程如图7所示，点对点映射的方式，依赖于从端机械臂运动学分析，通过D-H参数法可以得到六自由度机器人的运动学逆解。从端机械臂的末端位姿映射通常可采用引导控制和增量控制两种方式实现：引导控制即跟随控制，始终保持机器人基坐标系与手控器坐标系重合，使得机器人末端随手控器运动而运动；增量控制则通过将手控器运动的相对增量映射到机器人当前末端位姿来实现映射控制。Although the hand controller and the slave robot arm used in this embodiment have the same degrees of freedom, their structures are different and belong to heterogeneous human-computer interaction control. Therefore, the output of the hand controller is mapped to the slave robot in a point-to-point manner. The pose of the arm end and the mapping control process are shown in Figure 7. The point-to-point mapping method depends on the kinematics analysis of the slave-end robotic arm. The inverse kinematics solution of the six-degree-of-freedom robot can be obtained by the D-H parameter method. The end pose mapping of the slave manipulator can usually be realized in two ways: guided control and incremental control: guided control, that is, follow control, always keep the robot base coordinate system coincident with the hand controller coordinate system, so that the robot end moves with the hand controller. The movement; incremental control realizes the mapping control by mapping the relative increment of the hand controller movement to the current end pose of the robot.
作为一种优选的实施方式，本实施例中，在预测移动步骤中，根据接收到的移动信号生成下一时刻的指令，具体包括：As a preferred implementation manner, in this embodiment, in the step of predicting movement, an instruction for the next moment is generated according to the received movement signal, which specifically includes:
采用增量控制方式，将手控器输出的移动信号映射为从端机械臂的末端位姿，并生成相应的指令；Using the incremental control method, the movement signal output by the hand controller is mapped to the end pose of the slave robot arm, and the corresponding command is generated;
本实施例采用增量控制方式将手控器输出的移动信号映射为从端机械臂的末端位姿，能够防止从端机械臂在运动的起始阶段，其末端位姿出现突变，由此能够保证从端机械臂的安全。In this embodiment, the incremental control method is used to map the movement signal output by the hand controller to the terminal posture of the slave robot arm, which can prevent the terminal posture of the slave robot arm from changing abruptly at the initial stage of movement. Ensure the safety of the slave robot arm.
由于手控器的工作空间远小于从端机械臂的运动空间，而为了保证控制精度，手控器进行比例因子调整的比例因子的大小受限，在此情况下，从端机械臂的运动也可能超出手控器的工作空间；作为一种优选的实施方式，本实施例中，当映射后从端机械臂的末端位姿的变化量超过映射前手控器的工作空间时，采用映射空间漂移策略完成移动信号到从端机械臂末端位姿的映射；Since the working space of the hand controller is much smaller than the movement space of the slave manipulator, and in order to ensure the control accuracy, the size of the scale factor for the hand controller to adjust the scale factor is limited. In this case, the movement of the slave manipulator is also limited. It may exceed the workspace of the hand controller; as a preferred implementation, in this embodiment, when the change in the posture of the end of the slave manipulator after the mapping exceeds the workspace of the hand controller before the mapping, the mapping space is used. The drift strategy completes the mapping of the movement signal to the end pose of the slave manipulator;
映射空间漂移策略如图8所示，具体按照如下步骤执行：The mapping space drift strategy is shown in Figure 8, and the specific steps are as follows:
(S1)根据预设的比例因子计算手控器的总运动距离，作为目标距离；(S1) calculate the total movement distance of the hand controller according to the preset scale factor, as the target distance;
(S2)使手控器连续运动，直至手控器已运动的总距离达到目标距离，或者手控器运动到极限位置；(S2) make the hand controller move continuously, until the total distance that the hand controller has moved reaches the target distance, or the hand controller moves to the limit position;
(S3)若手控器已运动的总距离达到目标距离，则转入步骤(S4)；若手控器运动到极限位置，则将手控器复位到初始位置，并转入步骤(S2)；(S3) if the total distance that the hand controller has moved reaches the target distance, then go to step (S4); If the hand controller moves to the limit position, then reset the hand controller to the initial position, and go to step (S2);
(S4)映射结束。(S4) Mapping ends.
本实施例在映射后从端机械臂的末端位姿的变化量较大时，采用映射空间漂移策略完成从端机械臂末端位姿的映射，能够在保证控制精度的同时，保证映射空间的有效覆盖。In this embodiment, when the variation of the end pose of the slave manipulator after the mapping is large, the mapping space drift strategy is used to complete the mapping of the slave arm end pose, which can ensure the control accuracy and the effectiveness of the mapping space. cover.
作为一种可选的实施方式，本实施例中，预测移动步骤还包括：As an optional implementation manner, in this embodiment, the predicting movement step further includes:
根据人工神经网络的预测结果，将从端机械臂的运动以虚拟现实的方式在主端进行呈现，由此能够便于实时监测从端机械臂的运动情况。According to the prediction result of the artificial neural network, the motion of the slave manipulator is presented on the master side in virtual reality, which facilitates real-time monitoring of the motion of the slave manipulator.
带力觉信息的手控器是一种重要的地面直接交互控制机械臂设备，同时具有运动输出和力反馈功能：一方面，手控器将操作者手部的运动信息传给远端的机器人控制器，以控制机器人的运动；另一方面，将远端机器人与环境的相互作用力/力矩或“虚拟”力/力矩信息反馈给主端，通过手控器的力反馈实现作用于操作者手部，为操作者提供力觉临场感。本实施例利用手控器输出根据人工神经网络的预测结果生成的移动信号，提高了人机交互性。The hand controller with force sense information is an important ground-based direct interactive control robotic arm device, and has both motion output and force feedback functions: on the one hand, the hand controller transmits the motion information of the operator's hand to the remote robot. The controller is used to control the movement of the robot; on the other hand, the force/torque or "virtual" force/torque information of the interaction between the remote robot and the environment is fed back to the main end, and the force feedback of the hand controller is used to act on the operator. The hand provides the operator with a sense of force presence. In this embodiment, the hand controller is used to output the movement signal generated according to the prediction result of the artificial neural network, which improves the human-computer interaction.
应当说明的是，本实施例中采用手控器基于人工神经网络的预测结果产生移动信号，仅为本发明的优选实施例，不应理解为对本发明的唯一限定；在本发明其他的一些实施例中，也可以使用空间鼠标、眼动仪、体感输入设备等人机交互设备来产生移动信号，在此将不作一一列举。It should be noted that in this embodiment, the hand controller is used to generate the moving signal based on the prediction result of the artificial neural network, which is only a preferred embodiment of the present invention and should not be construed as the only limitation of the present invention; in some other implementations of the present invention In an example, a human-computer interaction device such as a spatial mouse, an eye tracker, and a somatosensory input device can also be used to generate the movement signal, which will not be listed here.
实施例3：Example 3:
一种变时延条件下非合作目标抓捕遥操作系统，包括：通过天地链路通信的主端和从端；主端位于地面，包括主端控制器；从端位于空间，包括从端控制器和从端机械臂；A non-cooperative target capture teleoperation system under the condition of variable delay, comprising: a master terminal and a slave terminal communicating through a sky-earth link; the master terminal is located on the ground, including a master controller; the slave terminal is located in space, including slave terminal control controller and slave robot arm;
主端控制器，用于执行预测移动步骤；移动预测步骤包括：在当前时刻，在主端根据移动信号生成当前时刻的指令并发送至从端；基于主端向从端发送的指令序列和从端的状态序列，利用人工神经网络预测从端机械臂执行主端在当前时刻发送的指令后的状态；指令用于指示从端机械臂各关节的移动角度；移动信号为基于前一时刻人工神经网络的预测结果生成的信号，用于指示从端机械臂末端移动的位置信息；主端位于地面，包括主端控制器；从端位于空间，包括从端控制器和从端机械臂；主端和从端通过天地链路通信；The master-end controller is used to execute the predicted movement step; the movement prediction step includes: at the current moment, the master-end generates an instruction at the current moment according to the movement signal and sends it to the slave-end; based on the instruction sequence sent from the master-end to the slave-end and the slave-end The state sequence of the end, using artificial neural network to predict the state of the slave arm after executing the command sent by the master at the current moment; the command is used to indicate the movement angle of each joint of the slave arm; the movement signal is based on the artificial neural network at the previous moment. The signal generated by the prediction result is used to indicate the position information of the movement of the end of the slave robot arm; the master end is located on the ground, including the master end controller; the slave end is located in space, including the slave end controller and the slave end robot arm; the master end and The slave communicates through the sky-earth link;
主端控制器，还用于执行地面直接操作步骤；地面直接操作步骤包括：重复执行预测移动步骤，直至从端机械臂的末端与非合作目标之间的距离位于预设范围内；The master-end controller is also used to perform the ground direct operation step; the ground direct operation step includes: repeating the predicted movement step until the distance between the end of the slave-end robotic arm and the non-cooperative target is within a preset range;
从端控制器，用于执行局部自主操作步骤；局部自主操作步骤包括：在从端机械臂的末端与非合作目标之间的距离位于预设范围内时，控制从端机械臂自主识别非合作目标并抓捕非合作目标；The slave-end controller is used to execute the local autonomous operation steps; the local autonomous operation steps include: when the distance between the end of the slave-end robotic arm and the non-cooperative target is within a preset range, controlling the slave-end robotic arm to autonomously identify the non-cooperative target target and capture non-cooperative targets;
本实施例中，各控制器的具体实施方式，可参考上述方法实施例中的描述，在此将不作复述。In this embodiment, for the specific implementation of each controller, reference may be made to the descriptions in the foregoing method embodiments, which will not be repeated here.
本领域的技术人员容易理解，以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, etc., All should be included within the protection scope of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010888700.4A CN112060088A (en) | 2020-08-28 | 2020-08-28 | Non-cooperative target accurate capturing teleoperation method under variable time delay condition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010888700.4A CN112060088A (en) | 2020-08-28 | 2020-08-28 | Non-cooperative target accurate capturing teleoperation method under variable time delay condition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112060088A true CN112060088A (en) | 2020-12-11 |
Family
ID=73660186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010888700.4A Pending CN112060088A (en) | 2020-08-28 | 2020-08-28 | Non-cooperative target accurate capturing teleoperation method under variable time delay condition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112060088A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113927596A (en) * | 2021-10-07 | 2022-01-14 | 西北工业大学 | Time-varying output constraint robot teleoperation finite time control method based on width neural learning |
| CN114905478A (en) * | 2021-02-08 | 2022-08-16 | 腾讯科技（深圳）有限公司 | Bilateral teleoperation system and control method |
| CN115229798A (en) * | 2022-08-22 | 2022-10-25 | 桂林电子科技大学 | A force-impedance control method combining feedforward compensation and variable damping modeling |
| CN115446828A (en) * | 2022-08-09 | 2022-12-09 | 北京空间飞行器总体设计部 | Large cabin section autonomous capturing or transposition method based on space manipulator |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101733746A (en) * | 2009-12-22 | 2010-06-16 | 哈尔滨工业大学 | Autonomously identifying and capturing method of non-cooperative target of space robot |
| US20120325972A1 (en) * | 2007-03-09 | 2012-12-27 | Macdonald Dettwiler & Associates Inc. | Robotic satellite refuelling tool |
| CN106864776A (en) * | 2017-01-18 | 2017-06-20 | 哈尔滨工业大学深圳研究生院 | A kind of method and system of the capture target satellite based on butt joint ring |
| CN107717994A (en) * | 2017-11-08 | 2018-02-23 | 西安交通大学 | Principal and subordinate's heterogeneous robot universal control method and system based on principal and subordinate's space reflection |
| CN107877517A (en) * | 2017-11-16 | 2018-04-06 | 哈尔滨工业大学 | Motion mapping method based on CyberForce remote operating mechanical arms |
| CN109131956A (en) * | 2018-10-18 | 2019-01-04 | 哈尔滨工业大学 | A kind of noncooperative target satellite-rocket docking ring capture mechanism and its catching method |
-
2020
- 2020-08-28 CN CN202010888700.4A patent/CN112060088A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120325972A1 (en) * | 2007-03-09 | 2012-12-27 | Macdonald Dettwiler & Associates Inc. | Robotic satellite refuelling tool |
| CN101733746A (en) * | 2009-12-22 | 2010-06-16 | 哈尔滨工业大学 | Autonomously identifying and capturing method of non-cooperative target of space robot |
| CN106864776A (en) * | 2017-01-18 | 2017-06-20 | 哈尔滨工业大学深圳研究生院 | A kind of method and system of the capture target satellite based on butt joint ring |
| CN107717994A (en) * | 2017-11-08 | 2018-02-23 | 西安交通大学 | Principal and subordinate's heterogeneous robot universal control method and system based on principal and subordinate's space reflection |
| CN107877517A (en) * | 2017-11-16 | 2018-04-06 | 哈尔滨工业大学 | Motion mapping method based on CyberForce remote operating mechanical arms |
| CN109131956A (en) * | 2018-10-18 | 2019-01-04 | 哈尔滨工业大学 | A kind of noncooperative target satellite-rocket docking ring capture mechanism and its catching method |
Non-Patent Citations (3)
| Title |
|---|
| 王宇; 宋爱国; 徐效农: "基于任务空间双模式结合的遥操作机器人系统", 《电子测量技术》 * |
| 甘凯: "基于虚拟夹具的空间机器人遥操作与示教技术研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 赵迪: "变时延环境下的任务级遥操作关键技术研究", 《中国博士学位论文全文数据库信息科技辑》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114905478A (en) * | 2021-02-08 | 2022-08-16 | 腾讯科技（深圳）有限公司 | Bilateral teleoperation system and control method |
| CN113927596A (en) * | 2021-10-07 | 2022-01-14 | 西北工业大学 | Time-varying output constraint robot teleoperation finite time control method based on width neural learning |
| CN115446828A (en) * | 2022-08-09 | 2022-12-09 | 北京空间飞行器总体设计部 | Large cabin section autonomous capturing or transposition method based on space manipulator |
| CN115446828B (en) * | 2022-08-09 | 2025-01-28 | 北京空间飞行器总体设计部 | Autonomous capture or transfer method of large cabin based on space manipulator |
| CN115229798A (en) * | 2022-08-22 | 2022-10-25 | 桂林电子科技大学 | A force-impedance control method combining feedforward compensation and variable damping modeling |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112060088A (en) | Non-cooperative target accurate capturing teleoperation method under variable time delay condition | |
| US9862090B2 (en) | Surrogate: a body-dexterous mobile manipulation robot with a tracked base | |
| CN110587600A (en) | Point cloud-based autonomous path planning method for live working robot | |
| JPH06314103A (en) | Controller and passive sensing device | |
| CN111421554B (en) | Robotic arm intelligent control system, method and device based on edge computing | |
| CN108638065A (en) | A kind of explosive-removal robot both arms cooperative control system | |
| CN111230873A (en) | A collaborative handling control system and method based on teaching and learning | |
| CN113829343B (en) | Real-time multitasking and multi-man-machine interaction system based on environment perception | |
| CN105911995B (en) | A kind of teleoperation robot anti-collision warning method based on position and speed control | |
| CN110039561A (en) | Hot line robot remote operating staff training system and method based on cloud | |
| CN116834014A (en) | Intelligent cooperative control method and system for capturing non-cooperative targets by space dobby robot | |
| CN111716352B (en) | Power distribution network live working mechanical arm navigation obstacle avoidance method and system | |
| CN116021516B (en) | Fault-tolerant control method for mechanical arm with multiple degrees of freedom | |
| CN111872938A (en) | Spatial three-dimensional large-scale kinematics simulation system and method | |
| Kennel-Maushart et al. | Multi-arm payload manipulation via mixed reality | |
| CN112959342B (en) | Teleoperation method for grasping operation of flying manipulator based on operator's intention recognition | |
| KR20220150700A (en) | System for controlling robot | |
| Duan et al. | Training of construction robots using imitation learning and environmental rewards | |
| CN107553485A (en) | The generation method of dynamic virtual fixture in a kind of interactive process | |
| Gold et al. | Catching objects with a robot arm using model predictive control | |
| CN115488880A (en) | Simulation test system for mechanical arm visual servo grabbing in space environment | |
| CN115256373A (en) | Mobile phone remote control mechanical arm grabbing method based on neural network | |
| Anderson et al. | Coordinated control and range imaging for mobile manipulation | |
| CN115357851A (en) | A master-slave mixed mapping method for human-computer interaction system and its application | |
| Zhou et al. | An Adaptive Path Planning Approach for Digital Twin-Enabled Robot Arm Based on Inverse Kinematics and Deep Reinforcement Learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201211 |