CN118046394B - A remote motion control method, device, system and storage medium - Google Patents

Abstract

The embodiment of the present application belongs to the field of teleoperation technology, and relates to a teleoperation motion control method, including: obtaining at least the wrist joint posture and elbow joint posture of the master end operating arm; mapping the end posture of the robotic arm based on the wrist joint posture; obtaining multiple sets of alternative solutions for the joint motions of the robotic arm based on the mapped end posture of the robotic arm; matching the elbow joint posture with multiple intermediate joint postures most approximately; and using the alternative solution corresponding to the matched most approximately joint posture as the target solution. The embodiment of the present application also relates to related devices, systems, storage media, etc. The technical solution of the present application can effectively improve the accuracy of teleoperation motion control.

Description

A remote motion control method, device, system and storage medium

Technical Field

The present application relates to the field of remote operation technology, and in particular to a remote operation motion control method, device, system and storage medium.

Background technique

With the development of science and technology, the application of robotic arms is becoming more and more extensive. In some complex and dangerous environments, robotic arms need to be more flexible and have higher human-like working capabilities. For this reason, teleoperated robotic arms have come into being. Teleoperated robotic arms are achieved by installing attitude sensors on the master operator, such as inertial sensors (Inertial Measurement Unit IMU). The operator performs the target task in another real or virtual scene. The attitude sensor captures the operator's movements during the operation and collects the corresponding motion data to send to the controller. The controller generates motion control instructions based on the motion data to control the slave robotic arm to complete the target task, thereby achieving the purpose of teleoperation.

However, in the prior art, there is often a certain error in mapping the motion data of the master operator to the movement of the robot during the teleoperation process, thereby causing a certain error in the robot following the master teleoperation process.

Summary of the invention

The purpose of the embodiments of the present application is to provide a teleoperation motion control method, device, system and storage medium to effectively improve the accuracy of teleoperation motion control.

In a first aspect, the present application provides a teleoperation motion control method, which adopts the following technical solution:

A teleoperation motion control method is applied to a teleoperation system, wherein the teleoperation system comprises a posture sensor, a mechanical arm and a controller arranged at an operator's joint, and the method comprises the following steps:

Obtain at least the wrist joint posture and elbow joint posture of the master end manipulation arm;

Mapping the end position of the robotic arm based on the wrist joint position; obtaining multiple sets of alternative solutions for the motion of each joint of the robotic arm based on the mapped end position of the robotic arm;

The elbow joint posture is most approximately matched with a plurality of intermediate joint postures; the alternative solution corresponding to the matched most approximate joint posture is used as the target solution; wherein each of the intermediate joint postures is the joint posture of the intermediate joint of the corresponding robotic arm in each group of the alternative solutions.

Furthermore, in one embodiment, the step of performing the most approximate matching of the elbow joint posture with a plurality of intermediate joint postures; and taking the candidate solution corresponding to the matched most approximate joint posture as the target solution comprises the following steps:

Determine the elbow joint arm angle based on the elbow joint posture and the wrist joint posture;

Calculate the joint arm angle of the intermediate joint corresponding to each set of alternative solutions respectively;

The elbow joint arm angle is most approximately matched with the joint arm angle, and a group of alternative solutions corresponding to the matched most approximate joint arm angle are used as the target solution.

Furthermore, in one embodiment, obtaining the elbow joint arm angle based on the elbow joint posture and the wrist joint posture comprises the following steps:

Constructing a first vector from the shoulder joint coordinate system to the wrist joint coordinate system; and establishing a first coordinate system based on the first vector;

Obtaining an elbow joint projection of the elbow joint in the first vector;

The elbow joint arm angle is calculated based on the elbow joint projection and the first coordinate system.

Furthermore, in one embodiment, obtaining the joint arm angle of the intermediate joint corresponding to each set of alternative solutions comprises the following steps:

Constructing a second vector from the base coordinate system of the robotic arm to the end coordinate system of the robotic arm; and establishing a second coordinate system based on the second vector;

Obtaining a joint projection of the middle joint of the robotic arm in the second vector;

The joint arm angle is calculated based on the joint projection and the second coordinate system.

Furthermore, in one embodiment, the robotic arm is a 7-axis robotic arm; the step of obtaining multiple sets of alternative solutions for the motion of each joint of the robotic arm based on the mapped end position comprises the following steps:

Acquire the current third joint rotation angle from within a preset rotation angle threshold range of the third joint of the robotic arm;

Based on the robot model, the end position of the robot and the current third joint angle, a set of candidate solutions corresponding to the current third joint angle are obtained in combination with an inverse kinematics equation;

Repeat the above steps until the preset conditions are met to obtain multiple sets of alternative solutions.

Furthermore, in one embodiment, before obtaining at least the wrist joint posture and the elbow joint posture of the master-end operating arm, the method further includes the following steps:

Get the preset arm model parameters;

In combination with the motion data of the arm joints, updating the arm model parameters;

Based on the updated arm model parameters, the elbow joint posture and the wrist joint posture are obtained in combination with the forward kinematics equation.

In a second aspect, an embodiment of the present application provides a remotely operated motion control device, the device comprising:

A posture acquisition module, used to acquire at least the wrist joint posture and elbow joint posture of the master end operating arm;

An alternative solution obtaining module is used to map the end position of the robot arm based on the wrist joint position; and obtain multiple sets of alternative solutions for the motion of each joint of the robot arm based on the mapped end position of the robot arm;

A target determination module is used to most approximately match the elbow joint posture with multiple intermediate joint postures; the alternative solution corresponding to the matched most approximate joint posture is used as the target solution; wherein each intermediate joint posture is the joint posture of the intermediate joint of the corresponding robotic arm in each group of the alternative solutions.

In a third aspect, an embodiment of the present application provides a teleoperation system, the system comprising: a posture sensor, a robotic arm, and a controller;

The controller is communicatively connected with the posture sensor and the robotic arm respectively;

The controller is used to implement the steps of the teleoperation motion control method described above.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the remote operation motion control method described above are implemented.

In a fifth aspect, an embodiment of the present application provides a controller, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the remote-operated motion control method described above when executing the computer program.

Compared with the prior art, the embodiments of the present application have the following beneficial effects:

The embodiment of the present application maps the wrist joint posture of the operator at the main end to the end posture of the robot arm; obtains multiple sets of alternative solutions for the joint movements of the robot arm based on the mapped end posture; and matches the elbow joint posture with multiple joint postures most approximately, so as to use a set of alternative solutions corresponding to the most approximate joint posture obtained as the target solution. While considering the mapping between the operator's wrist joint and the end of the robot arm, the operator's elbow joint posture and the middle joint of the robot arm are also combined for similarity mapping, so that the robot arm as a whole can more perfectly track the multi-joint movements of the human arm, etc., thereby improving the accuracy of remote operation motion control.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG. 1 is a diagram of an exemplary system architecture to which the present application may be applied.

FIG. 2 is a schematic diagram of an embodiment of the ArmElbow coordinate system constructed in an embodiment of the present application.

FIG. 3 is a schematic diagram of an embodiment of the present application for forming an elbow joint arm angle based on the coordinate system of FIG. 2 .

FIG. 4 is a flow chart of an embodiment of a teleoperation motion control method of the present application.

FIG. 5 is a schematic structural diagram of an embodiment of a teleoperated motion control device of the present application.

FIG. 6 is a schematic diagram of the structure of a computer device according to an embodiment of the present application.

Detailed ways

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

As shown in FIG. 1 , FIG. 1 is an exemplary system architecture diagram to which the present application can be applied.

The embodiment of the present application provides a teleoperation system 100 , which includes: a posture sensor 110 , a robotic arm 120 , and a controller 130 .

The controller 130 is connected to the posture sensor 110 and the robot arm 120 in a wired or wireless communication manner.

It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G/4G/5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

Gesture sensor 110

The posture sensor 110 is used to collect the motion data of the arm joints of the operator at the master end, and subsequently generate the posture information of the corresponding joints of the operator's arm based on the motion data. Specifically, the motion data may include but is not limited to: speed and acceleration; or image data that can reflect the motion state of the operator's part to be measured, etc.

The posture sensor described in the embodiment of the present application can be various posture sensors that are currently available or will be developed in the future and can achieve similar functions, such as: IMU, image sensor. For ease of understanding, the embodiment of the present application is mainly described in detail by taking the posture sensor as IMU110 as an example.

Among them, IMU is an inertial measurement unit. Usually, IMU includes: gyroscope and accelerometer. In addition, in one embodiment, the attitude sensor may also include other devices as needed, such as: magnetometer, etc. The gyroscope is used to output the three-axis angular velocity of the target object (such as: the joint fixed by the attitude sensor); the accelerometer is used to collect the three-axis acceleration of the target object.

Specifically, IMU110 can be directly fixed to the operator's joints, or fixed to the operator's joints through a wearable device. For example, the posture sensor is pre-fixed to an arm exoskeleton, and the arm exoskeleton is worn on the operator's arm, thereby fixing the posture sensor to the operator's arm.

For example, the IMU can be fixed to the shoulder joint, elbow joint and wrist joint of the operator's arm respectively, so as to measure the motion data of the shoulder joint, elbow joint and wrist joint of the arm respectively, and then obtain the position and posture of the elbow joint and wrist joint.

Robotic Arm 120

The robot arm 120 is used to imitate the operator to complete the target task based on the motion instructions sent by the controller.

Specifically, the robot arm described in the embodiment of the present application may include multiple axes (i.e., movable joints). For ease of understanding, the embodiment of the present application is mainly described in detail by taking the robot arm as a 7-axis robot arm as an example, and the 7-axis robot arm may include 7 movable joints.

It should be noted that the robotic arm described in the embodiment of the present application can be a separately set robotic arm, or it can be a partial component of a system such as a humanoid robot (for example: using the robotic arm as the arm of a humanoid robot), and this application does not limit this.

Controller 130

The controller 130 is used to execute the steps of the remote operation motion control method and the like described in the embodiments of the present application.

The teleoperation motion control method provided in the embodiment of the present invention can be applied to a computer terminal (Personal Computer, PC); an industrial control computer terminal (Industrial Personal Computer, IPC); a mobile terminal; a server; a system including a terminal and a server, and implemented through the interaction between the terminal and the server; a programmable logic controller (Programmable Logic Controller, PLC); a field programmable gate array (Field-Programmable Gate Array, FPGA); a digital signal processor (Digital Signal Processer, DSP) or a microcontroller unit (Microcontroller unit, MCU) and the like to be executed. The controller generates program instructions according to a pre-fixed program, combined with data signals collected by an external posture sensor 110 and/or a robotic arm 120, etc. For the specific definition of the controller, please refer to the definition of the teleoperation motion control method in the following embodiment. Specifically, the controller can be a computer device as shown in Figure 6.

In one embodiment, the remote operation system 100 described in the embodiment of the present application may also include an observation sensor and/or a demonstrator (not shown in the figure), etc.

The observation sensor is used to collect observation data of the robot arm 120. The observation data can reflect the working status of the robot arm, such as the environment around the robot arm or the completion status of the target task of the robot arm.

The demonstrator is used to demonstrate the observation data or the observation data that has been processed to be demonstrable to the operator.

Based on the teleoperation system described in the above embodiments, the embodiments of the present application provide a teleoperation motion control method, which is generally executed by the controller 130 . Accordingly, the teleoperation motion control device described in the following embodiments is generally disposed in the controller 130 .

As shown in FIG. 4 , FIG. 4 is a flow chart of an embodiment of a teleoperation motion control method of the present application; the teleoperation motion control method may include the following method steps:

Step 210 obtains at least the wrist joint posture and elbow joint posture of the master-end operating arm.

Step 220 maps the wrist joint posture to the end posture of the robotic arm; and obtains multiple sets of candidate solutions for the motion of each joint of the robotic arm based on the mapped end posture.

Step 230 performs the most approximate matching of the elbow joint posture with multiple intermediate joint postures; the alternative solution corresponding to the matched most approximate joint posture is used as the target solution; wherein each intermediate joint posture is the joint posture of the intermediate joint of the corresponding robotic arm in each group of alternative solutions.

The embodiment of the present application maps the wrist joint posture of the operator at the main end to the end posture of the robot arm; obtains multiple sets of alternative solutions for the joint movements of the robot arm based on the mapped end posture; and matches the elbow joint posture with multiple joint postures most approximately, so as to use a set of alternative solutions corresponding to the most approximate joint posture obtained as the target solution. While considering the mapping between the operator's wrist joint and the end of the robot arm, the operator's elbow joint posture and the middle joint of the robot arm are also combined for similarity mapping, so that the robot arm as a whole can more perfectly track the multi-joint movements of the human arm, etc., thereby improving the accuracy of remote operation motion control.

For ease of understanding, the above method steps are further described in detail below.

Step 210 obtains at least the wrist joint posture and elbow joint posture of the master-end operating arm.

In one embodiment, the controller obtains at least the wrist joint posture and elbow joint posture of the arm obtained based on the motion data of the arm joint measured and output by the posture sensor from the memory or the server according to the preset address.

It should be noted that in addition to the wrist joint posture and elbow joint posture, the root of the arm (that is, the shoulder joint posture) can also be obtained according to actual needs, which all fall within the scope of protection of this application.

It should be noted that, before step 210, the embodiment of the present application can obtain at least the wrist joint posture and elbow joint posture of the arm based on various methods currently available or to be developed in the future.

In one embodiment, before step 210, the teleoperation motion control method of the present application may further include the following method steps to obtain the wrist joint posture and elbow joint posture of the arm based on the motion data of the arm measured and output by the posture sensor.

Step 240: Obtain preset arm model parameters.

For example, the human arm can be abstracted in advance as needed into a 7-DOF SRS (ball joint-revolute joint-ball joint) robotic arm structure, in which the shoulder joint has 3 degrees of freedom, the elbow joint has 1 degree of freedom, and the wrist joint has 3 degrees of freedom, for a total of seven degrees of freedom.

Based on the 7 degrees of freedom, an MDH model of a 7-axis robot arm can be pre-built and used as an arm model. For details, see Table 1, which is an example of MDH parameters.

Table 1

Step 250: Update the arm model parameters based on the motion data of the arm joints.

In one embodiment, the controller obtains the motion data of the arm joint measured by the posture sensor from the memory or server according to the preset address; combined with the preset calibration result of the posture sensor, the motion data of the 7 degrees of freedom of the arm can be obtained, that is, - The rotation angle of each degree of freedom is compared with the preset MDH model parameters - By adding the original angle of - Angle.

In the embodiment of the present application, the motion data measured and output by the attitude sensor can directly obtain or further obtain the angles of the seven degrees of freedom of the arm of the master remote operator. - In addition, the lengths d3 and d5 of the upper and lower arms of a person's arm can be obtained in advance by measurement.

Step 260 obtains the elbow joint posture and the wrist joint posture based on the updated arm model parameters and the forward kinematics equation.

In one embodiment, the controller can obtain the positions and postures of the elbow joint coordinate system and the wrist joint coordinate system in the shoulder joint coordinate system based on the updated MDH model and the forward kinematics equation.

Further, in one embodiment, according to the forward kinematics:

, among which, take The first three rows of the 4th column of (elbow joint position) are The position of the elbow joint coordinate system in the shoulder joint coordinate system can be obtained (It can be referred to as the elbow joint position for short.) Since the elbow joint has only one degree of freedom, the position of the elbow joint is the elbow joint posture described in the embodiment of the present application.

Further, in one embodiment, based on forward kinematics, we obtain:

Therefore, based on The wrist joint position can be obtained.

In an embodiment of the present application, the controller obtains motion data sent by the posture sensor, and based on the preset calibration results of the posture sensor and combined with a preset arm model, the wrist joint posture and elbow joint posture of the arm can be obtained.

Step 220 maps the end position of the robotic arm based on the wrist joint position; and obtains multiple sets of candidate solutions for the motion of each joint of the robotic arm based on the mapped end position of the robotic arm.

In one embodiment, the controller can convert the wrist joint posture to the end posture of the robot arm in the robot arm coordinate system; in addition, based on the end posture of the robot arm (for example, the posture of the center of the output flange of the end joint of the robot arm) and a preset robot arm model (for example, the D-H model), the controller can obtain multiple sets of rotation angle solutions (i.e., multiple sets of alternative solutions) for each joint of the robot arm through inverse kinematics equations.

Taking a 7-axis robot as an example, through the above method steps, multiple sets of alternative solutions for the 1, 2, 4, 5, 6, and 7 rotation angles of the 7-axis robot can be obtained.

In one embodiment, the 7-axis robot has 7 degrees of freedom, and the end position of the robot only provides 6 constraints. In this way, after the angle of the third joint of the robot is given (because compared with the usual six-axis robot, the seven-axis robot often has a redundant third joint, that is, the intermediate joint described in the embodiment of the present application), the robot can be solved based on the inverse kinematics equation. Then, in step 220, multiple sets of alternative solutions for the motion of each joint of the robot based on the mapped end position can include the following method steps:

Step 221 obtains the current third joint rotation angle from within a preset rotation angle threshold range of the third joint.

In one embodiment, a sample point angle may be selected in sequence or randomly at preset intervals from within the preset rotation angle limit range of the third joint as the current third joint rotation angle.

Step 222 obtains a set of alternative solutions corresponding to the current third joint angle based on the robot model, the end posture of the robot and the current third joint angle in combination with the inverse kinematics equation.

Specifically, based on the robot model, the end posture of the robot and the given current third joint angle, the angles of joints 1, 2, 4, 5, 6, and 7 of the 7-axis robot can be obtained in combination with the inverse kinematics equation.

For example, during the solution process, it can be found that there are two sets of solutions for a certain joint, for example, there are two solutions for the second joint q2, recorded as q2_1 and q2_2, and the solution closest to the previous moment is selected. Similarly, the joint angles of other joints can be obtained to obtain a set of alternative solutions corresponding to the current third joint angle.

Step 223 repeats the above steps until the preset conditions are met to obtain multiple groups of candidate solutions.

It should be noted that the above-mentioned preset conditions can be set arbitrarily as needed, such as: presetting the number of repetitions.

In the embodiment of the present application, the current third joint rotation angle selected each time is different, and a set of candidate solutions corresponding to the current third joint rotation angle can be obtained accordingly. Therefore, by repeating the above method steps, multiple sets of candidate solutions can be obtained.

Through the above method steps, the embodiment of the present application can enable the robot to autonomously add constraints for the third joint when the teleoperated wrist joint only provides six constraints, thereby obtaining multiple groups of alternative solutions corresponding to each constraint.

It should be noted that, in addition to the methods described in the above embodiments, the embodiments of the present application can adopt various existing or future developed methods based on the number of joints actually included in the robotic arm to achieve multiple sets of alternative solutions for the motion of each joint of the robotic arm based on the mapped end posture of the robotic arm.

Step 230 performs the most approximate matching of the elbow joint posture with multiple intermediate joint postures; the alternative solution corresponding to the matched most approximate joint posture is used as the target solution; wherein each intermediate joint posture is the joint posture of the intermediate joint of the corresponding robotic arm in each group of the alternative solutions.

It should be noted that, for robotic arms with different numbers of axes, the intermediate joints can be changed accordingly as needed. Taking some 7-axis robotic arms as an example, the intermediate joints can usually be set to the third axis, and the position of the output end of the third axis (for example: the center of the output end flange) can be used as the intermediate joint position of the robotic arm.

In one embodiment, step 230 may include the following method steps:

Step 231 obtains the elbow joint arm angle based on the elbow joint posture and the wrist joint posture.

Step 232 obtains the joint arm angles of the intermediate joints corresponding to each set of alternative solutions.

Step 233 performs the most approximate matching of the elbow arm angle and the joint arm angle, and uses a set of candidate solutions corresponding to the matched most approximate joint arm angle as the target solution.

The embodiment of the present application compares the elbow joint arm angle with the intermediate joint arm angle corresponding to each alternative solution of the intermediate joint of the robotic arm, so as to find the joint arm angle that is most similar to the elbow joint arm angle, and uses a set of solutions corresponding to the joint arm angle as the target solution, so that the robotic arm can more accurately complete the remote operation tracking of multiple joint movements of the human arm.

In one embodiment, the above step 231 may include the following method steps:

Step 2311 constructs a first vector from the shoulder joint coordinate system (eg, shoulder joint center) to the wrist joint coordinate system (eg, wrist joint center); and establishes a first coordinate system based on the first vector.

Step 2312 obtains the elbow joint projection of the elbow joint center in the first vector.

Step 2313 calculates the elbow joint arm angle based on the elbow joint projection and the first coordinate system.

For ease of understanding, the following examples are given for illustration.

FIG2 is a schematic diagram of an embodiment of the ArmElbow coordinate system constructed in an embodiment of the present application; FIG3 is a schematic diagram of an embodiment of the present application forming an arm angle of an elbow joint based on the coordinate system of FIG2 .

As shown in FIG. 2 , a first vector SW can be obtained from the shoulder joint center S to the wrist joint center W, and the vertical upward unit vector is defined as the V=[0,0,1] axis, and the first coordinate system FArmElbow is established with the SW and V axes.

The XYZ axis can be set up in the following way:

The Z axis is the direction of the first vector SW and is normalized, the Y axis is the direction determined by the cross product of V and SW and is normalized, and the X axis is the direction determined by the cross product of the Y axis and the Z axis and is normalized.

Then the position of pArmElbow in Figure 2 is The position of the projection on SW. is the position pElbow (3*1 vector) of the elbow joint coordinate system in the shoulder joint coordinate system.

The XYZ axis can be organized into a rotation matrix by column, and then A homogeneous matrix can be formed, namely the coordinate system FArmElbow.

As shown in Figure 3, Projecting to the first coordinate system FArmElbow, we get , its Z component must be 0. Use the Y and X components to obtain the arm angle of the elbow joint. The similarity can be defined by this arm angle later.

The embodiment of the present application performs remote-operated motion control through the above method, and the movement of each joint of the robotic arm at the slave end is almost consistent with the human arm at the master end, which is in line with human intuition, making it easier to control the robotic arm.

In one embodiment, step 232 of respectively obtaining the joint arm angles of the intermediate joints corresponding to each group of alternative solutions may include the following method steps:

Step 3311 constructs a second vector from the base coordinate system of the robot arm to the end coordinate system of the robot arm; and establishes a second coordinate system based on the second vector.

Step 3312 obtains the joint projection of the middle joint center in the second vector.

Exemplarily, for a 7-axis robot arm, the middle joint is the third joint of the robot arm, and the center of the flange at the output end of the third joint can be taken as the center of the middle joint.

Step 3313 calculates the joint arm angle based on the joint projection and the second coordinate system.

The above method steps can refer to the method for obtaining the arm angle of the elbow joint, which will not be repeated here.

The embodiment of the present application performs remote-operated motion control through the above method, and the movement of each joint of the robotic arm at the slave end is almost consistent with the human arm at the master end, which is in line with human intuition, making it easier to control the robotic arm.

The embodiment of the present application maps the wrist joint posture of the operator at the master end with the end posture of the robotic arm at the slave end; obtains multiple sets of alternative solutions for the joint movements of the robotic arm based on the mapped end posture; and matches the elbow joint posture with multiple joint postures most approximately, so as to use a set of alternative solutions corresponding to the most approximate joint posture obtained by matching as the target solution. While considering the mapping between the operator's wrist joint and the end of the robotic arm, the operator's elbow joint posture and the middle joint of the robotic arm are also combined for similarity mapping, so that the robotic arm as a whole can more perfectly track the multi-joint movements of the human arm, thereby improving the accuracy of remote operation motion control.

Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

Further referring to FIG. 5 , as an implementation of the method shown in FIG. 4 , the present application provides an embodiment of a remotely operated motion control device, and the device embodiment corresponds to the method embodiment shown in FIG. 4 .

As shown in FIG5 , the teleoperated motion control device 300 described in this embodiment includes:

A posture acquisition module 310, used to acquire at least the wrist joint posture and the elbow joint posture of the master end operating arm;

An alternative solution obtaining module 320 is used to map the end position of the robot arm based on the wrist joint position; and obtain multiple sets of alternative solutions for the motion of each joint of the robot arm based on the mapped end position of the robot arm;

The target determination module 330 is used to most approximately match the elbow joint posture with multiple intermediate joint postures; the alternative solution corresponding to the matched most approximate joint posture is used as the target solution; wherein each intermediate joint posture is the joint posture of the intermediate joint of the corresponding robotic arm in each group of the alternative solutions.

In one embodiment, the target determination module 330 may include:

The elbow joint obtaining submodule is used to obtain the elbow joint arm angle based on the elbow joint posture and wrist joint posture;

The joint obtaining submodule is used to obtain the joint arm angles of the intermediate joints corresponding to each group of alternative solutions;

The target selection submodule is used to most approximately match the elbow arm angle with the joint arm angle, and use a set of alternative solutions corresponding to the matched most approximate joint arm angle as the target solution.

Furthermore, in one embodiment, the elbow joint obtaining submodule may include:

A first construction unit, configured to construct a first vector from the shoulder joint coordinate system to the wrist joint coordinate system; and to establish a first coordinate system based on the first vector;

A first acquisition unit, used for acquiring an elbow joint projection of the elbow joint in the first vector;

The first obtaining unit is used to obtain the elbow joint arm angle based on the elbow joint projection and the first coordinate system.

Furthermore, in one embodiment, the joint obtaining submodule may include:

A second construction unit, configured to construct a second vector from a base coordinate system of the robot arm to a tip coordinate system of the robot arm; and to establish a second coordinate system based on the second vector;

A second acquisition unit, used for acquiring a joint projection of the middle joint center in the second vector;

The second obtaining unit is used to obtain the joint arm angle based on the joint projection and the second coordinate system.

In one embodiment, the robotic arm is a 7-axis robotic arm; the alternative obtaining module 320 may include:

The rotation angle acquisition submodule is used to acquire the current rotation angle of the third joint of the robot arm from a preset rotation angle threshold range;

An alternative solution obtaining submodule, used for obtaining a set of alternative solutions corresponding to the current third joint angle based on the robot model, the end posture of the robot and the current third joint angle in combination with the inverse kinematics equation;

The step repetition submodule is used to repeat the above steps until the preset conditions are met to obtain multiple sets of alternative solutions.

In one embodiment, the teleoperated motion control device 300 described in this embodiment may further include:

A model acquisition module, used to obtain preset arm model parameters;

The parameter updating module is used to update the arm model parameters based on the motion data of the arm joints;

The posture calculation module is used to calculate the elbow joint posture and wrist joint posture based on the updated arm model parameters and the forward kinematics equation.

In order to solve the above technical problems, an embodiment of the present application further provides a controller, such as a computer device as shown in FIG6 .

The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 6 with components 61-63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 6. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is usually used to store the operating system and various application software installed on the computer device 6, such as the program code of the motion control method of teleoperation, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run the program code stored in the memory 61 or process data, such as running the program code of the teleoperation motion control method.

The network interface 63 may include a wireless network interface or a wired network interface. The network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores a remote-operated motion control program, and the remote-operated motion control program can be executed by at least one processor so that the at least one processor performs the steps of the remote-operated motion control method as described above.

Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims (8)

1. A teleoperation motion control method, applied to a teleoperation system, wherein the teleoperation system comprises a posture sensor, a mechanical arm and a controller arranged at an operator's joint, wherein the method comprises the following steps: Obtain at least the wrist joint posture and elbow joint posture of the master end manipulation arm; Mapping the end position of the robotic arm based on the wrist joint position; obtaining multiple sets of alternative solutions for the motion of each joint of the robotic arm based on the mapped end position of the robotic arm; Determine the elbow joint arm angle based on the elbow joint posture and the wrist joint posture; Calculate the joint arm angle of the intermediate joint corresponding to each set of alternative solutions respectively; The elbow joint arm angle is matched most approximately with the joint arm angle, and a group of alternative solutions corresponding to the matched most approximate joint arm angle is used as the target solution; wherein each of the intermediate joint postures is the joint posture of the intermediate joint of the corresponding robot arm in each group of the alternative solutions; Wherein, obtaining the elbow joint arm angle based on the elbow joint posture and the wrist joint posture comprises the following steps: Construct a first vector from the shoulder joint coordinate system to the wrist joint coordinate system; and establish a first coordinate system with the first vector and the V axis; wherein a vertically upward unit vector is defined as the V axis; the Z axis is the direction of the first vector and is normalized; the Y axis is the direction determined by the cross product of the V axis and the first vector and is normalized; the X axis is the direction determined by the cross product of the Y axis and the Z axis and is normalized; Obtain The position of the elbow joint projection in the first vector; wherein the is the position of the elbow joint coordinate system in the shoulder joint coordinate system; The elbow joint arm angle is calculated based on the Y and X components of the position of the elbow joint projection. 2. The teleoperation motion control method according to claim 1, wherein the step of obtaining the joint arm angle of the intermediate joint corresponding to each set of alternative solutions comprises the following steps: Constructing a second vector from the base coordinate system of the robotic arm to the end coordinate system of the robotic arm; and establishing a second coordinate system based on the second vector; Obtaining a joint projection of the middle joint of the robotic arm in the second vector; The joint arm angle is calculated based on the joint projection and the second coordinate system. 3. The teleoperation motion control method according to claim 1 or 2, characterized in that the robotic arm is a 7-axis robotic arm; the step of obtaining multiple sets of alternative solutions for the motion of each joint of the robotic arm based on the mapped end position comprises the following steps: Acquire the current third joint rotation angle from within a preset rotation angle threshold range of the third joint of the robotic arm; Based on the robot model, the end position of the robot and the current third joint angle, a set of candidate solutions corresponding to the current third joint angle are obtained in combination with an inverse kinematics equation; Repeat the above steps until the preset conditions are met to obtain multiple sets of alternative solutions. 4. The teleoperation motion control method according to claim 1 or 2, characterized in that before obtaining at least the wrist joint posture and elbow joint posture of the master end operating arm, the method further comprises the following steps: Get the preset arm model parameters; In combination with the motion data of the arm joints, updating the arm model parameters; Based on the updated arm model parameters, the elbow joint posture and the wrist joint posture are obtained in combination with the forward kinematics equation. 5. A teleoperated motion control device, characterized in that the device comprises: A posture acquisition module, used to acquire at least the wrist joint posture and elbow joint posture of the master end operating arm; An alternative solution obtaining module is used to map the end position of the robot arm based on the wrist joint position; and obtain multiple sets of alternative solutions for the motion of each joint of the robot arm based on the mapped end position of the robot arm; An elbow joint obtaining submodule, used for obtaining an elbow joint arm angle based on the elbow joint posture and the wrist joint posture; The joint obtaining submodule is used to obtain the joint arm angles of the intermediate joints corresponding to each group of alternative solutions; A target selection submodule is used to perform the most approximate matching between the elbow joint arm angle and the joint arm angle, and to use a group of alternative solutions corresponding to the matched most approximate joint arm angle as the target solution; wherein each of the intermediate joint postures is a joint posture of the intermediate joint of the corresponding robot arm in each group of the alternative solutions; Wherein, the elbow joint obtaining submodule includes: A first construction unit is used to construct a first vector from the shoulder joint coordinate system to the wrist joint coordinate system; and to establish a first coordinate system using the first vector and the V axis; wherein a vertically upward unit vector is defined as the V axis; the Z axis is the direction of the first vector and is unitized; the Y axis is the direction determined by the cross product of the V axis and the first vector and is unitized; and the X axis is the direction determined by the cross product of the Y axis and the Z axis and is unitized; The first acquisition unit is used to acquire The position of the elbow joint projection in the first vector; wherein the is the position of the elbow joint coordinate system in the shoulder joint coordinate system; The first obtaining unit is used to obtain the elbow joint arm angle based on the Y and X components of the position of the elbow joint projection. 6. A teleoperation system, characterized in that the system comprises: a posture sensor, a mechanical arm and a controller; The controller is communicatively connected with the posture sensor and the robotic arm respectively; The controller is used to implement the steps of the teleoperation motion control method according to any one of claims 1 to 4. 7. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the remote operation motion control method according to any one of claims 1 to 4 are implemented. 8. A controller comprising a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of the teleoperation motion control method according to any one of claims 1 to 4 when executing the computer program.