WO2024037658A1

WO2024037658A1 - Method and apparatus for controlling pointing action of robot, and electronic device and storage medium

Info

Publication number: WO2024037658A1
Application number: PCT/CN2023/118887
Authority: WO
Inventors: 黄秋兰; 谢安桓; 朱世强; 顾建军; 王鑫; 梁定坤; 留云
Original assignee: 之江实验室
Priority date: 2023-01-03
Filing date: 2023-09-14
Publication date: 2024-02-22
Also published as: CN115922728B; CN115922728A

Abstract

A method and apparatus for controlling a pointing action of a robot, and an electronic device and a storage medium. The method for controlling a pointing action of a robot comprises: when a target object exceeds a reachable range of a robotic arm end of a robot, selecting, according to target object coordinates of the target object in a robot coordinate system, a robotic arm, which is closer to the target object, from double robotic arms of the robot (110); according to the mapping of the target object coordinates, shoulder joint coordinates of a shoulder joint of the selected robotic arm in the robot coordinate system, and a predetermined rule of a pointing action, obtaining a pose of a robotic arm end of the selected robotic arm pointing to the target object within the reachable range (120); according to the obtained pose of the robotic arm end, determining a trajectory of the selected robotic arm along the current pose of each joint of the selected robotic arm to a target pose of each joint (130); and according to the trajectory, controlling the selected robotic arm to point to the target object (140).

Description

Robot pointing action control method, device, electronic equipment and storage medium

Technical field

The present application relates to the technical field of robot manipulator anthropomorphic movements, and in particular to a robot pointing movement control method, device, electronic equipment and storage medium.

Background technique

With the development of science and technology, the application of robots has entered all aspects of people's lives, such as many shopping malls, restaurants, exhibition halls, etc. have introduced service robots. However, in the current robot-human interaction technology, voice and visual technology are relatively mature, and physical The lack of movement control makes the robot dull when interacting with people. If body language is added to this basis, the robot will be more human-like when interacting with people.

In the related technology, when the robot's mechanical arm performs an action, when the target object is outside the reachable range of the robot arm, the robot will not perform any processing on the target object. In this way, the application scenarios of robotic arms are relatively limited.

Contents of the invention

This application provides a robot pointing action control method, device, electronic equipment and storage medium to realize the application of pointing to target object scenes and improve the applicability of the robot.

This application provides a robot pointing action control method, including:

When the target object exceeds the reachable range of the end of the robot's manipulator arm, according to the target coordinates of the target object in the robot coordinate system, select the one of the two manipulator arms of the robot that is closer to the target object. robotic arm;

According to the mapping of the target coordinates, the shoulder joint coordinates of the selected robot arm's shoulder joint in the robot coordinate system and the predetermined rules of the pointing action, it is obtained that the end of the selected robot arm points to the target object within the reachable range. posture;

Based on the obtained posture of the end of the robotic arm, determine the trajectory of the selected robotic arm from the current posture of each joint of the selected robotic arm to the target posture of each joint;

According to the trajectory, the selected robotic arm is controlled to point to the target object.

Further, based on the mapping of the target coordinates, the shoulder joint coordinates of the selected robotic arm's shoulder joint in the robot coordinate system and the predetermined rules of the pointing action, it is obtained that the robotic arm end of the selected robotic arm is within the reachable range The pose pointing to the target object includes:

Map the target coordinates to coordinates within the reachable range of the end of the robotic arm of the selected robotic arm;

Determine the designated direction of the end of the robotic arm of the selected robotic arm according to the predetermined rules of the pointing action;

Based on the mapped coordinates, the shoulder joint coordinates of the selected robotic arm's shoulder joint in the robot coordinate system, and the specified direction, the posture of the robotic arm end of the selected robotic arm is generated.

Further, mapping the target coordinates to coordinates within the reachable range of the end of the selected robotic arm includes: based on the robot coordinate system, the shoulder joint of the selected robotic arm points to the target object. The angle between the vector of and the The relationship between the Z-direction coordinates, the coordinates of the shoulder joint of the selected robotic arm and the arm length is used to obtain the Z-coordinate of the robotic arm end of the selected robotic arm;

and / or,

Generating the posture of the end of the robotic arm of the selected robotic arm based on the mapped coordinates, the shoulder joint coordinates of the selected robotic arm's shoulder joint in the robot coordinate system, and the specified direction, includes:

The unit vector of the vector of the target object pointing to the end of the robotic arm is determined as the three Z-direction components of the attitude matrix of the end of the selected robotic arm that are used to form the Z-direction vector. The end of the selected robotic arm's robotic arm posture The state matrix is a matrix with 3 rows and 3 columns;

When the end of the selected robot arm points to the target object, and the three Z-direction components of the attitude matrix are positive, and the selected robot arm is a right robot arm, determine the composition of the attitude matrix. The three Y-direction components of the Y-direction vector are positive;

When the end of the selected robot arm points to the target object, and the three Z-direction components of the attitude matrix are positive, and the selected robot arm is the left robot arm, determine the composition of the attitude matrix. The three Y-direction components of the Y-direction vector are negative;

According to the right-hand rule, the three X-direction components of the attitude matrix used to form the X-direction vector are determined based on the three Z-direction components and the three Y-direction components of the attitude matrix of the end of the selected robot arm.

Further, after controlling the selected robotic arm to point to the target object based on the trajectory, the method further includes:

Use a measuring device to test the obtained pose of the robotic arm end, and determine the error in the pointing action of the selected robotic arm's robotic arm end towards the target object.

Further, the use of measuring equipment to test the obtained pose of the end of the robotic arm and determine the error of the pointing action of the end of the selected robotic arm toward the target object includes:

Determine a vector pointing from the target object to the end of the robotic arm of the selected robotic arm;

The angle between the vector and the Z-direction vector of the attitude matrix of the robot end of the selected robot arm is determined as the error.

Further, based on the obtained posture of the end of the robotic arm, determining the trajectory of the selected robotic arm from the current posture of each joint of the selected robotic arm to the target posture of each joint includes:

Based on the obtained posture of the end of the robotic arm, solve the inverse kinematics and obtain the target posture of each joint of the robotic arm;

Through trajectory interpolation, the trajectory of the robotic arm end of the selected robotic arm from the current posture to the target posture of each joint is obtained.

Further, based on the obtained posture of the end of the robotic arm, the inverse kinematics is solved to obtain the target posture of each joint of the robotic arm, including:

Based on the obtained posture of the end of the robotic arm, solve the inverse kinematics to obtain the target configuration of the selected robotic arm, where the target configuration includes the target angle of each joint;

According to the current configuration of the robot and the target configuration, trajectory interpolation is performed to control the movement of the robot arm from the current configuration of the robot to the target configuration of the robot pointing to the target object. The current configuration of the robot is The configuration includes the current angles of each of the joints.

This application provides a robot pointing action control device, including:

The robot arm selection module is used to select the robot arm of the robot according to the target coordinates of the target object in the robot coordinate system when the target object exceeds the reachable range of the robot's robot arm end. The robot arm is closer to the target object;

The pose determination module at the end of the robotic arm is used to obtain the robotic arm of the selected robotic arm based on the mapping of the target coordinates, the shoulder joint coordinates of the selected robotic arm's shoulder joint in the robot coordinate system, and the predetermined rules of the pointing action. The end point points to the pose of the target object within the reachable range;

A trajectory determination module, configured to determine the trajectory of the selected robotic arm from the current posture of each joint of the selected robotic arm to the target posture of each joint based on the obtained posture of the end of the robotic arm;

An action control module is used to control the selected robotic arm to point to the target object based on the trajectory.

This application provides an electronic device, including:

one or more processors;

Memory, used to store one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the above robot pointing action control method.

The present application provides a computer-readable storage medium on which computer instructions are stored, and the instructions are processed by a processor. During execution, the above robot pointing action control method is implemented.

In some embodiments, the robot pointing action control method of the present application uses, when the target object exceeds the reachable range of the end of the robot's mechanical arm, based on the mapping of the target coordinates, the shoulder joint of the selected mechanical arm is in the robot coordinate system. According to the predetermined rules of shoulder joint coordinates and pointing actions, the position and posture of the end of the selected robotic arm pointing to the target object within the reachable range is obtained, so as to complete the pointing action of the selected robotic arm pointing to the target object. In this way, applications directed to target object scenes can be realized and the applicability of the robot can be improved.

Description of drawings

Figure 1 shows a schematic flowchart of a robot pointing action control method provided by an embodiment of the present application.

Figure 2 shows a specific flow diagram of a robot pointing action control method provided by an embodiment of the present application.

Figure 3 shows a simplified schematic diagram of the structure of the robot coordinate system and the target object of the robot pointing action control method provided by the embodiment of the present application.

Figure 4a shows a schematic diagram of the current configuration of the robot according to the robot pointing action control method provided by the embodiment of the present application.

Figure 4b shows a schematic diagram of the target configuration of the robot according to the robot pointing action control method provided by the embodiment of the present application.

FIG. 5 is a schematic flowchart of measuring action accuracy of the robot pointing action control method provided by the embodiment of the present application.

Figure 6 shows a module schematic diagram of a robot pointing action control device provided by an embodiment of the present application.

Figure 7 shows a module block diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of one or more embodiments of the present application as detailed in the appended claims.

It should be noted that in other embodiments, the steps of the corresponding methods are not necessarily performed in the order shown and described in this application. In some other embodiments, methods may include more or fewer steps than described herein. In addition, a single step described in this application may be broken down into multiple steps for description in other embodiments; and multiple steps described in this application may also be combined into a single step in other embodiments. describe.

In order to solve the technical problem of limited application scenarios of robotic arms, embodiments of the present application provide a robot pointing action control method. When the target object exceeds the reachable range of the end of the robot's robotic arm, the target object is positioned in the robot coordinate system according to the According to the target coordinates in the robot's two robot arms, select the robot arm that is closer to the target object; according to the mapping of the target coordinates, the shoulder joint coordinates of the selected robot arm's shoulder joint in the robot coordinate system and the reservation of the pointing action Rules are used to obtain the pose of the end of the selected manipulator pointing to the target object within the reachable range; based on the obtained pose of the end of the manipulator, determine the current pose of the selected manipulator along each joint to each joint. The trajectory of the target pose of the joint; based on the trajectory, control the selected robotic arm to point to the target object.

In the embodiment of the present application, when the target object exceeds the reachable range of the end of the robot's mechanical arm, according to the mapping of the target coordinates, the shoulder joint coordinates and pointing actions of the shoulder joint of the selected mechanical arm in the robot coordinate system are determined According to the predetermined rules, the position and posture of the end of the selected robot arm pointing to the target object within the reachable range is obtained, so as to complete the pointing action of the selected robot arm pointing to the target object. In this way, applications directed to target object scenes can be realized and the applicability of the robot can be improved.

The robot pointing action control method in the embodiment of the present application is applied to application scenarios in which the robot points to a target object. Application scenarios where the robot points to the target object may include but are not limited to scenarios where the robot explains or delivers demonstration content in the exhibition hall. Demonstration content may include but is not limited to videos, speeches, and teaching content. Target audiences may include, but are not limited to, the specific content of the demo content.

The specific implementation process of the robot pointing action control method is introduced in detail below.

As shown in Figure 1, the robot pointing action control method includes the following steps 110 to 140:

Step 110: When the target object exceeds the reachable range of the end of the robot's mechanical arm, select the robot arm closer to the target object among the two robot arms of the robot based on the target coordinates of the target object in the robot coordinate system.

In this way, if the target object exceeds the reachable range of the end of the robot's mechanical arm, it means that the target object is outside the reachable range of the end of the robot's mechanical arm. However, by determining the target coordinates of the target object in the robot coordinate system, the target object and the dual manipulator arms of the robot are in the same robot coordinate system. Therefore, the selected manipulator arm that is closer to the target object can be obtained conveniently and quickly.

The above step 110 can realize the use of a robotic arm that is closer to the target object to complete the pointing action, improve the convenience of pointing, and is more in line with user needs and more conducive to operation implementation.

There are many ways to implement the above step 110. In one implementation, the above-mentioned step 110 includes: determining the area where the target object is located from the two pre-divided areas corresponding to the two robot arms of the robot according to the target coordinates of the target object in the robot coordinate system; The robot arm corresponding to the area where the target object is located is determined as the robot arm that is closer to the target object than the two robot arms of the robot.

In another implementation, the above step 110 includes: determining the vectors between the target object and the shoulder joints of the robotic arm respectively, and the mapping distance on the horizontal coordinate plane of the robot coordinate system; , determined as the robotic arm that is closer to the target object than the dual robotic arms of the robot.

In this embodiment, according to the dimensions of each shoulder joint of the robot, such as the height h and shoulder width w of the shoulder joint, the left shoulder joint coordinates ^r p _ls = (0, w, h) and the right shoulder joint coordinates in the robot coordinate system can be obtained Joint coordinates ^r p _rs = (0, -w, h), respectively calculate the Euclidean distance (Euclidean Distance) between the left shoulder joint, right shoulder joint and the target object in the robot coordinate system, and select the robot arm with the smallest distance as the selected machine. arm to perform pointing actions.

Step 120: According to the mapping of the target coordinates, the shoulder joint coordinates of the selected robot arm's shoulder joint in the robot coordinate system and the predetermined rules of the pointing action, obtain the position of the selected robot arm's end point pointing to the target object within the reachable range. Posture.

Among them, the predetermined rules for pointing actions serve as constraints for pointing actions, which can provide reliable guarantee for the execution of subsequent specified actions.

The above step 120 can obtain the position point where the end of the selected robotic arm points to the target object within the reachable range through the mapping conversion of the target coordinates. This enables subsequent control of the selected robotic arm to point to the target object.

The position and posture of the robotic arm end of the selected robotic arm pointing to the target object within the reachable range includes the position and posture of the robotic arm end of the selected robotic arm in the robot coordinate system.

There are many embodiments for determining the position of the end of the robotic arm of the selected robotic arm in the robot coordinate system in the above step 120 . In some embodiments, a position point whose distance from the shoulder of the selected robot arm is less than the arm length of the selected robot arm can be determined as a target in a direction in which the end of the robot arm of the selected robot arm points toward the target object. Coordinates map to positions within reach of the arm end of the selected arm.

In other embodiments, in the direction in which the end of the selected robotic arm points to the target object, the arm length of the selected robotic arm can be multiplied by the reduction coefficient to obtain a reduced position point within the reachable range as the target Coordinates map to positions within reach of the arm end of the selected arm. Among them, the reduction coefficient is greater than 0 and less than 1, thereby ensuring that the end of the robotic arm is within the reachable range. See below for detailed instructions.

In some embodiments, the arm length of the selected robotic arm can be reduced by a predetermined length in the direction in which the end of the selected robotic arm points to the target object, to obtain a reduced position point within the reachable range, as The target coordinates map to a location within reach of the end of the selected robot's arm. Wherein, the predetermined length is greater than 0 and less than the arm length. In order to ensure that the selected robotic arm stretches as much as possible, increase the pointing range of the robotic arm end, and ensure that the robotic arm end is within the reachable range, the predetermined length should be as small as possible than the arm length. See below for detailed instructions.

Of course, other methods that can determine the position of the end of the selected robotic arm in the robot coordinate system fall within the scope of the embodiments of the present application, and will not be cited one by one here.

Step 130: Determine the trajectory of the selected robotic arm from the current posture of each joint of the selected robotic arm to the target posture of each joint based on the obtained posture of the end of the robotic arm. This trajectory can be used as the basis for controlling the selected robot arm towards the target object.

Step 140: Control the selected robotic arm to point to the target object according to the trajectory to complete the pointing action of the end of the selected robotic arm pointing to the target object.

As shown in Figure 2, before the above-mentioned step 110, the method may also include but is not limited to the following steps 101 to 104 to determine the reachable range of the target object beyond the end of the robot's manipulator arm:

Step 101: Establish the robot world coordinate system, the robot coordinate system, and the local coordinate system of each joint of the double robotic arm.

Among them, the robot coordinate system refers to the robot's base coordinate system.

Step 102: According to the angle between the vector pointing the origin of the robot coordinate system to the coordinates of the target object in the robot coordinate system and the X-axis of the robot coordinate system, obtain the angle at which the robot chassis needs to rotate so that the target object is within the front range of the robot.

Specifically, the above step 102 can be implemented through but is not limited to the following three steps:

(1) Use the following formula (1) to calculate the vector of the robot's coordinate ^w p _robot pointing to the target coordinate ^w p _target in the robot world coordinate system. The angle α with the X-axis of the robot world coordinate system:

in, is a vector The Y-direction component of is a vector The X-direction component.

(2) Use the following formulas (2) to (3) to calculate the relationship between the angle α and the orientation angle θ of the front of the robot in the world coordinate system, thereby finding the angle β that the robot chassis needs to rotate:

in, is a vector The Y-direction component of is a vector The X-direction component. in, It is the difference between the angle α and the orientation angle θ of the front of the robot in the world coordinate system. This value may exceed [-π, π]. The β obtained by the above calculation is the minimum angle of chassis rotation.

(3) Use the following formula (4) to obtain the orientation angle γ in the world coordinate system after the robot chassis is rotated:

Step 103: According to the robot's coordinates ^w p _robot in the world coordinate system and the orientation angle γ, convert the target object's target coordinate ^w p _target in the world coordinate system to the coordinates in the robot coordinate system. Specifically, the following formula can be used ( 5)～(6) Calculation:

Among them, ^r p _target is the coordinates of the target object in the robot coordinate system, R _robot is the attitude matrix of the robot in the world coordinate system, is the inverse matrix of the robot's posture matrix in the world coordinate system, target is the target object, and robot is the robot.

Step 104: Determine the reachable range of the target object beyond the end of the robot's manipulator arm. In the above embodiment of step 104, if the target object is outside the maximum reachable range of the end of the robot's manipulator arm, it is determined that the target object exceeds the reachable range of the end of the robot's manipulator arm. In another embodiment of the above step 104, if the distance between the target object and the shoulder joints of the robot is greater than the arm length of the robot's mechanical arm, it is determined that the target object is beyond the reachable range of the end of the robot's mechanical arm.

As shown in Figure 3, the XY plane coordinate system is the coordinate system projected on the XY plane by the robot's base coordinate system. Among them, the XY plane coordinate system includes the origin O of the XY plane coordinate system corresponding to the head position 21 of the robot, the left shoulder joint 22 of the left robot arm and the right shoulder joint 23 of the right robot arm, the target object 24, and the selected machine. The position point 25 and so on within the reachable range of the end of the robotic arm. The following describes in detail how to determine the three-dimensional coordinates of the position point 25 in the robot coordinate system, that is, the X coordinate, the Y coordinate, and the Z coordinate.

Continuing to explain with reference to FIG. 3 , the above step 120 may include but is not limited to the following steps 121 to 123 . Step 121: Map the target coordinates to coordinates within the reachable range of the end of the selected robotic arm.

In some embodiments of the above step 121, the first step is to determine the angle between the vector of the shoulder joint of the selected manipulator pointing to the target object in the robot coordinate system and the X-axis of the robot coordinate system, and the selected manipulator The arm length of the selected robot arm is obtained to obtain the X coordinate and Y coordinate of the end of the robot arm of the selected robot arm.

For example, according to the robot coordinate system, the shoulder joint ^r p _s of the selected manipulator points to the vector of the target object ^r p _target The angle η with the X-axis of the robot coordinate system, and the arm length L of the selected robotic arm, are used to obtain ^{the X and Y coordinates r} ^p _e ₍ x) and ^r p _e (y) of the end r p e of the robotic arm. Specifically, Calculated by the following formula (7):

Among them, the setting of K will make the distance between the position point 25 and the shoulder joint of the selected robotic arm smaller than the arm length L of the selected robotic arm.

The second step is to obtain the Z coordinate r of the end of the robotic arm of the selected robotic arm based on the relationship between the Z coordinate of the target object in the robot coordinate system, the coordinates of the shoulder joint of the selected robotic arm, and the arm length ^. p _e (z). Specifically, it can be calculated by the following formula (8).

Among them, δ is a value greater than zero and less than L, and can be a traversal value within a range.

The above formula (8-1) is used to illustrate that in the Z direction, the position of the target object is lower than the shoulder joint, and the end of the selected robotic arm will be lower than the shoulder joint. Subtracted by ^r p _s (z) of the shoulder joint δ, in line with the simulation of the end of the robot's manipulator pointing toward the target object below the shoulder joint.

The above formula (8-2) is used to explain that in the Z direction, the position of the target object is higher than the shoulder joint and lower than the total height of the shoulder joint and the robot arm. It means that the position of the target object itself is higher than the shoulder joint and within the reach of the robot arm. Within the range, subtract δ from ^r p _target (z) of the target object, so that the end of the robot's manipulator arm can point to the target object at a lower position than the target object, which is more consistent with the target object that the end of the robot's manipulator arm simulates pointing to. .

The above formula (8-3) is used to illustrate that in the Z direction, the position of the target object is higher than the total height of the shoulder joint and the robotic arm, subtracted by ^r p _s (z) + L from the total height of the shoulder joint and the robotic arm. δ, the point where the robot arm points is lower than the target object. In this way, the end of the selected robot arm can be within the reachable range, and the end of the selected robot arm can be pointed to the target object within the maximum reachable range of the selected robot arm.

Step 122: Determine the designated direction of the end of the robotic arm of the selected robotic arm according to the predetermined rules of the pointing action.

Step 123: Generate the posture of the end of the robotic arm of the selected robotic arm based on the mapped coordinates, the shoulder joint coordinates of the selected robotic arm's shoulder joint in the robot coordinate system, and the specified direction.

In some embodiments of the above step 123, the first step is to determine the unit vector of the vector pointing the target object toward the end of the robotic arm as 3 of the attitude matrix of the end of the robotic arm of the selected robotic arm used to form the Z direction vector. Z direction components, the attitude matrix of the end of the selected robotic arm is a matrix of 3 rows and 3 columns. For example, the attitude matrix R of the end of the selected robotic arm is represented by the following formula (9):

in, is the X-direction vector, ax, ay, and az are three X-direction components respectively. These three X-direction components belong to the first column. is the Y-direction vector, ox, oy, and oz are three Y-direction components respectively. These three Y-direction components belong to the second column. is the Z-direction vector, nx, ny and nz are three Z-direction components respectively. These three Z-direction components belong to the third column.

The first step above can be calculated by the following formulas (10) ~ (11). The vector from the target object ^r p _target to the end of the robot arm ^r p _e of the selected robot arm The unit vector of is the third column of the attitude matrix of the end of the selected robotic arm. Right now:

The second step is to determine the attitude matrix used to form Y when the end of the selected robot arm points to the target object, the three Z-direction components of the attitude matrix are positive, and the selected robot arm is the right robot arm. The three Y-direction components of the direction vector are positive;

The third step is to determine the attitude matrix used to form Y when the end of the robot arm of the selected robot arm points to the target object, the three Z-direction components of the attitude matrix are positive, and the selected robot arm is the left robot arm. The three Y-direction components of the direction vector are negative.

Figure 4a shows a schematic diagram of the current configuration of the robot according to the robot pointing action control method provided by the embodiment of the present application. Figure 4b shows a schematic diagram of the target configuration of the robot according to the robot pointing action control method provided by the embodiment of the present application.

As shown in Figure 4a and Figure 4b, the base center position of the robot's base coordinate system is _Or . The dual robotic arms include a left robotic arm (not labeled in the figure) and a right robotic arm 30 . The following takes the right robotic arm 30 as the selected robotic arm as an example. The right robotic arm 30 may include but is not limited to the right shoulder joint 31 , the right large arm 32 , the right small arm 33 and the right palm 34 . The same applies to the left robotic arm, so I won’t go into details here.

Continuing to illustrate with reference to Figure 4a and Figure 4b, the above steps 2 and 3 can be achieved through the following steps:

According to the rule that when the robotic arm points to the target object, the palm of the hand is tilted upward, it can be seen that when pointing to the target object, the Z-direction component of R of the attitude matrix of the end of the robotic arm is positive, when the right robotic arm points to the target object, the Y-direction component of the attitude matrix R at the end of the robotic arm If it is positive, it means the palm is facing upward. When the left manipulator points to the target object, the Y direction component is negative. Taking the right robotic arm as shown in Figure 4a and Figure 4b as an example, the following formula (12) is satisfied:

The value of oz affects the upward direction of the palm, and can be selected based on the pointing effect. In this implementation use case, oz=0.4 is selected. It can also float around this value based on the actual solution results, so that we can get:

set up:

Thus we can get:

The same applies to the left robotic arm, so I won’t go into details here.

Step 4: According to the right-hand rule, determine the three X-direction components of the attitude matrix used to form the X-direction vector based on the three Z-direction components and the three Y-direction components of the attitude matrix at the end of the selected robot arm. In this way, compared with the implementation process of solving nonlinear equations, it is more convenient and does not rely on existing libraries, such as matlab's library for solving nonlinear equations, which can increase the calculation speed.

The above-mentioned step 4 can be achieved through the following steps: According to the right-hand rule, the second column of the robot end attitude matrix can be and the third column Find the first column, expressed by the following formula (16):

The above step 130 may further include, but is not limited to, the following two steps. The first step is to solve the inverse kinematics based on the obtained posture of the end of the robotic arm to obtain the target posture of each joint of the robotic arm. The second step is to obtain the trajectory of the end of the selected robotic arm from the current posture to the target posture of each joint through trajectory interpolation.

Among them, the above-mentioned step 1 may further include but is not limited to:

Based on the obtained pose of the end of the robotic arm, solve the inverse kinematics and obtain the target configuration of the selected robotic arm. The target configuration includes the target angles of each joint;

Based on the current configuration of the robot and the target configuration, trajectory interpolation is performed to control the movement of the robot arm from the current configuration of the robot to the target configuration of the robot pointing to the target object. The current configuration of the robot includes the current angles of each joint.

As shown in FIGS. 1 to 5 , after the above-mentioned step 140 , the method may include but is not limited to: step 150 , using a measuring device to test the obtained pose of the end of the robotic arm, and determine the end of the robotic arm of the selected robotic arm. Error in pointing motion toward a target object. In this way, after controlling the robot arm to point to the target, the deviation of the robot arm pointing to the target is determined, thereby measuring the accuracy of the pointing action to facilitate subsequent adjustments and control optimization.

In some embodiments, using a measurement device to test the obtained pose of the end of the robotic arm, and determining the error of the pointing action of the end of the selected robotic arm toward the target object, includes: a first step, determining that the target object points to the selected Vector of the end of the robotic arm. Specifically, a measuring device can be used to test the posture of the end of the robotic arm, and based on the position of the target object and the measured position of the end of the robotic arm, the vector pointing from the target object to the end of the robotic arm is calculated. The second step is to determine the angle between the vector and the Z-direction vector of the attitude matrix of the robot end of the selected robot arm as the above error.

Further, the above step 150 may further include but is not limited to the following 1) to 3):

1) Use a laser tracker to test the posture of the end of the robotic arm. The coordinate system used for measurement coincides with the robot coordinate system. The test result is the actual posture of the end of the robotic arm in the robot coordinate system ( ^r p _e ′, ^r R _e ′);

2). Based on the test results in step 1) above and the coordinates ^r p _target of the target object in the robot coordinate system, calculate the vector of the target object pointing to the end of the robotic arm.

3), the vector represented by the third column of the attitude matrix ^r R _e ′ With the vector in step 2) above The angle between , which is the error δ of the pointing action, is calculated by the following formula (17):

As shown in Figure 6, the robot points to the action control device, including the following modules:

The robotic arm selection module 41 is used to select the one of the robot's dual robotic arms that is closer to the target object based on the target coordinates of the target object in the robot coordinate system when the target object exceeds the reachable range of the end of the robot's robotic arm. Nearby robotic arm;

The pose determination module 42 of the end of the robotic arm is used to obtain the end of the robotic arm of the selected robotic arm based on the mapping of the target coordinates, the shoulder joint coordinates of the selected robotic arm's shoulder joint in the robot coordinate system, and the predetermined rules of the pointing action. The pose pointing to the target object within the reachable range;

The trajectory determination module 43 is used to determine the trajectory of the selected robotic arm from the current posture of each joint of the selected robotic arm to the target posture of each joint based on the obtained posture of the end of the robotic arm;

The action control module 44 is used to control the selected robotic arm to point to the target object according to the trajectory.

In some embodiments, the pose determination module 42 at the end of the robotic arm includes:

The coordinate mapping submodule is used to map the target coordinates to coordinates within the reachable range of the end of the selected robotic arm;

The designated direction determination submodule is used to determine the designated direction of the end of the robotic arm of the selected robotic arm according to the predetermined rules of the pointing action;

The attitude generation submodule of the end of the robotic arm is used to generate the attitude of the end of the robotic arm of the selected robotic arm based on the mapped coordinates, the shoulder joint coordinates of the shoulder joint of the selected robotic arm in the robot coordinate system, and the specified direction. .

In some embodiments, the coordinate mapping submodule is specifically configured to: in the robot coordinate system, the intersection between the vector of the shoulder joint of the selected manipulator pointing to the target object and the X-axis of the robot coordinate system angle, and The arm length of the selected robotic arm is used to obtain the X coordinate and Y coordinate of the end of the robotic arm of the selected robotic arm; based on the Z coordinate of the target object in the robot coordinate system and the coordinates of the shoulder joint of the selected robotic arm, the According to the relationship between the arm lengths, the Z coordinate of the end of the robotic arm of the selected robotic arm is obtained;

and / or,

The attitude generation submodule at the end of the robotic arm is specifically used for:

The unit vector of the vector of the target object pointing to the end of the robotic arm is determined as the three Z-direction components of the attitude matrix of the end of the selected robotic arm that are used to form the Z-direction vector. The end of the selected robotic arm's robotic arm The posture matrix is a matrix with 3 rows and 3 columns;

In some embodiments, the device further includes: a pointing action error determination module, configured to use a measurement device to test the obtained pose of the end of the robotic arm, and determine the pointing action of the end of the selected robotic arm toward the target object. error.

In some embodiments, the error determination module of the pointing action is specifically used to: determine the vector of the target object pointing to the end of the robotic arm of the selected robotic arm; determine the attitude matrix between the vector and the end of the robotic arm of the selected robotic arm. The angle between the Z direction vectors is used as the above error.

In some embodiments, the trajectory determination module 43 includes:

The target posture determination submodule is used to solve the inverse kinematics based on the obtained posture of the end of the robotic arm, and obtain the target posture of each joint of the robotic arm;

The trajectory determination submodule is used to obtain the trajectory of the end of the selected robotic arm from the current posture to the target posture of each joint through trajectory interpolation.

In some embodiments, the pose determination module 42 of the end of the robotic arm is specifically used to: solve the inverse kinematics based on the obtained pose of the end of the robotic arm, and obtain the target configuration of the selected robotic arm. The target configuration includes the target angle of each joint; based on the current configuration of the robot and the target configuration, trajectory interpolation is performed to control the robot arm to move from the current configuration of the robot to point to the target object The target configuration of the robot, and the current configuration of the robot includes the current angles of each joint.

The specific implementation process of the functions and effects of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be described again here.

This application also provides a computer-readable storage medium on which computer instructions are stored. When the instructions are executed by a processor, the above-mentioned robot pointing action control method is implemented.

The present application also provides an electronic device. Figure 7 shows a module block diagram of the electronic device provided by an embodiment of the present application.

As shown in FIG. 7 , the electronic device 50 includes one or more processors 51 for implementing the robot pointing action control method as described above.

In some embodiments, the electronic device 50 may include a memory 59, which may store programs that can be called by the processor 51, and may include a non-volatile storage medium. In some embodiments, electronic device 50 may include memory 58 and interface 57 . In some embodiments, the electronic device 50 may also include other hardware depending on the actual application.

The memory 59 in the embodiment of the present application stores a program thereon, and when the program is executed by the processor 51, it is used to implement the above-mentioned robot pointing action control method.

The application may take the form of a computer program product implemented on one or more memories 59 (including but not limited to disk memory, CD-ROM, optical memory, etc.) having program code embodied therein. Memory 59 includes permanent and non-permanent, removable and non-removable media, and information storage may be accomplished by any method or technology. Information may be computer-readable instructions, data structures, modules of programs, or other data. Examples of memory 59 include, but are not limited to: phase change memory (Phase Change RAM, PRAM), static random access memory (Static Random-Access Memory, SRAM), dynamic random access memory (Dynamic Random Access Memory, DRAM), and others. Types of Random Access Memory (RAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory (Flash Memory) or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, magnetic cassettes, tapes and Disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the present application. within the scope of protection.

It should also be noted that the terms "comprises,""comprises," or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements not only includes those elements, but also includes Other elements are not expressly listed or are inherent to the process, method, article or equipment. Without further limitation, the statement "comprises a..." qualifies an element as not excluding the presence of other identical elements in a process, method, article, or device that includes the stated element.

Claims

A robot pointing action control method, which is characterized by including:

When the target object exceeds the reachable range of the end of the robot's manipulator arm, according to the target coordinates of the target object in the robot coordinate system, select the one of the two manipulator arms of the robot that is closer to the target object. robotic arm;

According to the mapping of the target coordinates, the shoulder joint coordinates of the selected robot arm's shoulder joint in the robot coordinate system and the predetermined rules of the pointing action, it is obtained that the end of the selected robot arm points to the target object within the reachable range. posture;

Based on the obtained posture of the end of the robotic arm, determine the trajectory of the selected robotic arm from the current posture of each joint of the selected robotic arm to the target posture of each joint;

According to the trajectory, the selected robotic arm is controlled to point to the target object.
The robot pointing action control method according to claim 1, characterized in that, based on the mapping of the target coordinates, the shoulder joint coordinates of the selected mechanical arm's shoulder joint in the robot coordinate system and the predetermined rules of the pointing action, Obtain the pose of the end of the selected robotic arm pointing to the target object within the reachable range, including:

Map the target coordinates to coordinates within the reachable range of the end of the robotic arm of the selected robotic arm;

Determine the designated direction of the end of the robotic arm of the selected robotic arm according to the predetermined rules of the pointing action;

Based on the mapped coordinates, the shoulder joint coordinates of the selected robotic arm's shoulder joint in the robot coordinate system, and the specified direction, the posture of the robotic arm end of the selected robotic arm is generated.
The robot pointing action control method according to claim 2, characterized in that:

Mapping the target coordinates to coordinates within the reachable range of the end of the selected robotic arm includes:

According to the angle between the vector of the selected robot arm's shoulder joint pointing to the target object in the robot coordinate system and the X-axis of the robot coordinate system, and the arm length of the selected robot arm, the value of the selected robot arm is obtained. The X coordinate and Y coordinate of the end of the robotic arm;

According to the relationship between the Z direction coordinate of the target object in the robot coordinate system, the coordinates of the shoulder joint of the selected robotic arm, and the arm length, the Z coordinate of the robotic arm end of the selected robotic arm is obtained;

and / or,

Generating the posture of the end of the robotic arm of the selected robotic arm based on the mapped coordinates, the shoulder joint coordinates of the selected robotic arm's shoulder joint in the robot coordinate system, and the specified direction, includes:

The unit vector of the vector of the target object pointing to the end of the robotic arm is determined as the three Z-direction components of the attitude matrix of the end of the selected robotic arm that are used to form the Z-direction vector. The end of the selected robotic arm's robotic arm The posture matrix is a matrix with 3 rows and 3 columns;

When the end of the selected robot arm points to the target object, and the three Z-direction components of the attitude matrix are positive, and the selected robot arm is a right robot arm, determine the composition of the attitude matrix. The three Y-direction components of the Y-direction vector are positive;

When the end of the selected robot arm points to the target object, and the three Z-direction components of the attitude matrix are positive, and the selected robot arm is the left robot arm, determine the composition of the attitude matrix. The three Y-direction components of the Y-direction vector are negative;

According to the right-hand rule, the three X-direction components of the attitude matrix used to form the X-direction vector are determined based on the three Z-direction components and the three Y-direction components of the attitude matrix of the end of the selected robot arm.
The robot pointing action control method according to claim 1, characterized in that, after controlling the selected robotic arm to point to the target object based on the trajectory, the method further includes:

Use a measuring device to test the obtained pose of the robotic arm end, and determine the error in the pointing action of the selected robotic arm's robotic arm end towards the target object.
The robot pointing action control method according to claim 4, characterized in that the measurement device is used to test the obtained posture of the end of the robotic arm to determine the direction in which the end of the robotic arm of the selected robotic arm points to the target object. Action errors include:

Determine a vector pointing from the target object to the end of the robotic arm of the selected robotic arm;

The angle between the vector and the Z-direction vector of the attitude matrix of the robot end of the selected robot arm is determined as the error.
The robot pointing action control method according to claim 1, characterized in that, based on the obtained posture of the end of the robotic arm, determining the current posture of each joint of the selected robotic arm along the selected robotic arm to the The trajectory of the target pose of each joint includes:

Based on the obtained posture of the end of the robotic arm, solve the inverse kinematics and obtain the target posture of each joint of the robotic arm;

Through trajectory interpolation, the trajectory of the robotic arm end of the selected robotic arm from the current posture to the target posture of each joint is obtained.
The robot pointing action control method according to claim 6, wherein the step of solving the inverse kinematics based on the obtained posture of the end of the robotic arm to obtain the target posture of each joint of the robotic arm includes:

Based on the obtained posture of the end of the robotic arm, solve the inverse kinematics to obtain the target configuration of the selected robotic arm, where the target configuration includes the target angle of each joint;

According to the current configuration of the robot and the target configuration, trajectory interpolation is performed to control the movement of the robot arm from the current configuration of the robot to the target configuration of the robot pointing to the target object. The current configuration of the robot is The configuration includes the current angles of each of the joints.
A robot pointing action control device, characterized by including:

The robot arm selection module is used to select the robot arm of the robot according to the target coordinates of the target object in the robot coordinate system when the target object exceeds the reachable range of the robot's robot arm end. The robot arm is closer to the target object;

The pose determination module at the end of the robotic arm is used to obtain the robotic arm of the selected robotic arm based on the mapping of the target coordinates, the shoulder joint coordinates of the selected robotic arm's shoulder joint in the robot coordinate system, and the predetermined rules of the pointing action. The end point points to the pose of the target object within the reachable range;

A trajectory determination module, configured to determine the trajectory of the selected robotic arm from the current posture of each joint of the selected robotic arm to the target posture of each joint based on the obtained posture of the end of the robotic arm;

An action control module is used to control the selected robotic arm to point to the target object according to the trajectory.
An electronic device, characterized by including:

one or more processors;

Memory, used to store one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method according to any one of claims 1-7.
A computer-readable storage medium on which computer instructions are stored, characterized in that when the instructions are executed by a processor, the method according to any one of claims 1-7 is implemented.