CN111319037A - Redundant robot uncalibrated visual servo control method and system - Google Patents
Redundant robot uncalibrated visual servo control method and system Download PDFInfo
- Publication number
- CN111319037A CN111319037A CN201811539384.9A CN201811539384A CN111319037A CN 111319037 A CN111319037 A CN 111319037A CN 201811539384 A CN201811539384 A CN 201811539384A CN 111319037 A CN111319037 A CN 111319037A
- Authority
- CN
- China
- Prior art keywords
- robot
- matrix
- jacobian matrix
- homography
- composite
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a projective homography-based redundant robot uncalibrated vision servo control method and a system, wherein the method comprises the following steps: step one, constructing a task function, namely constructing the task function by using a homography matrix with 8 degrees of freedom for mapping current image characteristics and expected image characteristics; step two, joint control, namely constructing a controller by utilizing a composite Jacobian matrix, and directly controlling the joints of the robot; step three, singular value filtering, namely limiting the minimum singular value of the composite Jacobian matrix by using a singular value filtering algorithm, so that the servo process is not interrupted due to singularity of the composite Jacobian matrix until the robot leaves a singularity interval smoothly; and step four, joint limiting, namely limiting the joints by using a gradient projection method by using the redundancy characteristic of the redundant robot. The invention relates to an improved projective homography-based visual servo control method and system applied to a redundant robot.
Description
Technical Field
The invention relates to the field of visual servo control methods, in particular to an improved visual servo control method based on projective homography and applied to a redundant robot.
Background
In the traditional image-based uncalibrated visual servo problem, a task function is often constructed by directly using image information to control a robot. In the actual working process, due to the problems of image noise, blurring caused by camera motion, target visual field defect, unobvious contrast between the target and the environment color and the like, the problems of loss of partial image features, wrong matching, larger image processing error and the like can be caused, so that the control fails. In the process of uncalibrated visual servo, the Jacobian matrix is easy to generate a singular condition, so that the robot cannot normally move. Meanwhile, the corresponding movement amount of the tail end of the robot is calculated by the general visual servo problem through image characteristics, but the solution of the complicated inverse kinematics problem of the redundant robot still depends on the calibration with relatively high precision on the parameters of the robot and influences the calculation speed. This means that the robustness of the visual servo system to calibration errors and environmental disturbances is still limited.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides an improved visual servo control method based on projective homography and applied to a redundant robot.
In order to achieve the purpose, the invention adopts the following technical scheme: the method for controlling the redundant robot uncalibrated vision servo based on projective homography comprises the following steps:
step one, constructing a task function, namely constructing the task function by using a homography matrix with 8 degrees of freedom for mapping current image characteristics and expected image characteristics;
step two, joint control, namely constructing a controller by utilizing a composite Jacobian matrix, and directly controlling the joints of the robot;
step three, singular value filtering, namely limiting the minimum singular value of the composite Jacobian matrix by using a singular value filtering algorithm, so that the servo process is not interrupted due to singularity of the composite Jacobian matrix until the robot leaves a singularity interval smoothly;
and step four, joint limiting, namely limiting the joints by using a gradient projection method by using the redundancy characteristic of the redundant robot.
In the first step, the homography matrix is a 3 × 3 mapping matrix between images of the same object formed on two different imaging planes, and when the homography matrix is obtained by using not less than 4 feature points, a certain element of the homography matrix can be constrained to obtain a determined result.
And in the second step, the composite Jacobian matrix of each period is obtained by utilizing online identification, and a controller is constructed by utilizing the composite Jacobian matrix to directly control the joints of the robot.
In the third step, when Jacobian enters a singular interval, each singular value of the composite Jacobian matrix identified in real time is restrained, and the singular value is limited to a reasonable threshold value, so that the servo process cannot be interrupted due to singularity of the composite Jacobian matrix until the robot smoothly leaves the singular interval, and the system does not need to pause and restart the servo process when Jacobian is different.
And in the fourth step, the optimization degree concept is utilized to perform online adjustment on the coefficient k of the gradient projection, so that the joint angle does not exceed the limit, and the servo process is smoothly performed.
In order to achieve the purpose, the invention also adopts the following technical scheme: the utility model provides a redundant robot does not have demarcation vision servo control system based on projective homography, includes:
a constructing task function unit for constructing a task function by using a homography matrix with 8 degrees of freedom for mapping the current image characteristics and the expected image characteristics;
the joint control unit is used for constructing a controller by utilizing the composite Jacobian matrix and directly controlling the joints of the robot;
the singular value filtering unit is used for limiting the minimum singular value of the composite Jacobian matrix by using a singular value filtering algorithm, so that the servo process is not interrupted due to singularity of the composite Jacobian matrix until the robot leaves a singularity interval smoothly;
and the joint limiting unit is used for limiting the joints by using a gradient projection method by utilizing the redundancy characteristic of the redundant robot.
In the joint control unit, a composite Jacobian matrix of each period is obtained by utilizing online identification, a controller is constructed by utilizing the composite Jacobian matrix, and the joints of the robot are directly controlled.
In the singular value filtering unit, when the Jacobian enters a singular interval, each singular value of a composite Jacobian matrix identified in real time is restrained, and each singular value is limited to a reasonable threshold value, so that the servo process cannot be interrupted due to singularity of the composite Jacobian matrix until the robot smoothly leaves the singular interval.
The joint limiting unit further comprises a step of adjusting the coefficient k of the gradient projection on line, so that the joint angle cannot exceed the limit, and the servo process can be smoothly carried out.
The invention has the beneficial effects that: the invention directly controls each joint of the redundant robot by using visual information, and can avoid the problem of servo failure possibly caused by visual field defect, characteristic point mismatching or image processing error by using more image characteristic points under the condition of not increasing the number of dimension of Jacobian matrix. The singular value filtering algorithm is applied to the visual servo, so that the stability of the singular value of the Jacobian matrix in the servo process is kept, and the interruption of the servo process is avoided. Meanwhile, the problem that joint angles possibly appear in the servo process exceed the limit is solved by utilizing the redundancy characteristic of the redundant robot.
Drawings
Fig. 1 is a flowchart illustrating a projective homography-based uncalibrated vision servo control method for a redundant robot according to the present invention.
Fig. 2 is a flow chart of a method for controlling the visual tracking servo of the target motion.
Fig. 3 is a block diagram illustrating a redundant robot uncalibrated vision servo control system based on projective homography according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention.
The invention aims to solve the technical problem of providing a projective homography-based redundant robot uncalibrated vision servo control method. The invention directly controls each joint of the redundant robot by using visual information, and can avoid the problem of servo failure possibly caused by visual field defect, characteristic point mismatching or image processing error by using more image characteristic points under the condition of not increasing the number of dimension of Jacobian matrix. The singular value filtering algorithm is applied to the visual servo, so that the stability of the singular value of the Jacobian matrix in the servo process is kept, and the interruption of the servo process is avoided. Meanwhile, the problem that joint angles possibly appear in the servo process exceed the limit is solved by utilizing the redundancy characteristic of the redundant robot.
As shown in fig. 1, a projective homography-based method for controlling an uncalibrated vision servo of a redundant robot according to an embodiment of the present invention includes:
step one, constructing a task function, namely constructing the task function by using a homography matrix with 8 degrees of freedom for mapping current image characteristics and expected image characteristics;
step two, joint control, namely constructing a controller by utilizing a composite Jacobian matrix, and directly controlling the joints of the robot;
step three, singular value filtering, namely limiting the minimum singular value of the composite Jacobian matrix by using a singular value filtering algorithm, so that the servo process is not interrupted due to singularity of the composite Jacobian matrix until the robot leaves a singularity interval smoothly;
and step four, joint limiting, namely limiting the joints by using a gradient projection method by using the redundancy characteristic of the redundant robot.
The visual information is used for directly controlling each joint of the redundant robot, and the problems of visual field defect, characteristic point mismatching or image processing error which possibly cause servo failure can be avoided by using more image characteristic points under the condition of not increasing the number of dimension of a Jacobian matrix. The singular value filtering algorithm is applied to the visual servo, so that the stability of the singular value of the Jacobian matrix in the servo process is kept, and the interruption of the servo process is avoided. Meanwhile, the problem that joint angles possibly appear in the servo process exceed the limit is solved by utilizing the redundancy characteristic of the redundant robot.
In one embodiment, the task function is constructed using the first 8 elements of the current homography matrix, the homography matrix between the current image feature and the desired image feature is solved using at least 4 feature points on the target that can be identified, the homography matrix refers to a 3 × 3 mapping matrix between images of the same object formed on two different imaging planes, and when the homography matrix is solved using not less than 4 feature points, a certain element of the homography matrix can be constrained to obtain a certain result, when the homography matrix of 8 degrees of freedom is solved, the last element of the homography matrix is constrained to be 1:
the first 8 elements e ═ h using homography1,h2,h3,h4,h5,h6,h7,h8]T-[1,0,0,0,1,0,0,0]TAs a function of the task. The essential purpose of control according to the image characteristics is to control the relative pose between the robot and the target. It can be shown that the machine is capable of performing a task function with a value e equal to 0, if and only if the value of the task function is e equal to 0The image features obtained by the human tip camera coincide with the expected image features, i.e. the relative pose between the robot tip and the target coincides with the expected values (the proof process is noted below).
In one embodiment, the complex Jacobian matrix of each period is obtained by online identification, and a controller is constructed by using the complex Jacobian matrix to directly control the robot joint, so that the complex redundant inverse kinematics control problem of the robot can be avoided. The recognition of the Jacobian matrix can use a classic Kalman-Bucy filtering method, and a linear model needs to be established in order to use the method. The jacobian matrix is known to have the following form:
defining a system state vector:
according to the definition of the Jacobian matrix, there are:
e(k)＝e(k-1)+Jtotally(k-1)·Δq(k-1)(7a)；
Δq(k)＝[Δq1(k) Δq2(k)…Δqn(k)]T(8a)；
there are the following system equations:
x (k) is the system state at time k, H is the measurement system parameter, i.e. the matrix formed by the angle variations of the joints, w (k) and v (k) represent the process and the measured noise, respectively. At the moment, the Kalman-Bucy filtering algorithm is used for identifying the state vector of the system, so that the on-line identification of the Jacobian matrix is realized, namely:
the covariance difference of the process and measurement noise is Q, R, x (k | k-1) is the result predicted using the previous state and is the optimal result for the previous state, P (k | k-1) is the covariance for x (k | k-1), P (k-1| k-1) is the covariance for x (k-1| k-1), where G (k) is the Kalman filter gain. The jacobian matrix can be reconstructed using the system state vectors identified for each iteration cycle.
In one embodiment, when Jacobian enters a singular interval, each singular value of a composite Jacobian matrix identified in real time is constrained and limited to a reasonable threshold value, so that the servo process is not interrupted due to singularity of the composite Jacobian matrix until the robot smoothly leaves the singular interval.
The Jacobian matrix is easy to generate a strange situation in the process of visual servo, so that the robot must stop working and perform the trial movement again. In order to improve the problem, a singular value filtering method for solving the singularity of the Jacobian matrix in the robot kinematics is introduced into the visual servo. Performing singular value decomposition on the identified composite Jacobian matrix in each period:
σiis the singular value of the Jacobian matrix, uiAnd viColumn vectors of U and V, respectively. When the composite jacobian matrix is close to the singularity, a singular value or singular values are close to 0, which causes the calculated control quantity to be close to infinity. Therefore, it is necessary to limit singular values smaller than a certain threshold value, as shown in equation (13 a):
v is the curvature adjustment factor, σ0Is the smallest singular value that can be tolerated. At this time, JacobianThe pseudo-inverse of the matrix is:
the minimum singular value of the Jacobian matrix can be limited through singular value filtering, the joint speed of the robot is controlled to be kept stable, the robot leaves a singular interval of the composite Jacobian matrix as smoothly as possible, and then the expected servo task is continuously completed.
The invention directly controls each joint of the redundant robot, and each joint of the robot has an angle limit which can be reached practically. Therefore, each joint of the robot must be limited to prevent failure due to the fact that the joint reaches the limit in the servo process. In this case, the joint angle is moved away from the limit by performing a self-motion without affecting the main task by redundancy. Gradient projection is the most common method by optimizing the motion
Projection onto the null space of the redundant complex Jacobian matrix to achieve:
wherein I-J+ totallyJtotallyIs a composite Jacobian matrix JtotallyHowever, in the method, the composite Jacobian matrix is an 8 × n matrix, if the projection operator is constructed by directly using the composite Jacobian matrix, the robot has at least 9 degrees of freedom, so that the application of the method loses universality, and the composite Jacobian matrix identified on line is easy to generate singularity, and the gradient projection method in the visual servo is improved by considering the reasons:
discretizing according to an iteration period delta t, and obtaining:
namely, the Jacobian matrix of the robot is used for replacing a composite Jacobian matrix to construct a projection operator, and the motion of the Jacobian matrix of the robot on the null space does not influence the terminal pose of the robot, and the image characteristics are not influenced. The dimensionality of the Jacobian matrix of the robot depends on the terminal degree of freedom and the joint space dimensionality of the robot, so that the redundant robot with any degree of freedom can be optimized; and the Jacobian matrix of the robot is often more accurate to be solved, and the adverse effect on the servo process can not be caused. The optimization motion is shown in equation (18a) (19 a):
the selection or online adjustment of the amplification factor k may refer to a method proposed by luya based on a redundant robot motion optimizability concept, which is not described herein in detail.
For the visual tracking problem of target motion, the improved control system is shown in fig. 2. At this time, the change of the homography matrix element is simultaneously caused by the movement of the target and the movement of the robot. Therefore, the change rate of the homography matrix elements caused by the target motion is included in the identification content, and the expanded composite Jacobian matrix is identified on line:
in the case of recognition, the equations (6a) (8a) (10a) are also modified accordingly:
Δqextend(k)＝[Δq|Δt]＝[Δq1(k) Δq2(k)…Δqn(k) Δt]T(22a)；
in calculating the control amount, it is necessary to take the error of this portion into account. At this time, the system error is:
where Δ t is the length of the iteration cycle, and the control amount at this time is calculated as:
etrack＝e-et(25a)；
in consideration of the limitation of the actual joint velocity, it is sometimes necessary to correct the control amount:
The above methods can be summarized as the following algorithms:
(1a) and (3) calculating a homography matrix between the current robot end camera and the expected feature by using formulas (1a) to (4a) according to the image features obtained by the current robot end camera, constructing a task vector e by using the first 8 elements of the matrix, jumping out of the loop when e is smaller than a threshold value, and otherwise, performing the step (2 a).
(2a) According to the variable quantity y (k) of the obtained task vector and the last cycle task vector and the variable quantity delta q of the joint angle obtained in the last cycleUsing formulas (6a) - (11a), the complex Jacobian matrix J obtained in the previous cycle is processedtotallyAnd (6) updating.
(3a) For J obtained in the step (2)totallyPerforming singular value decomposition, and filtering singular values smaller than the threshold value by using formulas (12a) to (14a) to obtain singular values
(4a) According to the pose T of the robot at the moment, the Jacobian matrix J of the robot is obtainedrobot。
(5a) At this time, the joint angle q of the robot is obtained from the equations (18a) and (19a)
(6a) And adjusting the amplification coefficient k of the gradient projection method on line.
(7a) And (4) obtaining the angle change quantity delta q of each joint of the robot according to the results obtained in the steps (3a), (4a), (5a) and (6a) and the formulas (17a), (26a) and (27a), and controlling the robot to move. Returning to the step (1 a).
The method relates to the tracking problem of target motion, and adjusts an algorithm, and comprises the following steps:
(1a) and (3) calculating a homography matrix between the current robot end camera and the expected feature according to the image features obtained by the current robot end camera by using formulas (1a) to (4a), and constructing a task vector e by using the first 8 elements of the matrix.
(2a) According to the variable quantity y (k) of the obtained task vector and the last cycle task vector and the variable quantity delta q of the joint angle obtained in the last cycleextend＝[Δq|Δt]The extended Jacobian matrix J obtained for the previous cycle using equations (7a) (9a) (11a) (21a) (22a) (23a)extendAnd (6) updating.
(3a) J from step (2a) according to the formula (20a)extendExtract J fromtotallyAnd
and performing singular value decomposition on the former, and performing singular value decomposition on the singular value smaller than the threshold value by using formulas (12a) to (14a)Line filtering to obtain
(4a) According to the pose T of the robot at the moment, the Jacobian matrix J of the robot is obtainedrobot。
(5a) At this time, the joint angle q of the robot is obtained from the equations (18a) and (19a)
(6a) And adjusting the amplification coefficient k of the gradient projection method on line.
(7a) And (4) obtaining the angle change quantity delta q of each joint of the robot according to the results obtained in the steps (3a), (4a), (5a) and (6a) and the formulas (25a), (26a) and (27a), and controlling the robot to move. Returning to the step (1 a).
As shown in fig. 3, a projective homography-based redundant robot uncalibrated vision servo control system provided by an embodiment of the present invention includes:
s001, constructing a task function unit, wherein the task function unit is used for mapping a homography matrix with 8 degrees of freedom of the current image characteristics and the expected image characteristics to construct a task function;
the S002 joint control unit is used for constructing a controller by utilizing the composite Jacobian matrix and directly controlling the joints of the robot;
the S003 singular value filtering unit is used for limiting the minimum singular value of the composite Jacobian matrix by using a singular value filtering algorithm, so that the servo process cannot be interrupted due to singularity of the composite Jacobian matrix until the robot leaves a singularity interval smoothly;
and the S004 joint limiting unit is used for limiting the joint by using a gradient projection method by utilizing the redundancy characteristic of the redundant robot.
The visual information is used for directly controlling each joint of the redundant robot, and the problems of visual field defect, characteristic point mismatching or image processing error which possibly cause servo failure can be avoided by using more image characteristic points under the condition of not increasing the number of dimension of a Jacobian matrix. The singular value filtering algorithm is applied to the visual servo, so that the stability of the singular value of the Jacobian matrix in the servo process is kept, and the interruption of the servo process is avoided. Meanwhile, the problem that joint angles possibly appear in the servo process exceed the limit is solved by utilizing the redundancy characteristic of the redundant robot.
In one embodiment, the task function is constructed using the first 8 elements of the current homography matrix, the homography matrix between the current image feature and the desired image feature is solved using at least 4 feature points on the target that can be identified, the homography matrix refers to a 3 × 3 mapping matrix between images of the same object formed on two different imaging planes, and when a certain number of feature points are used to solve the homography matrix, a certain element of the homography matrix can be constrained to obtain a certain result, when a homography matrix of 8 degrees of freedom is solved, the last element of the homography matrix is constrained to be 1:
the first 8 elements e ═ h using homography1,h2,h3,h4,h5,h6,h7,h8]T-[1,0,0,0,1,0,0,0]TAs a function of the task. The essential purpose of control according to the image characteristics is to control the relative pose between the robot and the target. It can be demonstrated that the image feature obtained by the robot tip camera coincides with the expected image feature if and only if the value of the task function is e-0, i.e. the relative pose between the robot tip and the target coincides with the expected value (the proving process is remarked below).
In one embodiment, the complex Jacobian matrix of each period is obtained by online identification, and a controller is constructed by using the complex Jacobian matrix to directly control the robot joint, so that the complex redundant inverse kinematics control problem of the robot can be avoided. The recognition of the Jacobian matrix can use a classic Kalman-Bucy filtering method, and a linear model needs to be established in order to use the method. The jacobian matrix is known to have the following form:
defining a system state vector:
according to the definition of the Jacobian matrix, there are:
e(k)＝e(k-1)+Jimage(k-1)·Δq(k-1)(7b)；
Δq(k)＝[Δq1(k) Δq2(k)…Δqn(k)]T(8b)；
there are the following system equations:
x (k) is the system state at time k, H is the measurement system parameter, i.e. the matrix formed by the angle variations of the joints, w (k) and v (k) represent the process and the measured noise, respectively. At the moment, the Kalman-Bucy filtering algorithm is used for identifying the state vector of the system, so that the on-line identification of the Jacobian matrix is realized, namely:
the covariance difference of the process and measurement noise is Q, R, x (k | k-1) is the result predicted using the previous state and is the optimal result for the previous state, P (k | k-1) is the covariance for x (k | k-1), P (k-1| k-1) is the covariance for x (k-1| k-1), where G (k) is the Kalman filter gain. The jacobian matrix can be reconstructed using the system state vectors identified for each iteration cycle.
In one embodiment, when Jacobian enters a singular interval, each singular value of a composite Jacobian matrix identified in real time is constrained and limited to a reasonable threshold value, so that the servo process is not interrupted due to singularity of the composite Jacobian matrix until the robot smoothly leaves the singular interval.
The Jacobian matrix is easy to generate a strange situation in the process of visual servo, so that the robot must stop working and perform the trial movement again. In order to improve the problem, a singular value filtering method for solving the singularity of the Jacobian matrix in the robot kinematics is introduced into the visual servo. Performing singular value decomposition on the Jacobian matrix identified in each period:
σiis the singular value of the Jacobian matrix, uiAnd viColumn vectors of U and V, respectively. When the jacobian matrix approaches singularity, a singular value or singular values approach 0, which may result in the calculated control amount approaching infinity. Therefore, it is necessary to limit singular values smaller than a certain threshold value, as shown in equation (13 b):
v is the curvature adjustment factor, σ0Is the smallest singular value that can be tolerated. At this time, the pseudo-inverse of the Jacobian matrix is:
the minimum singular value of the Jacobian matrix can be limited through singular value filtering, the joint speed of the robot is controlled to be kept stable, the robot leaves a singular interval of the composite Jacobian matrix as smoothly as possible, and then the expected servo task is continuously completed.
The invention directly controls each joint of the redundant robot, and each joint of the robot has an angle limit which can be reached practically. Therefore, each joint of the robot must be limited to prevent failure due to the fact that the joint reaches the limit in the servo process. In this case, the joint angle is moved away from the limit by performing a self-motion without affecting the main task by redundancy. Gradient projection is the most common method by optimizing the motion
Projection onto the null space of the redundant complex Jacobian matrix to achieve:
wherein I-J+ totallyJtotallyIs a composite Jacobian matrix JtotallyHowever, in the method, the composite Jacobian matrix is an 8 × n matrix, if the projection operator is constructed by directly using the composite Jacobian matrix, the robot has at least 9 degrees of freedom, so that the application of the method loses universality, and the composite Jacobian matrix identified on line is easy to generate singularity, and the gradient projection method in the visual servo is improved by considering the reasons:
discretizing according to an iteration period delta t, and obtaining:
namely, the Jacobian matrix of the robot is used for replacing a composite Jacobian matrix to construct a projection operator, and the motion of the Jacobian matrix of the robot on the null space does not influence the terminal pose of the robot, and the image characteristics are not influenced. The dimensionality of the Jacobian matrix of the robot depends on the terminal degree of freedom and the joint space dimensionality of the robot, so that the redundant robot with any degree of freedom can be optimized; and the Jacobian matrix of the robot is often more accurate to be solved, and the adverse effect on the servo process can not be caused. The optimization motion is shown in equations (18b) (19 b):
the selection or online adjustment of the amplification factor k may refer to a method proposed by luya based on a redundant robot motion optimizability concept, which is not described herein in detail.
For the visual tracking problem of target motion, the improved control system is shown in fig. 2. At this time, the change of the homography matrix element is simultaneously caused by the movement of the target and the movement of the robot. Therefore, the change rate of the homography matrix elements caused by the target motion is included in the identification content, and the expanded composite Jacobian matrix is identified on line:
in the case of recognition, the equations (6b) (8b) (10b) are also modified accordingly:
Δqextend(k)＝[Δq|Δt][Δq1(k) Δq2(k)…Δqn(k) Δt]T(22b)；
in calculating the control amount, it is necessary to take the error of this portion into account. At this time, the system error is:
where Δ t is the length of the iteration cycle, and the control amount at this time is calculated as:
etrack＝e-et(25b)；
in consideration of the limitation of the actual joint velocity, it is sometimes necessary to correct the control amount:
The above methods can be summarized as the following algorithms:
(1b) and (3) calculating a homography matrix between the current robot end camera and the expected feature by using formulas (1b) to (4b) according to the image features obtained by the current robot end camera, constructing a task vector e by using the first 8 elements of the matrix, jumping out of a loop when e is smaller than a threshold value, and otherwise, performing the step (2 b).
(2b) According to the variation y (k) of the task vector and the last cycle task vector and the variation delta q of the joint angle obtained in the last cycle, the formulas (6b) - (11b) are utilized to obtain a composite Jacobian matrix J in the last cycletotallyAnd (6) updating.
(3b) For J obtained in the step (2)totallyPerforming singular value decomposition by using a common equationEquations (12b) to (14b) are obtained by filtering singular values smaller than the threshold value
(4b) According to the pose T of the robot at the moment, the Jacobian matrix J of the robot is obtainedrobot。
(5b) At this time, the joint angle q of the robot is obtained from the equations (18b) and (19b)
(6b) And adjusting the amplification coefficient k of the gradient projection method on line.
(7b) And (4) obtaining the angle change quantity delta q of each joint of the robot according to the results obtained in the steps (3b), (4b), (5b) and (6b) and the formulas (17b), (26b) and (27b), and controlling the robot to move. Returning to the step (1 b).
The method relates to the tracking problem of target motion, and adjusts an algorithm, and comprises the following steps:
(1b) and (3) calculating a homography matrix between the current robot end camera and the expected feature according to the image features obtained by the current robot end camera by using formulas (1b) to (4b), and constructing a task vector e by using the first 8 elements of the matrix.
(2b) According to the variable quantity y (k) of the obtained task vector and the last cycle task vector and the variable quantity delta q of the joint angle obtained in the last cycleextend＝[Δq|Δt]The extended Jacobian matrix J obtained for the previous cycle using equations (7b) (9b) (11b) (21b) (22b) (23b)extendAnd (6) updating.
(3b) J from step (2b) according to the formula (20b)extendExtract J fromtotallyAnd
and performing singular value decomposition on the former, and filtering the singular value smaller than the threshold value by using formulas (12b) to (14b) to obtain
(4b) According to the machine at that timeThe pose T of the person, and a Jacobian matrix J of the robot are obtainedrobot。
(5b) At this time, the joint angle q of the robot is obtained from the equations (18b) and (19b)
(6b) And adjusting the amplification coefficient k of the gradient projection method on line.
(7b) And (4) obtaining the angle change quantity delta q of each joint of the robot according to the results obtained in the steps (3b), (4b), (5b) and (6b) and the formulas (25b), (26b) and (27b), and controlling the robot to move. Returning to the step (1 b).
Remarking:
according to the definition of the homography matrix G, the following:
G＝KHK-1(28)；
k is an internal parameter matrix of the camera, H represents the mapping relation between three-dimensional coordinates of several characteristic points on the target in a camera coordinate system with different poses, namely:
H＝R+tnT(30)；
wherein R and t represent the rotation transformation amount and the translation transformation amount between two poses respectively, and n represents the representation of the unit normal vector of the target plane in one coordinate system. And homography matrix obtained by image characteristics
Is based on a certain scale of a column, namely:
it has been demonstrated that the robot end pose coincides with the expected end pose, i.e. the relative pose between the robot and the target reaches the expected value, if and only if the value of the task function is e-0.
And (3) proving that:
the sufficiency:
when the task function e is 0, it is obvious that
G-I/α, in this case, H-K-1Taking two points P in three-dimensional spaceiAnd PjThe three-dimensional coordinates in the two coordinate systems have the following relationship
And (3) solving the distance between the two points according to the coordinates:
||Fi-Fj||＝||F* i-Fj *||＝||F* i-Fj *||/α(33)；
in different coordinate systems, the distance between two points is constant, α is equal to 1, G is equal to I, H is equal to I, when R is equal to I, t is equal to 0, and the relative pose of the robot and the target reaches the expected value.
The necessity:
when the pose reaches the expected value, R is I, and t is 0. In this case, G ═ I can be obtained. As can be seen from the formula (31),
because the homography matrix is obtained
When, restrain
Thus, it is possible to obtain
In this case, the task function e is 0.
The above-described embodiments of the present invention should not be construed as limiting the scope of the present invention. Any other corresponding changes and modifications made according to the technical idea of the present invention should be included in the protection scope of the claims of the present invention.
Claims (10)
1. A projective homography-based redundant robot uncalibrated vision servo control method is characterized by comprising the following steps:
step one, constructing a task function, namely constructing the task function by using a homography matrix with 8 degrees of freedom for mapping current image characteristics and expected image characteristics;
step two, joint control, namely constructing a controller by utilizing a composite Jacobian matrix, and directly controlling the joints of the robot;
step three, singular value filtering, namely limiting the minimum singular value of the composite Jacobian matrix by using a singular value filtering algorithm, so that the servo process is not interrupted due to singularity of the composite Jacobian matrix until the robot leaves a singularity interval smoothly;
and step four, joint limiting, namely limiting the joints by using a gradient projection method by using the redundancy characteristic of the redundant robot.
2. The method according to claim 1, wherein in step one, the homography matrix is a 3 × 3 mapping matrix between images of the same object formed on two different imaging planes, and when the homography matrix is obtained by using not less than 4 feature points, a certain element of the homography matrix is constrained to obtain a certain result.
3. The projective homography-based redundant robot uncalibrated vision servo control method of claim 1, wherein in the second step, a composite Jacobian matrix of each period is obtained by using online identification, and a controller is constructed by using the composite Jacobian matrix to directly control the robot joints.
4. The method as claimed in claim 1, wherein in step three, when the jacobian enters the singular region, the singular values of the composite jacobian matrix identified in real time are constrained, and the singular values are limited to a reasonable threshold value, so that the servo process is not interrupted by the singularity of the composite jacobian matrix until the robot smoothly leaves the singular region.
5. The projective homography-based redundant robot uncalibrated vision servo control method of claim 1, wherein in the fourth step, the method further comprises an on-line adjustment of the coefficient k of the gradient projection, so that the joint angle does not exceed the limit, and the servo process is performed smoothly.
6. A redundant robot uncalibrated vision servo control system based on projective homography is characterized by comprising:
the task function constructing unit is used for mapping the homography matrix with 8 degrees of freedom of the current image characteristic and the expected image characteristic to construct a task function;
the joint control unit is used for constructing a controller by utilizing the composite Jacobian matrix and directly controlling the joints of the robot;
the singular value filtering unit is used for limiting the minimum singular value of the composite Jacobian matrix by using a singular value filtering algorithm, so that the servo process is not interrupted due to singularity of the composite Jacobian matrix until the robot leaves a singularity interval smoothly;
and the joint limiting unit is used for limiting the joints by using a gradient projection method by utilizing the redundancy characteristic of the redundant robot.
7. The projective homography based redundant robot uncalibrated vision servo control system of claim 6, wherein in the constructed task function unit, the homography matrix is a 3 × 3 mapping matrix between images of the same object formed on two different imaging planes, and when the homography matrix is obtained by using not less than 4 feature points, a certain element of the homography matrix is constrained to obtain a certain result.
8. The projective homography-based redundant robot uncalibrated vision servo-control system of claim 6, wherein the joint control unit obtains a composite Jacobian matrix for each period by online identification, and constructs a controller by using the composite Jacobian matrix to directly control the robot joints.
9. The projective homography-based redundant robot uncalibrated visual servo control system of claim 6, wherein the singular value filtering unit constrains the singular values of the composite Jacobian matrix identified in real time as the Jacobian enters the singular region, limiting the singular values to a reasonable threshold value, such that the servo process is not interrupted by the singularity of the composite Jacobian matrix until the robot leaves the singular region smoothly.
10. The projective homography-based redundant robot uncalibrated vision servo control system of claim 6, wherein the joint limiting unit further comprises an on-line adjustment of a coefficient k of gradient projection, so that a joint angle does not exceed a limit, and a servo process is performed smoothly.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811539384.9A CN111319037A (en) | 2018-12-17 | 2018-12-17 | Redundant robot uncalibrated visual servo control method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811539384.9A CN111319037A (en) | 2018-12-17 | 2018-12-17 | Redundant robot uncalibrated visual servo control method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111319037A true CN111319037A (en) | 2020-06-23 |
Family
ID=71163132
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811539384.9A Pending CN111319037A (en) | 2018-12-17 | 2018-12-17 | Redundant robot uncalibrated visual servo control method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111319037A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114643583A (en) * | 2022-05-05 | 2022-06-21 | 中国科学院合肥物质科学研究院 | Method for avoiding singular points of redundant mechanical arm |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140107840A1 (en) * | 2009-10-30 | 2014-04-17 | Honda Motor Co., Ltd. | Information processing method, apparatus, and computer readable medium |
| CN104392081A (en) * | 2014-06-30 | 2015-03-04 | 管小清 | Redundant mechanical arm moving obstacle avoiding algorithm |
| CN105719314A (en) * | 2016-01-30 | 2016-06-29 | 西北工业大学 | Homography estimation and extended Kalman filter based localization method for unmanned aerial vehicle (UAV) |
| CN107097231A (en) * | 2017-07-06 | 2017-08-29 | 哈尔滨工业大学深圳研究生院 | A kind of concentric tube robot precise motion control method of view-based access control model servo |
| CN108628310A (en) * | 2018-04-27 | 2018-10-09 | 华中科技大学 | Robot based on projection homography matrix is without calibration visual servo method for planning track |
| CN111360820A (en) * | 2020-02-18 | 2020-07-03 | 哈尔滨工业大学 | Distance space and image feature space fused hybrid visual servo method |
-
2018
- 2018-12-17 CN CN201811539384.9A patent/CN111319037A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140107840A1 (en) * | 2009-10-30 | 2014-04-17 | Honda Motor Co., Ltd. | Information processing method, apparatus, and computer readable medium |
| CN104392081A (en) * | 2014-06-30 | 2015-03-04 | 管小清 | Redundant mechanical arm moving obstacle avoiding algorithm |
| CN105719314A (en) * | 2016-01-30 | 2016-06-29 | 西北工业大学 | Homography estimation and extended Kalman filter based localization method for unmanned aerial vehicle (UAV) |
| CN107097231A (en) * | 2017-07-06 | 2017-08-29 | 哈尔滨工业大学深圳研究生院 | A kind of concentric tube robot precise motion control method of view-based access control model servo |
| CN108628310A (en) * | 2018-04-27 | 2018-10-09 | 华中科技大学 | Robot based on projection homography matrix is without calibration visual servo method for planning track |
| CN111360820A (en) * | 2020-02-18 | 2020-07-03 | 哈尔滨工业大学 | Distance space and image feature space fused hybrid visual servo method |
Non-Patent Citations (2)
| Title |
|---|
| 王耀兵: "《空间机器人》", 31 May 2018, 北京理工大学出版社 * |
| 陶波: "机器人无标定视觉伺服控制研究进展", 《力学学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114643583A (en) * | 2022-05-05 | 2022-06-21 | 中国科学院合肥物质科学研究院 | Method for avoiding singular points of redundant mechanical arm |
| CN114643583B (en) * | 2022-05-05 | 2023-11-10 | 中国科学院合肥物质科学研究院 | Redundancy mechanical arm singular point avoiding method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jabbari et al. | An adaptive scheme for image-based visual servoing of an underactuated UAV | |
| Qiu et al. | Visual servo tracking of wheeled mobile robots with unknown extrinsic parameters | |
| Shademan et al. | Robust jacobian estimation for uncalibrated visual servoing | |
| CN107627303B (en) | PD-SMC control method of visual servo system based on eye-on-hand structure | |
| Assa et al. | A robust vision-based sensor fusion approach for real-time pose estimation | |
| Palmieri et al. | A Comparison between Position-Based and Image-Based Dynamic Visual Servoings in the Control of a Translating Parallel Manipulator. | |
| CN105096341B (en) | Mobile robot position and orientation estimation method based on trifocal tensor and key frame strategy | |
| CN109176531A (en) | A kind of tandem type robot kinematics calibration method and system | |
| EP1548534A2 (en) | Method and apparatus for using rotational movement amount of mobile device and computer-readable recording medium for storing computer program | |
| Grigorescu et al. | Robust camera pose and scene structure analysis for service robotics | |
| CN112000135B (en) | Three-axis holder visual servo control method based on human face maximum temperature point characteristic feedback | |
| Silveira | On intensity-based nonmetric visual servoing | |
| Ruifang et al. | Line-based monocular graph SLAM | |
| CN113910218A (en) | Robot calibration method and device based on kinematics and deep neural network fusion | |
| CN111319037A (en) | Redundant robot uncalibrated visual servo control method and system | |
| Lopez-Nicolas et al. | Nonholonomic epipolar visual servoing | |
| Song et al. | On-line stable evolutionary recognition based on unit quaternion representation by motion-feedforward compensation | |
| CN109582026B (en) | Autonomous underwater vehicle path tracking control method based on self-tuning sight line and drift angle compensation | |
| Parikh et al. | Homography based visual servo control with scene reconstruction | |
| Fahimi et al. | An alternative closed-loop vision-based control approach for Unmanned Aircraft Systems with application to a quadrotor | |
| Lopez-Nicolas et al. | Homography-based visual control of nonholonomic vehicles | |
| Aranda et al. | Omnidirectional visual homing using the 1D trifocal tensor | |
| CN113211433A (en) | Separated visual servo control method based on composite characteristics | |
| CN116533237A (en) | Fuzzy variable impedance control method integrating attitude measurement and one-dimensional force sensor information | |
| Spica et al. | Active structure from motion for spherical and cylindrical targets |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200623 |