CN114355953B - High-precision control method and system of multi-axis servo system based on machine vision - Google Patents
High-precision control method and system of multi-axis servo system based on machine vision Download PDFInfo
- Publication number
- CN114355953B CN114355953B CN202210271120.XA CN202210271120A CN114355953B CN 114355953 B CN114355953 B CN 114355953B CN 202210271120 A CN202210271120 A CN 202210271120A CN 114355953 B CN114355953 B CN 114355953B
- Authority
- CN
- China
- Prior art keywords
- information
- acquiring
- industrial robot
- target
- axis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Manipulator (AREA)
Abstract
The invention discloses a high-precision control method and a high-precision control system of a multi-axis servo system based on machine vision, wherein the method comprises the following steps: acquiring visual image information containing a target workpiece, judging the definition of the visual image information, and preprocessing the visual image information if the definition reaches a preset threshold value; the method comprises the steps of carrying out image segmentation on preprocessed visual image information, carrying out feature extraction and generating feature point image position information, carrying out coordinate transformation on the feature point image position information, leading the coordinate information after the coordinate transformation into a multi-axis servo system to obtain a planned path by combining kinematic parameters of the industrial robot, obtaining a target position point from the planned path, obtaining position errors of all axes of the industrial robot at the target position point, and realizing error compensation of the planned path according to the position errors. According to the invention, the multi-axis industrial robot is controlled in high precision through machine vision, and the motion control efficiency and the positioning precision of the multi-axis industrial robot are improved.
Description
Technical Field
The invention relates to the technical field of automation control, in particular to a high-precision control method and system of a multi-axis servo system based on machine vision.
Background
Nowadays, science and technology develops at a high speed, and robot, automatic control, intelligent technology constantly develop maturity, and industrial robot's popularization has not only liberated the labour, has still accelerated production speed, has improved production quality, and industrial robot based on vision uses the research focus that realizes industrial automation in recent years, and the industrial robot of collocation vision system can effectively improve the intelligent level of robot. At present, most industrial robots execute set action instructions in an off-line programming or teaching mode, and set placement and accurate position teaching by a teaching person to grasp a workpiece to work. However, this approach has significant limitations. When the external environment or the workpiece state changes and the grabbing condition cannot be met, the robot cannot adapt to the change, and the task is interrupted or fails.
In order to control a multi-axis industrial robot with high precision, a system needs to be developed to cooperate with the multi-axis industrial robot for realizing, and the system is realized by: acquiring visual image information, and preprocessing the visual image information; the method comprises the steps of carrying out image segmentation on preprocessed visual image information, carrying out feature extraction and generating feature point image position information, carrying out coordinate transformation on the feature point image position information, leading the coordinate information after the coordinate transformation into a multi-axis servo system to obtain a planned path by combining kinematic parameters of the industrial robot, obtaining target position points from the planned path, obtaining position errors of all axes of the industrial robot at the target position points, and realizing error compensation of the planned path according to the position errors. How to carry out high-precision control on a multi-axis industrial robot through visual image information in the implementation process of the system and how to realize the compensation of position errors in the control process are all problems which need to be solved urgently.
Disclosure of Invention
In order to solve the technical problem, the invention provides a high-precision control method and a high-precision control system of a multi-axis servo system based on machine vision.
The invention provides a high-precision control method of a multi-axis servo system based on machine vision, which comprises the following steps:
acquiring visual image information containing a target workpiece, and judging the definition of the visual image information;
if the definition of the visual image information reaches a preset threshold value, preprocessing the visual image information;
performing image segmentation on the preprocessed visual image information, performing feature extraction and generating feature point image position information;
coordinate transformation is carried out on the feature point image position information, the coordinate information after the coordinate transformation is led into a multi-axis servo system to be combined with kinematic parameters of a multi-axis industrial robot to obtain a planned path, and a target workpiece is operated according to the planned path;
and acquiring a target position point in the planned path, acquiring the position error of each axis of the industrial robot at the target position point, and realizing the error compensation of the planned path according to the position error.
In the scheme, the definition judgment of the visual image information is specifically as follows:
acquiring visual image information, and measuring the focusing degree of each pixel point in the visual image information by using a focusing evaluation operator;
generating a focus evaluation curve through the focus degree evaluation value after focus evaluation of each pixel point, and comparing the focus evaluation curve with an ideal focus evaluation curve to generate a curve offset error;
judging whether the curve offset error is smaller than a preset offset error threshold value or not, and if so, indicating that the image definition meets a preset standard;
and if not, controlling to obtain an in-focus position by a golden section searching method, and carrying out automatic focusing of the image sensor according to the in-focus position.
In this scheme, the image segmentation is performed on the preprocessed visual image information, the feature extraction is performed, and feature point image position information is generated, specifically:
acquiring the gray value of each pixel in the preprocessed visual image information, and calculating the amplitude and the direction of the gradient by using a differential operator;
comparing the gray value of the target pixel with the adjacent pixels in the gradient direction of the target pixel, if the gray value of the target pixel is greater than that of the adjacent pixels, setting the target pixel as an edge, otherwise, setting the target pixel as not an edge;
performing image segmentation according to an edge judgment result in the visual image information, extracting characteristic information of the segmented visual image information, and identifying a target workpiece according to the characteristic information;
acquiring contour information of a target workpiece, extracting feature points according to the contour information of the target workpiece, and acquiring image position information of the feature points.
In the scheme, the method comprises the following steps of leading coordinate information after coordinate transformation into a multi-axis servo system to obtain a planned path in combination with kinematic parameters of a multi-axis industrial robot, and operating a target workpiece according to the planned path, specifically:
establishing a path planning model of the multi-axis industrial robot based on a neural network and a particle swarm algorithm, and performing initialization training;
acquiring initial position information and initial joint angle information of a point where an end effector of the multi-axis industrial robot is located, and setting kinematic constraint and a fitness function of the multi-axis industrial robot;
initializing a particle population, and randomly giving speed and position information to the particles;
obtaining the current speed of the particles, comparing the current speed with the constraint, judging the advantages and disadvantages of the particles according to the fitness value if the constraint is met, removing the particles if the constraint is not met, and performing iterative training on the removed particles until the constraint is met;
and after the particle speed and position information is updated for a plurality of times, acquiring the optimal position searched by each particle and the optimal position in all the particles, and acquiring the planned path of the multi-axis industrial robot according to the optimal positions.
In this scheme, still include:
acquiring time information from an initial position to characteristic point image position information of each shaft end effector of the multi-shaft robot;
comparing and analyzing the time information of each axis to generate time deviation information, if the deviation information is greater than a preset time deviation threshold value, acquiring mechanical arm information corresponding to the time deviation information, and generating path correction information according to the mechanical arm information and the time deviation information;
and updating the planned path according to the path correction information.
In this scheme, the acquisition industrial robot each axle be in the position error of target position point, according to the error compensation of planning the route is realized to the position error, specifically do:
acquiring a target position point from the planned path, calibrating the target position point in a camera coordinate system to obtain an image position coordinate of the target position point, and transferring the image position coordinate to a base coordinate system of the multi-axis industrial robot to serve as theoretical position coordinate information;
establishing a mechanical arm connecting rod coordinate system by a D-H parameter method according to the degree of freedom of the multi-axis industrial robot and the position and posture relation of two adjacent mechanical arms of the multi-axis mechanical arm in a camera coordinate system;
acquiring relative position information of a point of each shaft end effector of the multi-shaft industrial robot in a mechanical arm connecting rod coordinate system of each shaft of the industrial robot, and acquiring a pose matrix of the end effector according to the relative position information;
acquiring actual position coordinate information of each axis in a target position point in a multi-axis industrial robot base coordinate system through the pose matrix, and comparing the actual position coordinate information with the theoretical position coordinate information to generate a position error;
generating a planned path compensation quantity according to the position error, acquiring a track path compensation point by combining period information, and generating error correction information according to the planned path compensation quantity and the planned path compensation point;
and carrying out error compensation on the planned path according to the error correction information.
The second aspect of the present invention also provides a high-precision control system for a multi-axis servo system based on machine vision, the system comprising: the high-precision control method program of the multi-axis servo system based on the machine vision realizes the following steps when being executed by the processor:
acquiring visual image information containing a target workpiece, and judging the definition of the visual image information;
if the definition of the visual image information reaches a preset threshold value, preprocessing the visual image information;
performing image segmentation on the preprocessed visual image information, performing feature extraction and generating feature point image position information;
coordinate transformation is carried out on the feature point image position information, the coordinate information after the coordinate transformation is led into a multi-axis servo system to be combined with kinematic parameters of a multi-axis industrial robot to obtain a planned path, and a target workpiece is operated according to the planned path;
and acquiring a target position point in the planned path, acquiring the position error of each axis of the industrial robot at the target position point, and realizing the error compensation of the planned path according to the position error.
In the scheme, the definition judgment of the visual image information is specifically as follows:
acquiring visual image information, and measuring the focusing degree of each pixel point in the visual image information by using a focusing evaluation operator;
generating a focus evaluation curve through the focus degree evaluation value after focus evaluation is carried out on each pixel point, and comparing the focus evaluation curve with an ideal focus evaluation curve to generate a curve offset error;
judging whether the curve offset error is smaller than a preset offset error threshold value or not, and if so, indicating that the image definition meets a preset standard;
and if not, controlling to obtain a focusing position by a golden section searching method, and carrying out automatic focusing on the image sensor according to the focusing position.
In this embodiment, the image segmentation is performed on the preprocessed visual image information, the feature extraction is performed, and feature point image position information is generated, specifically:
acquiring the gray value of each pixel in the preprocessed visual image information, and calculating the amplitude and the direction of the gradient by using a differential operator;
comparing the gray value of the target pixel with the adjacent pixels in the gradient direction of the target pixel, if the gray value of the target pixel is greater than that of the adjacent pixels, setting the target pixel as an edge, otherwise, setting the target pixel as not an edge;
performing image segmentation according to an edge judgment result in the visual image information, extracting characteristic information of the segmented visual image information, and identifying a target workpiece according to the characteristic information;
acquiring contour information of a target workpiece, extracting feature points according to the contour information of the target workpiece, and acquiring image position information of the feature points.
In the scheme, the method comprises the following steps of leading coordinate information after coordinate transformation into a multi-axis servo system to obtain a planned path in combination with kinematic parameters of a multi-axis industrial robot, and operating a target workpiece according to the planned path, specifically:
establishing a path planning model of the multi-axis industrial robot based on a neural network and a particle swarm algorithm, and performing initialization training;
acquiring initial position information and initial joint angle information of a point where an end effector of the multi-axis industrial robot is located, and setting kinematic constraint and fitness function of the multi-axis industrial robot;
initializing a particle population, and randomly giving speed and position information to the particles;
obtaining the current speed of the particles, comparing the current speed with the constraint, judging the advantages and disadvantages of the particles according to the fitness value if the constraint is met, removing the particles if the constraint is not met, and performing iterative training on the removed particles until the constraint is met;
and after the particle speed and position information is updated for a plurality of times, acquiring the optimal position searched by each particle and the optimal position in all the particles, and acquiring the planned path of the multi-axis industrial robot according to the optimal positions.
In this scheme, still include:
acquiring time information from an initial position to characteristic point image position information of each shaft end effector of the multi-shaft robot;
comparing and analyzing the time information of each axis to generate time deviation information, if the deviation information is greater than a preset time deviation threshold value, acquiring mechanical arm information corresponding to the time deviation information, and generating path correction information according to the mechanical arm information and the time deviation information;
and updating the planned path according to the path correction information.
In this scheme, the obtaining of the position error of each axis of the industrial robot at the target position point, and the error compensation of the planned path according to the position error are specifically as follows:
acquiring a target position point from the planned path, calibrating the target position point in a camera coordinate system to obtain an image position coordinate of the target position point, and transferring the image position coordinate to a base coordinate system of the multi-axis industrial robot to serve as theoretical position coordinate information;
establishing a mechanical arm connecting rod coordinate system by a D-H parameter method according to the degree of freedom of the multi-axis industrial robot and the position and posture relation of two adjacent mechanical arms of the multi-axis mechanical arm in a camera coordinate system;
acquiring relative position information of a point of each shaft end effector of the multi-shaft industrial robot in a mechanical arm connecting rod coordinate system of each shaft of the industrial robot, and acquiring a pose matrix of the end effector according to the relative position information;
acquiring actual position coordinate information of each axis in a target position point in a multi-axis industrial robot base coordinate system through the pose matrix, and comparing the actual position coordinate information with the theoretical position coordinate information to generate a position error;
generating a planned path compensation quantity according to the position error, acquiring a track path compensation point by combining period information, and generating error correction information according to the planned path compensation quantity and the planned path compensation point;
and carrying out error compensation on the planned path according to the error correction information.
The third aspect of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a program of a high-precision control method for a multi-axis servo system based on machine vision, and when the program of the high-precision control method for the multi-axis servo system based on machine vision is executed by a processor, the steps of the high-precision control method for the multi-axis servo system based on machine vision as described in any one of the above are implemented.
The invention discloses a high-precision control method and a high-precision control system of a multi-axis servo system based on machine vision, wherein the method comprises the following steps: acquiring visual image information, judging the definition of the visual image information, and preprocessing the visual image information if the definition of the visual image information reaches a preset threshold value; the method comprises the steps of carrying out image segmentation on preprocessed visual image information, carrying out feature extraction and generating feature point image position information, carrying out coordinate transformation on the feature point image position information, leading the coordinate information after the coordinate transformation into a multi-axis servo system to obtain a planned path by combining kinematic parameters of the industrial robot, obtaining target position points from the planned path, obtaining position errors of all axes of the industrial robot at the target position points, and realizing error compensation of the planned path according to the position errors. According to the invention, the multi-axis industrial robot is controlled in high precision through machine vision, and the motion control efficiency and the positioning precision of the multi-axis industrial robot are improved.
Drawings
FIG. 1 is a flow chart of a high-precision control method of a multi-axis servo system based on machine vision according to the invention;
fig. 2 shows a block diagram of a high-precision control system of a multi-axis servo system based on machine vision according to the present invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced otherwise than as specifically described herein and, therefore, the scope of the present invention is not limited by the specific embodiments disclosed below.
Fig. 1 shows a flow chart of a high-precision control method of a multi-axis servo system based on machine vision according to the invention.
As shown in fig. 1, a first aspect of the present invention provides a high-precision control method for a multi-axis servo system based on machine vision, including:
s102, acquiring visual image information containing a target workpiece, and judging the definition of the visual image information;
s104, if the definition of the visual image information reaches a preset threshold value, preprocessing the visual image information;
s106, carrying out image segmentation on the preprocessed visual image information, carrying out feature extraction and generating feature point image position information;
s108, carrying out coordinate transformation on the feature point image position information, importing the coordinate information subjected to the coordinate transformation into a multi-axis servo system to obtain a planned path in combination with kinematic parameters of a multi-axis industrial robot, and operating a target workpiece according to the planned path;
and S110, acquiring a target position point from the planned path, acquiring the position error of each axis of the industrial robot at the target position point, and realizing error compensation of the planned path according to the position error.
It should be noted that, the determining the sharpness of the visual image information specifically includes:
acquiring visual image information, and measuring the focusing degree of each pixel point in the visual image information by using a focusing evaluation operator; generating a focus evaluation curve through the focus degree evaluation value after focus evaluation of each pixel point, and comparing the focus evaluation curve with an ideal focus evaluation curve to generate a curve offset error; judging whether the curve offset error is smaller than a preset offset error threshold value or not, and if so, indicating that the image definition meets a preset standard; and if not, controlling to obtain a focusing position by a golden section searching method, and carrying out automatic focusing on the image sensor according to the focusing position.
The focusing degree of each pixel point is measured by using a focusing evaluation operator, the higher the focusing degree is, the larger the ashing degree is, the larger the edge sharpness degree is, and meanwhile, the larger the corresponding high-frequency information is, so that the detail information can be better embodied by a single pixel point in the subsequent preprocessing of the visual image information. Commonly used focus evaluation operators include: based on gradient operation, Laplace transform, wavelet transform and other methods. The golden section search method is a search method in which the length of a search interval is reduced continuously according to the golden ratio
Search range of
Two golden section points are selected
And
comparison of
And
image evaluation values obtained at two positions
And
if, if
Change the search range to
If, if
Change the search range to
And reducing the search range to approach the focus-aligning position until the search range reaches the unit step length requirement, and finishing automatic focusing.
The image segmentation of the visual image information after the preprocessing, the feature extraction, and the feature point image position information generation are specifically as follows:
acquiring the gray value of each pixel in the preprocessed visual image information, and calculating the amplitude and the direction of the gradient by using a differential operator; comparing the gray value of the target pixel with the adjacent pixels in the gradient direction of the target pixel, if the gray value of the target pixel is greater than that of the adjacent pixels, setting the target pixel as an edge, otherwise, setting the target pixel as not an edge; performing image segmentation according to an edge judgment result in the visual image information, extracting feature information of the segmented visual image information, and identifying a target workpiece according to the feature information; acquiring contour information of a target workpiece, extracting feature points according to the contour information of the target workpiece, and acquiring image position information of the feature points.
It should be noted that the importing of coordinate information after coordinate transformation into a multi-axis servo system and combining with kinematic parameters of a multi-axis industrial robot to obtain a planned path, and operating a target workpiece according to the planned path specifically includes:
establishing a path planning model of the multi-axis industrial robot based on a neural network and a particle swarm algorithm, and performing initialization training;
acquiring initial position information and initial joint angle information of a point where an end effector of the multi-axis industrial robot is located, and setting kinematic constraint and fitness function of the multi-axis industrial robot;
initializing a particle population, and randomly giving speed and position information to the particles;
obtaining the current speed of the particles, comparing the current speed with the constraint, judging the advantages and disadvantages of the particles according to the fitness value if the constraint is met, removing the particles if the constraint is not met, and performing iterative training on the removed particles until the constraint is met;
and after the particle speed and position information is updated for a plurality of times, acquiring the optimal position searched by each particle and the optimal position in all the particles, and acquiring the planned path of the multi-axis industrial robot according to the optimal positions.
The present invention further includes a step of performing synchronous correction of each axis based on time information from an initial position to feature point image position information of each axis end effector:
acquiring time information from an initial position to characteristic point image position information of each shaft end effector of the multi-shaft robot;
comparing and analyzing the time information of each axis to generate time deviation information, if the deviation information is greater than a preset time deviation threshold value, acquiring mechanical arm information corresponding to the time deviation information, and generating path correction information according to the mechanical arm information and the time deviation information;
and updating the planned path according to the path correction information.
It should be noted that the obtaining of the position error of each axis of the industrial robot at the target position point and the implementation of the error compensation of the planned path according to the position error specifically include:
acquiring a target position point from the planned path, calibrating the target position point in a camera coordinate system to obtain an image position coordinate of the target position point, and transferring the image position coordinate to a base coordinate system of the multi-axis industrial robot to serve as theoretical position coordinate information;
establishing a mechanical arm connecting rod coordinate system by a D-H parameter method according to the degree of freedom of the multi-axis industrial robot and the position and posture relation of two adjacent mechanical arms of the multi-axis mechanical arm in a camera coordinate system;
acquiring relative position information of a point of each shaft end effector of the multi-shaft industrial robot in a mechanical arm connecting rod coordinate system of each shaft of the industrial robot, and acquiring a pose matrix of the end effector according to the relative position information;
acquiring actual position coordinate information of each axis in a target position point in a multi-axis industrial robot base coordinate system through the pose matrix, and comparing the actual position coordinate information with the theoretical position coordinate information to generate a position error;
generating a planned path compensation quantity according to the position error, and generating error correction information according to the planned path compensation quantity and the planned path compensation point when a track path compensation point is obtained by combining period information;
and carrying out error compensation on the planned path according to the error correction information.
According to the embodiment of the invention, the invention further comprises the steps of carrying out non-contact identification inspection on the target workpiece through the image sensor, and checking whether the target workpiece is qualified, wherein the steps are as follows:
acquiring target workpiece image information of a target workpiece after processing operation of the multi-axis industrial robot, and acquiring central point coordinate information of the target workpiece;
presetting a target workpiece placing position area, acquiring distance information from the central point coordinate information to the edge of the target workpiece placing position area, and judging whether the position of the target workpiece is qualified or not according to the distance information;
acquiring expected processing information of a target workpiece, preprocessing the target processing image information, acquiring characteristic information according to the preprocessed target workpiece image information, comparing the expected processing information with the characteristic information, and calculating to generate similarity information;
judging whether the similarity information is larger than a similarity threshold value or not, if so, proving that the target workpiece is qualified for machining, and if not, marking the target workpiece to obtain the deviation between the marked target workpiece and the expected machining information;
and generating feedback information according to the deviation of the marked target workpiece and the expected processing information, and feeding back a vision system and a path planning system of the multi-axis industrial robot.
Fig. 2 shows a block diagram of a high-precision control system of a multi-axis servo system based on machine vision according to the present invention.
The second aspect of the present invention also provides a high-precision control system 2 of a multi-axis servo system based on machine vision, the system comprising: a memory 21 and a processor 22, wherein the memory includes a high-precision control method program of a multi-axis servo system based on machine vision, and the high-precision control method program of the multi-axis servo system based on machine vision realizes the following steps when being executed by the processor:
acquiring visual image information containing a target workpiece, and judging the definition of the visual image information;
if the definition of the visual image information reaches a preset threshold value, preprocessing the visual image information;
carrying out image segmentation on the preprocessed visual image information, carrying out feature extraction and generating feature point image position information;
coordinate transformation is carried out on the feature point image position information, the coordinate information after the coordinate transformation is led into a multi-axis servo system to be combined with kinematic parameters of a multi-axis industrial robot to obtain a planned path, and a target workpiece is operated according to the planned path;
and acquiring a target position point in the planned path, acquiring the position error of each axis of the industrial robot at the target position point, and realizing the error compensation of the planned path according to the position error.
It should be noted that, the determining the sharpness of the visual image information specifically includes:
acquiring visual image information, and measuring the focusing degree of each pixel point in the visual image information by using a focusing evaluation operator; generating a focus evaluation curve through the focus degree evaluation value after focus evaluation of each pixel point, and comparing the focus evaluation curve with an ideal focus evaluation curve to generate a curve offset error; judging whether the curve offset error is smaller than a preset offset error threshold value or not, and if so, indicating that the image definition meets a preset standard; and if not, controlling to obtain an in-focus position by a golden section searching method, and carrying out automatic focusing of the image sensor according to the in-focus position.
The focusing degree of each pixel point is measured by using a focusing evaluation operator, the higher the focusing degree is, the larger the ashing degree is, the larger the edge sharpness degree is, and meanwhile, the larger the corresponding high-frequency information is, so that the detail information can be better embodied by a single pixel point in the subsequent preprocessing of the visual image information. Commonly used focus evaluation operators include: based on gradient operation, Laplace transform, wavelet transform and other methods. The golden section search method is a search method in which the length of a search interval is reduced continuously according to the golden ratio
Search ofRange
Two golden section points are selected
And
comparison of
And
image evaluation values obtained at two positions
And
if, if
Then change the search range to
If, if
Change the search range to
And reducing the search range to approach the focus-aligning position until the search range reaches the unit step length requirement, and finishing automatic focusing.
The image segmentation of the visual image information after the preprocessing, the feature extraction, and the feature point image position information generation are specifically as follows:
acquiring the gray value of each pixel in the preprocessed visual image information, and calculating the amplitude and the direction of the gradient by using a differential operator;
comparing the gray value of the target pixel with the adjacent pixels in the gradient direction of the target pixel, if the gray value of the target pixel is greater than that of the adjacent pixels, setting the target pixel as an edge, otherwise, setting the target pixel as not an edge;
performing image segmentation according to an edge judgment result in the visual image information, extracting feature information of the segmented visual image information, and identifying a target workpiece according to the feature information;
acquiring contour information of a target workpiece, extracting feature points according to the contour information of the target workpiece, and acquiring image position information of the feature points.
It should be noted that the importing of coordinate information after coordinate transformation into a multi-axis servo system and combining with kinematic parameters of a multi-axis industrial robot to obtain a planned path, and operating a target workpiece according to the planned path specifically includes:
establishing a path planning model of the multi-axis industrial robot based on a neural network and a particle swarm algorithm, and performing initialization training;
acquiring initial position information and initial joint angle information of a point where an end effector of the multi-axis industrial robot is located, and setting kinematic constraint and fitness function of the multi-axis industrial robot;
initializing a particle population, and randomly giving particle speed and position information;
obtaining the current speed of the particles, comparing the current speed with the constraint, judging the advantages and disadvantages of the particles according to the fitness value if the constraint is met, removing the particles if the constraint is not met, and performing iterative training on the removed particles until the constraint is met;
and after the particle speed and position information is updated for a plurality of times, acquiring the optimal position searched by each particle and the optimal position in all the particles, and acquiring the planned path of the multi-axis industrial robot according to the optimal positions.
The present invention further includes a step of performing synchronous correction of each axis based on time information from an initial position to feature point image position information of each axis end effector:
acquiring time information from an initial position to characteristic point image position information of each shaft end effector of the multi-shaft robot;
comparing and analyzing the time information of each axis to generate time deviation information, if the deviation information is greater than a preset time deviation threshold value, acquiring mechanical arm information corresponding to the time deviation information, and generating path correction information according to the mechanical arm information and the time deviation information;
and updating the planned path according to the path correction information.
It should be noted that the obtaining of the position error of each axis of the industrial robot at the target position point and the implementation of the error compensation of the planned path according to the position error specifically include:
acquiring a target position point from the planned path, calibrating the target position point in a camera coordinate system to obtain an image position coordinate of the target position point, and transferring the image position coordinate to a base coordinate system of the multi-axis industrial robot to serve as theoretical position coordinate information;
establishing a mechanical arm connecting rod coordinate system by a D-H parameter method according to the degree of freedom of the multi-axis industrial robot and the position and posture relation of two adjacent mechanical arms of the multi-axis mechanical arm in a camera coordinate system;
acquiring relative position information of a point of each shaft end effector of the multi-shaft industrial robot in a mechanical arm connecting rod coordinate system of each shaft of the industrial robot, and acquiring a pose matrix of the end effector according to the relative position information;
acquiring actual position coordinate information of each axis in a target position point in a multi-axis industrial robot base coordinate system through the pose matrix, and comparing the actual position coordinate information with the theoretical position coordinate information to generate a position error;
generating a planned path compensation quantity according to the position error, acquiring a track path compensation point by combining period information, and generating error correction information according to the planned path compensation quantity and the planned path compensation point;
and performing error compensation on the planned path according to the error correction information.
According to the embodiment of the invention, the invention further comprises the steps of carrying out non-contact identification inspection on the target workpiece through the image sensor, and checking whether the target workpiece is qualified, wherein the steps are as follows:
acquiring target workpiece image information of a target workpiece after processing operation of the multi-axis industrial robot, and acquiring central point coordinate information of the target workpiece;
presetting a target workpiece placing position area, acquiring distance information from the central point coordinate information to the edge of the target workpiece placing position area, and judging whether the position of the target workpiece is qualified or not according to the distance information;
acquiring expected processing information of a target workpiece, preprocessing the target processing image information, acquiring characteristic information according to the preprocessed target workpiece image information, comparing the expected processing information with the characteristic information, and calculating to generate similarity information;
judging whether the similarity information is greater than a similarity threshold value or not, if so, proving that the target workpiece is qualified for machining, and if not, marking the target workpiece to obtain the deviation between the marked target workpiece and the expected machining information;
and generating feedback information according to the deviation of the marked target workpiece and the expected processing information, and feeding back a vision system and a path planning system of the multi-axis industrial robot.
The third aspect of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a program of a high-precision control method for a multi-axis servo system based on machine vision, and when the program of the high-precision control method for the multi-axis servo system based on machine vision is executed by a processor, the steps of the high-precision control method for the multi-axis servo system based on machine vision as described in any one of the above are implemented.
The invention discloses a high-precision control method and a high-precision control system of a multi-axis servo system based on machine vision, wherein the method comprises the following steps: acquiring visual image information, judging the definition of the visual image information, and preprocessing the visual image information if the definition of the visual image information reaches a preset threshold value; the method comprises the steps of carrying out image segmentation on preprocessed visual image information, carrying out feature extraction and generating feature point image position information, carrying out coordinate transformation on the feature point image position information, leading the coordinate information after the coordinate transformation into a multi-axis servo system to obtain a planned path by combining kinematic parameters of the industrial robot, obtaining target position points from the planned path, obtaining position errors of all axes of the industrial robot at the target position points, and realizing error compensation of the planned path according to the position errors. According to the invention, the multi-axis industrial robot is controlled in high precision through machine vision, and the motion control efficiency and the positioning precision of the multi-axis industrial robot are improved.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps of implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer-readable storage medium, and when executed, executes the steps including the method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (6)
1. A high-precision control method of a multi-axis servo system based on machine vision is characterized by comprising the following steps:
acquiring visual image information containing a target workpiece, and judging the definition of the visual image information;
if the definition of the visual image information reaches a preset threshold value, preprocessing the visual image information;
carrying out image segmentation on the preprocessed visual image information, carrying out feature extraction and generating feature point image position information;
coordinate transformation is carried out on the feature point image position information, the coordinate information after the coordinate transformation is led into a multi-axis servo system to be combined with kinematic parameters of a multi-axis industrial robot to obtain a planned path, and a target workpiece is operated according to the planned path;
acquiring a target position point from the planned path, acquiring the position error of each axis of the multi-axis industrial robot at the target position point, and realizing error compensation of the planned path according to the position error;
the method comprises the following steps of leading coordinate information after coordinate transformation into a multi-axis servo system, combining kinematic parameters of a multi-axis industrial robot to obtain a planned path, and operating a target workpiece according to the planned path, wherein the method specifically comprises the following steps:
establishing a path planning model of the multi-axis industrial robot based on a neural network and a particle swarm algorithm, and performing initialization training;
acquiring initial position information and initial joint angle information of a point where an end effector of the multi-axis industrial robot is located, and setting kinematic constraint and fitness function of the multi-axis industrial robot;
initializing a particle population, and randomly giving speed and position information to the particles;
obtaining the current speed of the particles, comparing the current speed with the constraint, judging the advantages and disadvantages of the particles according to the fitness value if the constraint is met, removing the particles if the constraint is not met, and performing iterative training on the removed particles until the constraint is met;
after the particle speed and position information is updated for a plurality of times, obtaining the optimal position searched by each particle and the optimal positions in all the particles, and obtaining the planned path of the multi-axis industrial robot according to the optimal positions;
further comprising: acquiring time information from an initial position to characteristic point image position information of each shaft end effector of the multi-shaft robot;
comparing and analyzing the time information of each axis to generate time deviation information, if the deviation information is greater than a preset time deviation threshold value, acquiring mechanical arm information corresponding to the time deviation information, and generating path correction information according to the mechanical arm information and the time deviation information;
updating the planned path according to the path correction information;
the non-contact identification inspection is carried out on the target workpiece, whether the target workpiece is qualified or not is checked, and the method specifically comprises the following steps:
acquiring target workpiece image information of a target workpiece after processing operation of the multi-axis industrial robot, and acquiring central point coordinate information of the target workpiece;
presetting a target workpiece placing position area, acquiring distance information from the central point coordinate information to the edge of the target workpiece placing position area, and judging whether the position of the target workpiece is qualified or not according to the distance information;
acquiring expected processing information of a target workpiece, preprocessing the target processing image information, acquiring characteristic information according to the preprocessed target workpiece image information, comparing the expected processing information with the characteristic information, and calculating to generate similarity information;
judging whether the similarity information is greater than a similarity threshold value or not, if so, proving that the target workpiece is qualified for machining, and if not, marking the target workpiece to obtain the deviation between the marked target workpiece and the expected machining information;
and generating feedback information according to the deviation of the marked target workpiece and the expected processing information, and feeding back a vision system and a path planning system of the multi-axis industrial robot.
2. The method according to claim 1, wherein the sharpness determination of the visual image information is specifically:
acquiring visual image information, and measuring the focusing degree of each pixel point in the visual image information by using a focusing evaluation operator;
generating a focus evaluation curve through the focus degree evaluation value after focus evaluation is carried out on each pixel point, and comparing the focus evaluation curve with an ideal focus evaluation curve to generate a curve offset error;
judging whether the curve offset error is smaller than a preset offset error threshold value or not, and if so, indicating that the image definition meets a preset standard;
and if not, controlling to obtain a focusing position by a golden section searching method, and carrying out automatic focusing on the image sensor according to the focusing position.
3. The method for controlling the multi-axis servo system based on machine vision according to claim 1, wherein the image segmentation, feature extraction and feature point image position information generation are performed on the preprocessed visual image information, and specifically, the method comprises the following steps:
acquiring the gray value of each pixel in the preprocessed visual image information, and calculating the amplitude and the direction of the gradient by using a differential operator;
comparing the gray value of the target pixel with the adjacent pixels in the gradient direction of the target pixel, if the gray value of the target pixel is greater than that of the adjacent pixels, setting the target pixel as an edge, otherwise, setting the target pixel as not an edge;
performing image segmentation according to an edge judgment result in the visual image information, extracting feature information of the segmented visual image information, and identifying a target workpiece according to the feature information;
acquiring contour information of a target workpiece, extracting feature points according to the contour information of the target workpiece, and acquiring image position information of the feature points.
4. The method according to claim 1, wherein the step of obtaining the position error of each axis of the multi-axis industrial robot at the target position point and the step of compensating the error of the planned path according to the position error comprises:
acquiring a target position point from the planned path, calibrating the target position point in a camera coordinate system to obtain an image position coordinate of the target position point, and transferring the image position coordinate to a base coordinate system of the multi-axis industrial robot to serve as theoretical position coordinate information;
acquiring relative position information of a point of each shaft end effector of the multi-shaft industrial robot in a mechanical arm connecting rod coordinate system of each shaft of the multi-shaft industrial robot, and acquiring a pose matrix of the end effector according to the relative position information;
acquiring actual position coordinate information of each axis in a target position point in a multi-axis industrial robot base coordinate system through the pose matrix, and comparing the actual position coordinate information with the theoretical position coordinate information to generate a position error;
generating a planned path compensation quantity according to the position error, acquiring a track path compensation point by combining period information, and generating error correction information according to the planned path compensation quantity and the planned path compensation point;
and carrying out error compensation on the planned path according to the error correction information.
5. A high accuracy control system for a multi-axis servo system based on machine vision, the system comprising: the high-precision control method program of the multi-axis servo system based on the machine vision is executed by the processor, and the following steps are realized:
acquiring visual image information containing a target workpiece, and judging the definition of the visual image information;
if the definition of the visual image information reaches a preset threshold value, preprocessing the visual image information;
performing image segmentation on the preprocessed visual image information, performing feature extraction and generating feature point image position information;
coordinate transformation is carried out on the feature point image position information, the coordinate information after the coordinate transformation is led into a multi-axis servo system to be combined with kinematic parameters of a multi-axis industrial robot to obtain a planned path, and a target workpiece is operated according to the planned path;
acquiring a target position point from the planned path, acquiring the position error of each axis of the multi-axis industrial robot at the target position point, and realizing error compensation of the planned path according to the position error;
the method comprises the following steps of leading coordinate information after coordinate transformation into a multi-axis servo system, combining kinematic parameters of a multi-axis industrial robot to obtain a planned path, and operating a target workpiece according to the planned path, wherein the method specifically comprises the following steps:
establishing a path planning model of the multi-axis industrial robot based on a neural network and a particle swarm algorithm, and performing initialization training;
acquiring initial position information and initial joint angle information of a point where an end effector of the multi-axis industrial robot is located, and setting kinematic constraint and a fitness function of the multi-axis industrial robot;
initializing a particle population, and randomly giving speed and position information to the particles;
obtaining the current speed of the particles, comparing the current speed with the constraint, judging the advantages and disadvantages of the particles according to the fitness value if the constraint is met, removing the particles if the constraint is not met, and performing iterative training on the removed particles until the constraint is met;
after the particle speed and position information is updated for a plurality of times, obtaining the optimal position searched by each particle and the optimal positions in all the particles, and obtaining the planned path of the multi-axis industrial robot according to the optimal positions;
further comprising: acquiring time information from an initial position to characteristic point image position information of each shaft end effector of the multi-shaft robot;
comparing and analyzing the time information of each axis to generate time deviation information, if the deviation information is greater than a preset time deviation threshold value, acquiring mechanical arm information corresponding to the time deviation information, and generating path correction information according to the mechanical arm information and the time deviation information;
updating the planned path according to the path correction information;
the non-contact identification inspection is carried out on the target workpiece, whether the target workpiece is qualified or not is checked, and the method specifically comprises the following steps:
acquiring target workpiece image information of a target workpiece after processing operation of the multi-axis industrial robot, and acquiring central point coordinate information of the target workpiece;
presetting a target workpiece placing position area, acquiring distance information from the central point coordinate information to the edge of the target workpiece placing position area, and judging whether the position of the target workpiece is qualified or not according to the distance information;
acquiring expected processing information of a target workpiece, preprocessing the target processing image information, acquiring characteristic information according to the preprocessed target workpiece image information, comparing the expected processing information with the characteristic information, and calculating to generate similarity information;
judging whether the similarity information is greater than a similarity threshold value or not, if so, proving that the target workpiece is qualified for machining, and if not, marking the target workpiece to obtain the deviation between the marked target workpiece and the expected machining information;
and generating feedback information according to the deviation of the marked target workpiece and the expected processing information, and feeding back a vision system and a path planning system of the multi-axis industrial robot.
6. The system of claim 5, wherein the step of obtaining the position error of each axis of the multi-axis industrial robot at the target position point and performing error compensation of the planned path according to the position error comprises:
acquiring a target position point through the planned path, calibrating the target position point in a camera coordinate system to obtain an image position coordinate of the target position point, and transferring the image position coordinate into a multi-axis industrial robot base coordinate system to serve as theoretical position coordinate information;
establishing a mechanical arm connecting rod coordinate system by a D-H parameter method according to the degree of freedom of the multi-axis industrial robot and the position and posture relation of two adjacent mechanical arms of the multi-axis mechanical arm in a camera coordinate system;
acquiring relative position information of a point of each shaft end effector of the multi-shaft industrial robot in a mechanical arm connecting rod coordinate system of each shaft of the industrial robot, and acquiring a pose matrix of the end effector according to the relative position information;
acquiring actual position coordinate information of each axis in a target position point in a multi-axis industrial robot base coordinate system through the pose matrix, and comparing the actual position coordinate information with the theoretical position coordinate information to generate a position error;
generating a planned path compensation quantity according to the position error, acquiring a track path compensation point by combining period information, and generating error correction information according to the planned path compensation quantity and the planned path compensation point;
and carrying out error compensation on the planned path according to the error correction information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210271120.XA CN114355953B (en) | 2022-03-18 | 2022-03-18 | High-precision control method and system of multi-axis servo system based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210271120.XA CN114355953B (en) | 2022-03-18 | 2022-03-18 | High-precision control method and system of multi-axis servo system based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114355953A CN114355953A (en) | 2022-04-15 |
| CN114355953B true CN114355953B (en) | 2022-07-12 |
Family
ID=81094484
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210271120.XA Active CN114355953B (en) | 2022-03-18 | 2022-03-18 | High-precision control method and system of multi-axis servo system based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114355953B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115026823B (en) * | 2022-06-14 | 2023-01-17 | 广东天太机器人有限公司 | Industrial robot control method and system based on coordinate welding |
| CN115365088B (en) * | 2022-09-05 | 2024-05-10 | 苏州光宝科技股份有限公司 | Dispensing method and device based on visual guidance |
| CN116038698A (en) * | 2022-12-27 | 2023-05-02 | 上海深其深网络科技有限公司 | Robot guiding method and device, electronic equipment and storage medium |
| CN116909208B (en) * | 2023-09-12 | 2023-11-24 | 深圳市钧诚精密制造有限公司 | Shell processing path optimization method and system based on artificial intelligence |
| CN117400256B (en) * | 2023-11-21 | 2024-05-31 | 扬州鹏顺智能制造有限公司 | Industrial robot continuous track control method based on visual images |
| CN117742338B (en) * | 2023-12-26 | 2024-06-21 | 天津河工大先进装备研究院有限公司 | Curtain wall repair robot scheduling system based on images |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103381603B (en) * | 2013-06-29 | 2015-10-07 | 湖南大学 | The active obstacle planing method of deicing high-voltage power transmission line inspection robot |
| CN103974011B (en) * | 2013-10-21 | 2017-07-11 | 浙江大学 | A kind of projected image obscures removing method |
| CN104476549B (en) * | 2014-11-20 | 2016-04-27 | 北京卫星环境工程研究所 | The manipulator motion path compensation method that view-based access control model is measured |
| CN108907526A (en) * | 2018-08-04 | 2018-11-30 | 苏州佩恩机器人有限公司 | A kind of weld image characteristic recognition method with high robust |
| CN109910010A (en) * | 2019-03-23 | 2019-06-21 | 广东石油化工学院 | A kind of system and method for efficient control robot |
| CN110039523A (en) * | 2019-05-20 | 2019-07-23 | 北京无远弗届科技有限公司 | A kind of industrial robot vision's servo-system, servo method and device |
| CN110919654B (en) * | 2019-12-02 | 2021-04-30 | 中国船舶工业系统工程研究院 | Automatic airplane butt joint method based on visual servo and mechanical arm |
| CN111462154B (en) * | 2020-02-27 | 2024-01-23 | 中电莱斯信息系统有限公司 | Target positioning method and device based on depth vision sensor and automatic grabbing robot |
| CN111360827B (en) * | 2020-03-06 | 2020-12-01 | 哈尔滨工业大学 | Visual servo switching control method and system |
| CN113524194B (en) * | 2021-04-28 | 2023-03-21 | 重庆理工大学 | Target grabbing method of robot vision grabbing system based on multi-mode feature deep learning |
-
2022
- 2022-03-18 CN CN202210271120.XA patent/CN114355953B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN114355953A (en) | 2022-04-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114355953B (en) | High-precision control method and system of multi-axis servo system based on machine vision | |
| CN111775146B (en) | Visual alignment method under industrial mechanical arm multi-station operation | |
| CN110640746B (en) | Method, system, equipment and medium for calibrating and positioning coordinate system of robot | |
| CN112801977B (en) | Assembly body part relative pose estimation and monitoring method based on deep learning | |
| Zou et al. | An end-to-end calibration method for welding robot laser vision systems with deep reinforcement learning | |
| CN112836558B (en) | Mechanical arm tail end adjusting method, device, system, equipment and medium | |
| CN107942949A (en) | A kind of lathe vision positioning method and system, lathe | |
| Šuligoj et al. | Object tracking with a multiagent robot system and a stereo vision camera | |
| CN112561886A (en) | Automatic workpiece sorting method and system based on machine vision | |
| CN112907683B (en) | Camera calibration method and device for dispensing platform and related equipment | |
| CN113379849A (en) | Robot autonomous recognition intelligent grabbing method and system based on depth camera | |
| CN110146017B (en) | Industrial robot repeated positioning precision measuring method | |
| CN115213896A (en) | Object grabbing method, system and equipment based on mechanical arm and storage medium | |
| CN110695996A (en) | Automatic hand-eye calibration method for industrial robot | |
| CN111360821A (en) | Picking control method, device and equipment and computer scale storage medium | |
| CN112164112B (en) | Method and device for acquiring pose information of mechanical arm | |
| CN115841516A (en) | Method and device for modeling dynamic intrinsic parameters of camera | |
| CN117260815A (en) | Precise positioning method and system for manipulator based on visual positioning | |
| CN116766194A (en) | Binocular vision-based disc workpiece positioning and grabbing system and method | |
| Ruan et al. | Feature-based autonomous target recognition and grasping of industrial robots | |
| CN110992416A (en) | High-reflection-surface metal part pose measurement method based on binocular vision and CAD model | |
| CN113433129B (en) | Six-axis robot deburring cutter detection mechanism and method thereof | |
| Xu et al. | Industrial robot base assembly based on improved Hough transform of circle detection algorithm | |
| CN117340879A (en) | Industrial machine ginseng number identification method and system based on graph optimization model | |
| CN112631200A (en) | Machine tool axis measuring method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |