CN114056704A - Feeding deviation rectifying method and device and storage medium - Google Patents
Feeding deviation rectifying method and device and storage medium Download PDFInfo
- Publication number
- CN114056704A CN114056704A CN202111471241.0A CN202111471241A CN114056704A CN 114056704 A CN114056704 A CN 114056704A CN 202111471241 A CN202111471241 A CN 202111471241A CN 114056704 A CN114056704 A CN 114056704A
- Authority
- CN
- China
- Prior art keywords
- image
- target
- control device
- deviation
- force control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65C—LABELLING OR TAGGING MACHINES, APPARATUS, OR PROCESSES
- B65C9/00—Details of labelling machines or apparatus
- B65C9/40—Controls; Safety devices
- B65C9/42—Label feed control
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65H—HANDLING THIN OR FILAMENTARY MATERIAL, e.g. SHEETS, WEBS, CABLES
- B65H23/00—Registering, tensioning, smoothing or guiding webs
- B65H23/02—Registering, tensioning, smoothing or guiding webs transversely
- B65H23/0204—Sensing transverse register of web
- B65H23/0216—Sensing transverse register of web with an element utilising photoelectric effect
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65H—HANDLING THIN OR FILAMENTARY MATERIAL, e.g. SHEETS, WEBS, CABLES
- B65H23/00—Registering, tensioning, smoothing or guiding webs
- B65H23/02—Registering, tensioning, smoothing or guiding webs transversely
- B65H23/032—Controlling transverse register of web
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65H—HANDLING THIN OR FILAMENTARY MATERIAL, e.g. SHEETS, WEBS, CABLES
- B65H7/00—Controlling article feeding, separating, pile-advancing, or associated apparatus, to take account of incorrect feeding, absence of articles, or presence of faulty articles
- B65H7/02—Controlling article feeding, separating, pile-advancing, or associated apparatus, to take account of incorrect feeding, absence of articles, or presence of faulty articles by feelers or detectors
- B65H7/14—Controlling article feeding, separating, pile-advancing, or associated apparatus, to take account of incorrect feeding, absence of articles, or presence of faulty articles by feelers or detectors by photoelectric feelers or detectors
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65H—HANDLING THIN OR FILAMENTARY MATERIAL, e.g. SHEETS, WEBS, CABLES
- B65H7/00—Controlling article feeding, separating, pile-advancing, or associated apparatus, to take account of incorrect feeding, absence of articles, or presence of faulty articles
- B65H7/20—Controlling associated apparatus
Abstract
The application discloses a feeding deviation rectifying method, a feeding deviation rectifying device and a storage medium, wherein the method comprises the following steps: obtaining contact force data of a to-be-positioned part through a force control device, and obtaining a first target difference value according to the contact force data; acquiring an image of a to-be-positioned part through an image acquisition device; performing first image analysis on the image to obtain a target local image to be analyzed; performing second image analysis on the target local image to determine a second target difference value between the target local image of the to-be-positioned element and the preset model; and controlling the position of a to-be-positioned part on the force control device to correct the deviation based on the first target difference value and the second target difference value. According to the positioning method and the positioning device, dual positioning is carried out based on the first target difference value and the second target difference value, and a secondary positioning mode is adopted when image analysis is adopted, so that the positioning accuracy of the positioning piece to be positioned is improved on the premise of ensuring the positioning efficiency.
Description
Technical Field
The application relates to the field of automatic control, in particular to a feeding deviation rectifying method, a feeding deviation rectifying device and a storage medium.
Background
Visual Servo, in the field of automation, positioning is divided into mechanical positioning and Visual positioning.
In mechanism positioning, a mechanism such as a clamp is generally adopted to clamp a to-be-positioned element, and the to-be-positioned element is positioned in a limiting manner by using the mechanism such as the clamp, but the positioning accuracy of the mechanism depends on the mechanical accuracy and the action progress of the mechanism, so that the problem that errors are difficult to quantify exists, and the high-accuracy mechanism has high manufacturing cost, needs frequent debugging and needs high maintenance cost.
In order to improve the automation rate, reduce the debugging difficulty of the mechanism and realize a general mechanism scheme, the positioning method based on vision is widely applied to various fields of automation. The maximum significance lies in that the defect that the difference of the supplied materials of the product cannot be compensated by mechanism positioning can be overcome, the data stability of the product is greatly improved, and the excellent rate of the product is ensured.
In the existing visual-based positioning method, an image acquisition device, such as a camera, is usually used to acquire an image of a to-be-positioned element, and then a certain image processing algorithm is used to determine whether the position of the to-be-positioned element meets the requirement according to the position of the to-be-positioned element in the image, and then a mechanism is used to correct the position. However, the simple visual-based positioning method has the disadvantages of high calibration difficulty, low precision, long calibration time, and the need of manual calibration again if the camera becomes loose.
Therefore, the existing positioning modes have the problem that the relative precision of the positioning process of the to-be-positioned part is difficult to improve.
Disclosure of Invention
The application provides a feeding deviation rectifying method, a feeding deviation rectifying device and a storage medium, which can improve the positioning precision of a to-be-positioned piece through a means of combining force control and image positioning.
The application discloses pay-off method of rectifying, be applied to feeding equipment, feeding equipment is including being used for snatching the power control device who treats the setting element and treating the image acquisition device that the setting element carried out image acquisition, the method includes:
acquiring contact force data of the to-be-positioned part through a force control device, and acquiring a first target difference value according to the contact force data;
acquiring an image of the to-be-positioned part through an image acquisition device;
performing first image analysis on the image to obtain a target local image to be analyzed;
performing second image analysis on the target local image to determine a second target difference value between the target local image of the to-be-positioned part and a preset model;
and controlling the position of a part to be positioned on the force control device to correct the deviation based on the first target difference and the second target difference.
In an embodiment, the performing a first image analysis on the image to obtain a target local image to be analyzed includes:
and removing noise characteristics in the image by adopting an algorithm based on morphology.
In an embodiment, the performing a second image analysis on the target local image to determine a second target difference between the current position parameter of the to-be-positioned element and a preset value includes:
the preset model is a neural network model related to the target local image characteristics.
In one embodiment, the neural network model is a neural network model trained based on an R-CNN network.
In an embodiment, before the obtaining, by the force control device, contact force data on the to-be-positioned piece, the method further includes:
the to-be-positioned piece is abutted against the material on the target position through the force control device;
after correcting the position of the to-be-positioned part on the force control device, the method further comprises the following steps:
obtaining contact force data of the to-be-positioned piece through the force control device again;
and determining whether to execute the deviation rectification again according to the contact force data obtained after the deviation rectification.
In an embodiment, the determining whether to perform the deviation rectification again according to the contact force data obtained after the deviation rectification includes:
judging whether the corrected contact force data accords with a preset value or not;
if yes, ending;
if not, the deviation correction is executed again according to the corrected contact force data and the corrected image analysis.
In an embodiment, after the correcting the position of the to-be-positioned element on the force control device based on the first target difference and the second target difference, the method further includes:
and analyzing and judging the deviation rectifying effect in real time through an image acquisition device.
The application also provides a feeding device, the device includes:
the force control device is used for acquiring contact force data of the to-be-positioned part and acquiring a first target difference value according to the contact force data;
the image acquisition device is used for acquiring an image of the to-be-positioned part; and
the processing device is used for carrying out first-time image analysis on the image to obtain a target local image to be analyzed; performing second image analysis on the target local image to determine a second target difference value between the target local image of the to-be-positioned part and a preset model; and controlling the position of a part to be positioned on the force control device to correct the deviation based on the first target difference and the second target difference.
In an embodiment, the processing apparatus is further specifically configured to:
before the obtaining of the contact force data of the to-be-positioned piece, the method further comprises the following steps:
the to-be-positioned piece is abutted against the material on the target position through the force control device;
after correcting the position of the to-be-positioned part on the force control device, the method further comprises the following steps:
obtaining contact force data of the to-be-positioned piece through the force control device again;
and determining whether to execute the deviation rectification again according to the contact force data obtained after the deviation rectification.
The application also discloses a storage medium, wherein a computer program is stored in the storage medium, and when the computer program runs on a computer, the computer is enabled to execute the feeding deviation rectifying method
From the above, in the feeding deviation rectifying method, the feeding deviation rectifying device and the storage medium, the force control device and the image acquisition device respectively obtain the first target difference value and the second target difference value, double positioning is performed based on the first target difference value and the second target difference value, and a secondary positioning mode is adopted when image analysis is adopted, so that the positioning accuracy of a to-be-positioned part is improved on the premise of ensuring the positioning efficiency.
Drawings
Fig. 1 is a schematic structural diagram of a feeding device provided in an embodiment of the present application.
Fig. 2 is a flowchart of an implementation of the feeding deviation rectifying method according to the embodiment of the present application.
Fig. 3 is a flowchart of another implementation of the feeding deviation rectifying method according to the embodiment of the present application.
Fig. 4 is a flowchart of an implementation of an image analysis process provided in an embodiment of the present application.
Fig. 5 is another schematic structural diagram of a feeding device provided in an embodiment of the present application.
Fig. 6 is an application scenario schematic diagram of a feeding device provided in the embodiment of the present application.
Detailed Description
Referring to the drawings, wherein like reference numbers refer to like elements, the principles of the present application are illustrated as being implemented in a suitable computing environment. The following description is based on illustrated embodiments of the application and should not be taken as limiting the application with respect to other embodiments that are not detailed herein.
The term "module" as used herein may be a software object that executes on the computing system. The different components, modules, engines, and services described herein may be implementation objects on the computing system. The apparatus and method described herein may be implemented in software, but may also be implemented in hardware, and are within the scope of the present application.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
Referring to fig. 1, a structure of a feeding device provided in an embodiment of the present application is shown.
As shown in fig. 1, the feeding device may at least include a force control device, an image acquisition device, and the like. The feeding equipment is used for positioning the to-be-positioned piece in the feeding process, and then the to-be-positioned piece is moved to a specified position after deviation correction is carried out according to a positioning result. The specific process name or type of the feeding equipment can be determined according to actual conditions, and can be scenes such as feeding, rubberizing or upper plate and the like for accurately fitting the to-be-positioned piece and another material. It can be understood that the feeding device is mainly used for accurately feeding or positioning the to-be-positioned piece, and the application does not limit the specific application scene.
The force control device can comprise a mechanical arm, a sensor can be arranged at the position of the mechanical arm, which is contacted with the part to be positioned, and the sensor can judge whether the position and the attitude of the part to be positioned are in place or not by detecting the contact force between the mechanical arm and the part to be positioned. In one embodiment, the force control device can be a six-axis force control manipulator which operates by utilizing the rotation and the movement of the X, Y, Z axes, when a sensor of the manipulator contacts with different positions of a product, different contact forces are generated during the contact, and the position and the attitude of the product are adjusted by comparing the contact forces generated by the sensor with target parameters.
Of course, the specific form and implementation manner of the force control device can be formulated according to the user requirements, as long as the force control effect of the positioning piece to be treated can be realized.
The image acquisition device can adopt a camera for image acquisition, and specifically adopts a camera of a photosensitive element such as a CCD (charge coupled device), a CMOS (complementary metal oxide semiconductor) and the like. The image acquisition device can be used for acquiring the current real-time image of the to-be-positioned part and also can analyze the current real-time image by acquiring continuous multi-frame images. After the image is obtained, the image may be transmitted to a Processing device for Processing, where the Processing device may include a Processing Unit such as an MCU (micro controller Unit)/a CPU (Central Processing Unit)/an SOC (System on Chip), and the like, to implement a specific algorithm operation by executing a pre-stored program instruction, so as to implement an analysis of the image by comparing the image with preset parameters, and to obtain positioning information of the to-be-positioned item.
In addition, the image acquisition device can also comprise a lens, a light source controller and other components. The device can be in a monocular vision mode or a binocular vision mode and the like, and images are collected in real time along with the change of the pose of the manipulator of the force control device.
It can be understood that, besides the implementation manners of the above-mentioned force control device and image acquisition device, different models or structural manners may be selected according to actual needs, which is not limited in the present application.
Referring to fig. 2, an implementation manner of the feeding deviation rectifying method provided by the embodiment of the present application is shown in the figure.
As shown in fig. 2, the feeding deviation rectifying method includes:
101. and acquiring contact force data of the to-be-positioned part through the force control device, and acquiring a first target difference value according to the contact force data.
Before the first target difference value is obtained, contact force data of the to-be-positioned piece in an ideal pose can be obtained, and the data are used as preset data to be compared.
The contact force data may include contact forces of the part to be positioned with the robot sensor of the force control device in at least one or more directions of the x-axis, y-axis, or z-axis. Because the different positions and postures of the to-be-positioned piece can cause the difference of the contact surface or the contact mode between the to-be-positioned piece and the mechanical arm of the force control device, the difference can be calculated by comparing the obtained contact force data with preset data, and therefore the first target difference value is obtained.
102. And acquiring an image of the to-be-positioned part by an image acquisition device.
The image acquisition device can acquire images for the to-be-positioned part, and can photograph the characteristics of the auxiliary to-be-positioned part, such as the positions of adhesive tapes and other places needing to be positioned, through the camera. Of course, the image position of the to-be-positioned member acquired by the specific image acquiring device may be determined according to actual situations.
103. And carrying out first image analysis on the image to obtain a target local image to be analyzed.
The first image analysis can be used for primarily screening the position irrelevant to the positioning of the positioning piece to be positioned so as to obtain a target local image to be analyzed.
In some cases, the image obtained by the image obtaining device is likely to have more image features, and if all the image features are analyzed in real time, a larger computing resource is required, which results in a longer processing time and affects the processing efficiency.
At the moment, the target local image to be analyzed is reserved by screening out the unwanted noise characteristics in the image, so that the processing efficiency of the subsequent analysis of the to-be-positioned part can be improved. For example, in the material positioning process of the pasting process, the noise point characteristic may be a suction hole on a suction head of a manipulator or a characteristic that glue overflowing from a gummed paper interferes with the subsequent image analysis and judgment result.
In an embodiment, in order to realize the preliminary screening of the features, the image acquisition device may select an image at a specific position as a target local image, and may also remove noise features in the image through a morphological algorithm.
104. And performing secondary image analysis on the target local image to determine a second target difference value between the target local image of the to-be-positioned part and the preset model.
After the target local image in the image is acquired, a second target difference value can be obtained by performing detailed analysis on the target local image.
In an embodiment, the position difference between the calibrated target feature and the preset model may be determined by an AI real-time calibration, which may be implemented by using a neural network model.
Compared with a general visual algorithm, the method has the advantages of high accuracy and high efficiency by an AI real-time calibration mode, and is favorable for improving the accuracy and the processing efficiency of the feeding deviation rectifying process.
105. And controlling the position of a to-be-positioned part on the force control device to correct the deviation based on the first target difference value and the second target difference value.
The positioning part to be positioned can be accurately positioned by combining the first target difference value and the second target difference value, and then the deviation rectification treatment is carried out on the manipulator of the force control device based on the difference value, so that the deviation rectification of the positioning part to be positioned is completed.
In an embodiment, the image acquisition device can monitor the manipulator of the force control device in real time in the deviation rectifying process, so as to ensure the deviation rectifying effect, and if the deviation rectification does not reach the expectation, the force control device and the image acquisition device can be controlled to perform the deviation rectifying step again.
According to the feeding deviation rectifying method, through the cooperation of the force control device and the image processing device, double positioning can be carried out by utilizing data of two different dimensions of contact force and vision, and a secondary positioning mode is adopted when image analysis is adopted, so that the positioning precision of a to-be-positioned part is improved on the premise of ensuring the positioning efficiency.
Referring to fig. 3, another implementation flow of the feeding deviation rectifying method according to the embodiment of the present application is shown.
As shown in fig. 3, in an embodiment, the feeding deviation rectifying method may further include the following steps:
201. and (4) abutting the part to be positioned against the material on the target position through the force control device.
The collision means that the positioning piece is attached to the material on the target position.
For example, if the to-be-positioned component is a component a, the feeding deviation correcting mechanism is used for attaching and positioning the component a and another component B through a mechanical arm. At this time, the robot may bring the module a and the module B into contact with each other in advance, so that a certain frictional force is generated between the module a and the module B.
202. And acquiring a first target difference value and a second target difference value of the to-be-positioned part, and controlling the position of the to-be-positioned part on the force control device to correct the deviation according to the first target difference value and the second target difference value.
The first target difference value can be realized by arranging a sensor at the contact position of a manipulator of the force control device and the to-be-positioned piece, and the contact force data of the to-be-positioned piece in an ideal pose is acquired and is used as preset data to be compared.
For example, the contact force data may include contact forces in at least one or more directions of an x-axis, a y-axis, or a z-axis of the robotic sensor of the force control device and the part to be positioned.
The specific obtaining manner of the first target difference may refer to an embodiment of obtaining the first target difference according to the contact force data, and the specific manner is not further described in this application.
The second target difference value can be obtained by image analysis and comparison with a preset model after the image of the to-be-positioned part is obtained by the image obtaining device.
In some embodiments, the position parameters of the pose of the to-be-positioned piece can be obtained through secondary image analysis. The first image analysis can be used for primarily screening the position irrelevant to the positioning of the to-be-positioned part so as to obtain a target local image to be analyzed. At the moment, the target local image to be analyzed is reserved by screening out the unwanted noise characteristics in the image, so that the processing efficiency of the subsequent analysis of the to-be-positioned part can be improved. For example, in the material positioning process of the pasting process, the noise point characteristic may be a suction hole on a suction head of a manipulator or a characteristic that glue overflowing from a gummed paper interferes with the subsequent image analysis and judgment result.
In an embodiment, in order to realize the preliminary screening of the features, the image acquisition device may select an image at a specific position as a target local image, and may also remove noise features in the image through a morphological algorithm. Specifically, an opening operation and a closing operation can be adopted to remove noise characteristics in the image, and other image processing algorithms can also be adopted, so that the noise characteristics are quickly removed through the image processing algorithms.
203. And obtaining the contact force data of the to-be-positioned piece through the force control device.
After the deviation correction is finished, in order to verify the completion effect of the deviation correction, the contact force data of the to-be-positioned part can be obtained again through the force control device, the contact force data of the to-be-positioned part after the deviation correction is obtained through the sensor of the force control device, and after the contact force is compared with a preset value, whether the posture of the to-be-positioned part after the deviation correction is in accordance with the expectation can be obtained.
204. And determining whether to execute the deviation rectification again according to the contact force data obtained after the deviation rectification.
Generally, the moving precision of the manipulator is insufficient or the possibility of deviation of data acquisition exists, and in order to further improve the deviation rectifying effect, the deviation rectifying effect can be determined through contact force data or image analysis data obtained after deviation rectifying.
In one embodiment, the deviation correction result can be judged to be in accordance with the expectation by respectively or simultaneously acquiring the real-time image by the image acquisition device, besides the force control device is used for acquiring the contact force data again to judge whether the deviation correction result is in accordance with the expectation.
When one or two values of the first target difference value and the second target difference value respectively obtained from the contact force data or the real-time image do not reach the expectation, the to-be-positioned part can be determined to still not realize deviation rectification, and at the moment, the deviation rectification program can be executed again to reduce the position error of the to-be-positioned part as much as possible until the position of the to-be-positioned part meets the expectation.
If the corrected contact force data and the real-time image analysis both accord with expectations, it can be considered that correction is not required to be performed again.
Through the technical means for confirming the deviation rectifying effect and executing deviation rectifying again when the effect is not in line with expectation, the deviation rectifying effect of the positioning piece to be treated can be ensured by adopting double positioning of contact force data, image analysis and the like, meanwhile, the requirement on the precision of equipment is reduced, and the positioning is faster and more reliable.
Referring to fig. 4, an implementation flow of an image analysis process provided by the embodiment of the present application is shown.
The image analysis process can be applied to the first image analysis as well as the second image analysis.
As shown in fig. 4, the image analysis process includes:
301. and removing noise characteristics in the image by adopting an algorithm based on morphology to obtain a target local image to be analyzed.
The first image analysis can be used for primarily screening the position irrelevant to the positioning of the to-be-positioned part so as to obtain a target local image to be analyzed. At the moment, the target local image to be analyzed is reserved by screening out the unwanted noise characteristics in the image, so that the processing efficiency of the subsequent analysis of the to-be-positioned part can be improved. For example, in the material positioning process of the pasting process, the noise point characteristic may be a suction hole on a suction head of a manipulator or a characteristic that glue overflowing from a gummed paper interferes with the subsequent image analysis and judgment result.
In an embodiment, an open operation and a close operation may be used to remove noise features in an image, or other image processing algorithms may be used, or of course, a mode of removing an image at a specific position may be used, so that noise features are quickly removed by the image processing algorithm.
By the aid of the first image analysis, invalid analysis of a neural network model algorithm can be avoided, so that the processing efficiency of subsequent second image analysis is improved, and the recognition accuracy can be improved.
302. And analyzing the target local image by adopting a neural network model related to the target local image characteristics.
When the noise characteristics are screened out, a target local image with reference value can be obtained.
The neural network model can effectively improve the recognition accuracy of the features, and the network model can quickly determine the distance difference between the features in the target local image and the preset model by performing feature recognition training on the neural network model, so that the second target difference is obtained.
In one embodiment, the neural network model is a neural network model trained based on an R-CNN network. Of course, besides the above models, other common neural network models for image comparison may be adopted, which is not limited in the present application.
303. And determining a second target difference value between the current position parameter of the to-be-positioned part and a preset value.
The current position parameter can be a parameter corresponding to the relative position of the characteristic of the part to be positioned identified by the neural network model in the image; the preset value is a preset parameter corresponding to the model characteristic corresponding to the characteristic of the part to be positioned in the preset model. And determining the deviation of the specific characteristic of the to-be-positioned part relative to the ideal value of the model through comparison, so as to obtain a second target difference value.
By screening out the noise point characteristics and then comparing the noise point characteristics with the neural network model, the calculation reliability and the processing speed of the second target difference value can be effectively improved, and the reliability and the processing speed of the subsequent deviation rectifying process can be further effectively improved.
Referring to fig. 5, a structure of a feeding device provided in an embodiment of the present application is shown.
As shown in fig. 5, the feeding device 1010 includes a force control device 1, an image acquisition device 2, and a processing device 3.
The force control device 1 is used for acquiring contact force data of a to-be-positioned part and acquiring a first target difference value according to the contact force data;
the image acquisition device 2 is used for acquiring an image of the to-be-positioned part; and
the processing device 3 is used for carrying out first image analysis on the image to obtain a target local image to be analyzed; performing second image analysis on the target local image to determine a second target difference value between the target local image of the to-be-positioned element and the preset model; and controlling the position of the part to be positioned on the force control device 1 to correct the deviation based on the first target difference and the second target difference.
The force control device 1 may include a manipulator, a sensor may be disposed at a position where the manipulator contacts with the member to be positioned, and the sensor may determine whether the pose of the member to be positioned is in place by detecting a contact force between the manipulator and the member to be positioned. In an embodiment, the force control device 1 may be a six-axis force control manipulator, which operates by using the rotation and movement of the X, Y, Z axes, when the sensor of the manipulator contacts with different positions of the product, different contact forces are generated during the contact, and the pose of the product is adjusted by comparing the contact force generated by the sensor with the target parameter.
Of course, the specific form and implementation manner of the force control device 1 can be set according to the user's needs, as long as the force control effect of the positioning element to be treated can be achieved.
The image capturing device 2 may employ a camera for image capturing, specifically a camera of a photosensitive element such as a CCD or a CMOS. The image acquiring device 2 may be configured to acquire a current real-time image of the to-be-positioned element, or may perform analysis by acquiring images of consecutive multiple frames.
After the image is obtained, the image may be transmitted to a Processing device 3 for Processing, where the Processing device 3 may include a Processing Unit such as an MCU (micro controller Unit)/a CPU (Central Processing Unit)/an SOC (System on Chip), and the like, so as to implement a specific algorithm operation by executing a pre-stored program instruction, and thus implement analysis of the image by comparing the image with preset parameters, so as to obtain positioning information of the to-be-positioned component.
In addition, the image capturing device 2 may further include a lens, a light source controller, and the like. The device can be in a monocular vision mode or a binocular vision mode and the like, and images are collected in real time along with the change of the pose of the manipulator of the force control device 1.
It can be understood that, besides the implementation manners of the above-mentioned force control device 1 and the image acquisition device 2, different models or structural manners may be selected according to actual needs, and the present application is not limited thereto.
Before the first target difference value is obtained, contact force data of the to-be-positioned piece in an ideal pose can be obtained, and the data are used as preset data to be compared.
The contact force data may comprise contact forces of the part to be positioned with the robot sensor of the force control device 1 in at least one or more directions of the x-axis, the y-axis or the z-axis. Because the different positions and postures of the to-be-positioned piece can cause the difference of the contact surface or the contact mode between the to-be-positioned piece and the mechanical arm of the force control device 1, the difference can be calculated by comparing the obtained contact force data with preset data, and therefore the first target difference is obtained.
The image acquisition device 2 can acquire images of the to-be-positioned part, and can photograph the characteristics of the auxiliary to-be-positioned part, such as the positions of adhesive tapes and other places needing to be positioned, through the camera. Of course, the image position of the to-be-positioned member acquired by the specific image acquiring device 2 may be determined according to actual situations.
The first image analysis can be used for primarily screening the position irrelevant to the positioning of the positioning piece to be positioned so as to obtain a target local image to be analyzed.
In some cases, the image obtained by the image obtaining apparatus 2 is likely to have many image features, and if all the image features are analyzed in real time, a large amount of computing resources are required, which results in a long processing time and affects the processing efficiency.
At the moment, the target local image to be analyzed is reserved by screening out the unwanted noise characteristics in the image, so that the processing efficiency of the subsequent analysis of the to-be-positioned part can be improved. For example, in the material positioning process of the pasting process, the noise point characteristic may be a suction hole on a suction head of a manipulator or a characteristic that glue overflowing from a gummed paper interferes with the subsequent image analysis and judgment result.
After the target local image in the image is acquired, a second target difference value can be obtained by performing detailed analysis on the target local image.
In an embodiment, the position difference between the calibrated target feature and the preset model may be determined by an AI real-time calibration, which may be implemented by using a neural network model.
Compared with a general visual algorithm, the method has the advantages of high accuracy and high efficiency by an AI real-time calibration mode, and is favorable for improving the accuracy and the processing efficiency of the feeding deviation rectifying process.
The first target difference value and the second target difference value can be combined to accurately position the to-be-positioned element, and then the manipulator of the force control device 1 is controlled to perform deviation rectification processing based on the difference value, so that the deviation rectification of the to-be-positioned element is completed.
In an embodiment, the image obtaining device 2 may monitor the manipulator of the force control device 1 in real time during the deviation rectification process, so as to ensure the deviation rectification effect, and if the deviation rectification does not reach the expectation, the force control device 1 and the image obtaining device 2 may be controlled to perform the deviation rectification step again.
Feeding equipment 10 in this application, through the cooperation of power accuse device 1 and image processing device 3, can utilize the data of two different dimensions of contact force and vision to carry out dual location to adopted the mode of secondary location when adopting image analysis, thereby improved the positioning accuracy of treating the setting element under the prerequisite of ensureing positioning efficiency.
Please refer to fig. 6, which illustrates an application scenario of the feeding device according to the embodiment of the present application.
The processing device 3 of the feeding equipment can be specifically used for: before the obtaining of the contact force data of the to-be-positioned part 4, the method further comprises the following steps: the to-be-positioned piece 4 is abutted against the material 5 on the target position through the force control device 1; after correcting the position of the to-be-positioned element 4 on the force control device 1, the method further comprises the following steps: obtaining contact force data of the to-be-positioned part 4 through the force control device 1 again; and determining whether to execute the deviation rectification again according to the contact force data obtained after the deviation rectification.
The collision means that the positioning piece 4 is attached to the material 5 at the target position.
For example, if the to-be-positioned element 4 is an assembly a, the feeding deviation-correcting mechanism performs fitting positioning on the assembly a and another assembly B through a mechanical arm. At this time, the robot may bring the module a and the module B into contact with each other in advance, so that a certain frictional force is generated between the module a and the module B.
After the deviation correction is finished, in order to verify the completion effect of the deviation correction, the contact force data of the to-be-positioned part 4 can be obtained again through the force control device 1, the contact force data of the to-be-positioned part 4 after the deviation correction is obtained by utilizing the sensor 11 of the force control device 1, and after the contact force is compared with a preset value, whether the posture of the to-be-positioned part 4 after the deviation correction is in accordance with the expectation can be obtained.
Generally, the moving precision of the manipulator is insufficient or the possibility of deviation of data acquisition exists, and in order to further improve the deviation rectifying effect, the deviation rectifying effect can be determined through contact force data or image analysis data obtained after deviation rectifying.
In an embodiment, in addition to obtaining the contact force data again by the force control device 1 to determine whether the deviation correction result is in accordance with the expectation, the image obtaining device 2 may be used to obtain the real-time image to determine whether the deviation correction result is in accordance with the expectation, respectively or simultaneously.
When one or two values of the first target difference value and the second target difference value respectively obtained from the contact force data or the real-time image do not reach the expectation, it can be determined that the to-be-positioned part 4 still does not realize deviation rectification, and at the moment, the deviation rectification program can be executed again to reduce the position error of the to-be-positioned part 4 as much as possible until the position meets the expectation.
If the corrected contact force data and the real-time image analysis both accord with expectations, it can be considered that correction is not required to be performed again.
Through the technical means for confirming the deviation rectifying effect and executing deviation rectifying again when the effect is not in line with expectation, the deviation rectifying effect of the positioning piece 4 to be treated can be ensured by adopting double positioning of contact force data, image analysis and the like, meanwhile, the requirement on the precision of equipment is reduced, and the positioning is faster and more reliable.
In the embodiment of the present application, the feeding device and the feeding deviation rectifying method in the above embodiments belong to the same concept, and any method step provided in the feeding deviation rectifying method embodiment may be run on the feeding device, and the specific implementation process thereof is described in detail in the feeding deviation rectifying method embodiment, and any combination may be adopted to form an optional embodiment of the present application, which is not described herein again.
Embodiments of the present application further provide a computer storage medium, where the computer storage medium may store a program, and when the program is executed by a processor, the program may perform some or all of the steps in the embodiments provided in the present application, for example:
acquiring contact force data of the to-be-positioned part through a force control device, and acquiring a first target difference value according to the contact force data; acquiring an image of the to-be-positioned part through an image acquisition device; performing first image analysis on the image to obtain a target local image to be analyzed; performing second image analysis on the target local image to determine a second target difference value between the target local image of the to-be-positioned part and a preset model; and controlling the position of a part to be positioned on the force control device to correct the deviation based on the first target difference and the second target difference.
The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a Random Access Memory (RAM).
The embodiments of the present application have been described in detail with reference to the drawings, but the present application is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present application within the knowledge of those skilled in the art.
Claims (10)
1. A feeding deviation rectifying method is applied to feeding equipment and is characterized in that the feeding equipment comprises a force control device used for grabbing a to-be-positioned piece and an image acquisition device used for acquiring an image of the to-be-positioned piece, and the method comprises the following steps:
acquiring contact force data of the to-be-positioned part through a force control device, and acquiring a first target difference value according to the contact force data;
acquiring an image of the to-be-positioned part through an image acquisition device;
performing first image analysis on the image to obtain a target local image to be analyzed;
performing second image analysis on the target local image to determine a second target difference value between the target local image of the to-be-positioned part and a preset model;
and controlling the position of a part to be positioned on the force control device to correct the deviation based on the first target difference and the second target difference.
2. The feeding deviation rectifying method according to claim 1, wherein the performing a first image analysis on the image to obtain a target local image to be analyzed includes:
and removing noise characteristics in the image by adopting an algorithm based on morphology.
3. The feeding deviation rectifying method according to claim 1, wherein the performing a second image analysis on the target local image to determine a second target difference between the current position parameter of the to-be-positioned element and a preset value comprises:
and performing secondary image analysis on the target local image by adopting a neural network model related to the target local image characteristics.
4. The feed correction method of claim 3, wherein the neural network model is a neural network model based on R-CNN network training.
5. The feed deviation correcting method according to claim 1, wherein:
before the contact force data of the to-be-positioned part is acquired by the force control device, the method further comprises the following steps:
the to-be-positioned piece is abutted against the material on the target position through the force control device;
after correcting the position of the to-be-positioned part on the force control device, the method further comprises the following steps:
obtaining contact force data of the to-be-positioned piece through the force control device again;
and determining whether to execute the deviation rectification again according to the contact force data obtained after the deviation rectification.
6. The feeding deviation correcting method according to claim 5, wherein the determining whether to perform deviation correction again according to the contact force data obtained after deviation correction comprises:
judging whether the corrected contact force data accords with a preset value or not;
if yes, ending;
if not, the deviation correction is executed again according to the corrected contact force data and the corrected image analysis.
7. The method for correcting the feeding deviation according to claim 1, wherein after correcting the position of the to-be-positioned member on the force control device based on the first target difference and the second target difference, the method further comprises:
and analyzing and judging the deviation rectifying effect in real time through an image acquisition device.
8. A feeding device, characterized in that it comprises:
the force control device is used for acquiring contact force data of the to-be-positioned part and acquiring a first target difference value according to the contact force data;
the image acquisition device is used for acquiring an image of the to-be-positioned part; and
the processing device is used for carrying out first-time image analysis on the image to obtain a target local image to be analyzed; performing second image analysis on the target local image to determine a second target difference value between the target local image of the to-be-positioned part and a preset model; and controlling the position of a part to be positioned on the force control device to correct the deviation based on the first target difference and the second target difference.
9. The feeding deviation correcting device of claim 8, wherein the processing device is further specifically configured to:
before the obtaining of the contact force data of the to-be-positioned piece, the method further comprises the following steps:
the to-be-positioned piece is abutted against the material on the target position through the force control device;
after correcting the position of the to-be-positioned part on the force control device, the method further comprises the following steps:
obtaining contact force data of the to-be-positioned piece through the force control device again;
and determining whether to execute the deviation rectification again according to the contact force data obtained after the deviation rectification.
10. A storage medium having stored therein a computer program which, when run on a computer, causes the computer to execute the feed correction method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111471241.0A CN114056704B (en) | 2021-12-03 | 2021-12-03 | Feeding deviation rectifying method and equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111471241.0A CN114056704B (en) | 2021-12-03 | 2021-12-03 | Feeding deviation rectifying method and equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114056704A true CN114056704A (en) | 2022-02-18 |
| CN114056704B CN114056704B (en) | 2022-11-11 |
Family
ID=80228638
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111471241.0A Active CN114056704B (en) | 2021-12-03 | 2021-12-03 | Feeding deviation rectifying method and equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114056704B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102602730A (en) * | 2012-03-23 | 2012-07-25 | 西南铝业（集团）有限责任公司 | Rectification control system |
| CN103359522A (en) * | 2013-07-11 | 2013-10-23 | 华中科技大学 | Film conveying deviation-rectifying device combining tension control |
| CN108163600A (en) * | 2017-12-25 | 2018-06-15 | 湖南顶立科技有限公司 | A kind of deviation correction control system and method |
| CN111055279A (en) * | 2019-12-17 | 2020-04-24 | 清华大学深圳国际研究生院 | Multi-mode object grabbing method and system based on combination of touch sense and vision |
| CN112100841A (en) * | 2020-09-09 | 2020-12-18 | 中铁二十局集团有限公司 | Shield tunneling machine attitude prediction method and device, terminal equipment and storage medium |
| CN112693947A (en) * | 2020-12-21 | 2021-04-23 | 江苏迪盛智能科技有限公司 | Pulling deviation processing method and device, electronic equipment and storage medium |
| CN113203376A (en) * | 2021-04-19 | 2021-08-03 | 深圳市安全守护科技有限公司 | Position deviation rectifying method, device, equipment and computer readable storage medium |
-
2021
- 2021-12-03 CN CN202111471241.0A patent/CN114056704B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102602730A (en) * | 2012-03-23 | 2012-07-25 | 西南铝业（集团）有限责任公司 | Rectification control system |
| CN103359522A (en) * | 2013-07-11 | 2013-10-23 | 华中科技大学 | Film conveying deviation-rectifying device combining tension control |
| CN108163600A (en) * | 2017-12-25 | 2018-06-15 | 湖南顶立科技有限公司 | A kind of deviation correction control system and method |
| CN111055279A (en) * | 2019-12-17 | 2020-04-24 | 清华大学深圳国际研究生院 | Multi-mode object grabbing method and system based on combination of touch sense and vision |
| CN112100841A (en) * | 2020-09-09 | 2020-12-18 | 中铁二十局集团有限公司 | Shield tunneling machine attitude prediction method and device, terminal equipment and storage medium |
| CN112693947A (en) * | 2020-12-21 | 2021-04-23 | 江苏迪盛智能科技有限公司 | Pulling deviation processing method and device, electronic equipment and storage medium |
| CN113203376A (en) * | 2021-04-19 | 2021-08-03 | 深圳市安全守护科技有限公司 | Position deviation rectifying method, device, equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114056704B (en) | 2022-11-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3486041A2 (en) | Gripping system, learning device, and gripping method | |
| CN107263468B (en) | SCARA robot assembly method using digital image processing technology | |
| CN111347411B (en) | Two-arm cooperative robot three-dimensional visual recognition grabbing method based on deep learning | |
| CN113146172B (en) | Multi-vision-based detection and assembly system and method | |
| CN109159113B (en) | Robot operation method based on visual reasoning | |
| CN109926817B (en) | Machine vision-based automatic transformer assembling method | |
| JP7306937B2 (en) | A control device for a robot device that adjusts the position of a member supported by a robot | |
| CN114912287B (en) | Robot autonomous grabbing simulation system and method based on target 6D pose estimation | |
| CN111923053A (en) | Industrial robot object grabbing teaching system and method based on depth vision | |
| CN111179233B (en) | Self-adaptive deviation rectifying method based on laser cutting of two-dimensional parts | |
| CN113379849A (en) | Robot autonomous recognition intelligent grabbing method and system based on depth camera | |
| JP2004001122A (en) | Picking device | |
| CN114355953B (en) | High-precision control method and system of multi-axis servo system based on machine vision | |
| CN114670189B (en) | Storage medium, and method and system for generating control program of robot | |
| CN115629066A (en) | Method and device for automatic wiring based on visual guidance | |
| Prezas et al. | AI-enhanced vision system for dispensing process monitoring and quality control in manufacturing of large parts | |
| CN111993422A (en) | Robot axis and hole alignment control method based on uncalibrated vision | |
| CN113319859B (en) | Robot teaching method, system and device and electronic equipment | |
| CN114056704B (en) | Feeding deviation rectifying method and equipment and storage medium | |
| JP2019077026A (en) | Control device, robot system, and control device operating method and program | |
| CN112935772A (en) | Method, device, storage medium and equipment for screwing screws by vision-guided robot | |
| CN116079732A (en) | Cabin assembly method based on laser tracker and binocular vision mixed guidance | |
| CN113763400A (en) | Robot vision guiding method, device, equipment and storage medium | |
| US20230386075A1 (en) | Method, System, And Computer Program For Recognizing Position And Posture Of Object | |
| Sun et al. | Further development of adaptable automated visual inspection—part i: concept and scheme |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |