CN117817211A - Welding automation control method and system based on machine vision - Google Patents

Abstract

The application discloses a welding automation control method and system based on machine vision, and relates to the technical field of welding processing, wherein the method comprises the following steps: acquiring an image of a part to be welded according to a machine vision system to obtain a part image acquisition result; image enhancement is carried out on the component image acquisition result based on an MSRCR algorithm, so that component image data are obtained; activating a cleanliness checker, and executing cleanliness verification of the parts to be welded; when the cleanliness is verified to be qualified, loading part welding requirement information of the part to be welded; based on the component welding demand information and the component image data, performing welding decision analysis of the component to be welded to obtain a component welding decision; and sending the component welding decision to a welding robot, and executing the welding control of the component to be welded by the welding robot according to the component welding decision. The technical problems of low automation degree, poor adaptability and unstable welding quality are solved, and the technical effects of improving the automation level and the adaptability and ensuring the welding quality are realized.

Description

Welding automation control method and system based on machine vision

Technical Field

The invention relates to the technical field of welding processing, in particular to a welding automation control method and system based on machine vision.

Background

Welding is widely used in various industries as an important manufacturing process in the manufacturing industry. The continual development of the manufacturing industry places higher demands on welding quality, efficiency and automation.

The existing automatic welding system generally depends on manual experience and fixed parameters, is difficult to cope with complex welding environment and material changes, and timely adjusts welding parameters to ensure welding quality. The technical problems of low automation degree, poor adaptability and unstable welding quality exist.

Disclosure of Invention

The application provides a welding automation control method and a welding automation control system based on machine vision, which are used for solving the technical problems of low automation degree, poor adaptability and unstable welding quality in the prior art and realizing the technical effects of improving the automation level and the adaptability and ensuring the welding quality.

In a first aspect, the present application provides a machine vision-based welding automation control method, wherein the method comprises:

acquiring an image of a part to be welded according to a machine vision system to obtain a part image acquisition result; image enhancement is carried out on the component image acquisition result based on an MSRCR algorithm, and component image data are obtained; activating a cleanliness checker, and executing the cleanliness verification of the parts to be welded by combining the part image data to obtain a part cleanliness verification result; when the component cleanliness verification result is qualified, loading component welding demand information of the component to be welded, wherein the component welding demand information comprises component model information and component welding seam forming demand data; activating a welding decision algorithm to execute welding decision analysis of the parts to be welded based on the part welding demand information and the part image data, and obtaining a part welding decision; and sending the component welding decision to a welding robot, and executing the welding control of the component to be welded by the welding robot according to the component welding decision.

In the method, the image of the part to be welded is acquired through a machine vision system, and the image is enhanced through an MSRCR algorithm. The method is beneficial to improving the quality and definition of the image and enabling the subsequent image processing and analysis to be more accurate. A cleanliness checker is activated, and cleanliness verification is performed in conjunction with the component image data. By verifying the cleanliness of the parts, no pollution and impurities are ensured in the welding process, so that the welding quality and reliability are improved. And making a proper welding decision according to the specific requirements of the components and the actual image information so as to improve the welding efficiency and quality. And sending the obtained part welding decision to a welding robot, and executing welding control of the parts to be welded by the welding robot according to the decision. The automatic control of the welding process is realized, the interference of manual operation is reduced, and the consistency and the precision of welding are improved.

In one possible implementation, activating a cleanliness checker, performing cleanliness verification of the component to be welded in combination with the component image data, to obtain a component cleanliness verification result, including:

the cleanliness checker comprises a cleanliness identification module and a cleanliness verification module;

inputting the component image data into the cleanliness recognition module to obtain a component cleanliness coefficient;

Inputting the component cleanliness coefficient into the cleanliness verification module to generate a component cleanliness verification result;

the cleanliness verification module comprises a cleanliness verification condition, wherein the cleanliness verification condition comprises that if the part cleanliness coefficient is larger than/equal to a preset cleanliness threshold value, the obtained part cleanliness verification result is qualified;

and the cleanliness verification condition further comprises that if the part cleanliness coefficient is smaller than the preset cleanliness threshold, the obtained part cleanliness verification result is unqualified, and a part cleanliness abnormality early warning signal is generated.

According to the method in the implementation mode, the cleanliness identification module and the cleanliness verification module are respectively constructed to form the cleanliness checker, so that the evaluation and constraint discrimination of the surface quality of the target raw material through the component image data are realized. The cleanliness checker provides a mechanism for monitoring and verifying the cleanliness of the welding parts in real time through the combination of the image recognition and verification modules, so that the cleanliness in the welding process can reach the expected level.

In one possible implementation, the method includes:

loading a sample part image data record and a sample part cleanliness factor record;

Taking the image data record of the sample part as input data, taking the cleanliness coefficient record of the sample part as output supervision data, training the convolutional neural network, and acquiring network output accuracy rate when training is performed for preset times;

and if the network output accuracy is greater than or equal to the preset accuracy, generating a cleanliness identification model, and embedding the cleanliness identification model into the cleanliness identification module.

In other words, the acquired historical component image data is recorded with the corresponding cleanliness factor. A complex association between the cleanliness factor and the image data is included. And performing supervised training by taking the data and the records as training data, so that the cleanliness recognition model based on the convolutional neural network acquires the related data of the cleanliness coefficient and the image data, and further, mapping between the data and the image data is established, and the end-to-end conversion from the image data of the component to the cleanliness coefficient is realized.

In one possible implementation, activating a welding decision algorithm to perform a welding decision analysis of the component to be welded based on the component welding demand information and the component image data, to obtain a component welding decision, including:

Performing feature recognition on the component image data according to an ORB algorithm to obtain component ORB feature data;

loading a plurality of sets of sample part weld records, wherein each set of sample part weld records includes sample part model information, sample part ORB feature data, sample part weld forming data, and a sample part weld plan;

based on preset feature twinning constraint, performing welding decision registration on the plurality of groups of sample part welding records according to the part model information and the part ORB feature data to obtain a part welding decision candidate domain;

and carrying out welding decision optimization on the part welding decision candidate domain according to the part welding seam forming requirement data based on preset optimizing constraint conditions, and generating the part welding decision.

Based on the preset feature twinning constraint, combining the part model information and the part ORB feature data, the method in the implementation mode carries out welding decision registration on the plurality of groups of sample part welding records so as to determine which sample part record is most suitable for the part to be welded currently. And then the preliminary screening of welding decision can be carried out according to the similarity. The preset feature twinning constraint is a threshold value and is used for judging whether the welding records of a plurality of groups of sample parts are suitable for the current task according to the part model information and the part ORB feature data.

In other words, if one or more of the plurality of groups of sample component welding records accords with the preset optimizing constraint condition, the fact that the component welding decision in the one or more sample component welding records has higher adaptation degree with the current task is indicated, a candidate decision range for subsequent decision optimization can be included, and then the self-adaptive generation of the component welding decision is realized through an optimization step.

In a possible implementation manner, based on a preset feature twinning constraint, performing welding decision registration on the plurality of groups of sample component welding records according to the component model information and the component ORB feature data to obtain a component welding decision candidate domain, including:

extracting a first set of sample part welding records according to the plurality of sets of sample part welding records, wherein the first set of sample part welding records comprises first sample part model information, first sample part ORB characteristic data, first sample part weld forming data and a first sample part welding scheme;

performing model twinning analysis based on the component model information and the first sample component model information to obtain a first model feature twinning coefficient;

performing an ORB feature twinning analysis based on the component ORB feature data and the first sample component ORB feature data, obtaining a first ORB feature twinning coefficient;

Weighting calculation is carried out on the first model feature twinning coefficient and the first ORB feature twinning coefficient based on a preset weight condition, and a first sample feature twinning coefficient is generated;

judging whether the first sample feature twinning coefficient meets the preset feature twinning constraint;

if the first sample feature twinning coefficient meets the preset feature twinning constraint, adding the first sample component weld forming data and the first sample component welding scheme to the component welding decision candidate domain;

and by analogy, based on the preset feature twinning constraint, continuing to perform welding decision registration on the plurality of groups of sample part welding records according to the part model information and the part ORB feature data, and constructing the part welding decision candidate domain.

In a possible implementation manner, based on a preset optimizing constraint condition, performing welding decision optimizing on the component welding decision candidate domain according to the component welding seam forming requirement data, and generating the component welding decision, including:

extracting a first candidate welding decision and a second candidate welding decision, and first decision sample weld forming data and second decision sample weld forming data respectively corresponding to the first candidate welding decision and the second candidate welding decision according to the component welding decision candidate domain;

Based on the component weld forming demand data, performing differential recognition on the first decision sample weld forming data and the second decision sample weld forming data respectively to obtain a first weld forming differential coefficient and a second weld forming differential coefficient;

screening a current optimal weld forming difference coefficient and a current optimal welding decision based on the first weld forming difference coefficient, the second weld forming difference coefficient, the first candidate welding decision and the second candidate welding decision;

and continuing to perform iterative optimization on the component welding decision candidate domain according to the component welding seam forming demand data based on the current optimal welding seam forming difference coefficient and the current optimal welding decision, and obtaining the component welding decision when the iterative optimization times meet the preset optimization constraint condition.

In a possible implementation, the welding robot performs welding control of the component to be welded according to the component welding decision, including:

welding monitoring is carried out on the parts to be welded according to the machine vision system, and part welding monitoring data are obtained;

performing weld forming characteristic identification based on the part welding monitoring data to acquire weld forming characteristic data;

Carrying out consistency analysis based on the weld forming demand data of the parts and the weld forming characteristic data to obtain a weld forming demand matching coefficient;

judging whether the weld joint forming requirement matching coefficient meets a preset requirement matching constraint;

and if the welding seam forming requirement matching coefficient does not meet the preset requirement matching constraint, generating a welding seam forming early warning signal.

In a second aspect, the present application also provides a machine vision based welding automation control system, wherein the system comprises:

the image acquisition module is used for acquiring images of the parts to be welded according to the machine vision system to obtain part image acquisition results;

the enhancement module is used for carrying out image enhancement on the component image acquisition result based on an MSRCR algorithm to obtain component image data;

the surface verification module is used for activating a cleanliness checker, and executing cleanliness verification of the parts to be welded in combination with the part image data to obtain a part cleanliness verification result;

the requirement acquisition module is used for loading the component welding requirement information of the component to be welded when the component cleanliness verification result is qualified, wherein the component welding requirement information comprises component model information and component welding seam forming requirement data;

The welding decision module is used for activating a welding decision algorithm to execute welding decision analysis of the parts to be welded based on the part welding demand information and the part image data to obtain a part welding decision;

and the control execution module is used for sending the component welding decision to a welding robot, and the welding robot executes the welding control of the component to be welded according to the component welding decision.

The invention discloses a welding automation control method and a system based on machine vision, comprising the following steps: acquiring an image of a part to be welded according to a machine vision system to obtain a part image acquisition result; image enhancement is carried out on the component image acquisition result based on an MSRCR algorithm, so that component image data are obtained; activating a cleanliness checker, and executing the cleanliness verification of the parts to be welded by combining the part image data to obtain a part cleanliness verification result; when the component cleanliness verification result is qualified, loading component welding requirement information of a component to be welded, wherein the component welding requirement information comprises component model information and component welding seam forming requirement data; based on the component welding demand information and the component image data, activating a welding decision algorithm to execute welding decision analysis of the component to be welded, and obtaining a component welding decision; and sending the component welding decision to a welding robot, and executing the welding control of the component to be welded by the welding robot according to the component welding decision. The welding automation control method and the welding automation control system based on the machine vision solve the technical problems of low automation degree, poor adaptability and unstable welding quality, and achieve the technical effects of improving the automation level and the adaptability and ensuring the welding quality.

Drawings

FIG. 1 is a flow chart of a machine vision-based welding automation control method of the present application;

fig. 2 is a schematic structural diagram of the welding automation control system based on machine vision.

Reference numerals illustrate: the device comprises an image acquisition module 11, an enhancement module 12, a surface verification module 13, a demand acquisition module 14, a welding decision module 15 and a control execution module 16.

Detailed Description

Some terms involved in the embodiments of the present application are explained in advance below.

Welding: manufacturing and assembly methods for achieving connection between components including arc welding, gas shield welding, laser welding, ultrasonic welding, and the like;

part welding demand information: specific requirements and information about the parts to be welded. May include the materials, dimensions, shape of the components, as well as the specific requirements of the weld, such as the location, dimensions, and quality criteria of the weld, etc.

The welding robot may be an arc welding robot, gas shield welding robot, laser welding robot, ultrasonic welding robot, or other device or apparatus that may implement the methods involved in welding.

The technical scheme who provides in the embodiment of this application is low, adaptability is poor, welding quality unstable technical problem for the degree of automation that solves prior art existence, and the whole thinking that adopts is as follows:

Firstly, carrying out image acquisition on a part to be welded through a machine vision system, and acquiring a part image acquisition result. And then, carrying out image enhancement on the component image acquisition result based on an MSRCR algorithm to obtain component image data. And then, activating a cleanliness checker, and executing the cleanliness verification of the parts to be welded by combining the image data of the parts to obtain a part cleanliness verification result. And when the component cleanliness verification result is qualified, loading component welding requirement information of the component to be welded, wherein the component welding requirement information comprises component model information and component weld forming requirement data. And activating a welding decision algorithm to execute welding decision analysis of the parts to be welded based on the part welding demand information and the part image data, and obtaining a part welding decision. And finally, sending the component welding decision to a welding robot, and executing the welding control of the component to be welded by the welding robot according to the component welding decision.

The foregoing aspects will be better understood by reference to the following detailed description of the invention taken in conjunction with the accompanying drawings and detailed description. It should be apparent that the described embodiments are only some of the embodiments of the present application and not all of the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments used to explain the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present application without making any inventive effort, are intended to be within the scope of the present invention. It should be noted that, for convenience of description, only some, but not all of the drawings related to the present invention are shown.

Example 1

FIG. 1 is a flow chart of a machine vision-based welding automation control method of the present application; wherein in one embodiment, the method comprises the following steps:

acquiring an image of a part to be welded according to a machine vision system to obtain a part image acquisition result;

specifically, after the image acquisition is performed on the part to be welded, the acquired part image acquisition result contains the image data of the welding position of the part to be welded. The machine vision system comprises a white light camera, an infrared camera, a structured light camera, a TOF and other image acquisition devices and corresponding ISP components, wherein the image data can be full-color images, infrared images, depth images and the like.

Exemplary, component image acquisition results are acquired, and first, component preparation is performed to ensure that components are placed in proper positions and that good lighting conditions are provided to ensure image quality. And then starting the machine vision system to perform relevant setting. Including adjusting camera parameters, setting acquisition modes, defining the position and orientation of the welded component, etc. And further image acquisition of the welded component using a machine vision system. This involves a camera or sensor scanning or photographing the welding area to acquire corresponding image data. Then, the acquired image data is preprocessed. This may include denoising, image enhancement, color correction, etc. to improve image quality and accuracy. Finally, the processed image result is stored as digitized data.

The above process utilizes a machine vision system to acquire, process and analyze image data to obtain detailed information about the parts to be welded. Such information may be used for quality control, positioning, path planning, etc. in an automated welding process.

Image enhancement is carried out on the component image acquisition result based on an MSRCR algorithm, and component image data are obtained;

the MSRCR (Multi-Scale Retinex with Color Restoration) is an algorithm for image enhancement, and is suitable for images with uneven illumination or lower contrast, so that less obvious detail features in the component image acquisition result can be better enhanced, and subsequent identification is facilitated.

Illustratively, first, an original image is decomposed into a plurality of scale images using a gaussian filter; then, carrying out contrast enhancement on the image of each scale; then, restoring the distorted color information after the contrast enhancement; and finally, synthesizing the image after contrast enhancement and color recovery into an enhanced image. The enhanced image ensures that the desired features are highlighted and that the critical information of the welded component is better visible.

Activating a cleanliness checker, and executing the cleanliness verification of the parts to be welded by combining the part image data to obtain a part cleanliness verification result;

Wherein the cleanliness check is used for analyzing the image and detecting surface defects, foreign objects or unclean areas. Is important to ensure the success of welding and other processes and the quality of products. By using a machine vision and cleanliness checker, this process can be performed automatically, improving efficiency and accuracy.

Illustratively, the cleanliness verifier determines possible contamination by detecting areas of different colors in the image. Analyzing textures in the image, searching abnormal or unusual texture features, and acquiring possible defects. Different shapes in the image are detected to determine whether a foreign object or lesion is present.

Further, activating a cleanliness checker, and executing the cleanliness verification of the to-be-welded component in combination with the component image data to obtain a component cleanliness verification result, wherein the method comprises the following steps:

the cleanliness checker comprises a cleanliness identification module and a cleanliness verification module;

inputting the component image data into the cleanliness recognition module to obtain a component cleanliness coefficient;

inputting the component cleanliness coefficient into the cleanliness verification module to generate a component cleanliness verification result;

the cleanliness verification module comprises a cleanliness verification condition, wherein the cleanliness verification condition comprises that if the part cleanliness coefficient is larger than/equal to a preset cleanliness threshold value, the obtained part cleanliness verification result is qualified;

And the cleanliness verification condition further comprises that if the part cleanliness coefficient is smaller than the preset cleanliness threshold, the obtained part cleanliness verification result is unqualified, and a part cleanliness abnormality early warning signal is generated.

Optionally, the cleanliness recognition module is configured to recognize features of the component image data that may have defects and the like and that are unfavorable for welding, and perform cleanliness evaluation based on the recognition result, to generate a cleanliness coefficient. Wherein the cleanliness factor is a quantitative evaluation index for the component image data, and is high

Illustratively, the component cleanliness factor is communicated to the cleanliness verification module to obtain a component cleanliness verification result, which includes passing or failing verification, which may be binary.

Specifically, the cleanliness verification is performed based on threshold detection: and setting a threshold value of the cleanliness according to the component cleanliness coefficient, and judging whether the cleanliness standard is met. And if the component cleanliness factor is greater than or equal to a preset cleanliness threshold, marking the component cleanliness verification result as qualified. And if the component cleanliness factor is smaller than the preset cleanliness threshold value, marking the component cleanliness verification result as unqualified. Meanwhile, generating a part cleaning abnormality early warning signal. This signal may be used to inform related personnel or systems to handle or investigate, including replacing components, cleaning components again, problem tracing, etc.

In some implementations, the method includes:

loading a sample part image data record and a sample part cleanliness factor record;

taking the image data record of the sample part as input data, taking the cleanliness coefficient record of the sample part as output supervision data, training the convolutional neural network, and acquiring network output accuracy rate when training is performed for preset times;

and if the network output accuracy is greater than or equal to the preset accuracy, generating a cleanliness identification model, and embedding the cleanliness identification model into the cleanliness identification module.

Optionally, the cleanliness recognition module is a supervised training result based on a convolutional neural network. Specifically, firstly, preparing a sample part image data record and a corresponding sample part cleanliness coefficient record in advance, wherein the sample part image data record and the corresponding sample part cleanliness coefficient record are related to each other; then, using a Convolutional Neural Network (CNN), the sample part image data is recorded as input and the sample part cleanliness coefficients are recorded as output. And performing multiple training, and acquiring the output accuracy of the network at the end of each training. Then, whether the output accuracy of the network reaches or exceeds the preset accuracy is checked, and whether the model is trained well enough is determined. Exemplary constraints include convergence of the continuous preset number of times accuracy, reaching or exceeding the continuous preset number of times accuracy, completion of iterative training of the preset number of times, and the like.

Optionally, if the network output accuracy rate reaches or exceeds the preset accuracy rate, generating a cleanliness recognition model. And embedding the generated cleanliness recognition model into a cleanliness recognition module so as to be used for realizing end-to-end cleanliness verification. The model for identifying the cleanliness is learned from sample data by using a neural network model, and the model is applied to carry out real-time cleanliness verification after a certain accuracy is achieved.

When the component cleanliness verification result is qualified, loading component welding demand information of the component to be welded, wherein the component welding demand information comprises component model information and component welding seam forming demand data;

alternatively, the component welding requirement information may be provided by parsing a target welding job book or the like or by a related technician. So as to improve the flexibility of the system and enable the system to adapt to task books and other data with different forms and sources. The method accords with the diversity and complexity of information acquisition in actual situations.

Wherein the part model information includes specific model, specification or other identifying information identifying the part to be welded. Such information is critical to the validation of welded components to ensure that the correct welding parameters and procedures are used. The component weld formation requirement data includes the shape, size, location, etc. requirements of the weld. To ensure that the quality of the welded joint meets the design requirements and that the required weld joint is accurately formed during the welding process.

Activating a welding decision algorithm to execute welding decision analysis of the parts to be welded based on the part welding demand information and the part image data, and obtaining a part welding decision;

further, based on the component welding requirement information and the component image data, activating a welding decision algorithm to execute welding decision analysis of the component to be welded to obtain a component welding decision, including:

performing feature recognition on the component image data according to an ORB algorithm to obtain component ORB feature data;

loading a plurality of sets of sample part weld records, wherein each set of sample part weld records includes sample part model information, sample part ORB feature data, sample part weld forming data, and a sample part weld plan;

based on preset feature twinning constraint, performing welding decision registration on the plurality of groups of sample part welding records according to the part model information and the part ORB feature data to obtain a part welding decision candidate domain;

and carrying out welding decision optimization on the part welding decision candidate domain according to the part welding seam forming requirement data based on preset optimizing constraint conditions, and generating the part welding decision.

ORB feature data of the component is obtained by feature recognition of the component image data using an ORB algorithm. Then, a plurality of sets of sample part weld records are loaded, wherein each set of records includes model information of the sample part, ORB characterization data, weld formation data, and a welding scheme. And then, a plurality of groups of sample data are utilized for learning and model establishment so as to support the follow-up more accurate analysis and decision-making on the welding of the parts.

The ORB algorithm (Oriented FAST and Rotated BRIEF) is a computer vision algorithm and is used for detecting and describing key feature points in an image, and has the characteristics of rotation invariance and high calculation efficiency. The component image data is processed through an ORB algorithm, key points in the image are identified and extracted, and ORB feature data describing the key points is generated. The ORB feature data is a compact and efficient representation for representing structures and textures in an image.

Specifically, each set of sample part weld records of the plurality of sets of sample part weld records includes various aspects of information including: identifying the model or class of the component for classifying the different types of components; image characteristic data processed by ORB algorithm provides position and description information of key points; sample component weld formation data describing the shape, structure, or other characteristics of the weld; planning or step of the actual welding operation.

In some implementations, based on a preset feature twinning constraint, performing welding decision registration on the plurality of sets of sample part welding records according to the part model information and the part ORB feature data to obtain a part welding decision candidate domain, including:

extracting a first set of sample part welding records according to the plurality of sets of sample part welding records, wherein the first set of sample part welding records comprises first sample part model information, first sample part ORB characteristic data, first sample part weld forming data and a first sample part welding scheme;

performing model twinning analysis based on the component model information and the first sample component model information to obtain a first model feature twinning coefficient;

performing an ORB feature twinning analysis based on the component ORB feature data and the first sample component ORB feature data, obtaining a first ORB feature twinning coefficient;

weighting calculation is carried out on the first model feature twinning coefficient and the first ORB feature twinning coefficient based on a preset weight condition, and a first sample feature twinning coefficient is generated;

judging whether the first sample feature twinning coefficient meets the preset feature twinning constraint;

If the first sample feature twinning coefficient meets the preset feature twinning constraint, adding the first sample component weld forming data and the first sample component welding scheme to the component welding decision candidate domain;

and by analogy, based on the preset feature twinning constraint, continuing to perform welding decision registration on the plurality of groups of sample part welding records according to the part model information and the part ORB feature data, and constructing the part welding decision candidate domain.

Specifically, first, a first set of sample part weld records is extracted from a plurality of sets of sample part weld records, including part model information, ORB characterization data, weld formation data, and a welding scheme. And then comparing the part model information with the first sample part model information, and executing model twinning analysis to obtain a first model characteristic twinning coefficient. The model feature twinning coefficient reflects the degree of matching between the model of the sample part and the model of the part to be welded. The ORB feature data is then compared to the first sample component ORB feature data, and an ORB feature twinning analysis is performed to obtain a first ORB feature twinning coefficient. The ORB feature twinning coefficient reflects a degree of similarity between the sample component features and the component features to be welded.

Optionally, based on a preset weight condition, the first model feature twinning coefficient and the first ORB feature twinning coefficient are weighted and calculated to generate a first sample feature twinning coefficient. Through weighted calculation, importance degree differences among different factors influencing welding quality are fully introduced, and further, a sample characteristic twin coefficient with higher effectiveness is obtained.

Optionally, if the first sample feature twinning coefficient does not meet the preset feature twinning constraint, removing the first sample component weld forming data and the first sample component welding scheme, and re-selecting the sample component welding scheme.

The steps are based on the part model information and ORB characteristic data, and whether the sample part meets the preset characteristic twinning constraint is judged through characteristic twinning analysis and weighted calculation. Only sample parts that meet the constraints will be added to the part weld decision candidate field.

In some implementations, based on a preset optimizing constraint condition, performing welding decision optimization on the component welding decision candidate domain according to the component welding seam forming requirement data, generating the component welding decision includes:

extracting a first candidate welding decision and a second candidate welding decision, and first decision sample weld forming data and second decision sample weld forming data respectively corresponding to the first candidate welding decision and the second candidate welding decision according to the component welding decision candidate domain;

Based on the component weld forming demand data, performing differential recognition on the first decision sample weld forming data and the second decision sample weld forming data respectively to obtain a first weld forming differential coefficient and a second weld forming differential coefficient;

screening a current optimal weld forming difference coefficient and a current optimal welding decision based on the first weld forming difference coefficient, the second weld forming difference coefficient, the first candidate welding decision and the second candidate welding decision;

and continuing to perform iterative optimization on the component welding decision candidate domain according to the component welding seam forming demand data based on the current optimal welding seam forming difference coefficient and the current optimal welding decision, and obtaining the component welding decision when the iterative optimization times meet the preset optimization constraint condition.

Optionally, a random extraction is performed in the component welding decision candidate domain to obtain first and second candidate welding decisions, and their corresponding first and second decision sample weld formation data. And then, based on the component weld forming requirement data, performing differential identification on the first and second decision sample weld forming data respectively to obtain first and second weld forming differential coefficients. The quantitative evaluation of the forming difference coefficient evaluates the deviation of the weld forming data of the first decision sample and the second decision sample from the target forming data. Candidate welding decisions with smaller molding variance coefficients have better weld quality. In other words, the decision sample weld forming data with low forming difference coefficient values is more likely to be close to the component weld forming requirements. And selecting the minimum value of the first welding seam forming difference coefficient and the second welding seam forming difference coefficient and the candidate welding decision corresponding to the current optimal welding seam forming difference coefficient in the first candidate welding decision and the second candidate welding decision, and storing the candidate welding decision as the current optimal welding seam forming difference coefficient and the current optimal welding decision. For subsequent decision optimization.

Optionally, based on the current optimal result, continuing to iteratively optimize the component welding decision candidate domain. In each iteration, according to the weld forming requirement data of the part, performing differential recognition to obtain a new weld forming differential coefficient and a corresponding welding decision, and judging whether the iterative optimizing times meet preset optimizing constraint conditions. And when the iteration times meet the optimizing constraint condition, namely the iteration optimizing times threshold, obtaining a final component welding decision.

Optionally, if the iteration times meet the optimizing constraint condition, namely the iteration optimizing times threshold, the new weld forming difference coefficient is obtained and is not superior to the current optimal weld forming difference coefficient, the current optimal weld forming difference coefficient is stored as a component welding decision.

And sending the component welding decision to a welding robot, and executing the welding control of the component to be welded by the welding robot according to the component welding decision.

Further, the welding robot performs welding control of the to-be-welded component according to the component welding decision, including:

welding monitoring is carried out on the parts to be welded according to the machine vision system, and part welding monitoring data are obtained;

Performing weld forming characteristic identification based on the part welding monitoring data to acquire weld forming characteristic data;

carrying out consistency analysis based on the weld forming demand data of the parts and the weld forming characteristic data to obtain a weld forming demand matching coefficient;

judging whether the weld joint forming requirement matching coefficient meets a preset requirement matching constraint;

and if the welding seam forming requirement matching coefficient does not meet the preset requirement matching constraint, generating a welding seam forming early warning signal.

Optionally, the welding robot monitors the parts to be welded through the machine vision system to acquire welding monitoring data. Wherein the welding monitoring data includes images, video, or other relevant data during the welding process. For performing weld formation feature identification based on the weld monitoring data.

Optionally, if the weld forming requirement matching coefficient does not meet the preset requirement matching constraint, generating a weld forming early warning signal. For alerting an operator or system administrator to potential welding problems, further adjustments or interventions are required.

In summary, the welding automation control method based on machine vision provided by the invention has the following technical effects:

Image acquisition is carried out on the parts to be welded through a machine vision system, and a part image acquisition result is obtained; image enhancement is carried out on the component image acquisition result based on an MSRCR algorithm, so that component image data are obtained; activating a cleanliness checker, and executing the cleanliness verification of the parts to be welded by combining the part image data to obtain a part cleanliness verification result; when the component cleanliness verification result is qualified, loading component welding requirement information of a component to be welded, wherein the component welding requirement information comprises component model information and component welding seam forming requirement data; based on the component welding demand information and the component image data, activating a welding decision algorithm to execute welding decision analysis of the component to be welded, and obtaining a component welding decision; and sending the component welding decision to a welding robot, and executing the welding control of the component to be welded by the welding robot according to the component welding decision. The welding automation control method and the welding automation control system based on the machine vision solve the technical problems of low automation degree, poor adaptability and unstable welding quality, and achieve the technical effects of improving the automation level and the adaptability and ensuring the welding quality.

Example two

Fig. 2 is a schematic structural diagram of the welding automation control system based on machine vision of the present application. For example, the flow diagram of the machine vision-based welding automation control method in fig. 1 may be implemented by the structure shown in fig. 2.

Based on the same conception as the welding automation control method based on machine vision in the embodiment, the welding automation control system based on machine vision further provided by the application comprises:

the image acquisition module 11 is used for acquiring images of the parts to be welded according to the machine vision system to obtain part image acquisition results;

an enhancing module 12, configured to perform image enhancement on the component image acquisition result based on an MSRCR algorithm, to obtain component image data;

a surface verification module 13, configured to activate a cleanliness checker, perform cleanliness verification of the component to be welded in combination with the component image data, and obtain a component cleanliness verification result;

a requirement acquisition module 14, configured to load component welding requirement information of the component to be welded when the component cleanliness verification result is qualified, where the component welding requirement information includes component model information and component weld formation requirement data;

The welding decision module 15 is configured to activate a welding decision algorithm to perform welding decision analysis of the component to be welded based on the component welding requirement information and the component image data, so as to obtain a component welding decision;

and a control execution module 16, configured to send the component welding decision to a welding robot, where the welding robot executes welding control of the component to be welded according to the component welding decision.

Wherein the surface verification module 13 comprises:

a cleanliness recognition unit that obtains a part cleanliness coefficient by inputting the part image data to the cleanliness recognition module;

a cleanliness verification unit that generates the part cleanliness verification result by inputting the part cleanliness coefficient to the cleanliness verification module;

the cleanliness verification module comprises a cleanliness verification condition, wherein the cleanliness verification condition comprises that if the part cleanliness coefficient is larger than/equal to a preset cleanliness threshold value, the obtained part cleanliness verification result is qualified;

and the cleanliness verification condition further comprises that if the part cleanliness coefficient is smaller than the preset cleanliness threshold, the obtained part cleanliness verification result is unqualified, and a part cleanliness abnormality early warning signal is generated.

Further, the surface verification module 13 further includes:

a sample acquisition unit for loading a sample part image data record and a sample part cleanliness factor record;

the iterative training unit is used for training the convolutional neural network by taking the image data record of the sample part as input data and taking the cleanliness coefficient record of the sample part as output supervision data, and acquiring network output accuracy rate when training for preset times;

and the constraint embedding unit is used for generating a cleanliness recognition model and embedding the cleanliness recognition model into the cleanliness recognition module if the network output accuracy is greater than or equal to the preset accuracy.

Wherein the welding decision module 15 comprises:

the feature extraction unit is used for carrying out feature recognition on the component image data according to an ORB algorithm to obtain component ORB feature data;

the system comprises a record acquisition unit, a record processing unit and a record processing unit, wherein the record acquisition unit is used for loading a plurality of groups of sample part welding records, and each group of sample part welding records comprises sample part model information, sample part ORB characteristic data, sample part weld forming data and a sample part welding scheme;

the region configuration unit is used for carrying out welding decision registration on the plurality of groups of sample part welding records according to the part model information and the part ORB characteristic data based on preset feature twinning constraint to obtain a part welding decision candidate domain;

The decision adaptation unit is used for carrying out welding decision optimization on the part welding decision candidate domain according to the part welding seam forming requirement data based on preset optimizing constraint conditions, and generating the part welding decision.

In some implementations, the area configuration unit includes:

a record extraction unit for extracting a first set of sample part weld records according to the plurality of sets of sample part weld records, wherein the first set of sample part weld records includes first sample part model information, first sample part ORB feature data, first sample part weld forming data, and a first sample part weld plan;

a model twinning analysis unit for performing model twinning analysis based on the component model information and the first sample component model information to obtain a first model feature twinning coefficient;

an ORB twinning analysis unit for performing an ORB feature twinning analysis based on the component ORB feature data and the first sample component ORB feature data, obtaining a first ORB feature twinning coefficient;

the twin adding unit is used for carrying out weighted calculation on the first model characteristic twin coefficient and the first ORB characteristic twin coefficient based on a preset weight condition to generate a first sample characteristic twin coefficient;

The twin constraint unit is used for judging whether the first sample characteristic twin coefficient meets the preset characteristic twin constraint;

a decision candidate domain construction unit configured to add the first sample component weld forming data and the first sample component welding scheme to the component welding decision candidate domain if the first sample feature twinning coefficient satisfies the preset feature twinning constraint;

and by analogy, based on the preset feature twinning constraint, continuing to perform welding decision registration on the plurality of groups of sample part welding records according to the part model information and the part ORB feature data, and constructing the part welding decision candidate domain.

In some implementations, the decision adaptation unit includes:

the decision extraction unit is used for extracting a first candidate welding decision and a second candidate welding decision as well as first decision sample weld forming data and second decision sample weld forming data respectively corresponding to the first candidate welding decision and the second candidate welding decision according to the component welding decision candidate domain;

a difference calculation unit, configured to perform difference recognition on the first decision sample weld forming data and the second decision sample weld forming data based on the component weld forming demand data, respectively, to obtain a first weld forming difference coefficient and a second weld forming difference coefficient;

The decision optimization unit is used for screening a current optimal welding seam forming difference coefficient and a current optimal welding decision based on the first welding seam forming difference coefficient, the second welding seam forming difference coefficient, the first candidate welding decision and the second candidate welding decision;

the decision optimizing unit is used for continuing to carry out iterative optimization on the component welding decision candidate domain according to the component welding forming demand data based on the current optimal welding seam forming difference coefficient and the current optimal welding decision, and obtaining the component welding decision when the iterative optimization times meet the preset optimizing constraint condition.

Wherein the control execution module 16 includes:

the welding monitoring unit is used for performing welding monitoring on the to-be-welded parts according to the machine vision system to obtain part welding monitoring data;

the forming characteristic identification unit is used for carrying out weld forming characteristic identification based on the part welding monitoring data and acquiring weld forming characteristic data;

the consistency analysis unit is used for carrying out consistency analysis based on the welding seam forming demand data of the parts and the welding seam forming characteristic data to obtain a welding seam forming demand matching coefficient;

The matching constraint unit is used for judging whether the welding seam forming requirement matching coefficient meets the preset requirement matching constraint;

and the forming early warning unit is used for generating a welding seam forming early warning signal if the welding seam forming requirement matching coefficient does not meet the preset requirement matching constraint.

It should be understood that the embodiments mentioned in this specification focus on the differences from other embodiments, and that the specific embodiment in the first embodiment is equally applicable to the machine vision-based welding automation control system described in the second embodiment, and is not further developed herein for brevity of description.

It should be understood that the embodiments disclosed herein and the foregoing description may enable one skilled in the art to utilize the present application. While the present application is not limited to the above-mentioned embodiments, it should be understood that: modifications of the technical solutions described in the foregoing embodiments or equivalent substitutions of some technical features thereof may be still performed by those skilled in the art; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.

Claims (8)

1. A machine vision-based welding automation control method, the method comprising:

acquiring an image of a part to be welded according to a machine vision system to obtain a part image acquisition result;

image enhancement is carried out on the component image acquisition result based on an MSRCR algorithm, and component image data are obtained;

activating a cleanliness checker, and executing the cleanliness verification of the parts to be welded by combining the part image data to obtain a part cleanliness verification result;

when the component cleanliness verification result is qualified, loading component welding demand information of the component to be welded, wherein the component welding demand information comprises component model information and component welding seam forming demand data;

activating a welding decision algorithm to execute welding decision analysis of the parts to be welded based on the part welding demand information and the part image data, and obtaining a part welding decision;

and sending the component welding decision to a welding robot, and executing the welding control of the component to be welded by the welding robot according to the component welding decision.

2. The method of claim 1, wherein activating a cleanliness checker to perform cleanliness verification of the parts to be welded in conjunction with the part image data, obtaining a part cleanliness verification result, comprises:

The cleanliness checker comprises a cleanliness identification module and a cleanliness verification module;

inputting the component image data into the cleanliness recognition module to obtain a component cleanliness coefficient;

inputting the component cleanliness coefficient into the cleanliness verification module to generate a component cleanliness verification result;

the cleanliness verification module comprises a cleanliness verification condition, wherein the cleanliness verification condition comprises that if the part cleanliness coefficient is larger than/equal to a preset cleanliness threshold value, the obtained part cleanliness verification result is qualified;

and the cleanliness verification condition further comprises that if the part cleanliness coefficient is smaller than the preset cleanliness threshold, the obtained part cleanliness verification result is unqualified, and a part cleanliness abnormality early warning signal is generated.

3. The method according to claim 2, wherein the method comprises:

loading a sample part image data record and a sample part cleanliness factor record;

taking the image data record of the sample part as input data, taking the cleanliness coefficient record of the sample part as output supervision data, training the convolutional neural network, and acquiring network output accuracy rate when training is performed for preset times;

And if the network output accuracy is greater than or equal to the preset accuracy, generating a cleanliness identification model, and embedding the cleanliness identification model into the cleanliness identification module.

4. The method of claim 1, wherein activating a welding decision algorithm to perform a welding decision analysis of the component to be welded based on the component welding demand information and the component image data, obtaining a component welding decision comprises:

performing feature recognition on the component image data according to an ORB algorithm to obtain component ORB feature data;

loading a plurality of sets of sample part weld records, wherein each set of sample part weld records includes sample part model information, sample part ORB feature data, sample part weld forming data, and a sample part weld plan;

based on preset feature twinning constraint, performing welding decision registration on the plurality of groups of sample part welding records according to the part model information and the part ORB feature data to obtain a part welding decision candidate domain;

and carrying out welding decision optimization on the part welding decision candidate domain according to the part welding seam forming requirement data based on preset optimizing constraint conditions, and generating the part welding decision.

5. The method of claim 4, wherein performing a welding decision registration of the plurality of sets of sample part weld records based on the part model information and the part ORB feature data based on a preset feature twinning constraint to obtain a part welding decision candidate domain, comprising:

extracting a first set of sample part welding records according to the plurality of sets of sample part welding records, wherein the first set of sample part welding records comprises first sample part model information, first sample part ORB characteristic data, first sample part weld forming data and a first sample part welding scheme;

performing model twinning analysis based on the component model information and the first sample component model information to obtain a first model feature twinning coefficient;

performing an ORB feature twinning analysis based on the component ORB feature data and the first sample component ORB feature data, obtaining a first ORB feature twinning coefficient;

weighting calculation is carried out on the first model feature twinning coefficient and the first ORB feature twinning coefficient based on a preset weight condition, and a first sample feature twinning coefficient is generated;

judging whether the first sample feature twinning coefficient meets the preset feature twinning constraint;

If the first sample feature twinning coefficient meets the preset feature twinning constraint, adding the first sample component weld forming data and the first sample component welding scheme to the component welding decision candidate domain;

and by analogy, based on the preset feature twinning constraint, continuing to perform welding decision registration on the plurality of groups of sample part welding records according to the part model information and the part ORB feature data, and constructing the part welding decision candidate domain.

6. The method of claim 4, wherein performing welding decision optimization on the component welding decision candidate domain based on the component weld formation requirement data based on a preset optimization constraint, generating the component welding decision comprises:

extracting a first candidate welding decision and a second candidate welding decision, and first decision sample weld forming data and second decision sample weld forming data respectively corresponding to the first candidate welding decision and the second candidate welding decision according to the component welding decision candidate domain;

based on the component weld forming demand data, performing differential recognition on the first decision sample weld forming data and the second decision sample weld forming data respectively to obtain a first weld forming differential coefficient and a second weld forming differential coefficient;

Screening a current optimal weld forming difference coefficient and a current optimal welding decision based on the first weld forming difference coefficient, the second weld forming difference coefficient, the first candidate welding decision and the second candidate welding decision;

and continuing to perform iterative optimization on the component welding decision candidate domain according to the component welding seam forming demand data based on the current optimal welding seam forming difference coefficient and the current optimal welding decision, and obtaining the component welding decision when the iterative optimization times meet the preset optimization constraint condition.

7. The method of claim 1, wherein the welding robot performs welding control of the parts to be welded according to the part welding decision, comprising:

welding monitoring is carried out on the parts to be welded according to the machine vision system, and part welding monitoring data are obtained;

performing weld forming characteristic identification based on the part welding monitoring data to acquire weld forming characteristic data;

carrying out consistency analysis based on the weld forming demand data of the parts and the weld forming characteristic data to obtain a weld forming demand matching coefficient;

judging whether the weld joint forming requirement matching coefficient meets a preset requirement matching constraint;

And if the welding seam forming requirement matching coefficient does not meet the preset requirement matching constraint, generating a welding seam forming early warning signal.

8. A machine vision based welding automation control system, the system comprising:

the image acquisition module is used for acquiring images of the parts to be welded according to the machine vision system to obtain part image acquisition results;

the enhancement module is used for carrying out image enhancement on the component image acquisition result based on an MSRCR algorithm to obtain component image data;

the surface verification module is used for activating a cleanliness checker, and executing cleanliness verification of the parts to be welded in combination with the part image data to obtain a part cleanliness verification result;

the requirement acquisition module is used for loading the component welding requirement information of the component to be welded when the component cleanliness verification result is qualified, wherein the component welding requirement information comprises component model information and component welding seam forming requirement data;

the welding decision module is used for activating a welding decision algorithm to execute welding decision analysis of the parts to be welded based on the part welding demand information and the part image data to obtain a part welding decision;

And the control execution module is used for sending the component welding decision to a welding robot, and the welding robot executes the welding control of the component to be welded according to the component welding decision.