CN116168031B - Welding code generation method based on three-dimensional image, welding system and related equipment - Google Patents
Welding code generation method based on three-dimensional image, welding system and related equipment Download PDFInfo
- Publication number
- CN116168031B CN116168031B CN202310452472.XA CN202310452472A CN116168031B CN 116168031 B CN116168031 B CN 116168031B CN 202310452472 A CN202310452472 A CN 202310452472A CN 116168031 B CN116168031 B CN 116168031B
- Authority
- CN
- China
- Prior art keywords
- welding
- point cloud
- welded
- cloud image
- dimensional assembly
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K31/00—Processes relevant to this subclass, specially adapted for particular articles or purposes, but not covered by only one of the preceding main groups
- B23K31/02—Processes relevant to this subclass, specially adapted for particular articles or purposes, but not covered by only one of the preceding main groups relating to soldering or welding
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K37/00—Auxiliary devices or processes, not specially adapted to a procedure covered by only one of the preceding main groups
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Mechanical Engineering (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Robotics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Optics & Photonics (AREA)
- Manipulator (AREA)
Abstract
The invention provides a welding code generation method, a welding system and related equipment based on a three-dimensional image, and relates to the technical field of three-dimensional image processing, wherein the method comprises the following steps: acquiring a three-dimensional assembly point cloud image corresponding to the three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained recognition model, and acquiring a member recognition result to be welded output by the recognition model; determining welding point information corresponding to each welding component pair in the three-dimensional assembly point cloud image based on the identification result of the components to be welded; generating a connection path based on the welding edges, and connecting the welding edges based on the connection path to generate a welding path; generating a welding code based on the welding path, wherein the welding code is used for controlling the welding robot to weld the components to be welded, and the welding code comprises the welding path, coordinate information of each welding spot in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding spot. The invention can realize automatic generation of welding codes and reduce labor cost.
Description
Technical Field
The present invention relates to the field of three-dimensional image processing technologies, and in particular, to a welding code generating method, a welding system and related devices based on three-dimensional images.
Background
Because of the convenience and visibility of three-dimensional modeling, three-dimensional software is adopted to draw a three-dimensional model when many mechanical parts are designed, and then the parts are manufactured based on the three-dimensional model. At present, a welding robot is used for carrying out mass welding more and more, but in the prior art, based on a three-dimensional model drawn by a designer, an experienced person is required to determine a welding process according to specific connection requirements of each part (for example, what part needs what degree of connection), manually codes are compiled, and the codes are input to the welding robot to control the welding robot to carry out automatic welding. This approach requires high labor costs.
Disclosure of Invention
The invention provides a welding code generation method, a welding system and related equipment based on a three-dimensional image, which are used for solving the defects of high labor cost and the need of manually compiling a control code of a welding robot in the prior art and realizing the reduction of labor cost in the welding code generation process.
The invention provides a welding code generation method based on a three-dimensional image, which comprises the following steps:
acquiring a three-dimensional assembly point cloud image corresponding to a three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained identification model, and acquiring a component to be welded identification result output by the identification model, wherein the component to be welded identification result comprises names of components to be welded contained in the three-dimensional assembly model and areas occupied by the components to be welded in the three-dimensional assembly point cloud image;
determining welding point information corresponding to each welding member pair in the three-dimensional assembly point cloud image based on the identification result of the members to be welded, wherein the welding point information comprises a welding edge, welding point positions on the welding edge and welding process parameters corresponding to each welding point, and each welding member pair comprises two members to be welded which need to be welded;
generating a connection path based on the welding edges, and connecting the welding edges based on the connection path to generate a welding path;
generating a welding code based on the welding path, wherein the welding code is used for controlling a welding robot to weld the components to be welded, and the welding code comprises the welding path, coordinate information of each welding spot in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding spot.
According to the welding code generation method based on the three-dimensional image, the method for determining the welding spot information corresponding to each welding component pair in the three-dimensional assembly point cloud image based on the identification result of the component to be welded comprises the following steps:
determining a connecting edge in the three-dimensional assembly point cloud image based on the identification result of the to-be-welded member, wherein the connecting edge is an edge connecting two to-be-welded members in the welding member pair;
generating a three-dimensional mark point cloud image based on a connecting edge in the three-dimensional assembly point cloud image, wherein the three-dimensional mark point cloud image comprises coordinate information of each point in the three-dimensional assembly point cloud image and connecting edge mark information corresponding to each point, and the connecting edge mark information reflects whether the point is on the connecting edge or not;
determining semantic vectors corresponding to the components to be welded in the three-dimensional assembly point cloud image based on the identification result of the components to be welded, wherein the semantic vectors reflect names and materials of the components to be welded, generating a three-dimensional semantic point cloud image based on the semantic vectors, wherein the three-dimensional semantic point cloud image comprises coordinate information of each point in the three-dimensional assembly point cloud image and semantic information corresponding to each point, and the semantic information reflects semantic vectors of names of the components to be welded corresponding to areas where the points are located;
Inputting the three-dimensional mark point cloud image and the three-dimensional semantic point cloud image into a trained point cloud processing model, and acquiring the welding spot information output by the point cloud processing model;
the point cloud processing model comprises a fusion layer and an output layer, wherein the fusion layer is used for fusing the three-dimensional mark point cloud image and the three-dimensional semantic point cloud image to obtain a target point cloud image, and the output layer is used for outputting the welding spot information based on the target point cloud image.
According to the welding code generation method based on the three-dimensional image, the semantic vector corresponding to each member to be welded in the three-dimensional assembly point cloud image is determined based on the recognition result of the member to be welded, and the method comprises the following steps:
determining the material corresponding to the member to be welded based on a pre-established mapping relation between the name and the material;
inputting the names of the components to be welded and the materials of the components to be welded into a trained semantic extraction model, and obtaining semantic vectors corresponding to the components to be welded, which are output by the semantic extraction model;
the semantic extraction model and the point cloud processing model are obtained based on combined training of multiple sets of training data, and each set of training data comprises a sample three-dimensional assembly point cloud image, a sample welding component recognition result and a welding point information label.
According to the welding code generation method based on the three-dimensional image, the method for acquiring the three-dimensional assembly point cloud image corresponding to the three-dimensional assembly model comprises the following steps:
setting point densities corresponding to a first area and a second area in the three-dimensional assembly model based on the three-dimensional assembly model, wherein the point density corresponding to the first area is smaller than the point density corresponding to the second area, the first area is an area in a preset range around an upper edge of the surface of the three-dimensional assembly model, and the second area is an area of the surface of the three-dimensional assembly model except for the first area;
and carrying out shell extraction processing on the three-dimensional assembly model, and carrying out point cloud conversion on the three-dimensional assembly model after shell extraction based on the point densities corresponding to the first area and the second area respectively to obtain the three-dimensional assembly point cloud image.
According to the welding code generation method based on the three-dimensional image, which is provided by the invention, a connection path is generated based on the welding edge, and the welding code generation method comprises the following steps:
acquiring first coordinate information and second coordinate information corresponding to each welding edge, wherein the first coordinate information is the coordinate of the starting point of the welding edge in the three-dimensional assembly point cloud image, and the second coordinate information is the coordinate of the ending point of the welding edge in the three-dimensional assembly point cloud image;
Optimizing a connection path scheme based on the degree of freedom of movement of a welding arm of the welding robot, and generating the connection path based on a connection path scheme corresponding to an optimization result;
in the process of optimizing the connection path scheme, the optimization target is that the total length of the welding path corresponding to the connection path scheme is shortest.
According to the welding code generation method based on the three-dimensional image, after the welding code is generated based on the welding path, the method further comprises the following steps:
combining the welding codes with the three-dimensional assembly point cloud image to obtain welding guidance data;
and sending the welding guidance data to the welding robot, so that when the welding robot performs welding, coordinate conversion is performed on coordinate information of each welding point in the welding code based on a point cloud image of a real object to be welded and the three-dimensional assembly point cloud image, and welding is performed on the welding point after coordinate conversion based on welding process parameters in the welding code.
The invention also provides a welding code generating device based on the three-dimensional image, which comprises the following steps:
the component recognition module is used for acquiring a three-dimensional assembly point cloud image corresponding to the three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained recognition model, and acquiring a component recognition result to be welded output by the recognition model, wherein the component recognition result to be welded comprises names of components to be welded contained in the three-dimensional assembly model and areas occupied by the components to be welded in the three-dimensional assembly point cloud image;
The welding point information generation module is used for determining welding point information corresponding to each welding member pair in the three-dimensional assembly point cloud image based on the recognition result of the members to be welded, wherein the welding point information comprises a welding edge, welding point positions on the welding edge and welding process parameters corresponding to each welding point, and each welding member pair comprises two members to be welded which need to be welded;
the welding path generation module is used for generating a connection path based on the welding edges, connecting the welding edges based on the connection path and generating a welding path;
the welding code generation module is used for generating a welding code based on the welding path, the welding code is used for controlling a welding robot to weld the components to be welded, and the welding code comprises the welding path, coordinate information of each welding spot in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding spot.
The invention also provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the welding code generation method based on the three-dimensional image when executing the program.
The invention also provides a welding system which comprises the electronic equipment and the welding robot;
the electronic equipment is used for acquiring a three-dimensional assembly point cloud image corresponding to a three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained identification model, and acquiring a component to be welded identification result output by the identification model, wherein the component to be welded identification result comprises names of components to be welded contained in the three-dimensional assembly model and areas occupied by the components to be welded in the three-dimensional assembly point cloud image; determining welding point information corresponding to each welding member pair in the three-dimensional assembly point cloud image based on the identification result of the members to be welded, wherein the welding point information comprises a welding edge, welding point positions on the welding edge and welding process parameters corresponding to each welding point, and each welding member pair comprises two members to be welded which need to be welded; generating a connection path based on the welding edges, and connecting the welding edges based on the connection path to generate a welding path; generating a welding code based on the welding path, wherein the welding code is used for controlling a welding robot to weld the components to be welded, and the welding code comprises the welding path, coordinate information of each welding spot in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding spot;
The welding robot is used for acquiring the three-dimensional assembly point cloud image and the welding code, carrying out point cloud identification on objects to be welded to generate an object point cloud image, carrying out point cloud matching on the object point cloud image and the three-dimensional assembly point cloud image, carrying out coordinate conversion on coordinate information of each welding point in the welding code based on a matching result to obtain the object coordinates of each welding point in the welding code, and carrying out welding on the object coordinates of each welding point based on welding process parameters corresponding to each welding point.
The present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a three-dimensional image-based welding code generation method as described in any one of the above.
According to the welding code generation method, the welding system and the related equipment based on the three-dimensional image, the three-dimensional assembly model is converted into the three-dimensional assembly point cloud image, the three-dimensional assembly point cloud image is firstly input into the trained identification model, the identification result of the to-be-welded component including the name and the material of the to-be-welded component in the three-dimensional assembly model and the area occupied in the three-dimensional assembly point cloud image is identified, then the welding spot information including the welding edge, the welding position on the welding edge and the welding process parameters of welding in the three-dimensional assembly point cloud is determined based on the identification result of the to-be-welded component, the connection path is generated based on the welding edge, and then the welding code including the welding path, the coordinate information of each welding spot in the three-dimensional assembly point cloud image and the welding process parameters corresponding to each welding spot in the welding path is generated based on the welding path, so that a welding robot can automatically weld based on the welding code.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a welding code generation method based on three-dimensional images provided by the invention;
FIG. 2 is a schematic structural view of a welding code generating device based on three-dimensional images provided by the invention;
fig. 3 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The method for generating a welding code based on a three-dimensional image according to the present invention is described below with reference to fig. 1, and as shown in fig. 1, the method for generating a welding code based on a three-dimensional image according to the present invention includes the steps of:
s110, acquiring a three-dimensional assembly point cloud image corresponding to a three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained identification model, and acquiring a component to be welded identification result output by the identification model, wherein the component to be welded identification result comprises names and materials of components to be welded, which are included in the three-dimensional assembly model, and areas occupied by the components to be welded in the three-dimensional assembly point cloud image.
The three-dimensional assembly model may be a model drawn based on three-dimensional modeling software, such as UG, solidworks, etc., including the shape, size, mutual combination relation, etc., of each member to be welded. In order to facilitate automatic coordinate conversion of a welding code generated in the later stage when a welding robot welds a real object, the method provided by the invention firstly converts the three-dimensional assembly model into a three-dimensional point cloud image, and the three-dimensional point cloud image can facilitate point cloud matching of the later stage welding robot based on the point cloud image of the real object, so that correspondence between virtual data and actual data is realized, and accurate actual welding is realized.
Specifically, the acquiring the cloud image of the three-dimensional assembly point corresponding to the three-dimensional assembly model includes:
setting point densities corresponding to a first area and a second area in the three-dimensional assembly model based on the three-dimensional assembly model, wherein the point density corresponding to the first area is smaller than the point density corresponding to the second area, the first area is an area in a preset range around an upper edge of the surface of the three-dimensional assembly model, and the second area is an area of the surface of the three-dimensional assembly model except for the first area;
and carrying out shell extraction processing on the three-dimensional assembly model, and carrying out point cloud conversion on the three-dimensional assembly model after shell extraction based on the point densities corresponding to the first area and the second area respectively to obtain the three-dimensional assembly point cloud image.
Since the welding is performed at the joint of the two members, detailed information on the joint edge of the two members is required, and identification of the members to be welded can be achieved only by the shape of the members to be welded. In order to reduce the data processing amount, in the scheme provided by the invention, a first area and a second area are defined in the three-dimensional assembly model, wherein the first area is an area within a preset range around a border in the three-dimensional assembly model, the preset range can be determined based on the size of each model component in the three-dimensional assembly model, the second area is an area except the first area, the point density corresponding to the first area is smaller than the point density corresponding to the second area, for example, the point density corresponding to the first area can be set to be N times of the point density corresponding to the second area, and N is greater than 1. That is, the density of points in the corresponding portion of the edge portion of the model in the three-dimensional assembly point cloud image is greater, so that sufficient edge information can be ensured, recognition of the welding edge and the welding spot on the welding edge is facilitated, and sufficient information of the shape boundary can be ensured, so that the shape of an accurate member can be reflected. The density of the points in the corresponding part of the three-dimensional assembly point cloud image in other parts is smaller, so that the number of the points in the three-dimensional assembly point cloud image can be effectively reduced, and the data calculation amount is reduced.
Furthermore, in the process of generating the welding code, only the shape information of the member to be welded is needed, so that how to the internal structure of the member to be welded is not needed, and in order to further reduce the data quantity, in the method provided by the application, the three-dimensional assembly model is further subjected to shell extraction processing, so that only the surface point cloud can be generated, and the number of points in the three-dimensional assembly point cloud image is further reduced.
In the same field, for example, in the field of building elements, the same type of elements tend to have similar shapes, such as steel columns, corner column L-shaped pieces, etc., and therefore the names of the elements to be welded included in the three-dimensional assembly point cloud image, and the occupied areas, can be identified by the shape features in the three-dimensional assembly point cloud image. The identification model can be obtained by adopting the existing framework of a model for processing the three-dimensional point cloud image and adopting data training comprising a sample three-dimensional assembly point cloud image of the field of the member to be welded and a corresponding identification result label of the member to be welded.
As shown in fig. 1, the method provided by the present application further includes the steps of:
s120, determining welding point information corresponding to each welding member pair in the three-dimensional assembly point cloud image based on the recognition result of the members to be welded, wherein the welding point information comprises a welding edge, welding point positions on the welding edge and welding process parameters corresponding to each welding point, and each welding member pair comprises two members to be welded which need to be welded;
The identification result of the to-be-welded piece comprises names of to-be-welded components included in the three-dimensional assembly model and areas occupied by the to-be-welded components in the three-dimensional assembly point cloud image, more than two to-be-welded components in the three-dimensional assembly model are likely to be needed to be connected, welding edges of the two to-be-welded components, positions of welding points on the welding edges and welding technological parameters corresponding to each welding point are to be identified, specifically, the welding technological parameters corresponding to the welding points comprise welding step length and welding temperature, and for arc welding, the welding temperature can be represented by current pulse intensity.
Based on the identification result of the member to be welded, determining welding spot information corresponding to each welding member pair in the three-dimensional assembly point cloud image comprises the following steps:
determining a connecting edge in the three-dimensional assembly point cloud image based on the identification result of the to-be-welded member, wherein the connecting edge is an edge connecting two to-be-welded members in the welding member pair;
generating a three-dimensional mark point cloud image based on a connecting edge in the three-dimensional assembly point cloud image, wherein the three-dimensional mark point cloud image comprises coordinate information of each point in the three-dimensional assembly point cloud image and connecting edge mark information corresponding to each point, and the connecting edge mark information reflects whether the point is on the connecting edge or not;
Determining semantic vectors corresponding to the components to be welded in the three-dimensional assembly point cloud image based on the identification result of the components to be welded, wherein the semantic vectors reflect names and materials of the components to be welded, generating a three-dimensional semantic point cloud image based on the semantic vectors, wherein the three-dimensional semantic point cloud image comprises coordinate information of each point in the three-dimensional assembly point cloud image and semantic information corresponding to each point, and the semantic information reflects semantic vectors of the components to be welded corresponding to areas where the points are located;
inputting the three-dimensional mark point cloud image and the three-dimensional semantic point cloud image into a trained point cloud processing model, and acquiring the welding spot information output by the point cloud processing model;
the point cloud processing model comprises a fusion layer and an output layer, wherein the fusion layer is used for fusing the three-dimensional mark point cloud image and the three-dimensional semantic point cloud image to obtain a target point cloud image, and the output layer is used for outputting the welding spot information based on the target point cloud image.
Specifically, based on the area occupied by each member to be welded in the three-dimensional assembly point cloud image in the recognition result of the member to be welded, each member to be welded can be distinguished in the three-dimensional assembly point cloud image, so that the connecting edges of two members to be welded in the pair of welding members are determined, and the welding edges can only be edges in the connecting edges. And associating connection side information with each point in the three-dimensional assembly point cloud image to generate the three-dimensional mark point cloud image, namely associating connection side mark information with each point in the three-dimensional mark point cloud image in addition to the associated coordinate information. In this way, the three-dimensional mark point cloud image includes information such as the position and the length of the connecting edge.
Further, the determination of the welding parameters is further determined by comprehensively considering the names and the materials of the components to be welded, wherein the names of the components to be welded can reflect the connection strength requirements of the components to be welded, and the materials of the components to be welded can reflect the welding temperature requirements of the components to be welded. Therefore, in the method provided by the invention, besides the three-dimensional mark point cloud image, the three-dimensional semantic point cloud image which can reflect the name and the material of the component to be welded is generated. And fusing the three-dimensional mark point cloud image and the three-dimensional semantic point cloud image to obtain a target point cloud image, and outputting the welding spot information based on the target point cloud image.
Specifically, the determining, based on the recognition result of the to-be-welded member, the semantic vector corresponding to each to-be-welded member in the three-dimensional assembly point cloud image includes:
determining the material corresponding to the member to be welded based on a pre-established mapping relation;
inputting the names of the components to be welded and the materials of the components to be welded into a trained semantic extraction model, and obtaining semantic vectors corresponding to the components to be welded, which are output by the semantic extraction model;
The semantic extraction model and the point cloud processing model are obtained based on combined training of multiple sets of training data, and each set of training data comprises a sample three-dimensional assembly point cloud image, a sample welding component recognition result and a welding point information label.
And presetting various materials corresponding to the to-be-welded components, namely establishing a mapping relation between names and the materials, so that after the names corresponding to the to-be-welded components are identified, the materials corresponding to the to-be-welded components can be determined based on the mapping relation.
The welding spot information labels comprise welding edge labels, welding spot position labels and welding process parameter labels corresponding to welding spots.
Inputting the name of the member to be welded and the material of the member to be welded into a trained semantic extraction model, obtaining a semantic vector output by the semantic extraction model, and associating the corresponding semantic vector with points in the three-dimensional assembly point cloud image to obtain the three-dimensional semantic point cloud image. Specifically, for a point in the three-dimensional assembly point cloud image, associating a semantic vector of the member to be welded corresponding to the area where the point is located. It is easy to see that the three-dimensional semantic point cloud image can reflect the names and materials of the components to be welded existing in the three-dimensional assembly point cloud image, and identification of welding information can be achieved by utilizing the information.
The welding edge recognition result in the welding spot information can be a probability vector, and the probability vector is the probability that each connecting edge is a welding edge. The welding spot position recognition result may be the density of the welding spot on the welding edge, and the welding process parameter corresponding to the welding spot may be the welding process parameter corresponding to the welding edge where the welding spot is located, that is, the welding process parameters corresponding to the welding spots on the same welding edge are the same.
Specifically, fusing the three-dimensional mark point cloud image and the three-dimensional semantic point cloud image to obtain a target point cloud image, including:
determining points of the three-dimensional semantic point cloud image, which are associated with two semantic vectors;
fusing two semantic vectors associated with the same point, so that at most one semantic vector is associated with the point in the three-dimensional semantic point cloud image to obtain a processed three-dimensional semantic point cloud image;
and fusing the processed three-dimensional semantic point cloud image and the three-dimensional mark point cloud image to obtain the target point cloud image.
The semantic extraction model and the point cloud processing model are trained together, and the training process of the semantic extraction model and the point cloud processing model comprises the following steps:
Selecting target training data from the plurality of groups of training data;
inputting the names and the materials of the sample to-be-welded components in the sample to-be-welded component identification results in the target training data into the semantic extraction model to obtain sample semantic vectors output by the semantic extraction model;
generating a sample three-dimensional mark point cloud image and a sample three-dimensional semantic point cloud image based on the sample to-be-welded component identification result, the sample semantic vector and the sample three-dimensional assembly point cloud image;
inputting the sample three-dimensional mark point cloud image and the sample three-dimensional semantic point cloud image into the point cloud processing model to obtain sample welding spot information output by the point cloud processing model;
determining training loss based on the sample welding spot information, the sample semantic vector and a welding spot information label in the target training data, and updating parameters of the semantic extraction model and the point cloud processing model based on the training loss;
and repeating the step of selecting target training data from the plurality of sets of training data until the parameters of the semantic extraction model and the point cloud processing model are converged.
Specifically, the determining the training loss based on the sample welding spot information, the sample semantic vector and the welding spot information label in the target training data includes:
Inputting the sample semantic vector into a prediction model, obtaining a prediction name and a prediction material outputted by the prediction model, and determining a first loss based on the difference between the prediction name and the name of the member to be welded in the sample member to be welded identification result and the difference between the prediction material and the material of the member to be welded in the sample member to be welded identification result;
determining a second loss based on a difference between the sample weld information and the weld information tag;
the training loss is determined based on the first loss and the second loss.
Based on the first loss and the second loss, determining the training loss, and based on the training loss, updating parameters of the semantic extraction model and the point cloud processing model can accelerate the training rate, so that model parameters approach an optimal solution more quickly.
The updating parameters of the semantic extraction model and the point cloud processing model based on the training loss comprises the following steps:
and updating parameters of the semantic extraction model, the point cloud processing model and the prediction model based on the training loss.
In order to better evaluate whether the semantic vector extracted by the semantic extraction model can reflect the name and the material of the component to be welded, in the method provided by the invention, a prediction model is adopted to evaluate the semantic vector extracted by the semantic extraction model. Specifically, when the semantic vector extracted by the semantic extraction model can reflect the name and the material of the member to be welded, the name and the material of the member to be welded can be reversely deduced based on the semantic vector. Therefore, when the semantic extraction model and the point cloud processing model are trained, semantic vectors extracted by the semantic extraction model are input into a prediction model, a prediction name and a prediction material output by the prediction model are obtained, and a first loss is determined based on the prediction name and the prediction material. The predictive model may be an existing classification model.
As can be seen from the foregoing description, the solder joint information includes various data: welding edges, welding spot positions on the welding edges and welding process parameters corresponding to welding spots. The determining a second loss based on a difference between the sample weld information and the weld information tag comprises:
obtaining a first probability vector based on the sample welding spot information, wherein the first probability vector comprises probabilities that each sample connecting edge is a welding edge, obtaining a first partial loss based on the distance between the first probability vector and a second probability vector, wherein the second probability vector is a vector obtained based on the welding spot information label, and in the second probability vector, the probability value corresponding to the welding edge in the welding spot information label is 1, and the probability values corresponding to the rest connecting edges are 0;
obtaining second loss based on the welding spot position information in the sample welding spot information and the welding spot position information in the welding spot information label;
and obtaining third loss based on the welding process parameters in the sample welding spot information and the welding process parameters in the welding spot information label.
And carrying out weighted summation on the first partial loss, the second partial loss and the third partial loss to obtain the second loss.
Specifically, the determining the training loss based on the first loss and the second loss includes:
carrying out weighted summation on the first loss and the second loss to obtain the training loss;
when the first partial loss is larger than a preset threshold value, multiplying the weight value of the second loss by a preset coefficient, wherein the preset coefficient is larger than 1.
The recognition of the welding edge is the basis of welding process parameters corresponding to the welding point position and the welding point, so when the difference between the recognition result of the welding edge and the real result is large, penalty measures are necessary to be added, namely the weight value of the second loss is increased, and the weight value of the first loss is kept unchanged, so that the training loss is increased, and better parameters can be learned more quickly in the model training process.
Referring again to fig. 1, the welding code generating method based on the three-dimensional image provided by the invention further comprises the following steps:
and S130, generating a connection path based on the welding edges, and connecting the welding edges based on the connection path to generate a welding path.
After the welding edges are obtained, the welding sequence of the welding edges needs to be planned, so that the welding robot can realize continuous welding of a plurality of weldments. Specifically, the generating a connection path based on the welding edge includes:
Acquiring first coordinate information and second coordinate information corresponding to each welding edge, wherein the first coordinate information is the coordinate of the starting point of the welding edge in the three-dimensional assembly point cloud image, and the second coordinate information is the coordinate of the ending point of the welding edge in the three-dimensional assembly point cloud image;
optimizing a connection path scheme based on the degree of freedom of movement of a welding arm of the welding robot, and generating the connection path based on a connection path scheme corresponding to an optimization result;
in the process of optimizing the connection path scheme, the optimization target is that the total length of the welding path corresponding to the connection path scheme is shortest.
In the method provided by the invention, an optimization algorithm is adopted to realize optimization of the connection path schemes, each connection path scheme comprises a connection path for connecting each welding edge, and the optimization target is that the shortest welding path is formed after each welding edge is connected by adopting the connection path scheme. Existing optimization algorithms, such as genetic algorithms, simulated annealing algorithms, etc., may be employed. In the optimization process, the method provided by the invention is considered to be applied to automatic welding of the welding robot, so that constraint conditions based on the welding robot are added in the optimization process, wherein the constraint conditions are that the welding path corresponding to the connection path scheme cannot exceed the range capable of being realized by the motion freedom degree of the welding arm, namely, after each connection edge is connected based on the connection path, the obtained welding path cannot exceed the range of the motion freedom degree of the welding robot, and therefore, the applicability of an optimization result in practice can be ensured.
And S140, generating a welding code based on the welding path, wherein the welding code is used for controlling a welding robot to weld the members to be welded, and comprises the welding path, coordinate information of each welding point in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding point.
Based on the welding path and the weld spot information, the welding code may be generated, which may be written in a code language that the welder robot may parse.
Further, after generating the welding code based on the welding path, the method provided by the invention further comprises:
combining the welding codes with the three-dimensional assembly point cloud image to obtain welding guidance data;
and sending the welding guidance data to the welding robot, so that when the welding robot performs welding, coordinate conversion is performed on coordinate information of each welding point in the welding code based on a point cloud image of a real object to be welded and the three-dimensional assembly point cloud image, and welding is performed on the welding point after coordinate conversion based on welding process parameters in the welding code.
In order to realize full-automatic welding, in the method provided by the invention, besides sending the welding codes to the welding robot so as to control the welding robot to carry out welding based on the welding codes, the three-dimensional assembly point cloud image is sent to the welding robot, when the welding robot actually executes welding, the welding robot carries out computer vision imaging on the real object to be welded, generates the point cloud image of the real object to be welded, carries out point cloud matching on the point cloud image of the real object to be welded and the three-dimensional assembly point cloud image, converts the coordinate position of a welding spot in the three-dimensional assembly point cloud image into the coordinate position in the point cloud image of the real object to be welded based on the matching result, and further converts the coordinate position into the coordinate position in the operation environment of the welding robot so as to realize the fact facing different visit positions of the welding robot, and also can carry out automatic identification welding.
In summary, according to the welding code generation method based on the three-dimensional image, the three-dimensional assembly model is converted into the three-dimensional assembly point cloud image, the three-dimensional assembly point cloud image is input into the trained identification model, the identification result of the to-be-welded component including the name and the material of the to-be-welded component in the three-dimensional assembly model and the area occupied in the three-dimensional assembly point cloud image is identified, the welding point information including the welding edge, the welding position on the welding edge and the welding process parameters of welding in the three-dimensional assembly point cloud is determined based on the identification result of the to-be-welded component, the connection path is generated based on the welding edge, and the welding code including the welding path, the coordinate information of each welding point in the three-dimensional assembly point cloud image and the welding process parameters corresponding to each welding point in the welding path is generated based on the welding path, so that the welding robot can automatically weld based on the welding code.
The welding code generating device based on the three-dimensional image provided by the invention is described below, and the welding code generating device based on the three-dimensional image and the welding code generating method based on the three-dimensional image described above can be correspondingly referred to each other. As shown in fig. 2, the welding code generating device based on three-dimensional image provided by the invention comprises: component identification module 210, weld information generation module 220, weld path generation module 230, and weld code generation module 240.
The component recognition module 210 is configured to obtain a three-dimensional assembly point cloud image corresponding to a three-dimensional assembly model, input the three-dimensional assembly point cloud image into a trained recognition model, and obtain a recognition result of a component to be welded output by the recognition model, where the recognition result of the component to be welded includes a name of the component to be welded included in the three-dimensional assembly model and an area occupied by each component to be welded in the three-dimensional assembly point cloud image.
The welding spot information generating module 220 is configured to determine, based on the identification result of the to-be-welded member, welding spot information corresponding to each welding member pair in the three-dimensional assembly point cloud image, where the welding spot information includes a welding edge, a welding spot position on the welding edge, and a welding process parameter corresponding to each welding spot, and each welding member pair includes two to-be-welded members to be welded.
The welding path generating module 230 is configured to generate a connection path based on the welding edges, and connect the welding edges based on the connection path, so as to generate a welding path.
The welding code generating module 240 is configured to generate a welding code based on the welding path, where the welding code is configured to control a welding robot to weld the member to be welded, and the welding code includes the welding path, coordinate information of each welding point in the welding path in the three-dimensional assembly point cloud image, and welding process parameters corresponding to each welding point.
Fig. 3 illustrates a physical schematic diagram of an electronic device, as shown in fig. 3, where the electronic device may include: processor 310, communication interface (Communications Interface) 320, memory 330 and communication bus 340, wherein processor 310, communication interface 320, memory 330 accomplish communication with each other through communication bus 340. The processor 310 may invoke logic instructions in the memory 330 to perform a three-dimensional image based welding code generation method comprising: acquiring a three-dimensional assembly point cloud image corresponding to a three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained identification model, and acquiring a component to be welded identification result output by the identification model, wherein the component to be welded identification result comprises names of components to be welded contained in the three-dimensional assembly model and areas occupied by the components to be welded in the three-dimensional assembly point cloud image;
Determining welding point information corresponding to each welding member pair in the three-dimensional assembly point cloud image based on the identification result of the members to be welded, wherein the welding point information comprises a welding edge, welding point positions on the welding edge and welding process parameters corresponding to each welding point, and each welding member pair comprises two members to be welded which need to be welded;
generating a connection path based on the welding edges, and connecting the welding edges based on the connection path to generate a welding path;
generating a welding code based on the welding path, wherein the welding code is used for controlling a welding robot to weld the components to be welded, and the welding code comprises the welding path, coordinate information of each welding spot in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding spot.
Further, the logic instructions in the memory 330 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
On the other hand, the invention also provides a welding system, which comprises the electronic equipment and the welding robot, wherein the electronic equipment is used for acquiring a three-dimensional assembly point cloud image corresponding to a three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained recognition model, and acquiring a recognition result of a member to be welded, which is output by the recognition model, wherein the recognition result of the member to be welded comprises names of members to be welded, which are included in the three-dimensional assembly model, and areas occupied by the members to be welded in the three-dimensional assembly point cloud image; determining welding point information corresponding to each welding member pair in the three-dimensional assembly point cloud image based on the identification result of the members to be welded, wherein the welding point information comprises a welding edge, welding point positions on the welding edge and welding process parameters corresponding to each welding point, and each welding member pair comprises two members to be welded which need to be welded; generating a connection path based on the welding edges, and connecting the welding edges based on the connection path to generate a welding path; generating a welding code based on the welding path, wherein the welding code is used for controlling a welding robot to weld the components to be welded, and the welding code comprises the welding path, coordinate information of each welding spot in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding spot;
The welding robot is used for acquiring the three-dimensional assembly point cloud image and the welding code, carrying out point cloud identification on objects to be welded to generate an object point cloud image, carrying out point cloud matching on the object point cloud image and the three-dimensional assembly point cloud image, carrying out coordinate conversion on coordinate information of each welding point in the welding code based on a matching result to obtain the object coordinates of each welding point in the welding code, and carrying out welding on the object coordinates of each welding point based on welding process parameters corresponding to each welding point.
In another aspect, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of executing the three-dimensional image-based welding code generation method provided by the above methods, the method comprising: acquiring a three-dimensional assembly point cloud image corresponding to a three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained identification model, and acquiring a component to be welded identification result output by the identification model, wherein the component to be welded identification result comprises names of components to be welded contained in the three-dimensional assembly model and areas occupied by the components to be welded in the three-dimensional assembly point cloud image;
Determining welding point information corresponding to each welding member pair in the three-dimensional assembly point cloud image based on the identification result of the members to be welded, wherein the welding point information comprises a welding edge, welding point positions on the welding edge and welding process parameters corresponding to each welding point, and each welding member pair comprises two members to be welded which need to be welded;
generating a connection path based on the welding edges, and connecting the welding edges based on the connection path to generate a welding path;
generating a welding code based on the welding path, wherein the welding code is used for controlling a welding robot to weld the components to be welded, and the welding code comprises the welding path, coordinate information of each welding spot in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding spot.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the three-dimensional image-based welding code generation method provided by the above methods, the method comprising: acquiring a three-dimensional assembly point cloud image corresponding to a three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained identification model, and acquiring a component to be welded identification result output by the identification model, wherein the component to be welded identification result comprises names of components to be welded contained in the three-dimensional assembly model and areas occupied by the components to be welded in the three-dimensional assembly point cloud image;
Determining welding point information corresponding to each welding member pair in the three-dimensional assembly point cloud image based on the identification result of the members to be welded, wherein the welding point information comprises a welding edge, welding point positions on the welding edge and welding process parameters corresponding to each welding point, and each welding member pair comprises two members to be welded which need to be welded;
generating a connection path based on the welding edges, and connecting the welding edges based on the connection path to generate a welding path;
generating a welding code based on the welding path, wherein the welding code is used for controlling a welding robot to weld the components to be welded, and the welding code comprises the welding path, coordinate information of each welding spot in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding spot.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A welding code generation method based on a three-dimensional image, comprising:
acquiring a three-dimensional assembly point cloud image corresponding to a three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained identification model, and acquiring a component to be welded identification result output by the identification model, wherein the component to be welded identification result comprises names of components to be welded contained in the three-dimensional assembly model and areas occupied by the components to be welded in the three-dimensional assembly point cloud image;
determining welding point information corresponding to each welding member pair in the three-dimensional assembly point cloud image based on the identification result of the members to be welded, wherein the welding point information comprises a welding edge, welding point positions on the welding edge and welding process parameters corresponding to each welding point, and each welding member pair comprises two members to be welded which need to be welded;
generating a connection path based on the welding edges, and connecting the welding edges based on the connection path to generate a welding path;
generating a welding code based on the welding path, wherein the welding code is used for controlling a welding robot to weld the components to be welded, and the welding code comprises the welding path, coordinate information of each welding spot in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding spot.
2. The welding code generation method based on the three-dimensional image according to claim 1, wherein the determining, based on the recognition result of the member to be welded, the welding spot information corresponding to each welding member pair in the three-dimensional assembly point cloud image includes:
determining a connecting edge in the three-dimensional assembly point cloud image based on the identification result of the to-be-welded member, wherein the connecting edge is an edge connecting two to-be-welded members in the welding member pair;
generating a three-dimensional mark point cloud image based on a connecting edge in the three-dimensional assembly point cloud image, wherein the three-dimensional mark point cloud image comprises coordinate information of each point in the three-dimensional assembly point cloud image and connecting edge mark information corresponding to each point, and the connecting edge mark information reflects whether the point is on the connecting edge or not;
determining semantic vectors corresponding to the components to be welded in the three-dimensional assembly point cloud image based on the identification result of the components to be welded, wherein the semantic vectors reflect names and materials of the components to be welded, generating a three-dimensional semantic point cloud image based on the semantic vectors, wherein the three-dimensional semantic point cloud image comprises coordinate information of each point in the three-dimensional assembly point cloud image and semantic information corresponding to each point, and the semantic information reflects semantic vectors of the components to be welded corresponding to areas where the points are located;
Inputting the three-dimensional mark point cloud image and the three-dimensional semantic point cloud image into a trained point cloud processing model, and acquiring the welding spot information output by the point cloud processing model;
the point cloud processing model comprises a fusion layer and an output layer, wherein the fusion layer is used for fusing the three-dimensional mark point cloud image and the three-dimensional semantic point cloud image to obtain a target point cloud image, and the output layer is used for outputting the welding spot information based on the target point cloud image.
3. The welding code generation method based on the three-dimensional image according to claim 2, wherein the determining semantic vectors corresponding to the respective members to be welded in the three-dimensional assembly point cloud image based on the recognition result of the members to be welded includes:
determining the material corresponding to the member to be welded based on a pre-established mapping relation between the name and the material;
inputting the names of the components to be welded and the materials of the components to be welded into a trained semantic extraction model, and obtaining semantic vectors corresponding to the components to be welded, which are output by the semantic extraction model;
the semantic extraction model and the point cloud processing model are obtained based on combined training of multiple sets of training data, and each set of training data comprises a sample three-dimensional assembly point cloud image, a sample welding component recognition result and a welding point information label.
4. The method for generating a welding code based on a three-dimensional image according to claim 1, wherein the acquiring a three-dimensional assembly point cloud image corresponding to a three-dimensional assembly model comprises:
setting point densities corresponding to a first area and a second area in the three-dimensional assembly model based on the three-dimensional assembly model, wherein the point density corresponding to the first area is smaller than the point density corresponding to the second area, the first area is an area in a preset range around an upper edge of the surface of the three-dimensional assembly model, and the second area is an area of the surface of the three-dimensional assembly model except for the first area;
and carrying out shell extraction processing on the three-dimensional assembly model, and carrying out point cloud conversion on the three-dimensional assembly model after shell extraction based on the point densities corresponding to the first area and the second area respectively to obtain the three-dimensional assembly point cloud image.
5. The three-dimensional image-based welding code generation method according to claim 1, wherein the generating a connection path based on the welding edge comprises:
acquiring first coordinate information and second coordinate information corresponding to each welding edge, wherein the first coordinate information is the coordinate of the starting point of the welding edge in the three-dimensional assembly point cloud image, and the second coordinate information is the coordinate of the ending point of the welding edge in the three-dimensional assembly point cloud image;
Optimizing a connection path scheme based on the degree of freedom of movement of a welding arm of the welding robot, and generating the connection path based on a connection path scheme corresponding to an optimization result;
in the process of optimizing the connection path scheme, the optimization target is that the total length of the welding path corresponding to the connection path scheme is shortest.
6. The three-dimensional image-based welding code generation method according to claim 1, wherein after the welding code is generated based on the welding path, the method further comprises:
combining the welding codes with the three-dimensional assembly point cloud image to obtain welding guidance data;
and sending the welding guidance data to the welding robot, so that when the welding robot performs welding, coordinate conversion is performed on coordinate information of each welding point in the welding code based on a point cloud image of a real object to be welded and the three-dimensional assembly point cloud image, and welding is performed on the welding point after coordinate conversion based on welding process parameters in the welding code.
7. A welding code generation apparatus based on a three-dimensional image, comprising:
the component recognition module is used for acquiring a three-dimensional assembly point cloud image corresponding to the three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained recognition model, and acquiring a component recognition result to be welded output by the recognition model, wherein the component recognition result to be welded comprises names of components to be welded contained in the three-dimensional assembly model and areas occupied by the components to be welded in the three-dimensional assembly point cloud image;
The welding point information generation module is used for determining welding point information corresponding to each welding member pair in the three-dimensional assembly point cloud image based on the recognition result of the members to be welded, wherein the welding point information comprises a welding edge, welding point positions on the welding edge and welding process parameters corresponding to each welding point, and each welding member pair comprises two members to be welded which need to be welded;
the welding path generation module is used for generating a connection path based on the welding edges, connecting the welding edges based on the connection path and generating a welding path;
the welding code generation module is used for generating a welding code based on the welding path, the welding code is used for controlling a welding robot to weld the components to be welded, and the welding code comprises the welding path, coordinate information of each welding spot in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding spot.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the three-dimensional image based welding code generation method according to any one of claims 1 to 6 when executing the computer program.
9. A welding system comprising the electronic device of claim 8 and a welding robot;
the electronic equipment is used for acquiring a three-dimensional assembly point cloud image corresponding to a three-dimensional assembly model, inputting the three-dimensional assembly point cloud image into a trained identification model, and acquiring a component to be welded identification result output by the identification model, wherein the component to be welded identification result comprises names of components to be welded contained in the three-dimensional assembly model and areas occupied by the components to be welded in the three-dimensional assembly point cloud image; determining welding point information corresponding to each welding member pair in the three-dimensional assembly point cloud image based on the identification result of the members to be welded, wherein the welding point information comprises a welding edge, welding point positions on the welding edge and welding process parameters corresponding to each welding point, and each welding member pair comprises two members to be welded which need to be welded; generating a connection path based on the welding edges, and connecting the welding edges based on the connection path to generate a welding path; generating a welding code based on the welding path, wherein the welding code is used for controlling a welding robot to weld the components to be welded, and the welding code comprises the welding path, coordinate information of each welding spot in the welding path in the three-dimensional assembly point cloud image and welding process parameters corresponding to each welding spot;
The welding robot is used for acquiring the three-dimensional assembly point cloud image and the welding code, carrying out point cloud identification on objects to be welded to generate an object point cloud image, carrying out point cloud matching on the object point cloud image and the three-dimensional assembly point cloud image, carrying out coordinate conversion on coordinate information of each welding point in the welding code based on a matching result to obtain the object coordinates of each welding point in the welding code, and carrying out welding on the object coordinates of each welding point based on welding process parameters corresponding to each welding point.
10. A non-transitory computer readable storage medium having stored thereon a computer program, characterized in that the computer program, when executed by a processor, implements the three-dimensional image-based welding code generation method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310452472.XA CN116168031B (en) | 2023-04-25 | 2023-04-25 | Welding code generation method based on three-dimensional image, welding system and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310452472.XA CN116168031B (en) | 2023-04-25 | 2023-04-25 | Welding code generation method based on three-dimensional image, welding system and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116168031A CN116168031A (en) | 2023-05-26 |
| CN116168031B true CN116168031B (en) | 2023-08-29 |
Family
ID=86413552
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310452472.XA Active CN116168031B (en) | 2023-04-25 | 2023-04-25 | Welding code generation method based on three-dimensional image, welding system and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116168031B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111360785A (en) * | 2020-02-25 | 2020-07-03 | 季华实验室 | Robot-based welding method and device, industrial robot and storage medium |
| CN114473309A (en) * | 2022-03-18 | 2022-05-13 | 南京欧睿三维科技有限公司 | Welding position identification method for automatic welding system and automatic welding system |
| CN115131268A (en) * | 2021-03-25 | 2022-09-30 | 南京知谱光电科技有限公司 | Automatic welding system based on image feature extraction and three-dimensional model matching |
| CN115810133A (en) * | 2023-02-09 | 2023-03-17 | 中建科技集团有限公司 | Welding control method based on image processing and point cloud processing and related equipment |
-
2023
- 2023-04-25 CN CN202310452472.XA patent/CN116168031B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111360785A (en) * | 2020-02-25 | 2020-07-03 | 季华实验室 | Robot-based welding method and device, industrial robot and storage medium |
| CN115131268A (en) * | 2021-03-25 | 2022-09-30 | 南京知谱光电科技有限公司 | Automatic welding system based on image feature extraction and three-dimensional model matching |
| CN114473309A (en) * | 2022-03-18 | 2022-05-13 | 南京欧睿三维科技有限公司 | Welding position identification method for automatic welding system and automatic welding system |
| CN115810133A (en) * | 2023-02-09 | 2023-03-17 | 中建科技集团有限公司 | Welding control method based on image processing and point cloud processing and related equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116168031A (en) | 2023-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4153528B2 (en) | Apparatus, program, recording medium and method for robot simulation | |
| EP3863791B1 (en) | System and method for weld path generation | |
| CN111461398A (en) | Welding process parameter optimization method and device and readable storage medium | |
| CN114599488B (en) | Machine learning data generation device, machine learning device, work system, computer program, machine learning data generation method, and work machine manufacturing method | |
| EP3786736A1 (en) | Control for a laser cutting head movement in a cutting process | |
| CN112757284B (en) | Robot control device, method, and storage medium | |
| CN112518756B (en) | Motion trajectory planning method and device for mechanical arm, mechanical arm and storage medium | |
| CN115810133B (en) | Welding control method based on image processing and point cloud processing and related equipment | |
| CN111476771A (en) | Domain self-adaptive method and system for generating network based on distance countermeasure | |
| CN113205070A (en) | Visual perception algorithm optimization method and system | |
| CN116188526A (en) | Track generation method, device, equipment and medium | |
| CN115461199A (en) | Task-oriented 3D reconstruction for autonomous robotic operation | |
| CN115063768A (en) | Three-dimensional target detection method, encoder and decoder | |
| CN116168031B (en) | Welding code generation method based on three-dimensional image, welding system and related equipment | |
| CN116276973A (en) | Visual perception grabbing training method based on deep learning | |
| CN112220405A (en) | Self-moving tool cleaning route updating method, device, computer equipment and medium | |
| JP2021135977A (en) | Apparatus and method for processing information | |
| CN116152335A (en) | Apparatus and method for training machine learning model to generate descriptor images | |
| US20240013542A1 (en) | Information processing system, information processing device, information processing method, and recording medium | |
| US20210299795A1 (en) | System and Method for Detecting and Correcting Laser-Cutting Distortion | |
| CN114029940B (en) | Motion path planning method, device, equipment, medium and mechanical arm | |
| JP2021047797A (en) | Machine learning device, machine learning method, and program | |
| CN113592800B (en) | Image scanning method and device based on dynamic scanning parameters | |
| JP2019197441A (en) | Learning device, learning method, and learning program | |
| JP7074723B2 (en) | Learning equipment and programs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |