CN116596883A - Metal structural part weld joint identification method, system and equipment based on machine vision - Google Patents
Metal structural part weld joint identification method, system and equipment based on machine vision Download PDFInfo
- Publication number
- CN116596883A CN116596883A CN202310562954.0A CN202310562954A CN116596883A CN 116596883 A CN116596883 A CN 116596883A CN 202310562954 A CN202310562954 A CN 202310562954A CN 116596883 A CN116596883 A CN 116596883A
- Authority
- CN
- China
- Prior art keywords
- straight line
- dimensional model
- image
- metal structural
- machine vision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000002184 metal Substances 0.000 title claims abstract description 76
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000003466 welding Methods 0.000 claims abstract description 81
- 238000006243 chemical reaction Methods 0.000 claims abstract description 20
- 238000000605 extraction Methods 0.000 claims abstract description 12
- 238000012545 processing Methods 0.000 claims description 30
- 238000001514 detection method Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012216 screening Methods 0.000 claims description 7
- 238000012937 correction Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 6
- 230000000877 morphologic effect Effects 0.000 claims description 6
- 238000005304 joining Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 description 9
- 238000006073 displacement reaction Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 229910000831 Steel Inorganic materials 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 239000010959 steel Substances 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012916 structural analysis Methods 0.000 description 1
- 239000002341 toxic gas Substances 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30152—Solder
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a machine vision-based metal structural part weld joint identification method, a machine vision-based metal structural part weld joint identification system and machine vision-based metal structural part weld joint identification equipment, and relates to the technical field of welding; the ROI area is a line intersecting the surface in the three-dimensional model and a joint line between the subsection pieces; capturing an image of the ROI area on the metallic structure; performing linear extraction on the captured image to obtain a linear set; according to the position of each straight line in the straight line set, straight lines meeting the position conditions are screened out to form a welding line set; and converting the image coordinates of each welding seam in the welding seam set into world coordinates of a three-dimensional model, and transmitting the welding seam subjected to coordinate conversion into the three-dimensional model. The invention realizes the identification and positioning of the welding seam, and lays a foundation for the automatic welding of large-scale metal structural parts.
Description
Technical Field
The invention relates to the technical field of welding, in particular to a machine vision-based metal structural part weld joint identification method, system and equipment.
Background
With the rapid development and continuous progress of the manufacturing industry of China, the components occupied by the welding technology in industrial production are larger and larger, the quality of products is directly influenced by the excellent degree of the welding technology, the welding plays a very important role in the production and processing process of large-scale steel gates, the quantity of products produced by many small and medium-sized gate manufacturing enterprises is small and various, a manual welding mode is often adopted in the production process, thus the production cost is increased and the production efficiency is low due to the fact that personnel relying on professional skills excessively are involved, and toxic gas and heat radiation are generated during welding, so that the human body is harmed to a certain extent.
Therefore, development of automatic welding equipment is important, the important point of automatic processing of welding is to acquire the position of a region to be welded, and some traditional industrial robots generally adopt a teaching mode for welding, but steel gates produced by small and medium enterprises are various in variety and unfixed in shape, and the method needs to take a lot of time for teaching and is low in efficiency.
For a single large-scale metal structural part in small batches, a special positioning tool is not generally adopted to ensure the relative positioning precision between all the welded sub-parts, the sub-parts are often manually placed, and then the positions of all the sub-parts are fixed by spot welding (positioning welding). Thus, the relative positioning error of parts of the large-sized welding part is larger than the allowable welding seam position deviation of a general automatic welding machine. Therefore, the automatic welding program programmed based on the three-dimensional model cannot function properly.
Disclosure of Invention
The invention aims to provide a machine vision-based metal structural part weld joint identification method, a machine vision-based metal structural part weld joint identification system and machine vision-based metal structural part weld joint identification equipment.
In order to achieve the above object, the present invention provides the following solutions:
a machine vision-based metal structural part weld joint identification method comprises the following steps:
extracting an ROI region from a three-dimensional model of the metal structural member; the ROI area is an area including an intersection line between faces or a joining line between sub-components in the three-dimensional model;
capturing an image of the ROI area on the metallic structure;
performing linear extraction on the captured image to obtain a linear set;
according to the position of each straight line in the straight line set, straight lines meeting the position conditions are screened out to form a welding line set;
and converting the image coordinates of each welding seam in the welding seam set into world coordinates of a three-dimensional model, and transmitting the welding seam subjected to coordinate conversion into the three-dimensional model.
Optionally, extracting the ROI region from the three-dimensional model of the metallic structural member specifically includes:
obtaining STEP/STP files of the three-dimensional model;
and (3) using a STEP/STP file reader to read the STEP/STP file in terms of points, lines and planes, and extracting the ROI area in the three-dimensional model.
Optionally, image capturing is performed on the ROI area on the metal structural part, which specifically includes:
moving an area array camera to a corresponding position of the ROI area on the metal structural part by using a mechanical arm;
and adopting the area array camera to capture the image of the ROI area on the metal structural part.
Optionally, the method comprises the steps of extracting straight lines from the captured image to obtain a straight line set, and specifically comprises the following steps:
sequentially carrying out brightness correction, median filtering and morphological processing on the captured image to obtain a preprocessed image;
adopting a Canny algorithm to perform contour detection on the preprocessed image;
and (3) carrying out straight line searching on the image after contour detection by adopting a Hough straight line searching algorithm, storing the searched straight lines and the distances between each searched straight line and the area array camera into the straight line set, and marking the straight lines positioned at the top, the side or the bottom of the large-scale metal structural member according to the ROI area.
Optionally, brightness correction, median filtering and morphological processing are sequentially performed on the captured image, and before the preprocessed image is obtained, the method further includes:
a camera coordinate system of the captured image was determined using a Zhang Zhengyou calibration method.
Optionally, the position condition includes that a straight line is located at the bottom of the metal structural member, an intersecting line of two sides of the straight line, and the straight line is located at the top of the metal structural member.
The invention also discloses a machine vision-based metal structural part weld joint identification system, which comprises:
the ROI region extraction module is used for extracting an ROI region from the three-dimensional model of the metal structural part; the ROI area is an area including an intersection line between faces or a joining line between sub-components in the three-dimensional model;
the image capturing module is used for capturing images of the ROI area on the metal structural part by adopting an area array camera;
the straight line extraction module is used for carrying out straight line extraction on the captured image to obtain a straight line set;
the welding seam screening module is used for screening out the straight lines meeting the position conditions according to the position of each straight line in the straight line set to form a welding seam set;
the coordinate conversion module is used for converting the image coordinates of each welding seam in the welding seam set into world coordinates of a three-dimensional model, and transmitting the welding seam subjected to coordinate conversion into the three-dimensional model.
The invention also discloses an electronic device, which comprises a memory and a processor, wherein the memory is used for storing a computer program, and the processor runs the computer program to enable the electronic device to execute the machine vision-based metal structural part weld joint identification method.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
according to the invention, the ROI area is extracted from the three-dimensional model, then the ROI area is subjected to image capturing, a straight line is extracted from the captured image, and then the welding line is screened out according to the position of the extracted straight line, so that the method is simple and easy to implement, and the application range of the welding line identification is enlarged. The invention realizes the positioning and identification of the welding seam existing in the welding seam area image, thereby acquiring the position information of the welding seam, and laying a foundation for the automatic welding of large-scale metal structural parts.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a machine vision-based method for identifying a weld joint of a metal structural member according to an embodiment of the present invention;
FIG. 2 is a flow chart of file parsing according to an embodiment of the present invention;
FIG. 3 is a region of interest (ROI) based on three-dimensional model analysis according to an embodiment of the present invention;
FIG. 4 is a flowchart of image capturing according to an embodiment of the present invention;
FIG. 5 is a flowchart of image processing according to an embodiment of the present invention;
FIG. 6 is a flowchart of weld identification provided by an embodiment of the present invention;
FIG. 7 is a flowchart of a weld result process provided by an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a machine vision-based metal structural part weld joint recognition system according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention aims to provide a machine vision-based metal structural part weld joint identification method, a machine vision-based metal structural part weld joint identification system and machine vision-based metal structural part weld joint identification equipment, which realize positioning and identification of weld joints existing in a weld joint area image, so that position information of the weld joints is obtained, and a foundation is laid for automatic welding of large-scale metal structural parts.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Example 1
As shown in fig. 1, the present embodiment provides a machine vision-based method for identifying a weld seam of a metal structural member, including:
step 101: extracting an ROI region from a three-dimensional model of the metal structural member; the ROI area is an area including an intersection line between faces or a joining line between sub-components in the three-dimensional model.
Step 102: and capturing an image of the ROI area on the metal structural part.
Step 103: and carrying out linear extraction on the captured image to obtain a linear set.
Step 104: and screening out the straight lines meeting the position conditions according to the position of each straight line in the straight line set to form a welding line set.
Step 105: and converting the image coordinates of each welding seam in the welding seam set into world coordinates of a three-dimensional model, and transmitting the welding seam subjected to coordinate conversion into the three-dimensional model.
In the embodiment, based on three-dimensional rapid reconstruction to obtain a three-dimensional model of a metal structural member outline, the metal structural member is a large nonstandard structural member built by a T-shaped structural member, the rapid reconstruction of the three-dimensional model structural information is an outline inaccurate position, and because the weld characteristic information of the three-dimensional model is fuzzy, in order to obtain specific weld information, the image information needs to be further processed by combining an area array camera based on the three-dimensional model.
The step 101 specifically includes:
creating a STEP/STP (STEP or STP) file reader specifically includes: and (3) introducing a corresponding API class library, constructing environment variables required by the reader, creating a file reader, and reading all points, lines and surfaces of the three-dimensional model.
The STEP/STP file for obtaining the three-dimensional model specifically comprises the following STEPs: and selecting a rapidly reconstructed STEP/STP graphic file locally, reading the STEP/STP file based on a SolidWorks open source code library, and reading the shape characteristics of the three-dimensional model into a local memory.
And (3) using a STEP/STP file reader to read the points, the lines and the planes of the STEP/STP file, and extracting a region of interest (ROI region) in the three-dimensional model.
And converting coordinates of the ROI area, and converting world coordinates of the three-dimensional model into a coordinate system of the robot, so that the follow-up image capturing is convenient to execute.
The step 102 specifically includes:
the manipulator is adopted to move the area array camera to the corresponding position of the ROI area on the metal structural part, and the method specifically comprises the following steps: and setting joint parameters of the manipulator, and moving the area array camera to the ROI area obtained by analyzing the three-dimensional model.
Adopting the area array camera to capture the image of the ROI on the metal structural part, specifically comprising the following steps: the area array camera adopts a soft triggering mode to capture images of a target area, and the captured images are stored in a local memory for subsequent image processing.
Step 103 specifically includes:
a Zhang Zhengyou calibration method is used to determine the camera coordinate system of the captured image to facilitate the conversion of the subsequent image processing result data.
And reading the image captured by the area-array camera from the memory.
And carrying out brightness correction, median filtering and morphological processing on the captured image in sequence, eliminating noise in the image, and ensuring that lines in the image are identified as much as possible to obtain a preprocessed image.
And the contour detection is carried out on the preprocessed image by adopting a Canny algorithm, so that rough contour information is obtained, and the follow-up weld joint recognition and positioning module can be operated conveniently.
Performing feature processing on the image after contour detection processing, performing straight line searching on the image after contour detection by adopting a Hough straight line searching algorithm, storing the searched straight lines and the distances between each searched straight line and an area array camera into the straight line set, and sequentially marking that the straight lines in the straight line set are positioned at the top, the side or the bottom of a large-scale metal structural member by the ROI region, wherein the method specifically comprises the following steps: and acquiring a result of a laser displacement sensor at the tail end of the area array camera, combining the end point and length information of the straight line li, the distance data of the corresponding laser displacement sensor and the marked position information into information of the straight line li, and loading the information into the straight line set L.
In step 104, according to the structural specificity of the metal structural member, the straight lines meeting the position condition are screened out according to the position of each straight line in the straight line set to form a weld joint set.
The position conditions include a straight line at the bottom of the metal structural member, an intersection line of two sides of the straight line, and a straight line at the top of the metal structural member.
Step 104 specifically includes:
step 1041, from the straight line set L { L } 1 ,l 2 ,l 3 Read a straight line l in … i ∈L。
Step 1042, finding straight line l i The position of the straight line is obtained from the position information marked in the drawing.
Step 1043, if the read straight line l i At the bottom of the large metal structure, the intersection of the web and the bottom plate is taken, and the straight line is a fillet weld, step 1046 is performed.
Step 1044, if the read straight line l i Located on the side of the large metal structural member and the straight line is the intersection of the two webs, then the straight line is a vertical weld, step 1046 is performed.
Step 1045, if the read straight line l i At the top of the large metal structure, and the weld is at the junction of the two flanges, then the straight line is a flat weld, otherwise, step 1047 is performed.
Step 1046, loading the straight line information into the weld set H.
Step 1047, determining whether the straight line set L is completely read, and if not, repeating steps 1041 to 1047.
In step 105, as shown in fig. 7, classification and partition processing are performed on the weld information, coordinate transformation is performed on the processing result, the result is transmitted to the three-dimensional model, and the weld information on the three-dimensional model is perfected so as to facilitate the realization of subsequent automatic welding, which specifically includes:
step 1051, according to line l i The marked position information in the same area is divided into the same type.
The area in step 1051 includes bottom, sides, top, according to line l i Is used for partitioning the straight line.
And step 1052, sequentially performing coordinate transformation on all the welding seams, and transforming the image coordinates of the welding seams into world coordinates of the three-dimensional model.
And 1053, transmitting the information of the welding seam in the world coordinate system to the three-dimensional model, and calling a three-dimensional model processing module to perfect the information of the welding seam on the three-dimensional model so as to facilitate the follow-up welding.
According to the machine vision-based metal structural part weld joint identification method, after all the steps are executed, characteristic analysis is carried out on a three-dimensional model of the outline of the large-sized metal structural part obtained through quick reconstruction, and an area array camera is used for positioning and identifying weld joint information of the ROI area to obtain position information of the area to be welded on the large-sized metal structural part, namely position point coordinates and length information of the weld joint. The method can identify and position the area to be welded on the large-scale metal structural member without specific model information of the structural member, and the processing objects of small and medium-sized gate processing enterprises are mostly small in batches and various, often have no special positioning tool, and errors between the structure and the three-dimensional model after manual positioning and fixing of workers exceed the welding seam allowance deviation of the automatic welding robot, so that the programming of a welding program cannot be performed based on the three-dimensional model. And the conventional teaching is used for welding, so that time and labor are wasted, and the efficiency is low. The method can adapt to different types of large-scale metal structural parts and calculate accurate weld joint position information, solves the problem of low efficiency caused by frequent teaching when a welding object is frequently replaced, has the characteristics of wide application, simplicity, easiness, high degree of automation and the like, is based on NET platform development, uses an algorithm source code which is completely free, and has the advantages of simple structure, convenience in operation and high recognition rate of more than 85 percent.
Example 2
The embodiment provides a machine vision-based metal structural part weld joint identification method, which comprises the following steps:
(1) File analysis: and analyzing and reading STEP/STP files of the three-dimensional model obtained by rapid reconstruction by using a program, carrying out structural analysis on the three-dimensional model, and searching an ROI (region of interest) where a weld joint possibly exists.
As shown in fig. 2, the analysis flow after the STEP/STP file is read is as follows:
(11) A STEP/STP file reader is created.
(12) And (3) introducing a corresponding API class library, constructing environment variables required by the reader, creating a file reader, and reading all points, lines and surfaces of the three-dimensional model.
(13) And selecting a STEP/STP graphic file of the quick-reconstructed rough three-dimensional model locally, and reading the STEP/STP file based on a SolidWorks open source code library.
(14) The shape features of the model are read into memory and analyzed to obtain the ROI area, as shown in fig. 3.
(15) And converting coordinates of the ROI area, and converting world coordinates of the three-dimensional model into a coordinate system of the robot, so that the follow-up image capturing is convenient to execute.
(2) Image capturing: and (5) performing image capturing on the ROI area by moving the area-array camera through the manipulator.
As shown in fig. 4, the steps of image capturing are as follows:
(21) And setting joint parameters of a manipulator, and moving the area array camera to an ROI area obtained by analyzing the quick-reconstruction rough three-dimensional model.
(22) The camera adopts a soft triggering mode to capture images of the target area.
(23) The captured image is saved to local memory for subsequent image processing.
(3) Image processing: the captured image is analyzed using an image processing tool to derive positional information for all straight lines of the ROI area.
As shown in fig. 5, the image processing steps are as follows:
(31) And calibrating the camera, and determining a camera coordinate system by adopting a Zhang Zhengyou calibration method so as to facilitate the conversion of subsequent image processing result data. According to Zhang Zhengyou calibration method, the internal and external parameter matrix of the camera can be obtained, and the four coordinate systems are connected.
When the coordinates are from the image coordinate system UO 0 V conversion to imaging plane coordinate System O P X P Y P Z P Then, a relation between the pixel and the physical unit is established, and the pixel coordinate of a point on the image is set as (c x ，c y ) Then the point (u, v) on the image coordinate system is compared with the point (x) on the imaging plane p ，y p ) The relationship of (2) is as follows:
wherein s is x ，s y The number of pixels per millimeter of the imager in the x and y directions, respectively.
When the camera coordinate system O c X c Y c Z c And imaging plane coordinate system O P X P Y P Z P After conversion, the distance between the coordinates matches the real world distance, and a point (x p ，y p 0) with the point (x) of the camera coordinate system c ，y c ，z c ) The relationship of (2) is as follows:
where f denotes a focal length of a camera (area camera).
Last world coordinate system (x w ，y w ，z w ) With the camera coordinate system (x c ，y c ，z c ) The conversion relation of (2) is as follows:
where R represents a rotation matrix in the camera outlier matrix and T represents a translation matrix in the camera outlier matrix.
The simultaneous conversion matrix from pixel coordinates to world coordinates is:
Z C representing the physical distance of the camera to the target object, which needs to be measured on site, a represents the internal reference matrix of the camera.
(32) And reading the target image captured by the area-array camera from the memory.
(33) And (3) performing operations such as brightness correction, median filtering, morphological processing and the like on the captured image, eliminating noise in the image, and ensuring that as many straight lines as possible exist in the image are identified.
(34) And carrying out contour detection on the preprocessed image by using a Canny algorithm to obtain rough contour information, so that the follow-up weld joint recognition and positioning module can be operated conveniently.
(35) And carrying out feature processing on the image subjected to contour detection processing, and calling a Hough straight line searching algorithm in a welding seam recognition and positioning module to search straight lines so as to obtain straight lines in the image.
(36) Obtaining the result of a laser displacement sensor at the tail end of an area array camera, and making a straight line l i Is combined into a straight line l by the endpoint and length information of the laser displacement sensor and the distance data of the corresponding laser displacement sensor and the marked position information i Is loaded into the collection L.
(4) And (3) weld joint identification: according to the actual situation: the image processing can obtain a straight line set, and some straight lines in the set do not belong to welding lines, so that in order to improve the accuracy of the position of the welding lines, the straight line set is screened, and because of the specificity of a large-scale metal structural member, the existing area of the welding lines is relatively fixed, the straight lines belonging to the welding lines are screened according to the information of the straight lines in the straight line set, and the straight lines are loaded into a welding line set H.
As shown in fig. 6, the specific procedure for weld recognition is as follows:
(41) From the straight line set L { L 1 ,l 2 ,l 3 Read a straight line l in … i ∈L。
(42) Find straight line l i The position of the straight line is obtained from the position information marked in the drawing.
(43) If the straight line I is read i The bottom of the large metal structural member is the intersection line of the web plate and the bottom plate, and the straight line is a fillet weld, so that step S46 is performed.
(44) If the straight line I is read i Located on the side of the large metal structural member, and the straight line is the intersection line of the two webs, the straight line is a vertical weld, and step S46 is performed.
(45) If the straight line I is read i At the top of the large metal structural member and at the junction of the two flanges, the straight line is a flat weld, otherwise step S47 is performed.
(46) And loading the straight line information into a welding line straight line set H.
(47) Judging whether the straight line set L is read completely, and if not, repeating the steps S41 to S47.
(5) And (3) weld joint result processing: and carrying out classified partition treatment on the weld joint information, carrying out coordinate conversion on the treatment result, and transmitting the result to the three-dimensional model to perfect the weld joint information on the three-dimensional model so as to facilitate the realization of subsequent automatic welding.
The specific steps of the weld result treatment are as follows:
(51) According to straight line l i The marked position information divides the straight lines in the same area into the same type
(52) And sequentially carrying out coordinate conversion on all the welding seams, and converting the image coordinates of the welding seams into world coordinates of the three-dimensional model.
According to a conversion matrix from the pixel coordinates obtained by camera calibration to world coordinates:
inverse solving for world coordinates (x w ,y w ,z w ) The following steps are changed from the above formula:
setting intermediate parametersMat 2 ＝R -1 T, only the third term on both sides of the above equation is observed:
z w ＝Z c *Mat 1 (2,0)-Mat 2 (2,0)；
Z c ＝(z w +Mat 2 (2,0))/Mat 1 (2,0)；
wherein Mat 2 (2, 0) represents a third term of the 3×1 matrix, the distance dis between the camera and the target object can be obtained by the laser displacement sensor, and the height of the camera is h c Thus, the world coordinate z of the target point to be measured can be known w ＝h c Dis, substituting the formula to find Z c The world coordinates can be obtained according to the pixel coordinates.
(53) Transmitting the information of the welding seam in the world coordinate system to the approximate three-dimensional model, and calling a three-dimensional model processing module to perfect the information of the welding seam on the three-dimensional model so as to facilitate the follow-up welding.
In summary, after all the five steps are executed, the weld information of the ROI area can be identified by using an area array camera according to the characteristics of the rough three-dimensional model obtained by fast reconstructing the large-scale metal structural member, so as to perfect the position information of the area to be welded on the three-dimensional model of the large-scale metal structural member, and obtain the position point coordinates and the length information of the weld.
For practical use, firstly, a rapidly reconstructed outline three-dimensional model STEP/STP graphic file is read locally by means of a SolidWorks open source code library, meanwhile, structural information of the three-dimensional model is analyzed, position information of an ROI (region of interest) where a welding line possibly exists is obtained, position information is provided for subsequent image capturing of the ROI, and as shown in fig. 3, an intersection line between a bottom plate 1 and a web plate 2 and an intersection line between the web plate 2 and a wing plate 3 are the ROI where the welding line possibly exists. The manipulator is used for moving the area array camera to the target position for image capturing, the image processing tool is used for processing the captured welding seam image to obtain specific information of all straight lines in the ROI area, the positions of the straight lines are judged, the information of the welding seam straight lines is obtained through screening, the coordinate conversion is carried out on the welding seam straight line information, the welding seam straight line information is converted into a world coordinate system of a three-dimensional model from pixel coordinates, the converted welding seam information is transmitted to a three-dimensional model plate, and then accurate information of welding seams on the three-dimensional model is perfected.
Example 3
As shown in fig. 8, the present embodiment provides a machine vision-based metal structural part weld joint recognition system, which includes:
an ROI region extraction module 201, configured to extract an ROI region from a three-dimensional model of a metal structure; the ROI area is a line intersecting a plane in the three-dimensional model.
An image capturing module 202 is configured to capture an image of the ROI area on the metal structure using an area array camera.
The straight line extracting module 203 is configured to perform straight line extraction on the captured image to obtain a straight line set.
And the weld joint screening module 204 is configured to screen out the straight lines meeting the position condition according to the position of each straight line in the straight line set to form a weld joint set.
The coordinate conversion module 205 is configured to convert the image coordinates of each weld in the weld set into world coordinates of a three-dimensional model, and transmit the weld subjected to coordinate conversion into the three-dimensional model.
Example 3
An embodiment of the present invention provides an electronic device including a memory for storing a computer program and a processor that runs the computer program to cause the electronic device to perform the method of embodiment 1.
Alternatively, the electronic device may be a server.
In addition, the embodiment of the present invention also provides a computer-readable storage medium storing a computer program, which when executed by a processor, implements the method of embodiment 1.
Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (8)
1. The machine vision-based metal structural part weld joint identification method is characterized by comprising the following steps of:
extracting an ROI region from a three-dimensional model of the metal structural member; the ROI area is an area including an intersection line between faces or a joining line between sub-components in the three-dimensional model;
capturing an image of the ROI area on the metallic structure;
performing linear extraction on the captured image to obtain a linear set;
according to the position of each straight line in the straight line set, straight lines meeting the position conditions are screened out to form a welding line set;
and converting the image coordinates of each welding seam in the welding seam set into world coordinates of a three-dimensional model, and transmitting the welding seam subjected to coordinate conversion into the three-dimensional model.
2. The machine vision based metal structure weld joint identification method of claim 1, wherein extracting the ROI area from the three-dimensional model of the metal structure comprises:
obtaining STEP/STP files of the three-dimensional model;
and (3) using a STEP/STP file reader to read the points, the lines and the planes of the STEP/STP file, and extracting the ROI area in the three-dimensional model.
3. The machine vision based metal structure weld seam identification method of claim 1, wherein image capturing of the ROI area on the metal structure specifically comprises:
moving an area array camera to a corresponding position of the ROI area on the metal structural part by using a mechanical arm;
and adopting the area array camera to capture the image of the ROI area on the metal structural part.
4. The machine vision-based metal structure weld joint identification method according to claim 1, wherein the method comprises the steps of extracting straight lines from the captured image to obtain a straight line set, and specifically comprises the following steps:
sequentially carrying out brightness correction, median filtering and morphological processing on the captured image to obtain a preprocessed image;
adopting a Canny algorithm to perform contour detection on the preprocessed image;
and (3) carrying out straight line searching on the image after contour detection by adopting a Hough straight line searching algorithm, storing the searched straight lines and the distances between each searched straight line and the area array camera into the straight line set, and marking the straight lines positioned at the top, the side or the bottom of the metal structural member in the straight line set in sequence according to the ROI area.
5. The machine vision based metal structure weld joint identification method of claim 4, wherein the captured image is subjected to brightness correction, median filtering and morphological processing in sequence, and before the preprocessed image is obtained, the method further comprises:
a camera coordinate system of the captured image was determined using a Zhang Zhengyou calibration method.
6. The machine vision based metal structure weld identification method of claim 4, wherein the positional conditions include a straight line at the bottom of the metal structure, an intersection of two sides of the straight line, and a straight line at the top of the metal structure.
7. A machine vision-based metal structure weld recognition system, comprising:
the ROI region extraction module is used for extracting an ROI region from the three-dimensional model of the metal structural part; the ROI area is an area including an intersection line between faces or a joining line between sub-components in the three-dimensional model;
the image capturing module is used for capturing images of the ROI area on the metal structural part by adopting an area array camera;
the straight line extraction module is used for carrying out straight line extraction on the captured image to obtain a straight line set;
the welding seam screening module is used for screening out the straight lines meeting the position conditions according to the position of each straight line in the straight line set to form a welding seam set;
the coordinate conversion module is used for converting the image coordinates of each welding seam in the welding seam set into world coordinates of a three-dimensional model, and transmitting the welding seam subjected to coordinate conversion into the three-dimensional model.
8. An electronic device comprising a memory for storing a computer program and a processor that runs the computer program to cause the electronic device to perform the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310562954.0A CN116596883A (en) | 2023-05-17 | 2023-05-17 | Metal structural part weld joint identification method, system and equipment based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310562954.0A CN116596883A (en) | 2023-05-17 | 2023-05-17 | Metal structural part weld joint identification method, system and equipment based on machine vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116596883A true CN116596883A (en) | 2023-08-15 |
Family
ID=87598739
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310562954.0A Pending CN116596883A (en) | 2023-05-17 | 2023-05-17 | Metal structural part weld joint identification method, system and equipment based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116596883A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117576094A (en) * | 2024-01-15 | 2024-02-20 | 中铁科工集团有限公司 | 3D point cloud intelligent sensing weld joint pose extraction method, system and equipment |
-
2023
- 2023-05-17 CN CN202310562954.0A patent/CN116596883A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117576094A (en) * | 2024-01-15 | 2024-02-20 | 中铁科工集团有限公司 | 3D point cloud intelligent sensing weld joint pose extraction method, system and equipment |
| CN117576094B (en) * | 2024-01-15 | 2024-04-19 | 中铁科工集团有限公司 | 3D point cloud intelligent sensing weld joint pose extraction method, system and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Nahangi et al. | Automated 3D compliance checking in pipe spool fabrication | |
| DE102018109774B4 (en) | Image processing system, image processing device and image processing program | |
| CN108917593B (en) | Intelligent measurement system and method based on element configuration of workpiece to be measured | |
| CN104923593B (en) | Vision-based positioning method for top layer bending plate | |
| CN116596883A (en) | Metal structural part weld joint identification method, system and equipment based on machine vision | |
| US20130060369A1 (en) | Method and system for generating instructions for an automated machine | |
| Safa et al. | Automating measurement process to improve quality management for piping fabrication | |
| CN102157004A (en) | Automatic image mosaicking method for high-accuracy image measuring apparatus of super-view field part | |
| CN112238304A (en) | Method for automatically welding small-batch customized special-shaped bridge steel templates by mechanical arm based on image visual recognition of welding seams | |
| JP6621351B2 (en) | Image processing apparatus and image processing method for laser processing | |
| CN112238292A (en) | Method for tracking space curve track of friction stir welding robot based on vision | |
| JP2022018144A (en) | Groove shape measurement method, automatic welding method and automatic welding apparatus | |
| CN115619738A (en) | Detection method for module side seam welding after welding | |
| US20230274407A1 (en) | Systems and methods for analyzing weld quality | |
| CN114749848A (en) | Steel bar welding automatic system based on 3D vision guide | |
| Yan et al. | Multi-line laser structured light fast visual positioning system with assist of TOF and CAD | |
| Jin et al. | A new welding seam recognition methodology based on deep learning model MRCNN | |
| Ko et al. | Developing a Free and Open-Source Semi-Automated Building Exterior Crack Inspection Software for Construction and Facility Managers | |
| Cibicik et al. | Robotic weld groove scanning for large tubular T-joints using a line laser sensor | |
| Seçil et al. | 3-d visualization system for geometric parts using a laser profile sensor and an industrial robot | |
| CN114663360A (en) | Welding deformation measuring method based on vision measurement | |
| Wang et al. | A binocular vision method for precise hole recognition in satellite assembly systems | |
| Wu et al. | Research on Welding Guidance System of Intelligent Perception for Steel Weldment | |
| CN117073543B (en) | Appearance measurement method, device and equipment of double-rotation flatness measuring machine | |
| Cai et al. | A structured light-based visual sensing system for detecting multi-layer and multi-track welding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |