KR100823549B1 - Recognition method of welding line position in shipbuilding subassembly stage - Google Patents

Abstract

The present invention relates to a shipyard assembly welding member position recognition method. According to the present invention, the image processing unit receives a photographed image; Determining an appropriate threshold value according to the input image by the image processor; The image processing unit making the image as a binary image using an auto threshold holding technique; Performing image processing by preprocessing an image acquired by a camera attached to the gantry using a median filter to remove noise on the image; Detecting, by the image processor, edges from a binary image from which noise components are removed using a Sobel Operator; Performing an Hough transform on the detected edges to detect linear components and thinning the edges; Detecting a straight line through the thinned edge by the image processor; And calculating a distance value from the linear grouping (Blob Center) coordinates detected by the image processor to measure positions of two member vertices of the two straight lines determined as outlines of the member. A member position recognition method is provided.

Image processing unit, welding robot, Hough transform unit, storage unit

Description

Recognition method of welding line position in shipbuilding subassembly stage}

1 is a view showing a conventional welding member position recognition method.

2 is a view showing a shipyard assembly welding member position recognition system of the present invention.

3 is a flowchart illustrating a welding member position recognition method of the image processor of FIG. 2;

4A to 4C are diagrams illustrating threshold values of an image processor according to the present invention;

5A to 5C are diagrams illustrating images of labeling / filtering by an image processor according to the present invention;

6A and 6B illustrate image states before and after edge thinning by an image processor according to the present invention;

7 is a view showing the image state before and after the Hough transform image processing unit according to the present invention.

8 is a view showing a state in which the image processing unit measures the position of the member vertices of the two straight lines in accordance with the present invention.

<Explanation of symbols for the main parts of the drawings>

100: welding robot 200: image processing unit

220: image pickup unit 221: threshold hold setting unit

223: Hough transform unit 227: location information measuring unit

229 storage unit 230 control unit

The present invention relates to a method for recognizing the position of a small assembly welding member of a shipyard, and more specifically, to capture an image of two vertices (diagonal directions) of a target member to be welded before welding by a welding robot, and to coordinate the vertices of the member through image processing. The shipbuilding subassembly welding member derives the value (pixel coordinates), converts this point into spatial coordinates, finds two vertex coordinate values per member, and compares the CAD data of the member with the exact weld line coordinates in the member. It relates to a location recognition method.

In general, when welding a large structure using a welding robot, it takes a lot of time and manpower to accurately detect and weld the location of the welding place.

At this time, the size of the surface plate on which the welding target member is placed is very large, several M x several tens of M, and it is necessary to grasp the exact position and posture of the welding target member to perform automatic welding using a welding robot.

As such a conventional technique, a method of measuring the position and attitude of a shipyard assembly member through two-step image processing, which is our patent, is disclosed in Korean Patent Publication No. 1999-66116.

Referring to FIG. 1 of the related art, a second camera mounted on the gantry 80 after moving is obtained after an overall image of the surface plate is acquired by a first camera attached to the ceiling in order to enable welding line measurement for all regions of the surface plate. An image was acquired with a camera (welding process camera, 81) to find out the position of the member 10.

However, conventionally, when the image acquired by the camera (welding process camera) attached to the gantry is made into a binary image, the division of the object and the bottom surface to be detected is determined according to what value the threshold is determined. Since the threshold value is not variable according to the image state input to the camera mounted on the camera, the object and floor cannot be effectively distinguished, which causes a problem in that the position and posture of the member are not accurate.

In order to solve the above problems, the present invention provides a shipyard sub-assembly welding member position recognition method that finds the exact weld line coordinate value of the member through the photographed image, the CAD data of the member and the exact weld line coordinate value in the member. Its purpose is to.

In order to achieve the above object, the shipyard small assembly welding member position recognition method according to the present invention comprises the steps of (a) receiving an image captured by the image processor (S1); (b) determining, by the image processor, an appropriate threshold value according to the input image (S3); (c) the image processing unit making an image as a binary image using an auto threshold holding technique (S5); (d) performing, by the image processing unit, preprocessing the image acquired by the camera attached to the gantry using a median filter (S7); (e) performing image blob labeling / filtering to remove noise on the preprocessed image (S8); (S9) the image processing unit detects edges from a binary image from which the noise components are removed using a Sobel Operator (S9); (g) performing a Hough Transform on the detected edge of the image processor to detect linear components and thinning the edge (S11); (h) detecting, by the image processor, a straight line through the thinned edge (S13); And (i) step (S15) of the image processing unit calculating a distance value from the linear grouping (Blob Center) coordinates detected above to measure the positions of the two vertices of the member vertices determined as the outlines of the members; It includes.

Hereinafter, with reference to the accompanying drawings there may be a plurality of embodiments of the shipyard small assembly welding member position recognition method according to the present invention, the following describes the most preferred embodiment.

2 is a diagram illustrating a system for performing a welding member position recognition method according to the present invention.

Referring to FIG. 2, the welding member position recognition system of the present invention may perform welding according to a welding line scheduling unit for calculating weld line coordinates of a welding object using the photographed image and stored CAD data of the welding member, and the image processing unit. And a welding robot 100 for performing welding to the member.

The image processing unit 200 obtains an image photographing unit 220 for acquiring an image for extracting member vertex information, and obtains a threshold value for extracting welding line information for finding a welding line by acquiring an image photographed through the image capturing unit. The threshold setting unit 221 for determining and the Hough transform unit 223 for converting using the result of the Hough transform and the direction vector information of the member obtained from the mounting through the value set from the threshold setting unit, the Hough A position information measuring unit 227 for acquiring a binarized image based on the threshold value set by the converter and measuring position information from the obtained image, a storage unit 229 for storing position information of the welding member, and each unit And a controller 230 for controlling and comparing the weld line information photographed by the image photographing unit with data stored in the storage unit.

The image capturing unit 220 acquires an image with cameras mounted on the ceiling in the attaching process to extract the weld line information, and moves the gantry to the position identified in the image processing step to attach the gantry in the direction of the gantry axis of rotation. The video with the old camera.

The threshold setting unit 221 determines an appropriate threshold value for binarizing the image using the image input from the image capturing unit.

The threshold setting unit 221 collects histogram data by measuring the frequency of each gray level (0 to 255) from the image obtained by the image capturing unit.

In addition, the threshold setting unit 221 measures the frequency according to the gray level to set the valley value between the first peak point and the second peak point as an optimal threshold value. do.

The Hough transform unit 223 automatically recognizes the vertex of the member by using the result of the Hough transform and the direction vector information of the member obtained from the mounting through the value set by the threshold setting unit 221.

이내에 있는 직선들만 필터링 한 후 필터링한 직선들을 Blob Center 좌표값으로부터의 거리를 계산하여 이 거리값 별로 정렬 시킨 후 직선들을 그룹핑 한다.(단, 15 픽셀 이상 떨어진 직선은 다른 그룹으로 인식함) 그러면, 최종 선택된 직선군 중 중심에 존재하는 두 직선의 교점을 최종 꼭지점으로 인식하게 된다. The Hough transform unit 223 sorts the line extracted as a result of the Hough transform by angle and then, from the angle of the linear component given from the two direction vectors constituting the vertex and the vertex of the member stored in the mounting process.

After filtering only the straight lines within it, the filtered straight lines are calculated by calculating the distance from the Blob Center coordinate value and sorted by this distance value, and the straight lines are grouped (but the straight lines more than 15 pixels apart are recognized as other groups). The intersection point of two straight lines existing in the center among the last selected straight group is recognized as the final vertex.

The location information measuring unit 227 extracts points to be measured by the camera from the acquired image, measures location information, and converts two-dimensional image coordinates into three-dimensional real coordinates and transmits them to the control unit of the image processing unit. . Here, the two-dimensional image coordinates indicate the coordinates of the pointer corresponding to the point to be measured in the camera, and the three-dimensional real coordinates indicate the coordinates of the point to be measured when the coordinates of the location where the position is located are the origin. Indicates.

The storage unit 229 stores the CAD data of the position information welding member and the position of the welding line of the welding member to be welded.

The controller 230 controls the units and compares and calculates weld line information and data stored in the storage unit.

The controller 230 receives weld line information about a center point through the acquired image and stores the weld line information in the storage unit 229.

3 is a flowchart illustrating a method for recognizing a welding member position through the image processor of FIG. 2, FIGS. 4A to 4C are diagrams showing a threshold value according to the present invention, and FIGS. 6A and 6B are diagrams showing image states before and after edge thinning, FIG. 7 is a diagram showing image states before and after Hough transform, and FIG. 8 is according to the present invention. It is a figure which shows the state which the image processing part determines the position of the member vertex of two straight lines.

Hereinafter, the method for recognizing a welding member position performed by the image processor will be described with reference to FIGS. 4A to 8 according to each step along with FIG. 3.

First, the image processing unit acquires an image with two cameras in order to enable measurement of the entire area of the surface plate, and then performs an image synthesis process through the obtained two images to make one image. )

In the step S1, the entire image of the welding member is taken by using a fixed camera fixed to the ceiling, and the image is moved to the gantry welding to find the vertex of the welding member.

Next, the image processor determines an appropriate threshold value according to the input image (S3).

The step S3 is described with reference to FIGS. 4A to 4C. The small peak through a low pass filter is performed by performing fast Fourier transform on the histogram data of the image. The values are ignored and the valley between the maximum peaks is found.

Next, the image processor performs a step of making the image into a binary image by using an auto threshold holding technique (S5).

In step S5, the image processor obtains an image by binarizing the sensing threshold value set by the threshold setting unit.

In this case, the auto threshold holding technique changes the gray level to 255 (white) when the threshold value is 55 or more in the entire image, and sets the gray level to 0 (black) when the threshold value is less than 55. Is changed to).

That is, the threshold value is not 55 for all images, and the threshold value is determined from the distribution of the histogram of the input image to binarize the image above or below the value.

Next, a pre-processing of the image acquired by the camera attached to the gantry through a median filter is performed (S7), and blob labeling / filtering is performed to remove noise components from the pre-processed image. / Filtering) step (S8).

The step S7 is described with reference to FIGS. 5A and 5C. The image processor removes unwanted noise components from a binary image by using a blob labeling / filtering technique.

In this case, the image processor assigns a number to the blob of the binary image, obtains the center coordinates of each blob, and roughly identifies (1 to 4 quadrants) the position where the member is placed using the mounting information, Remove blobs in quadrant.

In addition, the image processing unit may include one quadrant (337 degrees to 120 degrees), two quadrants (60 degrees to 203 degrees), and three quadrants (157 degrees to 300 degrees) in consideration of a slight twist or rotation when the welding member is placed on the surface plate. ), The quadrant (240 degrees to 23 degrees) determines the area.

That is, the captured image shown in the drawing of FIG. 5A is converted into a binarized image (FIG. 5B), and an angle of a sum vector corresponding to two lines 61 and 62 constituting a member vertex of the mounting direction vector is quadrant 2. If present, the 2, 3 blobs except for the first blobs existing in the two quadrant lines 63 and 64 considering the rotation angle of the welding member are deleted to obtain an image after labeling / filtering (FIG. 5C).

Next, the image processor detects edges from a binary image from which the noise components are removed using a Sobel Operator (S9).

In this case, the Sobel Operator is a method used when detecting an edge and detecting the size and direction of the edge.

Next, the image processing unit performs a hough transform on the detected edge to detect various linear components to thin the edges (S11).

The step S11 is described with reference to FIGS. 6A and 6B, which reduces the thickness of the line at the edge portion before edge thinning (FIG. 6A) and after edge thinning (FIG. 6B). Reduce the number of linear components later.

Next, the image processor performs a step of performing a transform to detect a plurality of straight lines (S13).

In this case, a hough transform is a method of finding points lying on the same straight line in an image composed of N pixels, and after the hough transform of the straight line detected after the hough transform through the thinning step ( Finally, the vertex of the member is automatically recognized using the result of the Hough transform and the direction vector of the member obtained from the mounting.

내의 직선을 추출하면, θ1 -5 < θ < θ1 +5와, θ2 -5 < θ < θ2 +5 인 직선만이 추출된다. The step S13 is described with reference to FIG. 7, wherein the extracted straight line has a slope of one of the linear components at two mounting direction vector slopes θ1 and θ2.

When the straight line inside is extracted, only straight lines with θ1 -5 <θ <θ1 +5 and θ2 -5 <θ <θ2 +5 are extracted.

Next, the image processor calculates a distance value from the detected linear grouping coordinates (Blob Center) coordinates to determine the positions of the member vertices of the two straight lines determined as the outline of the member (S15).

이내에 있는 직선들만 필터링 하고, 필터링한 직선들을 Blob Center 좌표값으로부터 거리를 계산하여 이 거리값 별로 정렬 시킨 후 직선들을 그룹핑 한다.(단, 15 픽셀 이상 떨어진 직선은 다른 그룹으로 인식함)Referring to FIG. 8, the image processing unit sorts the lines extracted as a result of the Hough transform for each angle, and then forms two vertices and vertices of the members stored in the attaching process. From the angle of the vector and the given linear component

Filter only the straight lines that are within, and sort the filtered straight lines by calculating the distance from the Blob Center coordinates and grouping them by this distance value.

In this case, the image processing unit selects the outermost straight line from the Blob Center coordinate value, and the image processing unit recognizes the intersection of two straight lines existing in the center among the finally selected straight line group as the final vertex.

That is, the vertex recognition (selection of the straight line at the center of the region in the outermost straight line group) of the welding process image processing member is to find the outermost straight line components among the plurality of straight line components. Selecting a straight line ends the automatic recognition of the coordinates of the vertices.

In the above, the shipyard small assembly welding member position recognition method according to the present invention has been described. Such a technical configuration of the present invention will be understood by those skilled in the art that the present invention can be implemented in other specific forms without changing the technical spirit or essential features of the present invention.

Therefore, the above-described embodiments are to be understood in all respects as illustrative and not restrictive, and the scope of the present invention is indicated by the appended claims rather than the foregoing description, and the meanings of the claims and All changes or modifications derived from the scope and the equivalent concept should be construed as being included in the scope of the present invention.

As described above, the present invention converts the vertex coordinate value of the member into a spatial coordinate through the vertex image of the welding member, compares the CAD data of the corresponding member, and finds the coordinate value of the welding line in the member to improve the accuracy of the work. There is.

In addition, the present invention has the effect of improving the productivity by enabling the image processing even if a plurality of members in the image at the same time in the image photographed in the welding process by using the position information placed by the member in the mounting process.

In addition, the present invention has an effect of improving the reliability of image processing by selecting the outermost straight group based on the blob center coordinates of the member direction vector region obtained in the attaching process among several straight groups after the Hough Transform. .

Claims (3)

In the shipyard small assembly welding member position recognition method, (a) receiving an image captured by the image processor (S1); (b) determining a minimum value between a first peak point and a second peak point as a threshold value by measuring a frequency according to a gray level according to the input image by the image processor (S3); (c) the image processing unit making a binary image of the image input based on the threshold value using the threshold value determined in step (b); (d) performing image processing on the image acquired by the camera attached to the gantry using a median filter (S7); (e) After the image processing unit has labeled each of the blobs, the blob labeling / filtering is performed to obtain the center value of each blob and to delete the other blobs except for the blob of the quadrant where the mounting direction vector is placed. Removing noise on the preprocessed image (S8); (S9) the image processing unit detects edges from a binary image from which the noise components are removed using a Sobel Operator (S9); (g) thinning the detected edges of the image processor (S11); (h) detecting, by the image processing unit, linear components by performing a Hough Transform through the thinned edge (S13); And (i) the image processor performs grouping by dividing the plurality of linear components detected by the distance calculated from the blob center coordinate values by the distance calculated from the blob center coordinate values, and the outermost part farthest from the blob center coordinate values. Measuring a position of the member vertex by recognizing, as a member vertex, an intersection point of two center straight lines whose value of the slope among the straight lines of the straight line group is closest to the average slope of the straight lines in the outermost straight group region; Shipyard small assembly welding member position recognition method comprising a. The method of claim 1, (H) step, The slope of the linear component of the image processing unit in the mounting direction vector slope

Shipyard subassembly welding member position recognition method characterized in that it extracts only the straight line. The method of claim 1, In step (i), In order to determine the outline of the image processing member, the image processor selects a center straight line that is closest to an average slope of the straight lines of the outermost straight group among the straight lines of the outermost straight group region. Location awareness method.

Images