KR20180115646A - Bead recognition apparatus using vision camera and method thereof - Google Patents

Abstract

The present invention relates to an apparatus to recognize a bead with a vision camera and a method thereof. According to the present invention, the apparatus to recognize bead with a vision camera comprises: an input unit to receive an image including a test sample through a two-dimensional (2D) camera; an image processing unit to set a region of interest (ROI) in the image and convert the ROI into a grayscale image; a bead region extraction unit to detect a welding bead region of the test sample from the ROI; a first detection unit to detect feature information of a welding bead from a binarized image binarizing the welding bead region and the remaining background region in the ROI, and to determine an abnormal shape of the welding bead from the detected feature information; and a second detection unit to determine whether a hole or an undercut occurs in the welding bead, or to determine an occurrence position from an image combining the binarized image and the grayscale image. Accordingly, an image of a vision camera is used, thereby providing advantages of increasing welding bead detection performance and a data processing speed, and reducing a discrimination error in a defect test.

Description

[0001] The present invention relates to a bead recognition apparatus using a vision camera,

The present invention relates to a bead recognition apparatus using a vision camera and a method thereof, and more particularly, to a bead recognition apparatus using a vision camera capable of improving the detection performance of a weld bead and a method thereof.

The quality inspection of the welded part is carried out by the sample inspection and the whole inspection, and most of the whole inspection is carried out by visual inspection through the naked eye. Skilled technicians can detect weld defects through visual inspection, but usually have a lot of technical skill and vary from technician to technician.

In the automotive industry, which is highly affected by stability and quality, welding defects are fatal to accidents, and affect the appearance and quality of automobiles. Recently, advanced automakers have been inspecting the appearance inspection equipment using the vision of the welding part and applying the 2D or 3D based inspection system to reduce the deviation of the defect detection according to the engineer.

Vision inspection is mainly used in 3D-based inspection equipment, and 3D-based inspection method scans three-dimensional shape of product and performs inspection based on it. However, since the 3D-based inspection method inspects the product only by the shape, it is difficult to distinguish the boundary between the product and the welded part, and since it takes a lot of time to inspect the product by processing a large amount of data, There is a problem.

The technology that is the background of the present invention is disclosed in Korean Patent Laid-Open Publication No. 2009-0074964 (published on July 8, 2009).

An object of the present invention is to provide a bead recognition method using a vision camera capable of enhancing the detection performance of a weld bead.

An image processing apparatus includes an input unit for inputting an image including an inspection sample through a 2D camera, an image processing unit for setting an area of interest in the image and converting the area of interest into a gray image, A feature extraction unit for extracting feature information of a weld bead from a binarized image obtained by binarizing the weld bead area and the remaining background area in the ROI, And a second detecting unit for determining whether a hole or an undercut in a weld bead occurs or an occurrence position from an image obtained by combining the binarized image and the gray image, A recognition device is provided.

The feature information of the weld bead may include at least one of a start point and an end point of the weld bead, a length of the arc, and an average, a maximum and a minimum value of a plurality of bead widths obtained along the length direction of the arc.

The first detection unit may be configured to form a straight line in which a center point of the inspection sample and a point on the outer contour line meet each other with respect to a plurality of points on the outer contour line of the inner and outer contour lines of the weld bead, The plurality of bead widths can be obtained by obtaining the distance between the intersection point and the intersection between the straight line and the outer contour line.

The second detection unit may combine the binary image and the gray image in such a manner that a white region corresponding to the weld bead region in the binarized image is filled with an image of the corresponding region in the gray image.

The first detecting unit may determine whether or not the shape of the weld bead is abnormal by comparing the feature information of the weld bead with predetermined reference information, and the second detecting unit may calculate a depth value of the hole or undercut received from It is possible to determine whether or not the depth of the hole or the undercut is abnormal by comparing the reference value with each preset reference value.

The bead recognition apparatus may further include a determination unit that determines that the weld bead is normal if both the shape and the depth are normal and determines that the weld bead is defective if at least one of the shape and the depth is abnormal, And an output unit for providing a determination result to a display screen or an external client.

The present invention also provides a bead recognition method in a bead recognition apparatus using a vision camera, comprising the steps of: receiving an image including a test sample through a 2D camera; setting a region of interest in the image; Detecting a weld bead region of the test sample in the region of interest; detecting feature information of the weld bead from a binarized image obtained by binarizing the weld bead region and the remaining background region in the region of interest; Determining whether or not a hole or an undercut in the weld bead occurs or an occurrence position of the weld bead from the image combining the binarized image and the gray image; A bead recognition method using a vision camera.

The bead recognition method may further include determining whether the depth of the hole or the undercut is greater than or equal to the depth of the hole or the undercut by comparing the depth value of the hole or the undercut received from the predetermined reference value, The determining step may determine whether the shape of the weld bead is abnormal by comparing feature information of the weld bead with preset reference information.

Further, the bead recognition method may include determining that the weld bead is normal if both the shape and the depth are normal, determining that the weld bead is defective if at least one of the shape and the depth is abnormal, And providing the result to a display screen or an external client.

According to the present invention, there is an advantage that the detection performance of the weld bead and the data processing speed can be increased and the discrimination error of the defect inspection can be reduced by using the image of the vision camera.

1 is a diagram illustrating a bead recognition system according to an embodiment of the present invention.
FIG. 2 is a view illustrating an inspection sample mounted on the fixing jig of FIG. 1. FIG.
3 is a block diagram of a bead recognition apparatus according to an embodiment of the present invention.
4 is a diagram showing a result of binarizing a region of interest in an embodiment of the present invention.
5 is a diagram showing the principle of calculating the bead width in the embodiment of the present invention.
FIG. 6 is a view showing a result of combining two images of FIG.
7 is a view for explaining a bead recognition method using the bead recognition apparatus of FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention.

1 is a diagram illustrating a bead recognition system according to an embodiment of the present invention.

1, a bead recognition system according to an embodiment of the present invention includes a fixing jig 100, a 2D camera 200, an illumination 300, and a bead recognition apparatus 400. [

The fixing jig 100 mounts a sample to be inspected (hereinafter referred to as an inspection sample). For the accuracy and reliability of the bead recognition algorithm, the fixed state of the fixing jig 100 for fixing the test sample is important. In addition to the inspection sample, the fixing jig 100 may be equipped with a 2D camera 200 and an illumination 300 for imaging the inspection sample.

FIG. 2 is a view illustrating an inspection sample mounted on the fixing jig of FIG. 1. FIG. Such a typical example of the sample including the welding nut is shown in Fig.

The welding nuts are welded to the vehicle body and used to fix various parts and interior materials. As shown in Fig. 2, arc-shaped weld beads are formed at a peripheral portion (both sides) of the nut.

As described above, in the embodiment of the present invention, the inspection sample may correspond to a sample including a welding part (welding material) such as a welding nut. Of course, embodiments of the present invention can be applied to a variety of target specimens requiring welding status (welding bead) inspection.

The 2D camera 200 captures an image of a test sample mounted on the fixing jig 100 and acquires a captured image. The illumination 300 irradiates illumination toward the target sample to enhance the image quality of the 2D camera 200.

FIG. 2 shows an image of the test sample viewed from the perspective of the test sample for the purpose of illustration of the test sample only. The 2D camera 200 preferably obtains an image of the inspection sample taken at an angle looking down vertically from the top for the recognition accuracy of the weld bead.

In the embodiment of the present invention, a vision camera having a resolution of 1920 × 1200 (full HD grade) can be used to reduce detection errors such as the length, width, and height of the weld bead, and the bead recognition error In order to reduce the size of the image, it is possible to acquire the image using the illumination produced in an unshaded manner.

The bead recognizing apparatus 400 can detect and recognize weld beads from photographed images of the 2D camera 200, and can check various measured values, bad conditions, etc. of the weld beads and notify the client. The client may include a user terminal such as a PC, a desktop, a notebook, a smart phone, a smart pad, and a tablet PC accessible by an administrator, a user, .

3 is a block diagram of a bead recognition apparatus according to an embodiment of the present invention. 3, the bead recognizing apparatus 400 according to the embodiment of the present invention includes an input unit 410, an image processing unit 420, a bead region extracting unit 430, a first detecting unit 440, A determination unit 460, and an output unit 470.

The input unit 410 receives an image including a test sample through the 2D camera 200. Hereinafter, it is assumed that there are weld beads on both the left and right sides of the center of the test sample (ex, center of the nut).

The image processing unit 420 sets a region of interest (ROI) in the image and performs a preprocessing for converting the ROI into a gray image. Here, the region of interest may refer to an area including the inspection sample portion in the image or an area including the welding nut portion in the inspection sample.

The preprocessing of the image may include color space conversion (RGB → Gray) for converting a color (RGB) image into a gray image, noise elimination for removing noise in the image, and the like. Each pixel of the converted gray image may have a range of gray scale values between 0 and 255 (0? G? 255). The larger the G value, the lighter it is, so the pixel with G = 255 is white and the pixel with G = 0 is black.

Noise reduction represents a filtering process that removes Gaussian noise existing in an image that may be caused by illumination and an external environment. The use of a two-dimensional Gaussian kernel in noise filtering not only eliminates Gaussian noise in the image caused by the illumination, but also reduces the external effects of shadow or light. Of course, Gaussian filters and Median filters can be used to remove noise.

The bead region extracting unit 430 detects a weld bead region of the test sample in the pre-processed ROI. As shown in FIG. 2, the weld bead region is located around the weld nut and has an arc shape. By using this point, it is possible to detect the weld bead area of the inspection sample in the area of interest through the contour detection technique.

Of course, the bead region extracting unit 430 can detect the nut partial region of the test sample from the region of interest and the bead region around the nut region. For example, the bead region extraction unit 430 may extract the feature points such as the vertex of the nut in the region of interest, recognize the circumference and the shape of the nut, apply the bead outline contour detection algorithm to the periphery of the nut, Bead Edge) can be detected and a weld bead region composed of bead contours can be detected.

Of course, the feature information of the bead can be obtained by applying a calculation algorithm for measuring the width, length, etc. of the bead in the obtained weld bead region.

The first detection unit 440 detects the feature information of the weld bead from the binarized image obtained by binarizing the weld bead region and the remaining background region in the ROI, and can determine whether the shape of the weld bead is abnormal based on the detected feature information.

Here, the binarization process can be performed in the image processor 420. The image processing unit 420 performs a binarization process to distinguish the detected weld bead region from the remaining region in addition to the preprocessing for the initial input image.

4 is a diagram showing a result of binarizing a region of interest in an embodiment of the present invention. 4 (a) shows a gray image, and (b) shows a binarized image.

The image processor 420 performs color conversion of the color image to obtain a gray image as shown in FIG. 4A. The bead area extractor 430 applies an outline detection method to the obtained gray image, The bead region is detected.

The image processing unit 420 processes white pixels (G = 255) included in the previously detected weld bead region and treats all other pixels as black (G = 0) A binary image can be obtained. As a result, through the binarization process, only the weld bead area is processed in white in the image, and all remaining areas including the bolt are backgrounded in black.

The first detection unit 440 detects feature information of the weld bead from the binarized image. Here, the feature information of the weld bead may include at least one of a start point and an end point of the weld bead, a length of the arc, and an average, a maximum and a minimum value of a plurality of bead widths obtained along the length direction of the arc.

Here, the start and end points of the weld bead correspond to the start and end portions of the arc in the longitudinal direction. Also, the bead width can be measured at set intervals along the length direction of the beads (arcs), and the average, minimum, and maximum values of the bead width can be provided using the respective measurement results.

5 is a diagram showing the principle of calculating the bead width in the embodiment of the present invention.

As shown in Fig. 5, it can be seen that the bead has a arc shape and the bead width corresponds to the distance between the inner contour and the inner contour of the bead. The outer contour of the bead can be divided into an inner contour that is closer to the center of the nut and an outer contour that is farther away.

For calculating the bead width, first, the center point C of the test sample (nut) forms a straight line that meets a point A on the outer contour of the weld bead. That is, we derive a linear equation that meets the outer contour of the bead and the center of the nut. Then, the bead width W is calculated by finding a point (A) where a straight line and an outer contour meet and a point (B) where a straight line and an inner contour meet, and measuring a straight line distance between two points.

Therefore, in the embodiment of the present invention, the first detecting unit 440 detects the center point C of the test sample (welding nut) and the outer contour line of the test sample (weld nut) with respect to the plurality of points A on the outer contour line to the weld bead region, A plurality of bead widths w can be obtained by determining the distance between intersections between straight lines and two contour lines, that is, AB, after forming a straight line where the point A meets. This process can be performed sequentially along the length of the arc.

Further, the first detection unit 440 determines whether or not the shape of the weld bead is abnormal based on the feature information of the weld bead. Specifically, the first detection unit 440 can determine whether or not the shape of the weld bead is abnormal based on a result of comparing feature information of the weld bead with preset reference information.

For example, when the length of the arc deviates from the reference range, at least one of the plural pieces of feature information listed above has a reference range or a reference value for the minimum value or the average value thereof is smaller than the reference value In case of deviation, it can be judged that there is an abnormality in the shape of the weld bead

The first detection unit 450 determines whether a hole or an undercut in the weld bead occurs or an occurrence position from the combined image of the gray image and the binarized image. The hole means a hole portion in the bead, and the undercut means a depression portion from the bead surface.

FIG. 6 is a view showing a result of combining two images of FIG. Referring to FIG. 6, the second detector 450 can obtain an image as shown in FIG. 6 through mutual coupling of (a) and (b) shown in FIG. 4, It is possible to determine whether or not holes or undercuts are formed and the position where they occur.

Here, it can be seen that the second detection unit 450 combines the two images in such a manner that the white region (weld bead region) in the binarized image is filled with the image of the corresponding region in the gray image.

4 (a) and 4 (b), when a gray scale value of two pixels at the same position in an image is compared to select a pixel of a darker or lower gray scale value, . Of course, if the gray image of FIG. 4 (a) is stacked on the lower surface of the binarized image obtained by cutting out the white area in FIG. 4 (b), only the corresponding area of the gray image is exposed through the cut- The same result as 6 can be obtained.

The second detector 450 adaptively binarizes the image of FIG. 6, and then removes noise from the image using a Gaussian and a morphology filter algorithm. The hole or undercut portion in the weld bead can be precisely detected or highlighted through the adaptive binarization process. Such adaptive transfer processing is a method well known in the field of image processing, and a detailed description thereof will be omitted.

The second detector 450 uses the adaptive binarization process and the noise canceled image to determine whether or not the holes or undercuts are generated and the location where they occur. At this time, the occurrence position of holes or undercuts can be determined by using orthogonal and polar coordinate algorithms. The orthogonal and polar coordinate algorithms are used to detect positions, angles, directions, etc., and correspond to known techniques, and detailed description is omitted.

If holes or undercuts are not present, there is a high possibility that the welding condition is good. However, if holes or undercuts are present, there is a possibility that welding is bad. However, even if holes or undercuts are present, it may be regarded as a good state if the depth is below a critical value.

Further, the second detection unit 450 can determine whether the depth of the hole or the undercut is abnormal by using the depth measurement sensor. Here, the depth measurement sensor may be included in the bead recognition apparatus 100 or may be connected to the outside. The depth measurement sensor may include a laser sensor, an infrared sensor, a 3D scanner, and the like.

Specifically, the second detection unit 450 can determine whether the depth of the hole or the undercut is abnormal based on a result of comparing the depth value of the hole or undercut received from the depth measurement sensor with each predetermined reference value. For example, if the depth of the undercut is larger than the reference value, it can be judged that the depth of the undercut is abnormal.

The determination unit 460 determines whether the weld bead is defective based on the information received from the first detection unit 440 and the second detection unit 450. [ That is, if the shape of the weld bead and the hole / undercut depth are both normal, the weld bead is determined to be normal (weld normal), and if any of the shape or depth is abnormal, Defective).

 As described above, in the embodiment of the present invention, after confirming the morphological abnormality obtained based on the feature information of the weld bead region and the depth abnormality acquired based on the depth of the hole or undercut in the weld bead region, It is determined that the welding is defective and the result can be provided.

The output unit 470 provides the determination result of the determination unit 460 to a display screen or an external client. That is, the output unit 470 may directly output the determination result on the display screen of the bead recognition apparatus 400, or may transmit or transmit the determination result to the client (user terminal) connected to the wired / wireless network.

Of course, the output unit 470 can output information such as the characteristic information of the weld bead extracted from the test sample, whether or not holes or undercuts are generated, the occurrence position, the state of the weld bead shape, have.

7 is a view for explaining a bead recognition method using the bead recognition apparatus of FIG.

First, the input unit 410 receives an image including a test sample through a 2D camera (S710).

Next, the image processing unit 420 sets an ROI (ROI) on the image (S711). Then, the image processing unit 420 preprocesses the ROI. Specifically, the image of the ROI is converted into a gray image as shown in FIG. 4A through a color space conversion (RGB-> Gray) A Gaussian filter, a median filter, or the like) is applied to remove noise in the image (S712).

Then, the bead region extracting unit 430 determines the presence of the test sample in the region of interest converted into the gray image as shown in FIG. 4 (a), and then detects the weld bead region in the region of interest (S713).

As described above, the bead region extracting unit 430 can detect the weld bead region outside the nut and the weld bead region of the test sample using the extraction of the vertex of the nut and the contour detection.

In addition, the image processing unit 420 binarizes the pixels in the weld bead region detected by the bead region extracting unit 430 to '1' and the remaining region to '0' (G = 255), and applying a black color (G = 0) to a pixel of '0' to obtain a binarized image as shown in FIG. 4B (S714).

Thereafter, the bead recognition apparatus 400 includes a first process (S720 to S723) for determining whether the shape of the weld bead is abnormal based on the feature values of the weld bead measured from the binarized image, The process proceeds to steps S730 to S734 for detecting holes and undercuts and determining whether or not the holes and undercuts are abnormal. Of course, the two processes may be performed simultaneously in parallel or sequentially.

First, the first detection unit 440 detects the feature information of the weld bead from the binarized image (S720 to 722). Specifically, the first detection unit 440 searches for a start point and an end point of the weld bead (S720), calculates the arc length of the bead using the searched start point and end point (S721) The average / maximum / minimum values of the bead width are obtained (S722).

Here, in step S722, the distance between the inside / outside outlines of the beads that meet the straight line (distance between a and b) can be calculated as the width w of the bead. Of course, this process can be carried out at a set interval with respect to the circumferential direction of the nut to obtain the maximum, minimum and average values of the bead width, respectively.

Thereafter, the first detecting unit 440 compares the detected feature information of the weld bead with predetermined reference information to determine whether the shape of the weld bead is abnormal (S723).

The second detection unit 450 determines whether holes or undercuts occur in the weld bead using the image obtained by combining the binarized image and the gray image (S730 to S733). Specifically, the second detector 450 combines (a) and (b) of FIG. 4 to obtain the image of FIG. 6 (S730). Then, the obtained image is subjected to adaptive binarization and then noise is removed from the image using a Gaussian and a morphology filter algorithm (S731).

Thereafter, the second detector 450 determines whether a hole or an undercut has occurred in the image from which the noise has been removed (S732), and if the hole or the undercut is found, the detected position is determined (S733). At this time, the positions of holes or undercuts can be determined using orthogonal and polar coordinate algorithms.

If a hole or an undercut is found, the second detector 450 compares the depth value of the hole or the undercut received from the depth measurement sensor with each predetermined reference value, (S734). That is, when a hole or an undercut is found, the depth is measured using a separate depth measuring sensor to determine whether the depth is abnormal.

In step S740, the determination unit 460 determines whether there is a welding defect using the results of steps S723 and S734, and the output unit 470 outputs a determination result in step S750.

The determination unit 460 determines that the weld bead is defective when any one of the shape and the depth is abnormal. Of course, when the shape of the weld bead is normal and no holes or undercuts are found at all, the determination unit 460 can determine that the welding is normal or good, and even when the weld bead shape is normal and the depth values of holes and undercuts are normal Welding can be judged as normal or good.

The output unit 470 outputs the determination result to the display screen or the client so that the administrator can quickly and easily check the welding condition and the defectiveness of the inspection sample. In addition, the output unit 470 can provide the client with the results of the weld state inspection including the feature information of the weld bead, whether or not holes or undercuts are generated, and the location of occurrence.

According to the bead recognition apparatus and method using the vision camera according to the present invention, it is possible to increase the detection performance of the weld bead and the data processing speed by using the image of the vision camera, and to reduce the discrimination error of the defect inspection have.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

100: Fixing jig 200: 2D camera
300: Illumination 400: Bead recognition device
410: input unit 420: image processing unit
430: Bead region extracting unit 440: First detection unit
450: second detection unit 460:
470:

Claims (12)

An input unit for receiving an image including a test sample through a 2D camera;
An image processor for setting a region of interest in the image and converting the region of interest into a gray image;
A bead region extracting unit for detecting a weld bead region of the inspection sample in the region of interest;
A first detection unit for detecting feature information of a weld bead from a binarized image obtained by binarizing the weld bead region and the background region in the region of interest and determining whether the shape of the weld bead is abnormal based on the detected feature information; And
And a second detection unit for determining whether a hole or an undercut in the weld bead occurs or an occurrence position from the combined image of the binarized image and the gray image. The method according to claim 1,
The feature information of the weld bead includes:
A bead recognition device using at least one of a start point and an end point of the weld bead, a length of the arc, and an average, a maximum and a minimum value of a plurality of bead widths obtained along the length direction of the arc. The method of claim 2,
Wherein the first detection unit comprises:
Forming a straight line in which a center point of the inspection sample and a point on the outer contour line meet each other and then intersecting the intersection point of the straight line and the inner contour line with the straight line and the outer contour line, And acquiring the plurality of bead widths by obtaining a distance between the intersection points of the beads. The method according to claim 1,
Wherein the second detection unit comprises:
And combining the binarized image and the gray image by filling a white region corresponding to the weld bead region in the binarized image with the image of the corresponding region in the gray image. The method according to claim 1,
Wherein the first detection unit comprises:
Comparing feature information of the weld bead with preset reference information to determine whether the weld bead is abnormal or not,
Wherein the second detection unit comprises:
Wherein the depth value of the hole or the undercut received from the depth measurement sensor is compared with each predetermined reference value to determine whether the depth of the hole or the undercut is abnormal. The method of claim 5,
A determination unit for determining that the weld bead is normal if both the shape and the depth are normal and determining that the weld bead is defective if at least one of the shape and the depth is abnormal; And
And an output unit for providing the determination result to a display screen or an external client. A bead recognition method in a bead recognition apparatus using a vision camera,
Receiving an image including a test sample through a 2D camera;
Setting a region of interest in the image and converting the region of interest into a gray image;
Detecting a weld bead region of the test sample in the region of interest;
Detecting characteristic information of a weld bead from a binarized image obtained by binarizing the weld bead region and the background region in the region of interest and determining whether the shape of the weld bead is abnormal based on the detected feature information; And
And determining whether a hole or an undercut in a weld bead occurs or an occurrence position from an image combining the binarized image and the gray image. The method of claim 7,
The feature information of the weld bead includes:
A bead recognition method using at least one of a start point and an end point of a weld bead, a length of the arc, and an average, a maximum and a minimum of a plurality of bead widths obtained along the length direction of the arc. The method of claim 8,
Wherein the step of detecting the feature information comprises:
Forming a straight line in which a center point of the inspection sample and a point on the outer contour line meet each other and then intersecting the intersection point of the straight line and the inner contour line with the straight line and the outer contour line, And acquiring the plurality of bead widths by obtaining a distance between the intersection points. The method of claim 7,
Wherein the determining step comprises:
And combining the binarized image with the gray image by filling a white region corresponding to the weld bead region in the binarized image with an image of the corresponding region in the gray image. The method of claim 7,
Further comprising the step of comparing the depth value of the hole or the undercut received from the depth measurement sensor with each predetermined reference value to determine whether the depth of the hole or the undercut is abnormal,
The step of determining whether or not the shape abnormality is present comprises:
And comparing the feature information of the weld bead with predetermined reference information to determine whether or not the shape of the weld bead is abnormal. The method of claim 11,
Determining that the weld bead is normal if both the shape and the depth are normal, determining that the weld bead is defective if at least one of the shape and the depth is greater than the shape and depth; And
And providing the determination result to a display screen or an external client.

Images