KR102199505B1 - Apparatus and method for inspecting contamination of welding tip based on fisheye camera - Google Patents

Abstract

The present invention relates to a welding tip contamination inspection apparatus and method based on a fisheye camera, which includes a fisheye camera for simultaneously photographing a pair of welding tips facing each other; A camera calibration unit for obtaining and storing camera calibration information by photographing and analyzing the calibration plate attached to the pair of welding tip entrances through the fisheye camera; First and second for setting the first and second virtual cameras facing each of the pair of welding tips from the front, and for converting the viewpoint of the camera image from the viewpoint of the fisheye camera to the viewpoint of each of the first and second virtual cameras A viewpoint transformation information management unit for obtaining and storing viewpoint transformation information; A camera distortion removing unit configured to generate a corrected image by removing camera lens distortion of the camera image according to the camera correction information when a camera image in which a pair of welding tips are simultaneously photographed is obtained through the fisheye camera; After dividing the corrected image into a first image in which one of the pair of welding tips is photographed and a second image in which the remainder is photographed, the viewpoint of the first and second images according to the first and second viewpoint conversion information An image view converting unit for converting the image; And a welding tip inspection unit that simultaneously performs a contamination level inspection of each of the first and second welding tips based on each of the first and second images.

Description

Apparatus and method for inspecting contamination of welding tip based on fisheye camera}

The present invention relates to a welding tip contamination level inspection apparatus and method based on a fisheye camera, and more particularly, to a welding tip contamination level inspection apparatus and method based on a fisheye camera that enables the welding tip contamination level inspection operation to be performed with a minimized inspection equipment.

In general, spot welding is a type of resistance welding technology.After placing two or three metal sheets for welding and placing them between a pair of spot welding guns, pressurized with a pair of spot welding guns. This is a method of welding a metal plate by applying a current to a spot welding gun and using resistance heating generated therefrom. Such spot welding, also referred to as spot welding, has the advantage of short welding time and is widely used in thin plate welding.

During metal bonding in spot welding, a part of the base material after welding is buried at the tip of the spot welding gun, making it impossible to use the spot welding gun for a long time. To prevent this, the spot welding gun is protected by attaching a copper welding tip to the end of the spot welding gun.

In order to perform good quality welding, the condition of the welding tip is important. However, when welding 100 to 200 times, the end of the welding tip is contaminated and cannot be used. At this time, use a dresser (ATD: Auto Tip Dressor) to polish the end of the welding tip. After using a dresser (ATD) to cut off the contaminated welding tip end, inspect whether there are any foreign substances left on the end face of the welding tip through a welding tip inspection device. If the inspection result is defective, the welding tip is transferred to the dresser (ATD) with a welding tip transfer gun (GUN) to grind the welding tip. Typically, a dresser (ATD) and a welding tip inspection device are combined and installed in one package.

Since the welding tip contamination test is performed based on the numerical value of the contamination area, it is necessary to acquire an image of the welding tip from the front.

Accordingly, in the related art, as shown in FIG. 1, the two cameras 330 and 360 corresponding to each of the pair of welding tips 11 and 12, and the distal diameter images of each of the welding tips 11 and 12 are mirrored on each of the cameras 330 and 360. Reflectors 340 and 370 obtained and provided in a manner are provided, and an image of the welding tip viewed from the front is acquired and analyzed through these, so that the contamination level of each of the pair of welding tips 11 and 12 is inspected.

However, in the case of using two cameras and a reflector, there arises a problem that acts as an interference factor in the activity of the spot welding device due to the increased number of parts and the volume of the device.

Korean Patent Publication No. 10-2018-0064102 (Publication date: June 14, 2018)

Accordingly, in order to solve the above problems, the present invention is to provide a new type of welding tip contamination inspection apparatus and method for replacing two cameras and two reflectors with a single fisheye camera.

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

As a means for solving the above problem, according to an embodiment of the present invention, a fisheye camera for simultaneously photographing a pair of welding tips facing each other; A camera calibration unit for obtaining and storing camera calibration information by photographing and analyzing the calibration plate attached to the pair of welding tip entrances through the fisheye camera; First and second for setting the first and second virtual cameras facing each of the pair of welding tips from the front, and for converting the viewpoint of the camera image from the viewpoint of the fisheye camera to the viewpoint of each of the first and second virtual cameras A viewpoint transformation information management unit for obtaining and storing viewpoint transformation information; A camera distortion removing unit configured to generate a corrected image by removing camera lens distortion of the camera image according to the camera correction information when a camera image in which a pair of welding tips are simultaneously photographed is obtained through the fisheye camera; After dividing the corrected image into a first image in which one of the pair of welding tips is photographed and a second image in which the remainder is photographed, the viewpoint of the first and second images according to the first and second viewpoint conversion information An image view converting unit for converting the image; And it provides a welding tip contamination level inspection apparatus based on a fisheye camera including a welding tip inspection unit that simultaneously performs contamination level inspection of each of the first and second welding tips based on each of the first and second images.

The camera calibration unit extracts and analyzes a plurality of feature points from the grid pattern to obtain and store camera calibration information, and the camera calibration information includes at least one of a camera internal parameter, a camera external parameter, and a lookup table. To do.

The viewpoint conversion information management unit pre-acquires and stores setting information of each of the first and second virtual cameras, and the setting information includes information on a distance to a welding tip and a focal length of the camera.

The viewpoint conversion information management unit determines the physical positional relationship between the image coordinate system and the virtual camera coordinate system based on the setting information, and determines the physical positional relationship between the image coordinate system and the fisheye camera coordinate system from the camera external parameter, and then determines the image coordinate system. It is characterized by inferring and analyzing a positional relationship between a fisheye camera coordinate system and a virtual camera coordinate system as a medium to obtain each viewpoint transformation information.

As a means for solving the above problem, according to another embodiment of the present invention, photographing and analyzing a grid pattern attached to at least one of the pair of welding tips through a fisheye camera positioned between a pair of welding tips facing each other Obtaining and storing camera calibration information; First and second for setting the first and second virtual cameras facing each of the pair of welding tips from the front, and for converting the viewpoint of the camera image from the viewpoint of the fisheye camera to the viewpoint of each of the first and second virtual cameras Obtaining and storing view change information; Generating a correction image by removing camera lens distortion of the camera image according to the camera correction information when a camera image in which a pair of welding tips are simultaneously photographed is obtained through the fisheye camera; After dividing the corrected image into a first image in which one of the pair of welding tips is photographed and a second image in which the remainder is photographed, the viewpoint of the first and second images according to the first and second viewpoint conversion information Converting; And simultaneously performing a contamination level inspection of each of the first and second welding tips based on each of the first and second images.

The present invention makes it possible to replace two cameras and two reflectors with one fisheye camera, so that the cost and volume required to implement the device can be drastically reduced.

In addition, by removing lens distortion included in the camera image acquired through the fisheye camera and converting the shooting point of the fisheye camera to the front view point of the welding tip, the welding tip contamination level inspection can be performed. Make it possible to quantify. That is, the contamination area can be more accurately detected and quantified, thereby ensuring the reliability of the welding tip contamination level inspection.

1 is a view showing a welding tip contamination inspection apparatus according to the prior art.
2 is a view showing a welding tip contamination level inspection apparatus based on a fisheye camera according to an embodiment of the present invention.
3 is a diagram for describing a viewpoint of a virtual camera according to an embodiment of the present invention.
4 is a diagram illustrating a device initialization method according to an embodiment of the present invention.
5 and 6 are diagrams for explaining a physical positional relationship between a fisheye camera coordinate system and a virtual camera coordinate system applied to a welding tip contamination level inspection apparatus based on a fisheye camera according to an embodiment of the present invention.
7 is a view showing a welding tip contamination inspection method according to an embodiment of the present invention.

Objects and effects of the present invention, and technical configurations for achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. In describing the present invention, when it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted.

In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary depending on the intention or custom of users or operators.

However, the present invention is not limited to the embodiments disclosed below, and may be implemented in various different forms. These embodiments are provided only to make the disclosure of the present invention complete, and to fully inform the scope of the invention to those skilled in the art to which the present invention pertains, and the present invention is defined by the scope of the claims. It just becomes. Therefore, the definition should be made based on the contents throughout this specification.

2 is a view showing a welding tip contamination inspection apparatus based on a fisheye camera according to an embodiment of the present invention.

As shown in Figure 2, the welding tip contamination level inspection apparatus of the present invention is a fisheye camera 20 for side-viewing a pair of welding tips 11 and 12 facing each other, and attaching a grid pattern to the pair of welding tip entrances After photographing, a camera calibration unit 30 that extracts and analyzes feature points from a grid pattern in the camera image to obtain and stores camera calibration information, and first and second devices for converting the shooting time of the fisheye camera into the viewpoint of the virtual camera. A viewpoint conversion information management unit 40 that acquires and stores viewpoint conversion information, a camera distortion removal unit 50 that generates a correction image by removing camera lens distortion from a camera image obtained through a fisheye camera, and the correction After dividing the image into first and second images in which each of the welding tips was captured, the first image is converted into a viewpoint according to the first viewpoint conversion information, and the second image is converted into a viewpoint according to the second viewpoint conversion information. And an image viewpoint converting unit 60 for converting, and a welding tip inspection unit 70 for simultaneously performing contamination level inspection of each of the pair of welding tips based on each of the first and second images.

As described above, the present invention enables the contamination level inspection of a pair of welding tips to be performed at once using only one fisheye camera.

However, since the fisheye camera at this time photographs a pair of welding tips from the side, even after removing the lens distortion of the fisheye camera, the welding tip tip diameter area appearing in the camera image is not a circle or ellipse, but an asymmetrical ellipse. This becomes a factor that hinders accurate quantification of the contaminated area.

Accordingly, the present invention introduces a virtual camera viewpoint that assumes that the welding tip is photographed from the front, as shown in FIG. 3, and uses this to convert the viewpoint of the camera image from the side of the welding tip to the front of the welding tip. In other words, by converting the image taken from the side through the fisheye camera into an image that looks as if it was taken from the front, and then performing a pollution level inspection operation therefrom, the possibility of occurrence of an area calculation error is prevented in advance.

Hereinafter, a welding tip contamination inspection method based on the fisheye camera of the present invention will be described in more detail with reference to FIGS. 4 to 7.

4 is a diagram illustrating a device initialization method according to an embodiment of the present invention.

If device initialization is requested and a calibration plate (eg, grid pattern, chessboard pattern) for camera calibration is attached to a pair of welding tip entrances, the camera calibration unit 30 A camera image is obtained by photographing a pair of welding tips sideways through the camera 20 (S11).

For reference, it is preferable that the calibration plate be attached to each of the pair of welding tips, but if necessary, the camera calibration operation may be performed after attaching only to one of the pair of welding tips. This is because the fisheye camera includes lenses having a symmetrical structure in the left and right (or upper and lower sides), and the degree of distortion of one of the left and right lenses is the same as the degree of distortion of the other.

And after automatically extracting the feature points in the camera image by applying a feature point detection algorithm to the camera image acquired through step S11 (S12), camera calibration based on the feature points is performed to determine the internal/external parameters of the camera and a look-up table (or distortion model). Coefficients) are obtained, and these are set and stored as camera calibration information (S13).

At this time, the internal variables of the camera are the scale factor (kx, ky), the focal length (f), and the image center (u0, v0), and the external variables of the camera are the amount of rotation between the image coordinate system and the camera coordinate system (α=rx, β=ry). , γ=rz) and the amount of translation (tx,ty,tz). The lookup table is mapping data on the correspondence between pixels in the camera image and pixels in the corrected image. In addition, of course, such camera calibration information may be directly input or adjusted by the user when necessary.

The viewpoint conversion information management unit 40 is based on the setting information of the first and second virtual cameras set by the user (ie, the distance between the welding tip and the virtual camera, the virtual camera focal length) and the external parameters of the camera, After grasping the physical positional relationship between the camera coordinate system and the virtual camera coordinate system, the viewpoint conversion information for converting the camera image viewpoint from the fisheye camera to the virtual camera is obtained and stored therefrom (S14).

An image coordinate system, a fisheye camera coordinate system, and a virtual camera coordinate system according to an embodiment of the present invention have a physical positional relationship as shown in FIGS. 5 and 6.

As shown in FIG. 5, the physical positional relationship between the image coordinate system and the virtual camera coordinate system can be easily grasped from the distance between the welding tip and the virtual camera (Vd) and the virtual camera focal length (Vf).

And, as shown in Fig. 6, the physical positional relationship between the image coordinate system and the fisheye camera coordinate system is the amount of rotation between the image coordinate system and the camera coordinate system included in the camera external variable obtainable through the camera calibration operation (α=rx, β=ry, It can be seen that it can be easily identified based on γ=rz) and the amount of translation (tx,ty,tz).

In this case, the positional relationship between the fisheye camera coordinate system and the virtual camera coordinate system can be easily inferred through the image coordinate system.

Accordingly, in step S14, the image coordinate system is based on the external parameters of the camera having information about the amount of rotation (α=rx, β=ry, γ=rz) and the amount of translation (tx,ty,tz) between the image coordinate system and the camera coordinate system. The relationship between the image coordinate system and the virtual camera coordinate system is determined based on the position information of the first and second virtual cameras (ie, the distance between the welding tip and the virtual camera, and the virtual camera focal length). Try to understand the physical location relationship. Then, the physical positional relationship between the fisheye camera coordinates and the virtual camera coordinate system can be inferred through the image coordinate system, and from these relationships, the viewpoint conversion information for converting the fisheye camera coordinate system to the virtual camera coordinate system (that is, the viewpoint of the camera image). Calculate and store the viewpoint conversion information) that enables conversion from a fisheye camera to a virtual camera.

When all of the camera calibration information and viewpoint conversion information are obtained in this way, the device initialization is terminated, and the grid pattern attached to the pair of welding tip entrances is also removed.

7 is a view showing a welding tip contamination inspection method according to an embodiment of the present invention.

If a welding tip inspection is requested while the device initialization is completed, the camera distortion removing unit 50 photographs a pair of welding tips sideways through the fisheye camera 20 located between the pair of welding tips to obtain a camera image. Do (S21).

In addition, according to the camera correction information obtained through step S13, all camera lens distortions included in the camera image are removed to generate a correction image (S22).

And after the correction image is divided into a screen to generate a first image in which the first welding tip is photographed and a second image in which the second welding tip is photographed (S23), the first image is converted into a viewpoint according to the first viewpoint conversion information. Then, the second image is converted in view according to the second view conversion information (S24). That is, by converting the viewpoints of the first and second images from the fisheye camera viewpoint to the virtual camera viewpoint, and obtaining images as if the first and second welding tips were photographed from the front, the contaminated area can be accurately quantified in the future. Do (S24).

Finally, after performing a welding tip contamination level test based on each of the first and second images, the welding tip contamination level test result is visually guided to the user (S25).

The welding tip contamination level inspection of step S25 can be performed in various ways, for example, retrieving the position of the welding tip tip diameter from the grayscale image through a template matching technique; Detecting a size of a portion corresponding to the tip diameter of the welding tip found in the tip diameter searching step, and extracting a dressing surface of the welding tip based on the detected tip diameter; And analyzing the contamination degree of the dressing surface of the welding tip by comparing color difference information based on the welding tip after extracting color information corresponding to the detected dressing surface from the tip image in which CbCr has not been removed.

As described above, the present invention simultaneously photographs a pair of welding tips using only one fisheye camera, but performs image correction in consideration of lens distortion of the fisheye camera and the time of photographing, and then conducts a welding team contamination level inspection. The accuracy of the inspection can also be safely guaranteed.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains will be able to make various modifications and variations without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

Claims (5)

Fisheye camera for photographing a pair of welding tips facing each other at the same time;
A camera calibration unit for obtaining and storing camera calibration information by photographing and analyzing the grid pattern attached to the pair of welding tip entrances through the fisheye camera;
First and second for setting the first and second virtual cameras facing each of the pair of welding tips from the front, and for converting the viewpoint of the camera image from the viewpoint of the fisheye camera to the viewpoint of each of the first and second virtual cameras A viewpoint transformation information management unit for obtaining and storing viewpoint transformation information;
A camera distortion removing unit configured to generate a corrected image by removing camera lens distortion of the camera image according to the camera correction information when a camera image in which a pair of welding tips are simultaneously photographed is obtained through the fisheye camera;
After dividing the corrected image into a first image in which one of the pair of welding tips is photographed and a second image in which the remainder is photographed, the viewpoint of the first and second images according to the first and second viewpoint conversion information An image view converting unit for converting the image; And
A welding tip contamination level inspection apparatus based on a fisheye camera including a welding tip inspection unit that simultaneously performs contamination level inspection of each of the first and second welding tips based on each of the first and second images. The method of claim 1, wherein the camera calibration unit
A fisheye camera, characterized in that it extracts and analyzes a plurality of feature points from the grid pattern to obtain and store camera calibration information, wherein the camera calibration information includes at least one of a camera internal parameter, a camera external parameter, and a lookup table. Welding tip contamination inspection device based on. The method of claim 2, wherein the viewpoint conversion information management unit
Pre-acquisition and storage of setting information of each of the first and second virtual cameras, and the setting information includes information on a distance from the welding tip and a focal length of the camera. . The method of claim 3, wherein the viewpoint conversion information management unit
Based on the setting information, the physical positional relationship between the image coordinate system and the virtual camera coordinate system is determined, the physical positional relationship between the image coordinate system and the fisheye camera coordinate system is determined from the external parameters of the camera, and the fisheye camera coordinate system is used as a medium through the image coordinate system. A welding tip contamination level inspection apparatus based on a fisheye camera, characterized in that by inferring and analyzing a positional relationship between the coordinate system of the virtual camera and the viewpoint conversion information. Obtaining and storing camera calibration information by photographing and analyzing a grid pattern attached to at least one of the pair of welding tips through a fisheye camera positioned between a pair of welding tips facing each other;
First and second for setting the first and second virtual cameras facing each of the pair of welding tips from the front, and for converting the viewpoint of the camera image from the viewpoint of the fisheye camera to the viewpoint of each of the first and second virtual cameras Obtaining and storing view change information;
Generating a correction image by removing camera lens distortion of the camera image according to the camera correction information when a camera image in which a pair of welding tips are simultaneously photographed is obtained through the fisheye camera;
After dividing the corrected image into a first image in which one of the pair of welding tips is photographed and a second image in which the remainder is photographed, the viewpoint of the first and second images according to the first and second viewpoint conversion information Converting; And
A welding tip contamination level inspection method based on a fisheye camera comprising the step of simultaneously performing contamination level inspection of each of the first and second welding tips based on each of the first and second images.

Images