JP7176148B1 - Weld line detection system - Google Patents

Abstract

Kind Code: A1 A weld line detection system capable of improving work efficiency of instructing a welding robot to perform operations and improving welding quality is provided. A photographing unit (211) for photographing an image of an object to be welded, a coordinate system setting unit (212) for setting a user coordinate system based on a marker included in the photographed image, and a specific position of the marker based on the image. The detected specific position is set on the point cloud data acquired by the distance measurement sensor that measures the distance to the welding object, and the coordinates of the user coordinate system with the set specific position as the origin are given. A point cloud data drawing unit 213 that draws the obtained point cloud data in the user coordinate system, and a welding line detection unit 214 that detects the welding line to be welded based on the point cloud data drawn in the user coordinate system. Prepare. [Selection drawing] Fig. 2

Description

The present invention relates to a weld seam detection system.

Patent Literature 1 listed below discloses a motion teaching system for teaching a motion to a welding robot. In this motion teaching system, when teaching motion, the welding torch at the tip of the manipulator is replaced with an imaging unit. Then, based on the position information of the welding point generated based on the photographed image by the photographing unit and the position and orientation information of the manipulator when the photographed image was taken, the manipulator necessary for welding the welding point is determined. Position and posture trajectories are generated, and the trajectories at each welding location are combined to generate teaching data.

JP 2019-150930 A

By the way, in the motion teaching system of Patent Literature 1, before motion teaching, a line (welding line) indicating a welding point for which a trajectory is to be generated during motion teaching is drawn on the object to be welded. This work is performed by the operator using a pen to draw a weld line on the object to be welded. Therefore, the work is troublesome, and the line drawn with the pen may remain on the surface of the object to be welded.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a weld line detection system capable of improving work efficiency of instructing a welding robot to perform operations and improving welding quality.

A welding line detection system according to an aspect of the present invention includes a photographing terminal that photographs an image of a welding target, a coordinate system setting unit that sets a user coordinate system based on a marker included in the photographed image, and Detect a specific position of the marker based on the user A point cloud data drawing unit that draws the point cloud data to which the coordinates of the coordinate system are assigned in the user coordinate system, and a welding line that detects the welding line to be welded based on the point cloud data drawn in the user coordinate system. and a detector.

According to this aspect, the specific position of the marker is detected based on the captured image of the object to be welded and the marker, the detected specific position of the marker is set on the point cloud data, and the set specific position of the marker is set on the point cloud data. Point cloud data to which the coordinates of the user coordinate system as the origin are assigned is drawn on the image, and the welding line to be welded can be detected based on the point cloud data. Therefore, even if the operator does not draw the weld line on the object to be welded, the weld line on the object to be welded can be recognized.

In the above aspect, the welding line detection unit recognizes a plurality of surfaces corresponding to the object to be welded based on the point cloud data, and detects a line of intersection of two surfaces included in the plurality of surfaces as a welding line. good.

According to this aspect, of the surfaces represented by the point cloud data drawn in the user coordinate system, the line of intersection of two surfaces can be detected as the weld line, and the detection accuracy of the weld line can be improved. It becomes possible.

In the above aspect, the weld line detector may detect at least part of the line of intersection of the two surfaces as the weld line.

According to this aspect, at least a part of the line of intersection of the two surfaces can be detected as the weld line, and all candidates for the weld line can be detected.

The above aspect may further include a program creation unit that creates a work program for welding based on the weld line detected by the weld line detection unit.

According to this aspect, since a work program for welding the detected weld line can be created, it is possible to perform welding efficiently along the weld line specified by the work program.

In the above aspect, the marker may be an AR marker.

According to this aspect, when an AR marker is recognized, it is possible to start a program for superimposing and displaying the user coordinate system having the position of the AR marker as the origin on the actual image.

In the above aspect, the photographing terminal may include an image sensor for photographing an image and a distance measurement sensor.

According to this aspect, it is possible to fix the positional relationship between the image sensor and the distance measurement sensor, and to match the timing of acquiring data by each sensor.

In the above aspect, the imaging terminal further includes a display section for displaying an image, and a control section for controlling content to be displayed on the display section. They may be superimposed and displayed on the display unit.

According to this aspect, the operator who operates the imaging terminal can easily determine whether or not there is an undetected welding line while looking at the display section of the imaging terminal.

ADVANTAGE OF THE INVENTION According to this invention, the welding line detection system which can raise the working efficiency which makes a welding robot teach operation|movement, and can improve welding quality can be provided.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which illustrates the structure of the welding robot system containing the welding line detection system which concerns on embodiment. 1 is a diagram illustrating a functional configuration of a weld seam detection system; FIG. It is a figure which shows an example of a welding target. FIG. 10 is a diagram showing an example of a user coordinate system having a marker position as an origin; It is a figure which shows an example of a welding target. It is a figure which shows an example of the point cloud data drawn on a user coordinate system. FIG. 4 is a flowchart for explaining an example of operations when creating a work program in the weld line detection system; FIG.

Preferred embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that, in each figure, the same reference numerals have the same or similar configurations. Also, since the drawings are schematic, the dimensions and proportions of each component are different from the actual ones.

FIG. 1 is a diagram illustrating the configuration of a welding robot system including a weld line detection system according to an embodiment. The welding robot system 100 includes, for example, an imaging terminal 1, a robot control device 2, and a manipulator 3. The photographing terminal 1 and the robot control device 2 are connected via a network N, for example, and the robot control device 2 and the manipulator 3 are connected via a communication cable C, for example. The network N may be wired (including communication cables) or wireless. Welding robot system 100 may include a teaching pendant. The teaching pendant is an operating device that teaches the operation of the manipulator 3 by the operator.

The manipulator 3 is a welding robot that performs arc welding according to construction conditions set by the robot controller 2 . The manipulator 3 has, for example, an articulated arm 31 provided on a base member fixed to the floor of a factory or the like, and a welding torch 32 connected to the tip of the articulated arm 31 .

The robot control device 2 is a control unit that controls the operation of the manipulator 3, and includes, for example, a control section 21, a storage section 22, a communication section 23, and a welding power source section 24.

The control unit 21 controls the manipulator 3 and the welding power source unit 24 by causing the processor to execute a work program stored in the storage unit 22, for example.

The communication unit 23 controls communication with the imaging terminal 1 connected via the network N, and controls communication with the manipulator 3 connected via the communication cable C. FIG.

The welding power supply unit 24 supplies a welding current, a welding voltage, and the like to the manipulator 3 according to predetermined welding execution conditions, for example, in order to generate an arc between the tip of the welding wire and the work. The welding conditions include, for example, data items such as welding conditions, welding start position, welding end position, arc discharge time, welding distance, welding torch posture, and welding torch moving speed. The welding power source section 24 may be provided separately from the robot control device 2 .

The photographing terminal 1 is, for example, a digital camera, but may be a portable terminal with a digital camera. Portable terminals include, for example, portable terminals such as tablet terminals, smart phones, personal digital assistants (PDAs), and notebook PCs (personal computers). The imaging terminal 1 includes, for example, a control unit 11, an imaging unit 12, a communication unit 13, and a display unit 14.

The control unit 11 controls each unit of the photographing terminal 1 by causing the processor to execute a predetermined program stored in the memory.

The photographing unit 12 includes, for example, a lens and an imaging device (image sensor), and converts light from a subject received by the lens into an electric signal (digital image data).

The communication unit 13 controls communication with the robot control device 2 connected via the network N. FIG.

The display unit 14 is, for example, a display having a touch panel, displays an image of a subject captured by the imaging unit 12, and receives inputs such as operation instructions from the operator. The display unit 14 may be provided separately from the photographing terminal 1 as a display device having a touch panel, for example.

FIG. 2 is a diagram illustrating the functional configuration of the weld seam detection system according to the present invention. The welding line detection system has, for example, an imaging unit 211, a coordinate system setting unit 212, a point cloud data drawing unit 213, a welding line detection unit 214, and a program creation unit 215 as functional configurations. Among these functions, the imaging unit 211 is a function that the imaging terminal 1 has. On the other hand, the coordinate system setting unit 212, the point cloud data drawing unit 213, the welding line detection unit 214, and the program creation unit 215 may all be provided in either the imaging terminal 1 or the robot control device 2, or the imaging terminal 1 and each function may be distributed to the robot control device 2 . Also, a device other than the photographing terminal 1 and the robot control device 2 may have some or all of the above functions.

The photographing unit 211 is the same as the photographing unit 12 of the photographing terminal 1 described above. The photographing unit 211 according to the present embodiment photographs, for example, a structure composed of a plurality of iron plate members (workpieces) to be arc-welded as a welding target. FIG. 3 shows an example of an object to be welded. In the figure, a structure composed of one work Wa serving as a bottom plate, two works Wb and Wc serving as side plates, and one work Wd serving as a back plate is displayed as an object to be welded. ing. A marker M, which will be described later, is placed in the space formed by this structure.

The coordinate system setting unit 212 shown in FIG. 2 sets a user coordinate system whose origin is the position of the marker included in the image captured by the imaging unit 211 . FIG. 4 shows an example of a user coordinate system whose origin is the position of the marker. In the figure, the position of the marker M is the origin O, and a three-dimensional orthogonal coordinate system with the X, Y, and Z axes orthogonal to each other at the origin O is displayed as the user coordinate system.

It should be noted that the origin of the user coordinate system may be set with reference to the marker (for example, the corner of the marker, the center of the marker, etc.). The reason why the origin of the user coordinate system is set based on the marker instead of setting the origin of the user coordinate system based on the imaging terminal 1 is as follows. Since the photographing terminal 1 is carried and moved by the worker when photographing, it is difficult to specify the position of the photographing terminal 1 on the robot coordinate system. On the other hand, since the marker is fixedly arranged, it is relatively easy to specify the position of the marker on the robot coordinate system. Therefore, setting the user coordinate system with reference to the marker makes it easier to calibrate the positional relationship between the user coordinate system and the robot coordinate system than setting the user coordinate system with the imaging terminal 1 as the reference. becomes.

Here, the marker M may be an identifier that allows the imaging unit 211 to recognize that it is placed in the space. As a marker, it is preferable to use an AR marker, for example. By using AR markers, when an AR marker placed in space is recognized, it is possible to easily display the user coordinate system with the AR marker as the origin superimposed on the actual image. Become.

The user coordinate system based on the marker can be set by moving the origin of the camera coordinate system (for example, the center of the lens) to a specific position of the marker, which will be described later. Such a user coordinate system can be set, for example, by applying a known technique for setting the coordinate system of AR markers.

The point cloud data drawing unit 213 shown in FIG. 2 acquires coordinate data (point cloud data) corresponding to the object to be welded, and draws the acquired coordinate data on the user coordinate system.

A specific description will be given. The point cloud data drawing unit 213 detects the specific position of the marker (for example, the corner of the marker or the center of the marker) based on the image captured by the imaging unit 211, and converts the detected specific position of the marker to the distance described later. The point cloud data set on the point cloud data acquired by the measurement sensor and given the coordinates of the user coordinate system with the specific position of the set marker as the origin is drawn in the user coordinate system. For the specific position of the marker set on the point cloud data, for example, the specific position of the marker on the point cloud data may be automatically recognized by data analysis, or the specific position of the marker on the point cloud data may be determined by an operator. You can specify it by pointing to it.

Coordinate data corresponding to the object to be welded can be obtained by, for example, a distance measuring sensor. The distance measurement sensor may be any sensor capable of measuring the distance to the object to be welded. As the distance measurement sensor, for example, a LiDAR (Light Detection and Ranging) sensor, a millimeter wave sensor, an ultrasonic sensor, or the like can be used. Alternatively, the coordinate data corresponding to the object to be welded may be acquired by calculation based on a plurality of images of the object to be welded photographed from a plurality of different positions. In this case, a three-dimensional measurement method based on a known stereo method can be used.

Here, the imaging terminal 1 may include a distance measurement sensor. As a result, the positional relationship between the image sensor and the distance measurement sensor can be fixed, and the timing of acquiring data from each sensor can be matched. Therefore, it is possible to improve the processing efficiency of setting the above-described specific positions of the markers on the point cloud data. In addition, by providing the imaging terminal 1 with an image sensor and a distance measuring sensor, the operator who operates the imaging terminal 1 can freely move to an arbitrary position where the welding line to be welded and the marker can be photographed at the same time. work efficiency can be improved.

Furthermore, the photographing terminal 1 may include a sensor that has both the function of a sensor that captures and acquires an image and the function of a sensor that measures and acquires a distance. As a result, the image including the welding target and the distance to the welding target can be obtained from the same location and at the same timing, so that the processing efficiency of setting the above-described specific position of the marker on the point cloud data can be further improved. becomes possible.

The concept of drawing the coordinate data corresponding to the object to be welded as point group data on the user coordinate system will be described with reference to FIGS. 5 and 6. FIG.

FIG. 5 is a diagram showing an example of an object to be welded. The figure illustrates a welding object composed of a work We placed on a work table T and a work Wf placed so as to be substantially perpendicular to the work We. A marker M is placed near the object to be welded.

FIG. 6 is a diagram showing an example of point cloud data drawn on the user coordinate system. In the figure, coordinate data Wec and Wfc respectively corresponding to the work We and work Wf in FIG. 5, and coordinate data Tc corresponding to the work table T in FIG. 5 are drawn as point cloud data in the user coordinate system. there is

The welding line detection unit 214 shown in FIG. 2 detects the welding line to be welded based on the point cloud data drawn in the user coordinate system. Specifically, the weld line detection unit 214 recognizes a plurality of planes corresponding to the object to be welded based on the point cloud data, and detects a line of intersection of two planes included in the plurality of planes as a weld line. If there are multiple combinations of two planes, the weld line is detected for each combination. For example, in FIG. 6, the plane indicated by the coordinate data Wec corresponding to the work We and the plane indicated by the coordinate data Wfc corresponding to the work Wf are recognized, and the line of intersection of these two planes is detected as the weld line. . As a result, as shown in FIG. 5, the welding line L is detected as the line of intersection between the work Wf and the work We, and is superimposed on the actual image and displayed. When detecting the weld line, it is preferable to detect the weld line from end to end of the line of intersection of the two planes. The endpoints for detecting the weld line can be set under arbitrary conditions.

Here, what is detected as the weld line is not limited to the line of intersection of two planes. For example, a line of intersection of two surfaces may be detected as a weld line.

Further, when detecting a weld line, one or both of (1) and (2) below may be added to the conditions for detecting the weld line.
(1) If the detected weld line is less than a certain length, do not adopt the weld line (ignore the weld line).
(2) Excluding a predetermined length portion at both ends of the detected weld line from the weld line (ignoring part of the weld line).

Program creation unit 215 shown in FIG. 2 creates a work program for arc welding based on the weld line detected by weld line detection unit 214 . The program creation unit 215 stores the created work program in the storage unit 22 of the robot control device 2 . As a result, when the manipulator 3 performs arc welding, the control unit 21 of the robot control device 2 reads the work program, follows the welding procedure designated by the work program, and welds along the welding line designated by the work program. It is possible to control the manipulator 3 to perform welding.

Here, when creating the work program, the program creation unit 215 performs calibration to match the user coordinate system with the robot coordinate system set based on the position of the manipulator 3, for example. It is preferable to specify the position information of the manipulator 3 and the like so that the position on the user coordinate system and the position on the robot coordinate system can be associated with each other.

An example of the operation when the program creating unit 215 creates a work program will be described with reference to FIG.

First, the program creation unit 215 identifies a weld line on the user coordinate system based on the weld line detected by the weld line detection unit 214 (step S101).

Subsequently, the program creation unit 215 sets the attitude of the torch, such as the advancing angle of the torch, the receding angle of the torch, and the target angle of the torch, for the welding line specified in step S101 (step S102).

Subsequently, the program creation unit 215 determines a trajectory including the posture of each part of the manipulator 3 on the user coordinate system (step S103).

Subsequently, the program creation unit 215 converts the trajectory on the user coordinate system determined in step S103 into a trajectory on the robot coordinate system to create a work program (step S104). Then, this operation ends.

Here, each step of the above operation may be executed by either the photographing terminal 1 or the robot control device 2. Part of each step is executed by the photographing terminal 1, and the remaining part is executed by the robot control device 2. may be executed.

As described above, according to the weld line detection system according to the embodiment, the specific position of the marker is set as the origin based on the captured image of the object to be welded and the marker placed in the space formed by the object to be welded. A user coordinate system is set, point cloud data corresponding to the object to be welded is drawn in the user coordinate system, and the weld line of the object to be welded can be detected based on the point cloud data. Therefore, even if the operator does not draw the weld line on the object to be welded, the weld line on the object to be welded can be recognized.

Therefore, according to the weld line detection system according to the embodiment, it is possible to improve the work efficiency of instructing the operation to the manipulator 3 and improve the welding quality.

In addition, by making the photographing terminal 1 according to the above-described embodiment a portable terminal, the worker can photograph the object to be welded from any position, so that the welding line at the position desired by the worker can be detected. become able to. When the position of the photographing terminal 1 is fixed, it is impossible to detect the welding line in a place or range that cannot be photographed from the fixed position. It also includes such work. Therefore, by using the portable imaging terminal 1, for example, by moving the worker to a position where the welding line under the welding object can be photographed and photographing the welding object, the imaging terminal at a fixed position can be used. A welding line that cannot be detected with 1 can be detected.

[Variation]
It should be noted that the present invention is not limited to the embodiments described above, and can be implemented in various other forms without departing from the gist of the present invention. Therefore, the above-described embodiment is merely an example in all respects, and should not be construed as limiting.

For example, although the welding line detection unit 214 according to the above-described embodiment detects the welding line to be welded, it may detect welding line candidates. In this case, it is preferable to further include a welding line selection unit that allows the operator to select a welding line to be actually welded from the detected welding line candidates. In addition to this, it is preferable that the program creation unit 215 creates a work program for performing arc welding based on the weld line selected by the weld line selection unit.

Further, even if the control unit 11 of the imaging terminal 1 according to the above-described embodiment causes the display unit 14 to display the welding line detected by the welding line detection unit 214 superimposed on the image captured by the imaging unit 12. good. This allows the operator operating the imaging terminal 1 to easily determine whether or not there is an undetected welding line while looking at the display section 14 of the imaging terminal 1 . In other words, it is possible to provide the operator with effective information for determining whether or not to redo the photographing by the photographing terminal 1 .

DESCRIPTION OF SYMBOLS 1... Imaging terminal 2... Robot control apparatus 3... Manipulator 11... Control part 12... Imaging part 13... Communication part 14... Display part 21... Control part 22... Storage part 23... Communication part 24 Welding power supply unit 31 Articulated arm 32 Welding torch 100 Welding robot system 211 Imaging unit 212 Coordinate system setting unit 213 Point cloud data drawing unit 214 Welding line detection unit 215...Program creation part, C...Communication cable, M...Marker, N...Network

Claims (7)

a photographing terminal for photographing an image of a welding target;
a coordinate system setting unit that sets a user coordinate system based on a marker included in the photographed image;
A specific position of the marker is detected based on the image, the detected specific position is set on point cloud data acquired by a distance measurement sensor that measures the distance to the object to be welded, and the set a point cloud data drawing unit that draws, in the user coordinate system, the point cloud data to which coordinates of the user coordinate system having a specific position as an origin are assigned;
a welding line detection unit that detects the welding line of the welding target based on the point cloud data drawn in the user coordinate system;
Weld line detection system. The welding line detection unit recognizes a plurality of surfaces corresponding to the object to be welded based on the point cloud data, and detects a line of intersection of two surfaces included in the plurality of surfaces as the welding line.
The weld seam detection system of claim 1. The weld line detection unit detects at least a part of the line of intersection of the two surfaces as the weld line.
3. The weld seam detection system of claim 2. a program creation unit that creates a work program for welding based on the weld line detected by the weld line detection unit;
3. The weld seam detection system of claim 1 or 2, further comprising. the marker is an AR marker;
The weld seam detection system according to claim 1 or 2. The imaging terminal includes an image sensor that captures the image, and the distance measurement sensor.
The weld seam detection system according to claim 1 or 2. The shooting terminal is
a display unit that displays the image;
A control unit that controls the content to be displayed on the display unit,
The control unit superimposes the welding line detected by the welding line detection unit on the image and causes the display unit to display the image.
The weld seam detection system of claim 6.

Images