CN114986051B - Industrial robot welding control system and method based on template recognition - Google Patents

Industrial robot welding control system and method based on template recognition Download PDF

Info

Publication number
CN114986051B
CN114986051B CN202210669674.5A CN202210669674A CN114986051B CN 114986051 B CN114986051 B CN 114986051B CN 202210669674 A CN202210669674 A CN 202210669674A CN 114986051 B CN114986051 B CN 114986051B
Authority
CN
China
Prior art keywords
welding
coordinate
identification
template
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210669674.5A
Other languages
Chinese (zh)
Other versions
CN114986051A (en
Inventor
吴凌云
何志雄
陈统书
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Tiantai Robot Co Ltd
Original Assignee
Guangdong Tiantai Robot Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Tiantai Robot Co Ltd filed Critical Guangdong Tiantai Robot Co Ltd
Priority to CN202210669674.5A priority Critical patent/CN114986051B/en
Publication of CN114986051A publication Critical patent/CN114986051A/en
Application granted granted Critical
Publication of CN114986051B publication Critical patent/CN114986051B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
    • B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
    • B23K37/00—Auxiliary devices or processes, not specially adapted to a procedure covered by only one of the preceding main groups
    • B23K37/02—Carriages for supporting the welding or cutting element
    • B23K37/0252—Steering means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1679—Programme controls characterised by the tasks executed
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Robotics (AREA)
  • Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Manipulator (AREA)

Abstract

An industrial robot welding control system based on template recognition comprises a transmission crawler, a plurality of acquisition modules, an information processing module and an industrial robot, wherein the industrial robot comprises a receiving module and a plurality of welding mechanical arms; the information processing module comprises a coordinate dividing module; the coordinate dividing module is used for constructing a virtual cross coordinate system by taking the symmetrical center of the transmission crawler as the original point of the cross coordinate system, the transportation direction of the transmission crawler as the x-axis direction and the width direction of the transmission crawler as the y-axis direction; dividing a plurality of running areas along the y-axis direction at intervals; the method provided by the invention can track the welding point of the target to be welded in real time under the combined action of the construction of coordinates, the identification of a template graph and the movement speed of the transmission crawler, and can find out the welding point of the target to be welded even if the target to be welded is randomly placed on the transmission crawler, so that accurate welding is realized.

Description

Industrial robot welding control system and method based on template recognition
Technical Field
The invention relates to the technical field of robot control, in particular to an industrial robot welding control system and method based on template recognition.
Background
With the continuous progress of modern science and technology, especially the improvement of processing capacity of sensors and actuators, the development of computer technology, the progress of mechanical design and numerical control processing tools and the like jointly promote the rapid development of robots. The intelligent robot has been widely used in various fields such as engineering, manufacturing and life, and can replace human to perform tasks in welding work by virtue of the advantages of miniaturization, intelligence, high flexibility and the like.
In the method, because the positions of the clamping device and the welding robot are relatively fixed, only the welding point position of the welding robot needs to be set in advance, and the device to be transported transports the target to a specific position to immediately drive the welding robot to weld the target, but the system needs to be provided with the clamping device and the transporting device, and needs a larger production space to install the corresponding transporting device and the clamping device; a welding system that can reduce production space and maintain welding accuracy is in demand.
Disclosure of Invention
In view of the above-mentioned drawbacks, the present invention provides a system and a method for controlling welding of an industrial robot based on template recognition, which realize precise welding under the combined action of coordinate construction, template diagram recognition and movement speed of a transmission crawler.
In order to achieve the purpose, the invention adopts the following technical scheme: an industrial robot welding control system based on template identification comprises a transmission crawler, a plurality of acquisition modules, an information processing module and an industrial robot, wherein the industrial robot comprises a receiving module and a plurality of welding mechanical arms;
the information processing module comprises a coordinate dividing module, a matching module and a judging module;
the coordinate division module is used for constructing a virtual cross coordinate system by taking the symmetrical center of the transmission crawler as the original point of the cross coordinate system, the transportation direction of the transmission crawler as the x-axis direction and the width direction of the transmission crawler as the y-axis direction; dividing a plurality of running areas along the y-axis direction at intervals; the acquisition module and the welding mechanical arm are arranged in each operation area; the acquisition module is used for continuously shooting and acquiring image information in a corresponding operation area and sending the image information to the matching modules; the matching module is used for matching the image information by using a plurality of template pictures, judging whether the image information has an object to be welded, if so, acquiring a welding coordinate corresponding to a welding point of the object to be welded, and sending the welding coordinate to the coordinate dividing module; the judging module is used for receiving the welding coordinates, determining an operation area where the welding coordinates are located according to y-axis coordinates where the welding coordinates are located, generating control instructions for the welding coordinates and the operation area where the welding coordinates are located, and sending the control instructions to the receiving module; the receiving module is used for analyzing the control instruction to obtain corresponding operation area information, finding out a welding mechanical arm in the operation area according to the operation area information, and sending the welding coordinate to the welding mechanical arm in the operation area; and the welding mechanical arm updates the welding coordinate according to the movement speed of the transmission crawler, and when the welding coordinate enters the working range of the welding mechanical arm, the welding mechanical arm performs welding operation on the target to be welded according to the welding coordinate.
Preferably, the control device further comprises a rechecking module, wherein the rechecking module is configured to, when the object to be welded enters the rechecking area, acquire image information on the rechecking area, recall the matching module to acquire the welding coordinate of the object to be welded again, perform matching judgment on a y-axis coordinate in the welding coordinate located in the rechecking area and the first acquired y-axis coordinate, acquire a change value of the y-axis coordinate, if the change value of the y-axis coordinate is greater than a change threshold, recall the judging module, determine an operating area where the welding coordinate is located again according to the y-axis coordinate where the welding coordinate is located, and reproduce the control command.
Preferably, the review module further comprises a review region determination module; the rechecking area determining module comprises a reaction time determining submodule and a distance determining submodule; the reaction time determining submodule is used for acquiring coordinate calculation time, welding mechanical arm awakening time and updating time, wherein the coordinate calculation time is the time for the matching module to acquire a welding coordinate, the welding mechanical arm awakening time is the time for the receiving module to analyze the control instruction, and the updating time is the time for the judging module to generate the control instruction; the distance determining submodule is used for acquiring the sum of the coordinate calculation time length, the welding mechanical arm awakening time length, the updating time length and the time threshold value, and calculating a first reaction time distance according to the movement speed of the transmission crawler; the distance determining submodule is also used for acquiring coordinate calculation time, welding mechanical arm awakening time and updating time, and calculating a second reaction time distance according to the movement speed of the transmission crawler; determining the initial position of the reinspection area according to the distance between the welding mechanical arm and the first reaction time, determining the end position of the reinspection area according to the distance between the welding mechanical arm and the second reaction time, and constructing to obtain the reinspection area.
Preferably, the matching module comprises a preparation module; the preparation module comprises a template making submodule, an identification feature extraction submodule and a storage submodule; the template making submodule is used for making a plurality of template drawings, wherein each template drawing corresponds to a different integer angle; the identification feature extraction submodule comprises a gradient quantization unit and a lifting unit; the gradient quantization unit is used for performing first-layer pyramid direction gradient quantization and second-layer pyramid direction gradient quantization on the plurality of template pictures to respectively obtain identification features corresponding to the plurality of template pictures; the lifting unit is used for acquiring the identification features by taking the current angle as a list and storing the identification features; the storage submodule is used for storing all the identification features in the different angle table lists.
Preferably, the information processing module further comprises a correction module; the correction module comprises a to-be-welded target extraction submodule, an identification feature association submodule and a rotation translation submodule; the to-be-welded target extraction submodule is used for extracting the frame of the to-be-welded target from the image information in a sub-pixel point mode to obtain a target frame and sending the target frame to the identification feature association submodule; the identification feature association submodule is used for combining the identification features on the target frame and other identification features into a first identification point according to a ratio, and finding out a second identification point corresponding to the first identification point on the template graph according to the first identification point; acquiring distances between all first identification points and second identification points corresponding to the first identification points, judging whether the distances between all first identification feature points and the second identification points corresponding to the first identification feature points are larger than a distance threshold, if so, acquiring the number of first identification features meeting the distance threshold, judging whether the number meets a first number threshold, and if so, sending a correction instruction to the rotation and translation sub-module; and the rotation and translation sub-module receives the correction instruction, substitutes the first identification point and the second identification point into a change matrix, and corrects the pose of the template graph to obtain a corrected template graph.
A welding control method of an industrial robot based on template recognition is applied to the welding control system of the industrial robot based on the template recognition and is characterized in that the welding control system comprises a welding control system module and a welding control module; the method comprises the following steps: step S1: constructing a virtual cross coordinate system by taking the symmetrical center of the transmission crawler as the original point of the cross coordinate system, the transportation direction of the transmission crawler as the x-axis direction and the width direction of the transmission crawler as the y-axis direction; dividing a plurality of running areas along the y-axis direction at intervals; step S2: continuously shooting to obtain image information in a corresponding operation area, matching the image information by using a plurality of template pictures, judging whether an object to be welded exists in the image information, and if so, obtaining welding coordinates corresponding to a welding point of the object to be welded; and step S3: determining an operation area where the welding coordinate is located through a y-axis coordinate where the welding coordinate is located, finding out a welding mechanical arm in the operation area according to operation area information, and sending the welding coordinate to the welding mechanical arm in the operation area; and step S4: and updating the welding coordinates according to the movement speed of the transmission crawler, and when the welding coordinates enter the working range of the welding mechanical arm, carrying out welding operation on the target to be welded by the welding mechanical arm according to the welding coordinates.
Preferably, before executing step S4, the following steps are also executed: when the object to be welded enters a rechecking area, acquiring image information on the rechecking area, acquiring the welding coordinate of the object to be welded again, performing matching judgment on the y-axis coordinate in the welding coordinate when the object to be welded is located in the rechecking area and the first acquired y-axis coordinate, acquiring the change value of the y-axis coordinate, and determining the operation area where the welding coordinate is located again through the y-axis coordinate where the welding coordinate is located if the change value of the y-axis coordinate is greater than a threshold value.
Preferably, the determination process of the review area is as follows: acquiring coordinate calculation time, welding mechanical arm awakening time and updating time; acquiring the sum of the coordinate calculation time, the welding mechanical arm awakening time, the updating time and a time threshold, and calculating a first reaction time distance according to the movement speed of the transmission crawler; acquiring coordinate calculation time, welding mechanical arm awakening time and updating time, and calculating a second reaction time distance according to the movement speed of the transmission crawler; determining the initial position of the reinspection area according to the distance between the welding mechanical arm and the first reaction time, determining the end position of the reinspection area according to the distance between the welding mechanical arm and the second reaction time, and constructing to obtain the reinspection area.
Preferably, the following steps are also required to be executed before the step S2: step A1: making a plurality of template drawings, wherein each template drawing corresponds to a different integer angle; step A2: carrying out first-layer pyramid direction gradient quantization and second-layer pyramid direction gradient quantization on the plurality of template pictures, respectively obtaining identification features corresponding to the plurality of template pictures, obtaining the identification features by taking the current angle as a list, and storing the identification features; step A3: all identifying features in the different angle table columns are stored. The specific steps of selecting the coordinate selection template map in the step S2 are as follows: step B1: gradient extraction and quantification are carried out on the image information, a two-layer pyramid linear memory data container is created, and the image information line traverses the data of the two layers of pyramids; and step B2: performing bit-by-bit translation on the image information quantization gradient within the range of 4 x 4, and performing or operation on the obtained 16 images pixel by pixel to obtain a diffusion gradient matrix image of the image information after gradient diffusion; the similarity response matrix image acquisition unit is used for converting the diffusion gradient matrix image of the image information into gradient matrix images in the first four directions and the last four directions through AND operation, and finding out the maximum similarity of each gradient matrix image angle and each angle in a lookup table through a preset lookup table, wherein the lookup table is a pre-calculated table of various combinations in 8 directions; acquiring 8 similarity response matrix diagrams, converting all the similarity response matrix diagrams into a format of 16 or 64 orders, and storing the formats in the continuous linear memory data containers in a linearized manner(ii) a And step B3: finding an access entry of a linear memory of the two layers of pyramids by using the two layers of pyramids in the storage submodule according to the 8 similarity response matrix diagrams, and acquiring identification characteristics corresponding to each angle; and step B4: matching the image information and the identification features, acquiring the matching score of the image information and the identification features of each angle, judging whether the highest matching score is greater than a threshold value, if so, judging that the content of the image information is a target part, and acquiring a template corresponding to the highest matching score as a coordinate selection template; the matching score is calculated in the following way:
Figure GDA0003955096090000061
where Q is the input image information, T represents the template map, c is the position of the template map in the input image information, P represents the area centered on c, r is the offset position, and o is the identification feature.
Preferably, the specific steps of performing rotation and displacement on the coordinate selection template map in step S2 are as follows:
step C1: extracting the frame of the target part from the image information in a sub-pixel point mode to obtain a target frame;
and step C2: combining the recognition features on the target frame and the rest recognition features into a first recognition point according to a proportion, and finding out a second recognition point corresponding to the first recognition point on the template picture according to the first recognition point;
acquiring the distances between all the first identification points and the corresponding second identification points, judging whether the distances between all the first identification feature points and the corresponding second identification points are larger than a distance threshold, if so, acquiring the number of the first identification features meeting the distance threshold, and judging whether the number meets a first number threshold, if so, correcting the template graph;
step C3: and substituting the first identification point and the second identification point into a change matrix, and correcting the pose of the template graph to obtain a corrected template graph, wherein the calculation process of the change matrix is as follows:
coordinate of the first identification point andthe coordinates of the second recognition point are substituted into the following formula (1):
Figure GDA0003955096090000062
wherein R is a rotation matrix, and R is a rotation matrix,
Figure GDA0003955096090000063
to shift the matrix, q i And p i Coordinates, n, of the associated first and second identifying feature points, respectively i Is a feature vector, i is a natural integer greater than 1;
then, the minimum deflection angle R between the first identification point and the second identification point is obtained, the minimum deflection angle R is substituted into the following formula (2), and the minimum value of the rotation matrix R is obtained through calculation, wherein the formula (2) is as follows:
Figure GDA0003955096090000071
substituting the minimum value of the rotation matrix R back into equation (1) to obtain equation (3): (ii) a
Figure GDA0003955096090000072
Wherein c is i ＝p i ×n i ；
And (4) solving the deviation derivative of the formula (3), converting the deviation derivative into a linear equation and solving the angle r of the minimum deflection, the minimum horizontal offset x and the minimum vertical offset y, wherein the process is as follows:
the partial derivative formula four is as follows:
Figure GDA0003955096090000073
Figure GDA0003955096090000074
Figure GDA0003955096090000075
the transformation into the linear equation to find the angle of minimum deflection r, the minimum horizontal offset x and the minimum vertical offset y is as follows:
Figure GDA0003955096090000076
one of the above technical solutions has the following advantages or beneficial effects: according to the method provided by the invention, under the combined action of the coordinate construction, the template drawing identification and the movement speed of the transmission crawler, the welding point of the target to be welded can be tracked in real time, and even if the target to be welded is randomly placed on the transmission crawler, the welding point of the target to be welded can be found out, so that accurate welding is realized.
Drawings
FIG. 1 is a flow chart of one embodiment of a control method of the present invention.
Fig. 2 is a schematic structural diagram of an embodiment of the control system of the present invention.
Fig. 3 is a schematic diagram of the structure of the matching module in one embodiment of the control system of the present invention.
Fig. 4 is a schematic structural diagram of a review module in an embodiment of the control system of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "axial", "radial", "circumferential", and the like, indicate orientations and positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be considered as limiting the present invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless otherwise specified.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
As shown in fig. 1 to 4, an industrial robot welding control system based on template recognition comprises a transmission crawler, a plurality of acquisition modules, an information processing module and an industrial robot, wherein the industrial robot comprises a receiving module and a plurality of welding mechanical arms;
the information processing module comprises a coordinate dividing module, a matching module and a judging module;
the coordinate division module is used for constructing a virtual cross coordinate system by taking the symmetrical center of the transmission crawler as the original point of the cross coordinate system, the transportation direction of the transmission crawler as the x-axis direction and the width direction of the transmission crawler as the y-axis direction;
dividing a plurality of running areas along the y-axis direction at intervals;
the acquisition module and the welding mechanical arm are arranged in each operation area;
the acquisition module is used for continuously shooting and acquiring image information in a corresponding operation area and sending the image information to the matching module;
the matching module is used for matching the image information by using a plurality of template pictures, judging whether the image information has a target to be welded, if so, acquiring a welding coordinate corresponding to a welding point of the target to be welded, and sending the welding coordinate to the coordinate dividing module;
the judging module is used for receiving the welding coordinates, determining an operation area where the welding coordinates are located according to y-axis coordinates where the welding coordinates are located, generating control instructions for the welding coordinates and the operation area where the welding coordinates are located, and sending the control instructions to the receiving module;
the receiving module is used for analyzing the control instruction to obtain corresponding operation area information, finding out a welding mechanical arm in the operation area according to the operation area information, and sending the welding coordinate to the welding mechanical arm in the operation area;
and the welding mechanical arm updates the welding coordinate according to the movement speed of the transmission crawler, and when the welding coordinate enters the working range of the welding mechanical arm, the welding mechanical arm performs welding operation on the target to be welded according to the welding coordinate.
The conveying crawler belt replaces a clamping device and a conveying device in the existing robot welding system, and the installation space of the equipment is greatly reduced. When the welding robot is used, the object to be welded is placed on the conveying track, and the object to be welded is conveyed to the welding mechanical arm through the conveying track to be welded. However, the width of the conveying crawler is large, and the posture of the target to be welded may be changed when a worker or equipment places the target to be welded on the conveying crawler, so that the welding robot cannot perform fixed-point welding.
Therefore, the welding device is also provided with an acquisition module and an information processing module, wherein the acquisition module can be a camera or photographic equipment, can acquire image information of a target to be welded on the transmission track in real time, and then sends the image information to the information processing module;
the acquisition module only has the functions of acquisition, shooting and transmission, and cannot judge the operation area of the welding coordinate, so that the position of the welding coordinate cannot be directly and electrically connected with the welding mechanical arm to transmit, and only the image information can be processed through the information processing module on the cloud end;
the information processing module is internally provided with a judging module, a matching module and a coordinate dividing module, wherein the coordinate dividing module firstly takes the symmetrical center of the transmission crawler as the original point of a cross coordinate system, the transportation direction of the transmission crawler as the x-axis direction and the width direction of the transmission crawler as the y-axis direction to construct a virtual cross coordinate system, and the position of a target welding point to be welded is determined in a coordinate mode; and then matching the image information through the matching module, wherein the matching module is provided with a plurality of template pictures, the template pictures are obtained by training by taking the target to be welded as a training material, the position of the welding point on the template pictures is fixed, and the welding coordinate of the corresponding welding point can be found in a cross coordinate system through the template pictures. Then the welding coordinate is sent to a judging module, the judging module receives the welding coordinate, judges that the welding coordinate falls into an operation area according to a y-axis coordinate on the welding coordinate, and sends the operation area information of the coordinate to the receiving module; the receiving module is electrically connected with the plurality of welding mechanical arms, each welding mechanical arm is independently installed in an operation area, and the receiving module can judge the welding coordinate by receiving the information of the operation area and finally the welding mechanical arm is responsible for welding.
When the welding mechanical arm receives the welding coordinates, the x-axis coordinates of the welding coordinates can be updated according to the running speed of the transmission crawler belt, the direction of the object to be welded in the y-axis direction is unchanged when the object to be welded is transmitted by the transmission crawler belt, and only the direction of the x-axis changes along with the movement of the transmission crawler belt, so that the real-time position of the object to be welded corresponding to the welding coordinates can be obtained in real time only by updating the moving speed of the transmission crawler belt and the x-axis coordinates, and the welding mechanical arm can be driven to weld the welding coordinates of the object to be welded when the object to be welded falls into the welding range with the welding mechanical arm.
The following is an example for explanation:
firstly, the coordinate division module constructs a transmission crawler belt with the width of 100 and the length of 2000 into a virtual cross coordinate system, then a plurality of running areas are divided along the y-axis direction at intervals, wherein the interval is 20, at the moment, the transmission crawler belt is divided into 5 running areas along the width direction, wherein the first running area is (x, -50) to (x, -30), the second running area is (x, -30) to (x, -10), the third running area is (x, -10) to (x, 10), the fourth running area is (x, 10) to (x, 30), and the fifth running area is (x, 30) to (x, 50). And the welding mechanical arm and the acquisition module are arranged in each operation area, wherein the x coordinate of the acquisition module is (100, y), the x coordinate of the welding mechanical arm is (700, y), the values of the acquisition module and the y in the welding mechanical arm can be determined by the mechanical properties of the acquisition module and the y in the welding mechanical arm, and the values are usually the central points of the operation areas, for example, the coordinates of the acquisition module in the first operation area are (100, -40), and the coordinates of the welding robot in the first operation area are (700, -40). When the system is started, the transmission crawler drives an object to be welded to enter an acquisition range of the acquisition module, the acquisition module acquires image information in real time and sends the image information to the matching module, if the matching module identifies that one welding coordinate is (152, -48) by using a template diagram, at the moment, a value of a y axis of the welding coordinate acquired by the judgment module is-48, the y axis of the welding coordinate falls into a first operation area, the judgment module sends information of the first operation area and (152, -48) to the receiving module, the receiving module finds out a welding robot in the first operation area (700, -40) and sends (152, -48) to the welding robot according to the value of the y axis of-48, at the moment, the welding robot updates the welding coordinate by (152 x/S, -48), wherein x/S is the operation speed of the transmission crawler, and S is time. The belt 152+ x/S S falls into the working range of the welding robot in the first operation area, and the welding robot can weld (152 + x/S S-48) the welding coordinate point to realize new and accurate welding. Of course, the welding robot in the application is a conventional welding robot, is provided with a mechanical arm, and can swing in a certain area to meet the welding work in a subarea.
According to the method provided by the invention, under the combined action of the coordinate construction, the template drawing identification and the movement speed of the transmission crawler, the welding point of the target to be welded can be tracked in real time, and even if the target to be welded is randomly placed on the transmission crawler, the welding point of the target to be welded can be found out, so that accurate welding is realized.
Meanwhile, in order to ensure the unicity of the image information, repeated judgment cannot occur, each acquisition module only acquires the image information in the operation area where the acquisition module is located, and the repeated area is avoided when a plurality of acquisition modules acquire each other, so that the same control instruction is sent for multiple times to influence the normal welding of the welding mechanical arm.
Preferably, the system also comprises a rechecking module,
the rechecking module is used for acquiring image information on a rechecking area when the target to be welded enters the rechecking area, recalling the matching module to reacquire the welding coordinate of the target to be welded, performing matching judgment on the y-axis coordinate in the welding coordinate in the rechecking area and the first acquired y-axis coordinate to acquire the change value of the y-axis coordinate, recalling the judging module if the change value of the y-axis coordinate is greater than the change threshold, determining the operation area where the welding coordinate is located again through the y-axis coordinate where the welding coordinate is located, and recreating the control command.
In the operation process, the transportation crawler may be affected by an external force and shakes, so that the to-be-welded object sends a slight displacement, and a part of the to-be-welded object located at the edge of the operation area shakes to the edge of another operation area, so that the welding mechanical arm in the operation area corresponding to the control instruction obtained for the first time cannot be welded to the to-be-welded object. Therefore, the invention is also provided with a rechecking module, wherein when the target to be welded moves to the rechecking area, the image information in the rechecking area is obtained through the rechecking module, the matching module is called again to match the target to be welded in the rechecking area to obtain the welding coordinate of the target, and then the y-axis coordinate of the welding coordinate is compared with the y-axis coordinate of the previous welding coordinate to obtain the change value of the y-axis. However, if the y-axis coordinate change value is greater than the change threshold, it indicates that the object to be welded moves a large distance, and the operation area may change, so when the y-axis coordinate change value is greater than the change threshold, the determination module needs to be called again, the y-axis coordinate obtained in the rechecking module is used to perform another operation area determination, and the operation area where the object to be welded is located after the displacement is determined again, where the change threshold is 0.01 to 0.5 in one embodiment.
Preferably, the review module further comprises a review region determination module;
the rechecking area determining module comprises a reaction time determining submodule and a distance determining submodule;
the reaction time determining submodule is used for acquiring coordinate calculation time, welding mechanical arm awakening time and updating time, wherein the coordinate calculation time is the time for the matching module to acquire a welding coordinate, the welding mechanical arm awakening time is the time for the receiving module to analyze the control instruction, and the updating time is the time for the judging module to generate the control instruction;
the distance determining submodule is used for acquiring the sum of coordinate calculation time, welding mechanical arm awakening time, updating time and a time threshold, and calculating a first reaction time distance according to the movement speed of the transmission crawler;
the distance determining submodule is also used for acquiring coordinate calculation time, welding mechanical arm awakening time and updating time, and calculating a second reaction time distance according to the movement speed of the transmission crawler;
determining the initial position of the reinspection area according to the distance between the welding mechanical arm and the first reaction time, determining the end position of the reinspection area according to the distance between the welding mechanical arm and the second reaction time, and constructing to obtain the reinspection area.
In order to reduce the possibility that the transportation crawler is influenced by external force and sends jitter to influence the position of the object to be welded after the rechecking module rechecks. The distance from the rechecking area to the welding mechanical arm needs to be shortened as much as possible, so that the accuracy of welding coordinates is ensured.
Therefore, in the present invention, the location of the review area is determined by the review area determination module, and in the present invention, the review area is not a fixed area but is determined according to the network condition, the processing time of the matching module and the processing time of the determination module.
The rechecking area determining module firstly records the time length of the data processing, namely the coordinate calculation time length and the updating time length, when the matching module and the judging module carry out the data processing, and also records the time length of the receiving module for analyzing the control instruction. Since the above steps may need to be repeated in the review module, the above steps need to be left in time. The moving speed of the transportation crawler is unchanged, so that the position of the reinspection area can be determined by the moving speed of the transportation crawler when the time is left until needed. The rechecking area is an area which is confirmed together by the first reaction time distance and the second reaction time distance in the reverse movement direction of the transmission crawler; in addition, a time threshold is added in the calculation of the first reaction time distance, so that the fault tolerance is increased, and the problem that a receiving module cannot analyze the control instruction at the first time due to the influence of network delay is avoided. The following is explained with an example: for example, in one embodiment the coordinate calculation time period, the welding robot arm wake-up time period, the update time period, the time threshold are 0.5S, 0.1S, and 0.5S, respectively, while the movement speed of the transport tracks is 1m/S. The x coordinate of the welding mechanical arm is (700, y), the first reaction time distance is (0.5 + 0.1) =1.2m, and the first reaction time distance is (0.5 + 0.1) = 1=0.7m. The starting position of the review area is determined by the distance of the welding robot from the first reaction time, and then the starting position of the review area is (700-1.2, y), i.e. (698.8, y). The starting position of the rechecking region is (700-0.7, y), namely (699.3, y), so that the rechecking region is (698.8, y) - (699.3, y). Preferably, the matching module comprises a preparation module; the preparation module comprises a template making sub-module, an identification feature extraction sub-module and a storage sub-module; the template making submodule is used for making a plurality of template drawings, wherein each template drawing corresponds to a different integer angle; the identification feature extraction submodule comprises a gradient quantization unit and a lifting unit; the gradient quantization unit is used for performing first-layer pyramid direction gradient quantization and second-layer pyramid direction gradient quantization on the template pictures to respectively obtain identification features corresponding to the template pictures; the lifting unit is used for acquiring the identification features by taking the current angle as a list and storing the identification features; the storage submodule is used for storing all the identification features in the different angle table lists. Because the target to be welded is randomly placed on the conveying crawler belt, the placing angle of the target can influence the identification effect. There may be a certain placing angle, which cannot be identified, resulting in the target to be welded being marked as other target to be welded. For this reason, including the preparation module in the identification module of this application, be provided with template preparation submodule in the preparation module N template picture has been made in the template preparation submodule, N can be according to identification module's configuration is decided, works as identification module's configuration is high, and the value of N can correspondingly improve, increases the quantity of template picture, can not influence the recognition rate simultaneously, and after confirming N's quantity, uses 360/N, obtains the angle that each template picture corresponds. By increasing the number of the template drawings, the template drawings can cover the placing angle of each target object to be welded as much as possible, and the problem that the identification module identifies the target objects to be welded due to the placing angle is avoided.
The gradient quantization unit can perform gradient quantization on the template graph, so that the identification features in the template graph can be better acquired. In one embodiment, the first-layer pyramid direction gradient quantization and the second-layer pyramid direction gradient quantization are performed as follows:
calculating the gradient of a gradient image through sobel, wherein the template image is a three-channel image in one embodiment, and extracting a single-channel gradient amplitude maximum image matrix through a gradient square-sum non-maximum suppression algorithm in the X and Y directions;
obtaining an angle image matrix from the gradient image matrixes in the X direction and the Y direction;
quantizing the range of the angle image matrix from 0 to 360 degrees into an integer of 1 to 15, then continuously quantizing 7 remainder taking numbers in 8 directions, taking pixels which are larger than a threshold value in the amplitude image matrix, then taking a quantized image matrix corresponding to 3 x 3 in the pixel field to form a histogram, taking more than 5 same directions in the pixel field, assigning values to the directions, and carrying out shift coding on the index of 00000001 to 10000000;
wherein the gradient amplitude maximum image matrix calculation formula is as follows:
Figure GDA0003955096090000161
Figure GDA0003955096090000162
x represents the position of the object to be measured,
Figure GDA0003955096090000163
for x-position gradient values, { R, G, B } for R channel, G channel, B channel, ori for gradient direction.
After the gradient quantization is performed, the identification features in the template graph are obviously different from other pixel points in terms of pixel point values, and therefore, the process for identifying the features in the application is as follows: traversing the image matrix with the maximum gradient amplitude value, finding out pixel points with the maximum gradient amplitude value in each field in the image matrix with the maximum gradient amplitude value, and if finding out the pixel points with the maximum gradient amplitude value in the field, setting the gradient amplitude values of the pixel value points except the pixel points with the maximum gradient amplitude value in the field to be zero;
judging whether the gradient amplitude of the pixel point with the maximum gradient amplitude in all the fields is larger than a gradient amplitude threshold value or not, and if so, marking the pixel point as an identification feature;
acquiring the quantity of all the identification features, judging whether the quantity of all the identification features is larger than a quantity threshold value, if so, adding all the identification features into a feature set and storing the feature set in the storage submodule; if not, judging whether the identification features have at least another identification feature in the range of the distance threshold, if so, rejecting the identification features and the identification features in the distance threshold, and if not, storing the identification features in the storage submodule.
The identifying features in the storing submodules store groupings of identifying features in each group at an angle. In the process of identification by the identification module, the identification features in the storage sub-module are called, and the identification features of each group are matched with the objects to be welded in the image information.
Preferably, the matching module further comprises an information processing module;
the information processing module comprises a processing submodule and a score calculating submodule;
the processing submodule comprises: the system comprises a pyramid linear memory data container processing unit, a translation unit and a similarity response matrix diagram acquisition unit;
the pyramid linear memory data container processing unit is used for carrying out gradient extraction and quantification on the image information, creating two layers of pyramid linear memory data containers and traversing the image information line through the data of the two layers of pyramids;
the pyramid traversal process specifically comprises the following steps: acquiring the magnitude of a target gradient diffusion translation value, and acquiring a pyramid linear memory data container;
the translation unit is used for performing bit translation within a range of 4 × 4 on the image information quantization gradient, and performing pixel-by-pixel OR operation on the obtained 16 graphs to obtain a diffusion gradient matrix graph of the image information after gradient diffusion;
the similarity response matrix image acquisition unit is used for converting the diffusion gradient matrix image of the image information into gradient matrix images in the first four directions and the last four directions through AND operation, and finding out the maximum similarity of each gradient matrix image angle and each angle in a lookup table through a preset lookup table, wherein the lookup table is a pre-calculated table of various combinations in 8 directions;
obtaining a similarity response matrix diagram in a certain direction, obtaining 8 similarity response matrix diagrams due to 8 directions of the diffusion gradient matrix diagram, converting all the similarity response matrix diagrams into a 16-order or 64-order format, and storing the format in the continuous linear memory data container in a linearized manner;
the lookup manner of the lookup table with 8 similarity directions is as follows:
Figure GDA0003955096090000181
where i is the index of the quantization direction, L is the set of one direction in the diffusion gradient matrix map, 1 is the direction of the diffusion gradient matrix map, T is the index of the quantization direction i [L]Is a similarity response matrix diagram;
the score calculation submodule comprises a calling unit and a similarity calculation unit;
the calling unit is used for calling the two layers of pyramids in the storage submodule, finding access entries of linear memories of the two layers of pyramids according to the 8 similarity response matrix graphs, and acquiring identification features corresponding to all angles;
acquiring data of identification features corresponding to two layers of pyramids of a first template drawing from the storage submodule, simultaneously acquiring similarity response matrix drawings in 8 directions in a linear memory data container of a second layer of pyramids of image information, finding an access entry of the linear memory data container in the corresponding direction according to the information of the identification features of the second layer of pyramids, calculating the similarity of corresponding positions through iteration circulation of position range information of the template drawings obtained through calculation and MIPP accumulation, obtaining a matching similarity matrix of a single identification feature of the second layer of pyramids of the first template drawing, traversing all the identification features of the second layer of pyramids of the first template drawing, obtaining a matching similarity matrix dataset of the second layer of pyramids, converting the data in the matching similarity matrix dataset into a 100-system, obtaining a score of each matching similarity matrix, and removing the matching similarity matrix with the score smaller than a threshold value.
The similarity calculation unit is used for matching the image information and the identification features, obtaining the matching scores of the image information and the identification features of each angle, judging whether the highest matching score is larger than a threshold value, if so, judging that the content of the image information is a target to be welded, and obtaining a template corresponding to the highest matching score as a coordinate selection template.
Selecting the position of the identification feature of the second pyramid layer left by the calling unit from the identification feature position corresponding to the first pyramid layer, selecting a linear similarity matrix diagram in a certain direction of 8 directions of a first layer pyramid target detection diagram, finding a similarity response matrix diagram in 8 directions in a linear memory data container of the first layer pyramid, finding an access entry of the linear memory data container in the corresponding direction according to the information of the identification feature of the first layer pyramid, calculating the similarity of the corresponding position through iterative cycle of template diagram position range information obtained through calculation and MIPP accumulation to obtain a matching similarity matrix of a single identification feature of the first layer pyramid of the first template diagram, traversing all the identification features of the first layer pyramid of the first template diagram to obtain a first layer pyramid matching similarity matrix data set, converting data in the matching similarity matrix data set into a 100-system to obtain the score of each matching similarity matrix, and recording the scores of the first layer pyramid of the template diagram and the identification feature of the second layer as a matching score;
the matching score is calculated in the following manner:
Figure GDA0003955096090000191
wherein Q is the input image information, T represents the template drawing, c is the position of the template drawing in the input image information, and P represents the position centered on cThe domain, r is the offset position, o is the recognition feature, and ε (Q, T, c) is the match score.
In an embodiment in the present application, a score of similarity between the image information and the template map, i.e., a matching score, is calculated by acquiring the recognition features in the template map. And when the matching score is larger than the threshold value, the content of the image information is indicated as the object to be welded. And then extracting welding coordinates corresponding to the welding points according to the fixed relation between the template graph and the welding points.
Preferably, the information processing module further comprises a correction module;
the correction module comprises a to-be-welded target extraction submodule, an identification feature association submodule and a rotation translation submodule;
the to-be-welded target extraction submodule is used for extracting the frame of the to-be-welded target from the image information in a sub-pixel point mode to obtain a target frame and sending the target frame to the identification feature association submodule;
in one embodiment, the implementation process of obtaining the target frame is as follows:
the method comprises the steps of collecting an edge point set of an object to be welded in image information through a Canny operator, carrying out binary quadratic polynomial fitting on the edge point set, solving a binary quadratic polynomial through a facet model to obtain a Hessian matrix, solving a characteristic value and a characteristic vector of the edge point set for the Hessian matrix, deriving the characteristic value through a Taylor expansion formula to obtain sub-pixels of the edge point set, and extracting through a target frame of the object to be welded. Detecting an edge point set of a target to be welded through a Canny operator, fitting a binary quadratic polynomial, solving coefficients through a facet model to obtain a Hessian matrix, solving a characteristic value and a characteristic vector, wherein the characteristic vector is a direction vector of a second identification point, deriving through Taylor expansion, combining with a point direction vector, solving a corresponding sub-pixel point, and circularly solving a corresponding sub-pixel point set and a direction vector point set to store at a corresponding position of a kdtree data structure body. By constructing a KDTree algorithm, the storage sequence of the sub-pixel point sets and the direction vector point sets in the kdTree data structure is associated with leaf nodes of the KDTree tree, namely the storage sequence of the original sub-pixels and the original direction vectors in the data structure is changed. In addition, the sub-pixel points at the edge are extracted in the application, and the target to be welded is extracted. The edge points of the sub-pixels can improve the definition of the edge, the extracted targets to be welded can be more accurate, and the edge points or the feature points on the frame of the target can be more accurate.
The identification feature association submodule is used for combining the identification features on the target frame and other identification features into a first identification point according to a proportion and finding out a second identification point corresponding to the first identification point on the template picture according to the first identification point;
acquiring distances between all first identification points and second identification points corresponding to the first identification points, judging whether the distances between all first identification feature points and the second identification points corresponding to the first identification feature points are larger than a distance threshold, if so, acquiring the number of first identification features meeting the distance threshold, judging whether the number meets a first number threshold, and if so, sending a correction instruction to the rotation and translation sub-module;
one embodiment of the invention is characterized in that the ratio of 3: and 7, acquiring the identification features on the target frame and combining the rest identification features into a first identification point according to the proportion, wherein the rest identification features are identification features on a non-target edge, the proportion can reduce the time for picking out the identification features of the target frame and the rest identification features, and meanwhile, a large amount of the rest identification features can ensure the accuracy of the template pose correction.
The manner of acquiring the first identification point and the second identification point is as follows: and acquiring a tangent of the first recognition point, making a perpendicular line to the tangent of the first recognition point, connecting the perpendicular line with the second recognition point, and calculating the length of the perpendicular line, wherein the length of the perpendicular line is the distance between the first recognition point and the second recognition point.
And then, acquiring the distances between the first identification points and the second identification points which are in one-to-one correspondence after association, and judging whether the distances are greater than a distance threshold value. Only when the distance is greater than the distance threshold value, the situation that the position and the posture of the target to be welded are greatly different from the position and the posture of the template graph can be shown, and the position and the posture of the template graph need to be corrected. And after all the first identification points and the second identification points meeting the distance threshold are obtained, counting the number of the first identification points and the second identification points, and correcting the template when whether the number meets the number threshold or not. Because the first recognition point and the second recognition point are correlated with each other in the pose, but the first recognition point is likely to be a rotation edge point on the frame of the target, and the associated second recognition point edge point is only close in the pose, and the rotation edge point cannot be completely coincided with the edge point. Therefore, when the corrected pose of the template graph is close to the target to be welded, the first identification characteristic point and the second identification characteristic point of the class still meet the requirement of the distance threshold. If the distance threshold is only adopted to judge whether the template pose needs to be modified, the pose of the template graph can be corrected all the time, and the running resources of the system are wasted.
And the rotation and translation sub-module receives the correction instruction, substitutes the first identification point and the second identification point into a change matrix, and corrects the pose of the template graph to obtain a corrected template graph.
Wherein the change matrix comprises a translation matrix and a rotation matrix;
the coordinates of the first recognition point and the coordinates of the second recognition point are first substituted into the following formula (1):
Figure GDA0003955096090000211
wherein R is a rotation matrix, and R is a rotation matrix,
Figure GDA0003955096090000212
to translate the matrix, q i And p i Respectively the coordinates, n, of the associated first and second identifying feature points i Is a feature vector, i is a natural integer greater than 1, and epsilon is a variation matrix;
then, the minimum deflection angle R between the first identification point and the second identification point is obtained, the minimum deflection angle R is substituted into the following formula (2), and the minimum value of the rotation matrix R is obtained through calculation, wherein the formula (2) is as follows:
Figure GDA0003955096090000221
substituting the minimum value of the rotation matrix R back into equation (1) to obtain equation (3): (ii) a
Figure GDA0003955096090000222
Wherein c is i ＝p i ×n i ；
And (3) solving the partial derivatives of the formula (3), converting the partial derivatives into linear equations and solving the angle r of the minimum deflection, the minimum horizontal offset x and the minimum vertical offset y by the following process:
the partial derivative formula four is as follows:
Figure GDA0003955096090000223
Figure GDA0003955096090000224
Figure GDA0003955096090000225
the conversion into a linear equation to find the angle of minimum deflection r, the minimum horizontal offset x and the minimum vertical offset y is as follows:
Figure GDA0003955096090000226
preferably, the information processing module further comprises a verification sub-module;
the verification submodule is used for acquiring the number of times of modifying the pose of the current template drawing and the distances between all the first identification points and the second identification points, and judging the number of the second identification points meeting the distance threshold value on the modified template drawing and the number of times of modifying the template drawing;
and when the number of the second identification points meeting the distance threshold is less than the second number threshold and the template graph correction frequency is less than the frequency threshold, updating the current change matrix by using the last change matrix, continuously sending a correction instruction, and continuously correcting the template graph until the number of the second identification points meeting the distance threshold is greater than the second number threshold or the template graph correction frequency is equal to the frequency threshold.
The invention also sets the frequency threshold value for ending the frequency of the template graph correction, because the pose of the template graph can only be infinitely close to the pose of the target to be welded in the algorithm, after the limited correction, the pose of the template graph is already very close to the pose of the target image, and the template graph can be regarded as being overlapped with the target to be welded. The points extracted on the template map can also be very close to the corresponding points of the target image. And the memory resource is wasted by correcting the pose of the template graph. Since the template image and the target image have a linear relationship when the method is applied to the 2d image, the theoretical algorithm of the method is AX = B, and X is a linear relationship, namely a change matrix. After one pose correction is carried out, the epsilon = epsilon, the current epsilon is multiplied by the epsilon of the last time, and the change matrix is updated, so that the accuracy of the change matrix for the pose correction of the template graph is improved.
An industrial robot welding control method based on template recognition is applied to the industrial robot welding control system based on template recognition and is characterized in that the method comprises the following steps of (1) carrying out template recognition; the method comprises the following steps:
step S1: constructing a virtual cross coordinate system by taking the symmetrical center of the transmission crawler as the original point of the cross coordinate system, the transportation direction of the transmission crawler as the x-axis direction and the width direction of the transmission crawler as the y-axis direction;
dividing a plurality of running areas along the y-axis direction at intervals;
step S2: continuously shooting to obtain image information in a corresponding operation area, matching the image information by using a plurality of template pictures, judging whether a target to be welded exists in the image information, and if so, obtaining welding coordinates corresponding to a welding point of the target to be welded;
and step S3: determining an operation area where the welding coordinate is located through a y-axis coordinate where the welding coordinate is located, finding out a welding mechanical arm in the operation area according to operation area information, and sending the welding coordinate to the welding mechanical arm in the operation area;
and step S4: and updating the welding coordinates according to the movement speed of the transmission crawler, and when the welding coordinates enter the working range of the welding mechanical arm, carrying out welding operation on the target to be welded by the welding mechanical arm according to the welding coordinates.
Preferably, before executing step S4, the following steps are also executed:
when the object to be welded enters the rechecking area, acquiring image information on the rechecking area, acquiring the welding coordinate of the object to be welded again, performing matching judgment on the y-axis coordinate in the welding coordinate when the object to be welded is located in the rechecking area and the first acquired y-axis coordinate, acquiring the change value of the y-axis coordinate, and determining the operation area where the welding coordinate is located again through the y-axis coordinate where the welding coordinate is located if the change value of the y-axis coordinate is greater than the threshold value.
Preferably, the determination process of the review area is as follows:
acquiring coordinate calculation time, welding mechanical arm awakening time and updating time;
acquiring the sum of coordinate calculation time, welding mechanical arm awakening time, updating time and a time threshold, and calculating a first reaction time distance according to the movement speed of the transmission crawler;
acquiring coordinate calculation time, welding mechanical arm awakening time and updating time, and calculating a second reaction time distance according to the movement speed of the transmission crawler;
determining the initial position of the reinspection area according to the distance between the welding mechanical arm and the first reaction time, determining the end position of the reinspection area according to the distance between the welding mechanical arm and the second reaction time, and constructing to obtain the reinspection area. Preferably, the following steps are also required to be executed before step S2: step A1: making a plurality of template drawings, wherein each template drawing corresponds to a different integer angle; step A2: performing a first-level pyramid direction gradient quantization on a plurality of template graphsThe second layer of pyramid direction gradient quantization is carried out, identification features corresponding to a plurality of template pictures are obtained respectively, the identification features are obtained by taking the current angle as a list and are stored; step A3: all the identifying features in the different angle table columns are stored. The specific steps of selecting the coordinate selection template map in the step S2 are as follows: step B1: gradient extraction and quantification are carried out on the image information, a two-layer pyramid linear memory data container is created, and the image information line traverses the data of the two layers of pyramids; and step B2: performing bit-by-bit translation on the image information quantization gradient within the range of 4 x 4, and performing or operation on the obtained 16 images pixel by pixel to obtain a diffusion gradient matrix image of the image information after gradient diffusion; the similarity response matrix image acquisition unit is used for converting the diffusion gradient matrix image of the image information into gradient matrix images in the first four directions and the last four directions through AND operation, and finding out the maximum similarity of each gradient matrix image angle and each angle in a lookup table through a preset lookup table, wherein the lookup table is a pre-calculated table of various combinations in 8 directions; obtaining a similarity response matrix diagram in a certain direction, obtaining 8 similarity response matrix diagrams because the diffusion gradient matrix diagram has 8 directions, converting all the similarity response matrix diagrams into a format of 16 orders or 64 orders, and storing the format in the continuous linear memory data container in a linearized manner; and step B3: finding an access entry of a linear memory of the two layers of pyramids by using the two layers of pyramids in the storage submodule according to the 8 similarity response matrix diagrams, and acquiring identification characteristics corresponding to each angle; and step B4: matching the image information and the identification features, acquiring the matching scores of the image information and the identification features of each angle, judging whether the highest matching score is greater than a threshold value, if so, judging that the content of the image information is a target part, and acquiring a template corresponding to the highest matching score as a coordinate selection template; the matching score is calculated in the following way:
Figure GDA0003955096090000251
wherein Q is the input image information, T represents the template drawing, c is the position of the template drawing in the input image information, and P representsThe area centered at c, r is the offset position, and o is the identification feature. Preferably, the specific steps of performing rotation and displacement on the coordinate selection template map in step S2 are as follows: step C1: extracting the frame of the target part from the image information in a sub-pixel point mode to obtain a target frame; and C2: combining the recognition features on the target frame and the rest recognition features into a first recognition point according to a proportion, and finding out a second recognition point corresponding to the first recognition point on the template picture according to the first recognition point; acquiring the distances between all the first identification points and the corresponding second identification points, judging whether the distances between all the first identification feature points and the corresponding second identification points are larger than a distance threshold, if so, acquiring the number of the first identification features meeting the distance threshold, and judging whether the number meets a first number threshold, if so, correcting the template graph; step C3: and substituting the first identification point and the second identification point into a change matrix, and correcting the pose of the template graph to obtain a corrected template graph, wherein the calculation process of the change matrix is as follows: substituting the coordinates of the first recognition point and the coordinates of the second recognition point into the following formula (1):
Figure GDA0003955096090000261
wherein R is a rotation matrix, and R is a rotation matrix,
Figure GDA0003955096090000262
to translate the matrix, q i And p i Respectively the coordinates, n, of the associated first and second identifying feature points i I is a natural integer greater than 1;
then, the minimum deflection angle R between the first identification point and the second identification point is obtained, the minimum deflection angle R is substituted into the following formula (2), and the minimum value of the rotation matrix R is obtained through calculation, wherein the formula (2) is as follows:
Figure GDA0003955096090000263
the minimum value of the rotation matrix R is substituted back into equation (1), resulting in equation (3): (ii) a
Figure GDA0003955096090000264
Wherein c is i ＝p i ×n i ；
And (3) solving the partial derivatives of the formula (3), converting the partial derivatives into linear equations and solving the angle r of the minimum deflection, the minimum horizontal offset x and the minimum vertical offset y by the following process:
the partial derivative formula four is as follows:
Figure GDA0003955096090000265
Figure GDA0003955096090000266
Figure GDA0003955096090000271
the transformation into the linear equation to find the angle of minimum deflection r, the minimum horizontal offset x and the minimum vertical offset y is as follows:
Figure GDA0003955096090000272
10. the industrial robot welding control method based on template recognition according to claim 9, wherein the specific steps in step S2 until the coincidence degree of the coordinate selection template map and the target part satisfies the threshold value are as follows:
and C4: acquiring the number of times of modifying the pose of the current template graph and the distances between all the first identification points and the second identification points, and judging the number of the second identification points meeting the distance threshold value on the modified template graph and the number of times of modifying the template graph;
and when the number of the second identification points meeting the distance threshold is less than the second number threshold and the template graph correction times are less than the time threshold, updating the current change matrix by using the last change matrix, and continuously correcting the template graph until the number of the second identification points meeting the distance threshold is greater than the second number threshold or the template graph correction times are equal to the time threshold.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an illustrative embodiment," "an example," "a specific example," or "some examples" or the like mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims (8)

1. An industrial robot welding control system based on template recognition is characterized by comprising a transmission crawler, a plurality of acquisition modules, an information processing module and an industrial robot, wherein the industrial robot comprises a receiving module and a plurality of welding mechanical arms;
the information processing module comprises a coordinate dividing module, a matching module and a judging module;
the coordinate division module is used for constructing a virtual cross coordinate system by taking the symmetrical center of the transmission crawler as the original point of the cross coordinate system, the transportation direction of the transmission crawler as the x-axis direction and the width direction of the transmission crawler as the y-axis direction;
dividing a plurality of running areas along the y-axis direction at intervals;
the acquisition module and the welding mechanical arm are arranged in each operation area;
the acquisition module is used for continuously shooting and acquiring image information in a corresponding operation area and sending the image information to the matching modules;
the matching module is used for matching the image information by using a plurality of template pictures, judging whether the image information has a target to be welded, if so, acquiring a welding coordinate corresponding to a welding point of the target to be welded, and sending the welding coordinate to the coordinate dividing module;
the judging module is used for receiving the welding coordinates, determining an operation area where the welding coordinates are located according to y-axis coordinates where the welding coordinates are located, generating control instructions for the welding coordinates and the operation area where the welding coordinates are located, and sending the control instructions to the receiving module;
the receiving module is used for analyzing the control instruction to obtain corresponding operation area information, finding out a welding mechanical arm in the operation area according to the operation area information, and sending the welding coordinate to the welding mechanical arm in the operation area;
the welding mechanical arm updates welding coordinates according to the movement speed of the transmission crawler, and when the welding coordinates enter the working range of the welding mechanical arm, the welding mechanical arm performs welding operation on a target to be welded according to the welding coordinates;
also comprises a re-examination module which is used for carrying out re-examination,
the rechecking module is used for acquiring image information on a rechecking area when the target to be welded enters the rechecking area, recalling the matching module to reacquire the welding coordinate of the target to be welded, performing matching judgment on the y-axis coordinate in the welding coordinate in the rechecking area and the first acquired y-axis coordinate to acquire the change value of the y-axis coordinate, recalling the judging module if the change value of the y-axis coordinate is greater than the change threshold, determining the operation area where the welding coordinate is located again through the y-axis coordinate where the welding coordinate is located, and recreating the control command.
2. The industrial robot welding control system based on template identification as claimed in claim 1, wherein the review module further comprises a review area determination module;
the rechecking area determining module comprises a reaction time determining submodule and a distance determining submodule;
the reaction time determining submodule is used for acquiring coordinate calculation time, welding mechanical arm awakening time and updating time, wherein the coordinate calculation time is the time for acquiring a welding coordinate by the matching module, the welding mechanical arm awakening time is the time for analyzing the control instruction by the receiving module, and the updating time is the time for generating the control instruction by the judging module;
the distance determining submodule is used for acquiring the sum of coordinate calculation time, welding mechanical arm awakening time, updating time and a time threshold, and calculating a first reaction time distance according to the movement speed of the transmission crawler;
the distance determining submodule is also used for acquiring coordinate calculation time, welding mechanical arm awakening time and updating time, and calculating a second reaction time distance according to the movement speed of the transmission crawler;
and determining the initial position of the reinspection area according to the distance between the welding mechanical arm and the first reaction time, determining the end position of the reinspection area according to the distance between the welding mechanical arm and the second reaction time, and constructing to obtain the reinspection area.
3. An industrial robot welding control system based on template recognition according to claim 1, characterized in that the matching module comprises a preparation module;
the preparation module comprises a template making submodule, an identification feature extraction submodule and a storage submodule;
the template making submodule is used for making a plurality of template drawings, wherein each template drawing corresponds to a different integer angle;
the identification feature extraction submodule comprises a gradient quantization unit and a lifting unit;
the gradient quantization unit is used for performing first-layer pyramid direction gradient quantization and second-layer pyramid direction gradient quantization on the plurality of template pictures to respectively obtain identification features corresponding to the plurality of template pictures;
the lifting unit is used for acquiring the identification features by taking the current angle as a list and storing the identification features;
the storage submodule is used for storing all the identification features in the different angle table lists.
4. The industrial robot welding control system based on template recognition according to claim 3, wherein the information processing module further comprises a revision module;
the correction module comprises a to-be-welded target extraction sub-module, an identification feature association sub-module and a rotation translation sub-module;
the to-be-welded target extraction submodule is used for extracting the frame of the to-be-welded target from the image information in a sub-pixel point mode to obtain a target frame and sending the target frame to the identification feature association submodule;
the identification feature association submodule is used for combining the identification features on the target frame and other identification features into a first identification point according to a proportion and finding out a second identification point corresponding to the first identification point on the template picture according to the first identification point;
acquiring distances between all first identification points and second identification points corresponding to the first identification points, judging whether the distances between all first identification feature points and the second identification points corresponding to the first identification feature points are larger than a distance threshold, if so, acquiring the number of first identification features meeting the distance threshold, judging whether the number meets a first number threshold, and if so, sending a correction instruction to the rotation and translation sub-module;
and the rotation and translation sub-module receives the correction instruction, substitutes the first identification point and the second identification point into a change matrix, and corrects the pose of the template graph to obtain a corrected template graph.
5. An industrial robot welding control method based on template recognition is applied to the industrial robot welding control system based on the template recognition in any one of claims 1 to 4, and is characterized in that the method comprises the following steps of; the method comprises the following steps:
step S1: constructing a virtual cross coordinate system by taking the symmetrical center of the transmission crawler as the original point of the cross coordinate system, the transportation direction of the transmission crawler as the x-axis direction and the width direction of the transmission crawler as the y-axis direction;
dividing a plurality of running areas along the y-axis direction at intervals;
step S2: continuously shooting to obtain image information in a corresponding operation area, matching the image information by using a plurality of template pictures, judging whether a target to be welded exists in the image information, and if so, obtaining welding coordinates corresponding to a welding point of the target to be welded;
and step S3: determining an operation area where the welding coordinate is located through a y-axis coordinate where the welding coordinate is located, finding out a welding mechanical arm in the operation area according to operation area information, and sending the welding coordinate to the welding mechanical arm in the operation area;
and step S4: updating welding coordinates according to the movement speed of the transmission crawler, and when the welding coordinates enter the working range of the welding mechanical arm, performing welding operation on a target to be welded by the welding mechanical arm according to the welding coordinates;
before step S4 is executed, the following steps are also executed:
when the object to be welded enters a rechecking area, acquiring image information on the rechecking area, acquiring the welding coordinate of the object to be welded again, performing matching judgment on the y-axis coordinate in the welding coordinate when the object to be welded is located in the rechecking area and the first acquired y-axis coordinate, acquiring the change value of the y-axis coordinate, and determining the operation area where the welding coordinate is located again through the y-axis coordinate where the welding coordinate is located if the change value of the y-axis coordinate is greater than a threshold value.
6. The industrial robot welding control method based on template recognition according to claim 5, characterized in that the determination process of the recheck area is as follows:
acquiring coordinate calculation time, welding mechanical arm awakening time and updating time;
acquiring the sum of the coordinate calculation time, the welding mechanical arm awakening time, the updating time and a time threshold, and calculating a first reaction time distance according to the movement speed of the transmission crawler;
acquiring coordinate calculation time, welding mechanical arm awakening time and updating time, and calculating a second reaction time distance according to the movement speed of the transmission crawler;
determining the initial position of the reinspection area according to the distance between the welding mechanical arm and the first reaction time, determining the end position of the reinspection area according to the distance between the welding mechanical arm and the second reaction time, and constructing to obtain the reinspection area.
7. The industrial robot welding control method based on template recognition according to claim 6, characterized in that the following steps are executed before the step S2:
step A1: making a plurality of template drawings, wherein each template drawing corresponds to a different integer angle;
step A2: carrying out first-layer pyramid direction gradient quantization and second-layer pyramid direction gradient quantization on the plurality of template pictures, respectively obtaining identification features corresponding to the plurality of template pictures, obtaining the identification features by taking the current angle as a list, and storing the identification features;
step A3: storing all the identification features in the different angle list;
the specific steps of selecting the coordinate selection template map in the step S2 are as follows:
step B1: gradient extraction and quantification are carried out on the image information, a two-layer pyramid linear memory data container is created, and the image information line traverses the data of the two layers of pyramids;
and step B2: performing bit-by-bit translation on the image information quantization gradient within the range of 4 x 4, and performing or operation on the obtained 16 images pixel by pixel to obtain a diffusion gradient matrix image of the image information after gradient diffusion;
converting a diffusion gradient matrix image of image information into gradient matrix images in the first four directions and the last four directions through an and operation, and finding out the maximum similarity of each gradient matrix image angle and each angle in a lookup table through a preset lookup table, wherein the lookup table is a pre-calculated table of various combinations in 8 directions;
acquiring 8 similarity response matrix diagrams, converting all the similarity response matrix diagrams into a format of 16 orders or 64 orders, and storing the formats in the continuous linear memory data containers in a linearized manner;
and step B3: using two layers of pyramids and according to 8 similarity response matrix graphs, finding access entries of linear memories of the two layers of pyramids, and obtaining identification features corresponding to each angle;
and step B4: matching the image information and the identification features, acquiring the matching scores of the image information and the identification features of each angle, judging whether the highest matching score is greater than a threshold value, if so, judging that the content of the image information is a target part, and acquiring a template corresponding to the highest matching score as a coordinate selection template;
the matching score is calculated in the following manner:
Figure FDA0003955096080000061
where Q is the input image information, T represents the template map, c is the position of the template map in the input image information, P represents the area centered on c, r is the offset position, and o is the identification feature.
8. The industrial robot welding control method based on template recognition according to claim 7, wherein the specific steps of performing rotation and displacement on the coordinate selection template map in the step S2 are as follows:
step C1: extracting the frame of the target part from the image information in a sub-pixel point mode to obtain a target frame;
and step C2: combining the identification features on the target frame and the rest identification features into a first identification point according to a ratio, and finding out a second identification point corresponding to the first identification point on the template graph according to the first identification point;
acquiring the distances between all the first identification points and the corresponding second identification points, judging whether the distances between all the first identification feature points and the corresponding second identification points are larger than a distance threshold, if so, acquiring the number of the first identification features meeting the distance threshold, and judging whether the number meets a first number threshold, if so, correcting the template graph;
step C3: and substituting the first identification point and the second identification point into a change matrix, and correcting the pose of the template graph to obtain a corrected template graph, wherein the calculation process of the change matrix is as follows:
substituting the coordinates of the first recognition point and the coordinates of the second recognition point into the following formula (1):
Figure FDA0003955096080000071
wherein R is a rotation matrix, and R is a rotation matrix,
Figure FDA0003955096080000072
to shift the matrix, q i And p i Respectively the coordinates, n, of the associated first and second identifying feature points i Is a feature vector, i is a natural integer greater than 1;
then, the minimum deflection angle R between the first identification point and the second identification point is obtained, the minimum deflection angle R is substituted into the following formula (2), and the minimum value of the rotation matrix R is obtained through calculation, wherein the formula (2) is as follows:
Figure FDA0003955096080000073
the minimum value of the rotation matrix R is substituted back into equation (1), resulting in equation (3): (ii) a
Figure FDA0003955096080000074
Wherein c is i ＝p i ×n i ；
And (4) solving the deviation derivative of the formula (3), converting the deviation derivative into a linear equation and solving the angle r of the minimum deflection, the minimum horizontal offset x and the minimum vertical offset y, wherein the process is as follows:
the partial derivative formula is as follows:
Figure FDA0003955096080000081
Figure FDA0003955096080000082
Figure FDA0003955096080000083
the transformation into the linear equation to find the angle of minimum deflection r, the minimum horizontal offset x and the minimum vertical offset y is as follows:
Figure FDA0003955096080000084
CN202210669674.5A 2022-06-14 2022-06-14 Industrial robot welding control system and method based on template recognition Active CN114986051B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210669674.5A CN114986051B (en) 2022-06-14 2022-06-14 Industrial robot welding control system and method based on template recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210669674.5A CN114986051B (en) 2022-06-14 2022-06-14 Industrial robot welding control system and method based on template recognition

Publications (2)

Publication Number Publication Date
CN114986051A CN114986051A (en) 2022-09-02
CN114986051B true CN114986051B (en) 2023-01-24

Family

ID=83034425

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210669674.5A Active CN114986051B (en) 2022-06-14 2022-06-14 Industrial robot welding control system and method based on template recognition

Country Status (1)

Country Link
CN (1) CN114986051B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117681205B (en) * 2024-01-18 2024-04-26 武汉孚锐利自动化设备有限公司 Sensing and calibrating method for mechanical arm

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101829870A (en) * 2010-06-04 2010-09-15 欧朝旭 Welding fume collecting device and method capable of automatically tracking and positioning welding point
CN105058168B (en) * 2015-08-12 2017-07-28 西安费斯达自动化工程有限公司 The multi-axis machining center combination method of view-based access control model and mobile mechanical arm
CN106424998A (en) * 2016-08-31 2017-02-22 深圳市艾尔摩迪精密科技有限公司 Automatic laser soldering system based on welding spot recognition
CN109986258A (en) * 2017-12-30 2019-07-09 姜玉娟 A kind of automatic welding machinery arm
CN109658460A (en) * 2018-12-11 2019-04-19 北京无线电测量研究所 A kind of mechanical arm tail end camera hand and eye calibrating method and system
CN110757005B (en) * 2019-10-23 2023-04-28 苏州普热斯勒先进成型技术有限公司 Laser cutting equipment and method for plate partition cutting
CN111014975A (en) * 2019-12-04 2020-04-17 安徽赢创激光科技有限公司 Laser cutting device of synchronous cutting of multiple spot
CN213702215U (en) * 2020-10-30 2021-07-16 东莞市旺鑫精密工业有限公司 Online automatic radium carving equipment of cell-phone shell
CN114055438A (en) * 2022-01-17 2022-02-18 湖南视比特机器人有限公司 Visual guide workpiece follow-up sorting system and method
CN114535738B (en) * 2022-02-23 2023-11-07 大连爱智控制系统有限公司 Model and numerical control drilling file-based teaching programming method for automatic soldering machine

Also Published As

Publication number Publication date
CN114986051A (en) 2022-09-02

Similar Documents

Publication Publication Date Title
CN115008093B (en) Multi-welding-point welding robot control system and method based on template identification
CN114792373B (en) Visual identification spraying method and system of industrial robot
CN114986051B (en) Industrial robot welding control system and method based on template recognition
CN110853100A (en) Structured scene vision SLAM method based on improved point-line characteristics
CN112102372A (en) Cross-camera track tracking system for airport ground object
EP3745085A1 (en) Multi-device visual navigation method and system in variable scene
CN111667535B (en) Six-degree-of-freedom pose estimation method for occlusion scene
CN107909643B (en) Mixed scene reconstruction method and device based on model segmentation
CN111360821A (en) Picking control method, device and equipment and computer scale storage medium
CN115213896A (en) Object grabbing method, system and equipment based on mechanical arm and storage medium
CN114494462A (en) Binocular camera ranging method based on Yolov5 and improved tracking algorithm
CN115026823B (en) Industrial robot control method and system based on coordinate welding
CN116052046A (en) Dynamic scene binocular vision SLAM method based on target tracking
CN110533717B (en) Target grabbing method and device based on binocular vision
US20220072712A1 (en) Mix-size depalletizing
CN115601271B (en) Visual information anti-shake method, storage warehouse location state management method and system
CN117011660A (en) Dot line feature SLAM method for fusing depth information in low-texture scene
CN115026822B (en) Industrial robot control system and method based on feature point docking
CN116105721A (en) Loop optimization method, device and equipment for map construction and storage medium
CN114800508B (en) Grabbing control system and method of industrial robot
JP2004028811A (en) Device and method for correcting distance for monitoring system
CN110570473A (en) weight self-adaptive posture estimation method based on point-line fusion
CN116168306A (en) Unmanned aerial vehicle video target tracking method based on region re-search
CN115456870A (en) Multi-image splicing method based on external parameter estimation
CN114800533A (en) Sorting control method and system for industrial robot

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant