CN114454617B

CN114454617B - Code spraying total system

Info

Publication number: CN114454617B
Application number: CN202210059528.0A
Authority: CN
Inventors: 田浩楠; 冀春锟; 刘欢
Original assignee: Hunan Shibite Robot Co Ltd
Current assignee: Hunan Shibite Robot Co Ltd
Priority date: 2022-01-19
Filing date: 2022-01-19
Publication date: 2023-06-20
Anticipated expiration: 2042-01-19
Also published as: CN114454617A

Abstract

The invention relates to a code spraying total system, which is characterized by comprising a correction system, wherein the correction system can correct the initial information of the code spraying which is initially determined by detecting the position information of a code spraying main body to obtain the position of a code spraying workpiece to be sprayed, namely the code spraying correction information, so that error spraying and missing spraying are avoided, and the accuracy of the angle of the code spraying position and the like is ensured. The system can fully consider that the position information of the code spraying main body when reaching the code spraying point position has deviation in the aspects of position, pose, angle and the like, is not ideal in the situation that the code spraying main body is placed under the code spraying point position in a square direction, has angular points which are not the original points and angles which are also not zero, and has the key points that the real-time image of the code spraying main body is acquired through the image acquisition small module, the actual pose of the code spraying main body is identified, the code spraying position is adaptively adjusted, the code spraying position is ensured to be more accurate, and the system is a code spraying total system based on visual guidance and according to the pose of the code spraying main body, the code spraying point position is adaptively adjusted.

Description

Code spraying total system

Technical Field

The invention relates to the field of control, in particular to a code spraying total system based on visual identification.

Background

The main body frame of a construction machine such as a vehicle or a ship is welded by various workpieces, and the workpieces are mainly cut from steel plates. The workpieces cut by the steel plate are various in size and shape, and different in working procedures and purposes, so that the workpieces cut by the steel plate are marked, and the convenience for the subsequent recognition and tracking of the workpieces is particularly important.

At present, the reworking industry marks cut workpieces by code spraying (ink spraying, laser and the like). The code spraying mark mainly comprises two kinds of manual code spraying and mechanical code spraying. The manual code spraying is to mark the workpieces one by one after the workpieces are confirmed according to the trepanning chart, the mode is low in efficiency and easy to make mistakes, waste of manpower and materials is caused, and the code spraying, the identification and the tracking of the workpieces are inconvenient when the number of the workpieces is large and the types of the workpieces are complex; more importantly, the temperature of the cut steel plate is high, and safety accidents are easy to occur. The mechanical code spraying is generally to position the code spraying position (code spraying point position) and then to control the code spraying device to start the code spraying process from the appointed code spraying point position, so that the efficiency is faster than that of a manual mode, but the deviation of the code spraying position and even the missing spraying phenomenon can be caused due to the fact that the position angle deviation exists when the steel plate reaches the code spraying point position.

Therefore, how to adaptively adjust the code spraying position according to the real-time position, angle and pose deviation of the steel plate is an important technical problem to be solved in the mechanical code spraying of the workpiece.

Disclosure of Invention

Aiming at the technical problems, the invention provides a code spraying total system, which comprises: truss, code spraying system, control system and correction system;

The correction system includes:

the first acquisition device is used for acquiring code spraying initial information of the workpiece to be code sprayed on the code spraying main body;

the second acquisition device is used for acquiring the position information of the code spraying main body;

the correction device is connected with the first acquisition device and the second acquisition device and is used for determining code spraying correction information of the workpiece to be sprayed on the code spraying main body according to the code spraying initial information of the workpiece to be sprayed on the code spraying main body and the position information of the code spraying main body;

the control system is connected with the truss, the code spraying system and the correction system and is used for controlling the truss to drive the code spraying system to move to the code spraying point position designated by the code spraying correction information according to the code spraying correction information, and starting a code spraying procedure.

Further, the first acquisition device includes:

the first determining module is used for determining self code spraying information of the workpiece to be code sprayed;

the second determining module is used for determining the placement position information of the workpiece to be sprayed on the code spraying main body;

the third determining module is connected with the first determining module and the second determining module and is used for determining the initial code spraying information of the workpiece to be sprayed on the code spraying main body according to the self code spraying information of the workpiece to be sprayed and the placement position information of the workpiece to be sprayed on the code spraying main body.

Further, the first determining module includes:

the center-of-gravity calculating small module is used for calculating the center-of-gravity positions of the whole workpiece and the local workpiece of the workpiece to be sprayed with the code;

the solid area judging small module is connected with the gravity center calculating small module and is used for judging whether the gravity center position is the solid area of the workpiece to be sprayed with the code;

the deleting small module is connected with the solid area judging small module and is used for deleting the gravity center position of the solid area;

the code spraying information determining small module is connected with the solid area judging small module and is used for determining the code spraying information of the workpiece to be sprayed by taking the gravity center position of the solid area as a key point.

Further, the center of gravity calculation small module includes:

the first calculating unit is used for calculating the gravity center position of the whole workpiece of the workpiece to be sprayed with the code;

the dividing unit is used for dividing the whole workpiece of the workpiece to be sprayed into a plurality of local workpieces by taking the gravity center position of the whole workpiece as a dividing point and determining a dividing line according to the dividing point;

and the second calculating unit is used for calculating the gravity center position of the local workpiece of the workpiece to be sprayed with the code respectively.

Further, the code spraying information determining small module comprises:

the code-sprayable region determining unit is used for determining code-sprayable regions around the key points by taking the gravity center position of the solid region as the key points;

The area calculating unit is connected with the code-sprayable area determining unit and is used for calculating the area of the code-sprayable area;

the self-code spraying information determining unit is connected with the area calculating unit and is used for comparing the sizes of the code spraying areas and selecting key points corresponding to the maximum code spraying areas as self-code spraying information.

Further, the second acquisition device includes:

the sample collection small module is used for collecting sample images and labeling corner data on the sample images so as to construct a training sample set;

the model construction small module is used for constructing a neural network model for learning corner data;

the model training small module is used for inputting the training sample set into the neural network model to obtain a trained neural network model;

the image acquisition small module is used for acquiring real-time images of the workpiece to be detected;

the first side angle data output small module is used for inputting the real-time image into the trained neural network model and outputting first side angle data of the workpiece to be detected;

the image intercepting small module is used for intercepting an area image comprising the first side angle data in the real-time image according to the first side angle data;

the second corner data output small module is used for detecting the area image and obtaining second corner data of the workpiece to be detected;

And the correction small module is used for correcting and fusing the second corner data with the first corner data to obtain corrected corner data as a corner positioning output result.

And the determining small module is used for determining the position information of the code spraying main body according to the corrected corner data.

Further, the control system is also connected with the code spraying path planning device and is used for controlling the truss to drive the code spraying system to finish the code spraying process according to the code spraying path of the code spraying path planning device.

Further, the code spraying path planning device comprises:

the code spraying information acquisition small module is used for acquiring code spraying information of the code spraying main body;

selecting a small module as a first workpiece, selecting a workpiece to be sprayed, which is closest to the corner point, as a first workpiece to be sprayed by taking the corner point of the code spraying main body as a starting point, wherein i=1, and starting a code spraying process;

the workpiece deleting small module deletes the ith workpiece to be sprayed in the code spraying sequence when finishing the code spraying process of the ith workpiece to be sprayed;

the next workpiece selecting small module is used for selecting the workpiece to be sprayed, which is closest to the ith workpiece to be sprayed, as the next workpiece to be sprayed, and starting to spray the code, wherein the i=i+1;

the code spraying completion judging small module judges whether i is equal to N; if not, continuing code spraying; if yes, the code spraying is finished

The code spraying total system provided by the invention is characterized by comprising a correction system, wherein the correction system can correct the initial information of the code spraying which is initially determined by detecting the position information of the code spraying main body to obtain the position of the code spraying workpiece to be sprayed with the code, namely the code spraying correction information, so that error spraying and missing spraying are avoided, and the accuracy of the angle of the code spraying position and the like is ensured. The system can fully consider that the position information of the code spraying main body when reaching the code spraying point position has deviation in the aspects of position, pose, angle and the like, is not ideal in the situation that the code spraying main body is placed under the code spraying point position in a square direction, has angular points which are not the original points and angles which are also not zero, and has the key points that the real-time image of the code spraying main body is acquired through the image acquisition small module, the actual pose of the code spraying main body is identified, the code spraying position is adaptively adjusted, the code spraying position is ensured to be more accurate, and the system is a code spraying total system based on visual guidance and according to the pose of the code spraying main body, the code spraying point position is adaptively adjusted.

Drawings

FIG. 1 is a diagram of a part distribution on a steel plate illustrating an example of the present invention;

FIG. 2 is a flow chart of one embodiment of a method for correcting workpiece code spraying according to the present invention;

FIG. 3 is a schematic diagram of the code spraying of a workpiece to be code sprayed on an offset-free steel plate according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of the code spraying of the workpiece to be code sprayed on the offset steel plate according to the present invention;

FIG. 5 is a bitmap of code spraying points of a workpiece to be code sprayed on a steel plate according to an example of the invention;

FIG. 6 is a flowchart illustrating an embodiment of a step S1 of a method for correcting a workpiece code pattern according to the present invention;

FIG. 7 is a schematic diagram of self-code spraying information of a workpiece to be code sprayed according to the present invention;

FIG. 8 is a flowchart illustrating a step S11 of a method for correcting a workpiece code pattern, i.e., a method for identifying a workpiece code pattern according to an embodiment of the present invention;

FIG. 9 is a flowchart of one embodiment of step S111 of the workpiece code spraying recognition method of the present invention;

FIG. 10 is a flowchart illustrating an embodiment of step S114 of the workpiece code pattern recognition method according to the present invention;

FIG. 11 is a flowchart of step S2 of the workpiece code-spraying correction method of the present invention, namely, one embodiment of a corner positioning method based on a neural network and a detection algorithm;

FIG. 12 is a block diagram of one embodiment of a neural network model of the present invention;

FIGS. 13-14 are schematic diagrams of rectangular box corner points in the corner locating method of the present invention;

FIGS. 15-16 are schematic diagrams of marked edge points of rectangular frames in the corner positioning method of the present invention;

FIG. 17 is a flowchart of a corner locating method according to an embodiment of the present invention, step S23;

FIG. 18 is a flowchart of a corner locating method according to an embodiment of the present invention, step S25;

FIG. 19 is a flowchart of a corner locating method according to an embodiment of the present invention, step S27;

FIG. 20 is a flowchart of a corner locating method according to an embodiment of the present invention, step S272;

FIG. 21 is a flow chart of another embodiment of the corner locating method of the present invention;

FIG. 22 is a flow chart of an embodiment of a method for code spray path control according to the present invention;

FIG. 23 is a schematic diagram of a code spraying path according to the control method of the code spraying path of the present invention;

FIG. 24 is a block diagram illustrating one embodiment of a workpiece code correction system of the present invention;

FIG. 25 is a block diagram illustrating one embodiment of a first acquisition device of a workpiece code-spraying correction system of the present invention;

FIG. 26 is a block diagram illustrating an embodiment of a workpiece code-spraying recognition device of the present invention;

FIG. 27 is a block diagram of one embodiment of a corner locating device of the present invention;

FIG. 28 is a block diagram illustrating an embodiment of a code spray path planning apparatus according to the present invention;

fig. 29 is a schematic structural diagram of an embodiment of the code spraying system according to the present invention.

Detailed Description

In order to explain the technical idea of the present invention in detail, as shown in fig. 1, taking a code spraying main body as a steel plate, taking a part scattered on the steel plate as an example of a workpiece to be code sprayed, in the prior art, before or after the steel plate is cut into each part, code spraying marks are usually sprayed according to the types of the parts to distinguish different parts, and then the parts are assembled into engineering machinery such as ships, vehicles, etc. It should be noted that the above is only exemplary, and is not limited in particular. The code spraying main body is selected from, but not limited to, main materials such as steel plates, alloy pieces and the like, and single or multiple types of workpieces to be sprayed are scattered and distributed on the code spraying main body. For illustrating other application ranges of the invention, the code spraying main body is also optionally, but not exclusively, a plastic plate, parts for assembling the children toy are scattered on the code spraying main body, and name numbers of different parts are distinguished through code spraying marks, so that the children toy is convenient to assemble into a complete children toy subsequently; the code spraying main body is also optionally but not limited to cloth, and the upper surface of the code spraying main body is scattered with zero materials for assembling the whole clothes, local jigsaw for assembling the whole picture, and the like. Those skilled in the art will appreciate that examples for identifying loose parts by code spraying to assemble the parts into a single piece are within the scope of the present invention.

As shown in fig. 2-4, the present invention provides a method for correcting a workpiece spray code, comprising:

s1: the method comprises the steps of obtaining initial code spraying information of a workpiece to be sprayed on a code spraying main body (as illustrated in fig. 3, a rectangular outer frame in the drawing is the code spraying main body, such as a steel plate, the shape and the size of the rectangular outer frame in the drawing can be set arbitrarily, pentagonal patterns in the drawing are the workpiece to be sprayed, the shape and the size of the pentagonal patterns in the drawing can be set arbitrarily, other workpieces to be sprayed are not shown here, a dotted line frame in the drawing is a code spraying area, a solid round dot is a code spraying center, the initial code spraying information is the code spraying initial information to be obtained in our example, the initial code spraying information is indicated in the following example by a gravity center position, the initial code spraying information can be the code spraying center point/starting point/end point, etc., and the code spraying area can be a circle, a rectangular frame, etc. made by the gravity center position. Specifically, the code spraying initial information is, but not limited to, code spraying coordinates (which can be exemplified by code spraying center/start point/end point) of the workpiece to be code sprayed on the code spraying main body, code spraying angles, and the like, that is, when the corner point of the steel plate is assumed to be located at the origin (0, 0) and the angle is also 0 (the ideal case that the steel plate is placed without any deviation), the position (the initially determined position and angle when the steel plate is assumed to be free of deviation) of each workpiece to be code sprayed. Exemplary, code-spraying initiation information of the nth workpiece, optionally but not limited to, is recorded in coordinates (x _n ，y _n ) Angle r _n Represented as (x) _n ，y _n ，r _n )。

S2: position information of the code spraying main body (as shown in fig. 4, the displacement and the angle of the steel plate have deviation) is obtained. Specifically, the position information is optionally, but not limited to, a code spraying main body expressed by angular point coordinates, offset angles and the like. Taking the code spraying main body as a steel plate as an example, namely when the steel plate is conveyed to a code spraying point position through a movable tray, the angular point of the steel plate is not at an original point (0, 0), the angle is not 0, but a displacement deviation and an angle deviation exist, and the method is optional but not limited to the method which uses angular point coordinates (x _c ，y _c ) Angle r _c Represented as (x) _c ，y _c ，r _c ). More preferably, the angular point coordinates (x _c ，y _c ) And coordinates of the edge points (M _c ，N _c ) Calculate its offset angle r _c 。

S3: according to the code spraying of the workpiece to be sprayed on the code spraying main bodyInitial information (the code spraying information determined in an ideal state without deviation of the steel plate) and position information of a code spraying main body (the deviation of the real-time information, position, angle and the like of the steel plate) are used for determining code spraying correction information (corrected code spraying position angle and the like) of a workpiece to be sprayed on the code spraying main body. Namely, according to the position information (optionally but not limited to displacement deviation and angle deviation) of the code spraying main body, the initially determined code spraying information is corrected to obtain corrected code spraying information (code spraying correction information), namely, the position and angle of the current code spraying point position actually needing code spraying are expressed as (x) _n1 ，y _n1 ，r _n1 ) (as shown in fig. 4).

In this embodiment, a specific embodiment of the workpiece code spraying correction method is provided, which fully considers that the position information of the code spraying main body (optionally but not limited to a steel plate) when reaching the code spraying point position may have deviation (as shown in fig. 4) in the aspects of position, pose, angle and the like, the ideal situation that the non-square side is placed under the code spraying point position is that the angular point is not the origin and the angle is also non-zero, and the initially determined code spraying position is corrected by detecting the position information of the code spraying main body, so that the position where the code spraying of the workpiece to be sprayed is actually performed is obtained, and the accuracy of the code spraying position angle and the like is ensured. It should be noted that in the correction process, the number of the to-be-sprayed workpieces on the code spraying main body is actually required to be corrected, and the code spraying initial information of the N to-be-sprayed workpieces is sequentially corrected, so as to determine the code spraying correction information of each to-be-sprayed workpiece, and obtain the code spraying point bitmap as shown in fig. 5 (compared with fig. 1, the added point is the code spraying correction information of each to-be-sprayed workpiece, namely, the actual code spraying point position).

Specifically, as shown in fig. 6, step S1 may optionally include, but is not limited to:

s11: determining the self-code spraying information of the workpiece to be sprayed (as shown in fig. 7), namely, irrespective of the position of the workpiece to be sprayed in the code spraying main body, determining the position of the workpiece to be sprayed where the code to be sprayed should be marked according to the shape, the size and the like of the workpiece to be sprayed, optionally but not limited to the position of the workpiece to be sprayed in coordinates (x _na ，y _na ) And (3) representing.

S12: determining that a workpiece to be sprayed is on a spraying main bodyThe placement position information (shown in fig. 3, optionally but not limited to the coordinates of key points such as a certain corner point and a central point of the workpiece to be sprayed), specifically, the placement position of the workpiece to be sprayed on the spraying body, optionally but not limited to the coordinates (x) of feature points of the workpiece to be sprayed (such as the upper, lower, left and right corners, the central point, and the like of the workpiece, and the figure illustrates a corner point of the workpiece) _nb ，y _nb ) And (3) representing. More specifically, the placement location is optionally but not limited to resolving, by a data resolution module, extracted material data (optionally but not limited to information data including a number, name, etc. of each workpiece) and line data (optionally but not limited to graphic data including a shape, size, line segment composition, etc. of each workpiece) from a production trepanning graph (e.g., dxf, json, etc.); then storing the material data in a file form as file data, combining and converting the line data into frame data cut by the steel plate and workpiece data in a picture form as picture data; and finally, the file data and the picture data are in one-to-one correspondence to determine the position, shape, size and the like of each workpiece on the code spraying main body (steel plate).

S13: based on the self-code information of the workpiece to be code-coded (optionally but not exclusively using coordinates (x _na ，y _na ) Representation) and placement position information (x) of the workpiece to be code-sprayed on the code-spraying main body _nb ，y _nb ) Determining initial information (x) of the code spraying of the workpiece to be code sprayed on the code spraying main body _n ，y _n ). Angle r _n An angle may be set by those skilled in the art according to the appearance or the like, and generally, for convenience of viewing, is generally set parallel to the workpiece, i.e., the angle is 0.

In this embodiment, step S1 is provided, which is a specific embodiment of how to obtain the initial information of the code spraying, and by obtaining the self code spraying information of the workpiece to be code sprayed and the placement position information of the workpiece to be code sprayed on the code spraying main body, the initial information of the code spraying is determined, that is, the coordinate conversion from the small coordinate system of the workpiece to be code sprayed to the large coordinate system of the code spraying main body is completed, and the code spraying device has a simple structure, accurate calculation and can ensure that the code spraying position is accurate.

Specifically, in order to accurately identify and find the self code spraying information (as shown in fig. 7, the code spraying area (dashed line frame)/the code spraying start point/the end point/the center point (solid circle)) of the workpiece to be code sprayed, step S11 is another inventive concept of the present invention, and further provides a method for identifying the code spraying of the workpiece. In the specific step S11, the self-code spraying information of the workpiece to be code sprayed is optionally but not limited to determined according to the gravity center position of the workpiece to be code sprayed. More specifically, as shown in fig. 8, S11 may optionally include, but is not limited to:

S111: and calculating the gravity center positions of the whole workpiece and the local workpiece of the workpiece to be sprayed with the code. Specifically, the center of gravity is optionally, but not limited to, calculated according to a binary image of the workpiece to be coded. Specifically, the profile data of each workpiece is extracted from file data and picture data obtained from the production trepanning map, and then the binary map of each workpiece is obtained by filling the workpiece profile.

S112: judging whether the gravity center position is a solid area of the workpiece to be sprayed with the code; specifically, taking a local workpiece including 4 quadrants as an example, it is necessary to determine whether or not 5 barycentric positions (barycentric positions of 1 entire workpiece+barycentric positions of 4 local workpieces) are solid regions, respectively. More specifically, it is optional but not limited to determine whether each calculated center of gravity position (coordinate) is a solid area of the workpiece to be coded according to the file data and the picture data obtained from the production trepanning chart.

S113: if not, deleting the gravity center position of the non-solid area; specifically, if the center of gravity position is not a solid area, the center of gravity position is represented not to be on the workpiece to be sprayed (for example, the center of gravity position of the circular ring is in the center circle), and the position of the hole cannot be necessarily used as a preselected position of the code spraying area and needs to be deleted. Specifically, the storage sequence is optionally but not limited to be removed from the calculated plurality of center of gravity positions.

S114: if so, determining self code spraying information of the workpiece to be code sprayed by taking the gravity center position of the solid area as a key point. Specifically, the self code spraying information is optional but not limited to any information which characterizes the position of the code to be sprayed by taking the gravity center position of the solid area as the starting point/end point/center point of the area to be sprayed. More specifically, since in step S111, a plurality of barycentric positions are calculated, in step S114, there may be a plurality of barycentric positions in the solid area, and at this time, one of them is selected as the code spraying information as will be understood by those skilled in the art, and the following improved preferred scheme is provided.

In the embodiment, the identification method of the workpiece code spraying provided by the invention is provided, the gravity center positions of the whole workpiece and the local workpiece of the workpiece to be code sprayed are calculated, and the gravity center position of the solid area is selected as a key point.

Specifically, as shown in fig. 9, step S111 may optionally include, but is not limited to:

s1111: calculating the gravity center position of the whole workpiece of the workpiece to be sprayed with the code;

s1112: and taking the gravity center position of the whole workpiece as a dividing point, determining a dividing line according to the dividing point, and dividing the whole workpiece of the workpiece to be sprayed into a plurality of local workpieces. Specifically, but not limited to, the center of gravity position is used as a dividing point, and the center of gravity position (center of gravity coordinate point) is used as a horizontal axis parallel to the horizontal plane and a vertical axis perpendicular to the horizontal plane to divide the workpiece to be sprayed into four quadrants of local workpiece. It should be noted that the specific form, position, etc. of the dividing line may be optionally but not limited to any setting according to the actual shape, size, precision requirement of the code spraying position, etc. of the workpiece to be sprayed. Selecting only one dividing line, and dividing the workpiece to be sprayed into two quadrants of local workpieces; or selecting more dividing lines to divide the workpiece to be sprayed into more quadrants of local workpieces. More specifically, when the workpiece to be sprayed is in a symmetrical structure, the workpiece to be sprayed is optionally, but not limited to, an excessive center position, and the symmetrical axis of the workpiece to be sprayed is used as a dividing line to divide the workpiece to be sprayed into a plurality of local workpieces.

S1113: respectively calculating the gravity center positions of local workpieces of the workpieces to be sprayed; specifically, taking a local workpiece including 4 quadrants as an example, 5 center-of-gravity positions (center-of-gravity positions of 1 entire workpiece+center-of-gravity positions of 4 local workpieces) are obtained up to this point. More specifically, the center of gravity position is optionally, but not limited to, the same calculated by a binary image of the workpiece to be sprayed.

In this embodiment, step S111 is provided, which is a specific embodiment of how to calculate the center of gravity positions of the whole workpiece and the partial workpiece of the workpiece to be sprayed, and the workpiece to be sprayed is divided into a plurality of partial workpieces by selecting a dividing line through the center of gravity position of the whole workpiece as a dividing point, and then the center of gravity position of the partial workpiece is determined, so that the center of gravity position in a solid area can be selected from a plurality of center of gravity positions to be used as the code spraying information, the situation that the whole center of gravity position of some workpieces is not in the solid area (belongs to a hollow part) of the workpiece and cannot be used as a code sprayable area can be further avoided, the phenomena of missing spraying, misspraying and the like can be effectively avoided, and powerful guarantee is provided for the following processes of recognition, assembly and the like of the workpiece.

More specifically, as shown in fig. 10, step S114, optionally but not limited to, includes:

S1141: the barycenter position of the solid area is used as a key point, and a code-sprayable area (pre-code-spraying area) around the key point is determined; specifically, S1141, the position of the center of gravity of the solid area is optionally but not limited to a code spraying center point, r is a radius circle or a side length rectangular frame, and the solid area around the position of the center of gravity is determined to be a code spraying area. It should be noted that, the key point is selected as the code spraying center point, the surrounding solid area is selected as the code spraying area, and those skilled in the art can understand that the key point is selected as the starting point (foot drop point) of the code spraying area, and the code spraying area is calculated by the solid area at the left lower part (upper left, lower right, upper right) of the key point, which can be used as an alternative of the present invention.

More specifically, step S114 may optionally include, but is not limited to: s1142: and calculating the area of the code sprayable area. Specifically, the center of gravity of each solid area is taken as a key point, and the area of the code-sprayable area around the center of gravity is calculated, namely, the area of the solid area (the non-hollow area of the code-sprayable area) around the center of gravity is calculated and taken as the pre-code-spraying area.

S1143: comparing the sizes of the code sprayable areas, and selecting the key point corresponding to the maximum code sprayable area as self code spraying information. Namely, the maximum code-spraying area (maximum solid area) is selected as the area to be sprayed, and the corresponding key point is the center point/starting point/end point of the code spraying. It is noted that the code spraying information determined herein is the code spraying information of the workpiece to be sprayed, and is irrelevant to the position of the workpiece on the code spraying main body. More specifically, but not limited to, the center of gravity is selected as the center of the code spraying area, and a rectangular frame with r as the radius and a as the side length is selected as the code spraying area.

In this embodiment, step S114 is provided, which is a specific embodiment of selecting the best self-code spraying information by taking the center of gravity position of the solid area as a key point, and can select the area with the largest area of the code sprayable area and the corresponding key point thereof as the code spraying information, so that on one hand, the condition of incomplete code spraying (the partial code spraying is not shown due to insufficient code spraying position) can be avoided to the greatest extent; on the other hand, the code printing font and the code printing content can be expanded to the greatest extent, so that the work piece to be code printed can be identified more clearly and more completely by the staff. It should be noted that, the area is only an exemplary example of the best self-spraying code information, and those skilled in the art may also select, but not limited to, the most obvious position or the like as the evaluation criterion of the best self-spraying code information, so as to make the best selection for the gravity center positions of the solid areas.

Specifically, in order to obtain the position information of the code spraying main body (optionally, but not limited to, the position information is represented by, but not limited to, the coordinates of the corner points, the inclination angle, etc. representing the position of the code spraying main body), step S2 is another inventive concept of the present invention, and a corner positioning method based on a neural network and a detection algorithm is further provided for positioning the positions of the corner points and the corner points of the steel plate, the inclination angle of the steel plate, etc. (as shown in fig. 4, the code spraying of the steel plate is taken as an explanation here, the coordinates of the corner points of the steel plate are not the origin, the coordinates of the corner points are (X _c ，Y _c ) The coordinates of the edge points are (M _c ，N _c )). It should be noted that the corner positioning method based on the neural network and the detection algorithm is only a preferred example of the step S2, and is not limited thereto. Specifically, as shown in fig. 11, step S2 proposes a corner positioning method based on a neural network and a detection algorithm, which optionally but not exclusively includes:

s21: collecting a sample image, and labeling corner data (optionally but not limited to labeling with a rectangular frame, and subsequently returning to the rectangular frame to calculate and determine coordinates of edge points and corner points) on the sample image to construct a training sample set; specifically, a plurality of steel plate images of construction sites are optionally but not exclusively acquired as sample images by using image acquisition devices such as cameras, cameras and the like, and particularly, photographing images including but not limited to side points/corner points are included. More specifically, in order to fully restore the field environment and the actual situation, so as to improve the accuracy of the subsequent model training, the sample image can optionally but not only cover the conditions of illumination and dust as much as possible, so as to obtain sample images under various complex conditions, and enable the steel plate and the workpiece to have more real texture details, thus the neural network model obtained by the subsequent training also has better generalization.

S22: constructing a neural network model for learning corner data; specifically, a high-precision modified version yolov4 deep learning model (as shown in fig. 12) is optionally but not limited to be used. In the Prediction region, YOLOv4 outputs graphs with resolutions of 76×76/256, 38×38/512, and 19×19/1024, respectively, in response to the change in the number of feature graphs. In the present invention, small input resolutions are optionally but not limited to being used to speed up network training, yielding graphs at 52 x 52, 26 x 26 and 13 x 13 scale. More specifically, the invention can be used for optionally but not exclusively adjusting the last layer of activation function Mish, replacing the last layer of activation function Mish with a Relu activation function, and manufacturing a rapid model more suitable for on-site deployment under the condition of ensuring the accuracy.

S23: inputting the training sample set into a neural network model, and optimizing parameters to obtain a trained neural network model; specifically, a training sample set is input, and a loss function is continuously iterated and reduced to optimize parameters, so that a neural network model with better performance is obtained.

S24: collecting a real-time image of a main body to be detected; specifically, the method is optional but not limited to collecting the current real-time image of the steel plate (taking code spraying as an example, namely a code spraying main body) through image collecting equipment such as a camera, a camera and the like.

S25: inputting the real-time image into a trained neural network model, and outputting first edge data of a main body to be detected (a code spraying main body such as a steel plate); specifically, the neural network model, optionally but not limited to, returns the output data of the same type in a manner of marking the sample image according to step S21, and characterizes the corner data of the real-time image. Specifically, the method is optional but not limited to returning a rectangular frame representing the corner point and a rectangular frame representing the side point, and takes the central point of each rectangular frame as a first corner point coordinate and a first side point coordinate.

S26: according to the first side angle data, intercepting an area image comprising the first side angle data in the real-time image; specifically, a first corner point coordinate and a first side point coordinate are obtained according to first side angle data (optionally, but not limited to, a rectangular frame position and a size represented by a central point of a rectangular frame and a frame height or a diagonal coordinate of the rectangular frame), clipping processing with an identification point as a center is respectively performed, and a clipping region map with a fixed size is obtained as a region image of a real-time image.

S27: detecting the area image to obtain second corner data; specifically, the second corner data is obtained by inputting the region image obtained by cutting into an edge detection algorithm model optionally but not exclusively. More specifically, the edge detection algorithm model is optionally, but not limited to, a canny operator.

S28: and correcting and fusing the second corner data with the first corner data to obtain corrected corner data as a final corner positioning output result. In particular, but not exclusively, byDetecting area imageThe second corner data obtained, correct the fusion passNeural network modelAnd the obtained first corner data and the obtained second corner data are matched and fused with each other to obtain corrected corner data.

In this embodiment, a preferred embodiment of the step S2 of the present invention is given, which fuses the advantages of the neural network model and the conventional edge detection algorithm, first, the first edge data of the real-time image is obtained through the neural network model, then, the partial image (corner image) in the real-time image is intercepted according to the first edge data, so as to obtain the second edge data of the real-time image through the edge detection algorithm, and the two are corrected and fused, so as to obtain corrected edge data, that is, edge points, corner points, deflection angles, etc., the position information of the code spraying main body is fed back, the edge points and the corner points are calculated more accurately, and compared with the single neural network algorithm or the edge detection algorithm, the stability and the accuracy are higher, and the environmental interference is weaker. It should be noted that the method is merely a preferred example of step S2, and not limited thereto, and those skilled in the art will understand that any other method that can determine the position information of the code spraying main body (steel plate) may be used, such as a single neural network model method, a single edge detection algorithm, etc.

More specifically, in step S21, when labeling the corner data of the sample image, the corner points of the sample image are optionally, but not limited to, labeled with rectangular boxes as illustrated in fig. 13-14 (specifically, the solid line is a steel plate; the dotted line is a grid on which the steel plate is placed; and the dotted line is a rectangular box as labeled); the edge points of the sample image are labeled with rectangular boxes as illustrated in fig. 15-16. More specifically, when labeling the sample image with corner data using a rectangular box, the corner data may optionally be represented, but not limited to, by parameters that characterize the rectangular box position and the rectangular box size dimension. Illustratively, the position and size parameters are optionally, but not limited to, represented by the center point position and box height of a rectangular box; or in two pairs of angular coordinates of a rectangular box, etc.

More specifically, when the edge point data of the sample image is marked, the position and the size of the rectangular frame need to be paid special attention to, because the two parameters jointly determine the duty ratio of the target object (corner point or edge point) in the rectangular frame, the method has a crucial effect on training the neural network model and optimizing the parameters, and can improve the accuracy of detecting the edge angle data of the real-time image by using the trained neural network model.

For example, among the rectangular frames noted at the corner points, the rectangular frame selected in fig. 14 is superior to the rectangular frame selected in fig. 13. Specifically, as shown in fig. 13, a rectangular frame with a size of 100x100 pixels is selected as a left image, a pixel point is located at the center of a frame body, and a large area around a corner point is marked, which means that nearly 1/3 area in the frame comes from a grid of the background; the right graph marks the whole steel plate, and the large-area frame selection mode is influenced by the arrangement mode and the exposed area of the steel plate in the whole graph, and the arrangement mode and the exposed area of the steel plate can lead to mismatching/missed matching with a certain probability. The labeling mode of fig. 13 finds the area around the target point first, and then performs secondary refinement matching in the subsequent operation. This two-step calculation method brings more calculation amount, while the result of the second matching depends on the accuracy of the first coarse positioning, and if the target point is not recognized in the coarse positioning stage, the subsequent steps are blocked. In fig. 14, in the labeling of the rectangular frame of the left image, most of the rectangular frame is the main body of the steel plate, the rest part of the frame body needs to ensure that two right-angle edges forming the corner points are clearly contained, so that the deep learning network can distinguish the corner points of the steel plate from the corner points of the background grid without mismatching, and meanwhile, the rectangular frame can accurately find the corner points no matter what placement angle and exposure area the steel plate presents in the shooting image. The labeling mode of the right image is used for coping with the conditions of clear texture in the image and bright light source, but a plurality of 'suspected corner points' exist, the corner points in a plurality of backgrounds can be filtered by a smaller rectangular frame, and the small frame has the advantages that the pixels of the rectangular frame are fewer, the mismatching of each pixel value is equivalent to the larger loss in network calculation, and the deep learning network is promoted to review the matched result with a stricter standard. Therefore, the present invention is preferably to label corner data with rectangular boxes of the type of fig. 14 to cope with labeling different sample images. Specifically, a rectangular frame shown in the left diagram of fig. 14 is adopted for a sample image with low quality, the rectangular frame comprises corner points and two right-angle sides, the code spraying main body occupies more than 2/3 of the area of the rectangular frame, and when a neural network model is input and first corner data is returned, the middle points of the center points and the corner points of the rectangular frame are used as actual corner point coordinates; for the sample image with high quality, a rectangular frame shown in the right diagram of fig. 14 is adopted, which comprises corner points and two right-angle sides, but the pixel size (rectangular frame size) is smaller, and when the neural network model is input and the first side angle data is returned, the center point of the rectangular frame is taken as the actual corner point coordinate.

In the rectangular frame marked by the edge points, the most main technical difficulty is how to reduce mismatching of the network. Compared with the detection of the corner points, the edge points can have more interference points in the graph, such as all grids, background metal bars, scratches of the steel plate, water stains and the like, and all the edge points can be detected as edge points by mistake. Four labeling modes of fig. 15 were tried, but the trained neural network models all had a degree of mismatching. After improvement, a rectangular frame shown in fig. 16 is adopted as a new marking method of the edge points, the edge of a steel plate is marked by a long rectangular frame, the reserved positions of the left and right sides of the marking frame are kept equal, meanwhile, the marking frame cannot be too long, a certain space is reserved up and down, and the marking frame body and the marking frame of other long edge samples cannot have too large volume difference when the edge of the steel plate is exposed too short in the figure.

More specifically, as shown in fig. 17, step S23, optionally but not limited to, includes: s231: and (5) performing data cleaning on the training sample set. Namely: errors identifiable in the data file are found and corrected, and "dirty samples" that affect subsequent model training are removed or repaired. More specifically, the data cleansing step, optionally but not limited to, includes checking for data consistency, processing invalid and missing values, and the like. S232: and inputting the training sample set after data cleaning into a neural network model, continuously iterating to reduce a loss function output by the model, and then adjusting and optimizing model parameters by back propagation to obtain the trained neural network model. The trained neural network model has better performance.

More specifically, as shown in fig. 18, step S25 may further include, but is not limited to:

s251: inputting the real-time image into a trained neural network model;

s252: judging whether first side angle data are obtained or not;

s253: if yes, outputting first edge data of the workpiece to be detected; this is the case: the currently input real-time image has better quality, and the neural network model can actively identify first edge angle data such as edge points, corner points and the like.

S254: if not, the real-time image is enhanced, a trained neural network model is input, and first side angle data is output. This is the case: the current input real-time image quality is poor, the neural network model cannot automatically identify first edge angle data such as edge points and corner points, enhancement processing is needed to be carried out by using a plurality of synthesis algorithms with preset values according to the illumination characteristics of the current picture, a group of enhanced images are obtained, and then the enhanced images are input into the neural network model to obtain the first edge angle data.

More specifically, in step S254, the step of enhancing the real-time image may be, but is not limited to, an optional step of: brightness enhancement, HSV color space enhancement, up-down/horizontal inversion enhancement and Gaussian Laplace enhancement, effectively enhance calculation parameters, efficiently and pointedly enhance positioning and edge patrol original data, and achieve the purposes of expanding a data set and improving generalization and accuracy of a deep learning network.

In particular, as will be appreciated by those skilled in the art, there are hundreds of enhancement algorithms for processing images to augment the acquired data. The inventor selects several preferable enhancement modes to be combined according to the actual environment of the steel plate and performs enhancement treatment on the real-time image through continuous research and experiment and creative labor.

More specifically, the calculation formula of the brightness enhancement is:

g(x)＝αf(x)+β

where the parameters alpha >0 and beta are called gain and bias parameters, which control the brightness, respectively. f (x) is used as a source image pixel, g (x) is used as an output image pixel, and the enhancement calculation on the brightness of the input image is performed, so that a picture which is brighter or darker than the original picture is obtained.

In HSV color space enhancement, H represents hue, with a range of values (0, 180); s represents saturation, and the value range is (0, 255); v represents brightness, and the range of values is (0, 255). The algorithm firstly converts the real-time image from RGB space to HSV space, then carries out numerical amplification or reduction operation on the three components H, S, V according to experience, and finally converts the three components back to RGB space, thereby achieving the effect of image enhancement.

The up-down/horizontal inversion enhancement method can randomly cut out partial areas in the real-time image, and then splice n cut-out images together, so that one image contains as many concentrated objects as possible, the richness of the sample is increased, and a more complex identification scene is provided for a network. However, in particular, from experience, scaling is not suitable in the clipping process, and the clipping should be performed according to the original image size, so as to avoid sample missing caused by shifting of edge points and corner points from the clipping area of the image due to scaling of the image.

Laplace enhancement calculation formula:

wherein G (x, y) represents a Gaussian function, and delta represents a standard deviation, namely, the Gaussian is firstly used for checking an image for smoothing, so that high-frequency noise in a picture can be reduced, and subsequent Laplace operation is facilitated.

It means that a second partial division operation is performed on it to obtain a LoG function of laplace gaussian. Since laplace is a derivative operator, sharp gray changes in the image are emphasized, and areas of slow gray changes are suppressed, which tend to produce gray edges and discontinuous images in a dark background. And overlapping the Laplace image with the original image to obtain an image with the sharpening effect reserved.

More specifically, the step of enhancing the real-time image may optionally but not be limited to employing: contrast enhancement, zoom enhancement, mosaic enhancement, etc. However, because of the specificity of the scene, an inappropriate enhancement processing algorithm may be improved in the visual effect of human eyes, but is not suitable for subsequent algorithm calculation, and eventually, the performance of the neural network model is reduced. Especially under the location of steel sheet limit point, corner patrols limit scene, possess obvious advantage, mainly appear in: the contrast difference brought by the contrast enhancement method can increase the distinction between the steel plate and the background, thereby reducing the generalization of the deep learning network and being not beneficial to identifying the steel plate under complex conditions; the scaling enhancement and Mosaic enhancement methods are applicable in normal cases, but due to the fact that the positions of the edge points and the angular points of the steel plate are uncertain, the steel plate can appear at the edge positions of the image, and a part of marked information is easily lost in common scaling and splicing. When excessive blank unlabeled samples exist in training data of the deep learning network, the number of the training samples is reduced;

More specifically, the corner data, optionally but not limited to including corner data, edge data, optionally but not limited to being represented in corner coordinates (X _c ，Y _c ) And coordinates of the edge points (M _c ，N _c ) And (3) representing. More specifically, in the neural network model, the first corner coordinates are determined by using the center point coordinates of the corner marked rectangular frame, and the first edge coordinates are determined by using the center point coordinates of the edge marked rectangular frame. In the detection algorithm, the second corner point coordinates and the second edge point coordinates are determined by edge line segments. Specifically, as shown in fig. 19, step S27 may optionally include, but is not limited to, corner data determination:

s271: inputting the region image into an edge detection algorithm model to obtain edge line segments in the region image; it should be noted that the edge line may be a right angle edge (edge line that we need to detect) of the steel plate, or may be a scratch on the steel plate, a grid edge where the steel plate is placed, or the like, and needs to be removed later.

S272: according to the edge line segments, pre-selected corner points in the regional image are determined; specifically, optionally but not limited to finding all angular points (i.e. preselected angular points) existing in the region image according to the intersection condition of the right-angle sides; specifically, the corner point is the corner point determined by the edge line segment, and because the edge line segment may be a right-angle side of the steel plate, the determined corner point is an actual corner point of the steel plate and can be returned as second corner data; it is also possible to scratch the steel plate and place the grid edge of the steel plate, the determined corner point is not the actual corner point of the steel plate, and the determined corner point cannot be returned as the second corner data; therefore, in order to fully consider the conditions of scratches and the like of the steel plate, angular points which are not actual angular points of the steel plate are required to be removed from preselected angular points, so that the accuracy of corner positioning is improved; specific:

S273: judging whether a preselected angular point with the position difference with the first angular point in a first set range exists in the preselected angular points or not; specifically, because the preselected corner calculated in step S272 may be a corner of the steel plate (a corner to be found in practice), or may be a corner of a scratch of the steel plate, a corner of a grating, or the like (a corner that may be mistaken for a corner of the steel plate), the position difference between the preselected corner and the first corner needs to be compared, if the positions are close, one of the preselected corners determined by the characterization detection algorithm is a real corner of the steel plate, and fusion correction is needed for the two; if the position is far away, the preselected corner points determined by the characterization detection algorithm do not have the real corner points of the steel plate, but are corner points of other line segments such as scratches and the like, and the preselected corner points are required to be removed without fusion correction of the preselected corner points and the real corner points.

S274: if the position difference exists, returning a preselected angular point of which the position difference is in a first set range to be a second angular point; when the characterization detection algorithm detects the regional image, the actual angular point of the steel plate is found.

S275: if not, the second corner point is not returned. When the characterization detection algorithm detects the regional image, the actual angular point of the steel plate is not found.

More specifically, on the basis of the above-mentioned corner data determination, the step S28 of corner correction specifically includes: if the second corner point is returned, calculating a clustering center point of the first corner point and the second corner point to serve as corrected corner point data; and if the second corner point is not returned, taking the first corner point as corrected corner point data.

In this embodiment, the specific steps of calculating the second corner and correcting and fusing the first corner are given, so that unreasonable outliers (the corner points such as scratches may be mistaken as the steel plate corner) can be removed, the corner coordinates are further optimized, and the accuracy of corner positioning is improved.

More specifically, S27, optionally but not limited to, includes a side point data determination. Specifically, the edge point data determination has similar characteristics to the corner point data determination, and optionally but not limited to the following steps:

s276: according to the edge line segments, determining the point closest to the first edge point in the edge line segments as a preselected edge point;

s277: judging whether the position difference between the preselected edge point and the first edge point is within a second setting range;

s278: if yes, returning a preselected edge point of which the position difference is in a second setting range to be a second edge point;

s279: if not, the second edge point is not returned. When the characterization detection algorithm detects the regional image, the actual edge points of the steel plate are not found.

More specifically, based on the above-mentioned determination of the edge point data, the step S28 of edge point correction specifically includes: and calculating clustering center points of the first side points and the second side points to serve as corrected side point data.

In this embodiment, specific steps of calculating the second edge point and correcting and fusing the first edge point are given as edge point data, so that the coordinates of the edge point are further optimized, and the accuracy of positioning the edge is improved.

More specifically, S28 may further include, but is not limited to, further optimizing the obtained coordinate position by calculating the corner point and the edge point and further by various parameters such as distance, edge slope, and edge length where the point is located.

In this embodiment, a specific embodiment of how to integrate the first edge point data determined by the neural network model and the second edge point data determined by the detection algorithm is provided, so that the advantages of the first edge point data and the second edge point data can be integrated, the positioning accuracy of the corner points and the edge points can be improved, and the positioning accuracy of the code spraying point can be further optimized.

Specifically, as shown in fig. 21, in order to improve accuracy of corner determination, S272: also optionally but not limited to include: s2721: calculating the edge area wrapped by the edge points in the area image; s2722, finding the image center moment corresponding to the maximum edge area; s2723: calculating the distance between the center moment of the image and the edge line segment, fusing the edges with the distance within a certain range, and eliminating discontinuous edges; s2724: and calculating preselected angular points in the regional image according to the edge line segments for eliminating the discontinuous edges.

Specifically, in practical application, the corner positioning method further optionally but not exclusively includes: and converting the obtained corner data from the image coordinate system to a space coordinate system. Specifically, taking the acquisition of real-time images of the workpiece to be detected as a steel plate as an example, optionally but not exclusively inputting the calculated corner data into a conversion algorithm, calling an external parameter matrix of a current camera, calculating the real coordinates of the corner points/side points of the steel plate, and returning the result.

More specifically, in order to further obtain the current pose of the code spraying main body, as shown in fig. 21, the corner positioning method of the present invention further optionally but not limited to the following steps: s29, calculating the inclination angle of the code spraying main body (main body to be detected) according to the corrected corner data. In particular, but not exclusively, the corner data (X _c ，Y _c ) Side point data (M) _c ，N _c ) And the related parameters of the photographing position of the current camera, and measuring and calculating the inclination angle r of the steel plate _c 。

In this embodiment, the inclination angle of the main body to be detected (optionally but not limited to a steel plate) can be further determined according to the corner point data and the side point data, so as to provide effective guarantee for further accurate positioning of the code spraying points and further optimization of the code spraying effect

Specifically, in step S3, the code spraying correction information of the workpiece to be code sprayed on the code spraying main body is determined by using formulas (1) - (3).

x _n1 ＝x _n cos r _c -y _n sin r _c +x _c (1)

y _n1 ＝y _n sin r _c +y _n cosr _c +y _c (2)

r _n1 ＝r _n +r _c (3)

Specifically, as shown in fig. 22, the present invention further provides a method for controlling a code spraying path, including:

t1: and acquiring the code spraying information of the code spraying main body. Specifically, the code spraying information is selected from but limited to code spraying information of each workpiece to be code sprayed (N workpieces to be code sprayed with the same or different types and names) on the code spraying main body, and is selected from but not limited to code spraying information of each workpiece by serial number. More specifically, in order to ensure the accuracy of the code spraying position, avoid missing spraying and misspraying, the conditions of displacement, angle deviation and the like are optionally but not limited to fully considered when the code spraying main body reaches the code spraying point, the code spraying information is determined as the code spraying information after each workpiece to be code sprayed is corrected, namely the code spraying correction information, and the code spraying correction information is optionally but not limited to the information obtained by correcting the code spraying main body by the coordinate angle (x _n1 ，y _n1 ，r _n1 ) The specific manner of acquiring the code spraying information of each workpiece to be sprayed is optional but not limited to determining by the correction method of the code spraying of the workpiece, wherein the steps S1-S3 are optional but not limited to referring to the above, and are not repeated herein.

T2: taking the corner point of the code spraying main body as a starting point, selecting a workpiece to be code sprayed closest to the corner point as a first code spraying workpiece, enabling i to be 1, and starting a code spraying process; specifically, the distance calculation may be selected, but is not limited to, using the Euclidean formula.

T3: deleting the ith workpiece to be sprayed with the code in the sequence to be sprayed when finishing the code spraying process of the ith workpiece to be sprayed with the code; specifically, the subsequent workpiece to be sprayed with the code, which is understood by those skilled in the art, is the workpiece to be sprayed with the code which has not been finished in the code spraying process, that is, the workpiece to be sprayed with the code which has been finished in the code spraying process needs to be deleted at the position, and the subsequent distance comparison is not performed, so that the code spraying device is prevented from returning to the position where the code has been sprayed again, and the code is sprayed again, thereby reducing the time efficiency and increasing the cost on one hand; on the other hand, the code spraying device can be caused to go back and forth between certain two points, so that the code spraying is disordered and cannot be carried out.

T4: selecting the workpiece to be sprayed, which is closest to the ith workpiece to be sprayed, as the next workpiece to be sprayed, and starting to spray the code, wherein the i=i+1;

t5: judging whether i is equal to N; n is the total number of the workpieces to be sprayed; i.e. whether the work piece to be sprayed is completely finished with the code spraying process;

t6: if not, returning to the step T3;

t7: if yes, the code spraying is finished.

In this embodiment, a specific embodiment of the code spraying path planning of the present invention is provided, in which the shortest distance is used as an evaluation criterion, and the whole code spraying process of the code spraying device is arranged according to the optimal planned path (as shown in the line of fig. 23), so that the distance that the truss needs to move is minimized, the code spraying time is shortened, the code spraying efficiency is improved, and the code spraying cost consumption is reduced.

On the other hand, the invention also provides a corresponding system based on the method.

Specifically, as shown in fig. 24, the present invention further provides a correction system for workpiece code spraying, which optionally but not exclusively includes:

the first obtaining device is configured to obtain initial information of the code spraying on the code spraying main body of the workpiece to be sprayed (as illustrated in fig. 3, the rectangular outer frame is the code spraying main body, such as a steel plate, the shape and size of which can be set arbitrarily, the pentagon pattern therein is the workpiece to be sprayed, the shape and size of which can be set arbitrarily, the steel plate is also scattered with other workpieces to be sprayed, which are not shown here, the dotted frame is a code spraying area, the solid dot is a code spraying center, which is the initial information of the code spraying which needs to be obtained in our example, and is indicated by the gravity center position example, which can be the code spraying center point/start point/end point, etc., and the code spraying area can be a circle, a rectangular frame, etc. made by the gravity center position. Specifically, the code spraying initial information is selected, but not limited to, code spraying coordinates, code spraying angles and the like of the workpiece to be code sprayed on the code spraying main body. Exemplary, falseLet the corner point of the steel plate be located at the origin (0, 0), and the angle is also 0 (under the ideal condition that the steel plate is placed without any deviation), the position of each workpiece to be sprayed with codes should be sprayed with codes (the initially determined position and angle when the steel plate is assumed to be free of deviation). Exemplary, code-spraying initiation information of the nth workpiece, optionally but not limited to, is recorded in coordinates (x _n ，y _n ) Angle r _n Represented as (x) _n ，y _n ，r _n ). More specifically, the first obtaining module is optionally but not limited to being connected to the storage unit, and directly obtains the preset code spraying initial information in the trepanning chart.

And the second acquisition device is used for acquiring the position information (shown in fig. 4) of the code spraying main body. Specifically, the position information is optionally, but not limited to, position coordinates, offset angles, etc. of the code spraying main body. Taking the code spraying main body as a steel plate as an example, namely when the steel plate is conveyed to a code spraying point position through a movable tray, the angular point of the steel plate is not located at an original point (0, 0), and the angle is not 0, but has a displacement deviation and an angle deviation. Illustratively, the position information of the code-spraying main body is optionally but not limited to the position information in angular point coordinates (x _c ，y _c ) Angle r _c Represented as (x) _c ，y _c ，r _c ). More specifically, the second acquisition module optionally but not exclusively includes an image acquisition device, acquires a real-time image of the code spraying main body, and determines the current position, pose, etc. of the code spraying main body through image recognition.

And the correction device is connected with the first acquisition device and the second acquisition device and is used for determining code spraying correction information (corrected information) of the workpiece to be sprayed on the code spraying main body according to code spraying initial information (the steel plate has no deviation and the code spraying information is determined in an ideal state) of the workpiece to be sprayed on the code spraying main body and position information (the steel plate has deviation in real time information, position, angle and the like) of the code spraying main body. Namely, according to the position information (optionally but not limited to displacement deviation and angle deviation) of the code spraying main body, the initially determined code spraying information is corrected to obtain corrected code spraying information (code spraying correction information), namely, the position and angle of the current code spraying point position actually needing code spraying are expressed as (x) _n1 ，y _n1 ，r _n1 ) (as shown in fig. 4).

In this embodiment, a specific embodiment of the system for correcting the code spraying of the workpiece is provided, which fully considers the position information (as shown in fig. 4) of the code spraying main body (optionally, but not limited to, steel plates) when reaching the code spraying point position, the deviation may exist in the position, the pose, the angle and other aspects of the code spraying main body (as shown in fig. 4), the ideal situation that the non-square side is placed at the code spraying point position, the angular point is not the origin, the angle is also non-zero, the initially determined code spraying position is corrected by detecting the position information of the code spraying main body, the position of the code spraying main body which is actually supposed to be sprayed is obtained, the accurate error-free of the code spraying position angle and the like is ensured, in the correction process, the initial code spraying information of the N code spraying main bodies which are needed to be sprayed is sequentially corrected according to the number of the code spraying main body, the code spraying correction information of each code spraying workpiece is determined, and the code spraying point bitmap as shown in fig. 5 is obtained (compared with fig. 1, the added point is the code spraying correction information of each code spraying point position of the code spraying workpiece which is more worth noting, the following devices and the physical meanings, and the physical interaction mode is defined in the following systems, and the system is not limited to the specific form, and the system is defined in the following physical mode, and the method is not limited in any way.

Specifically, as shown in fig. 25, the first acquiring device may optionally include, but is not limited to:

a first determining module, configured to determine self-code spraying information (as shown in fig. 7) of the workpiece to be code sprayed, i.e. regardless of the position of the workpiece to be code sprayed in the code spraying main body, the determined position of the workpiece to be code sprayed should be identified according to the shape, the size, etc. of the workpiece to be code sprayed, optionally but not limited to the position of the workpiece to be code sprayed in coordinates (x _na ，y _na ) And (3) representing.

A second determining module, configured to determine the placement position information of the workpiece to be sprayed on the spraying code main body (as shown in fig. 3, optionally but not limited to the placement position information is represented by coordinates of a certain corner point, a central point, and other key points of the workpiece to be sprayed), specifically, the placement position of the workpiece to be sprayed on the spraying code main body, optionally but not limited to the placement position of the workpiece to be sprayed on the spraying code main bodyCoordinates (x) of feature points (e.g., upper, lower, left, right, center points, etc., of the workpiece, the drawing illustrates one corner point of the workpiece) _nb ，y _nb ) And (3) representing. More specifically, the placement location is optionally but not limited to resolving, by a data resolution module, extracted material data (optionally but not limited to information data including a number, name, etc. of each workpiece) and line data (optionally but not limited to graphic data including a shape, size, line segment composition, etc. of each workpiece) from a production trepanning graph (e.g., dxf, json, etc.); then storing the material data in a file form as file data, combining and converting the line data into frame data cut by the steel plate and workpiece data in a picture form as picture data; and finally, the file data and the picture data are in one-to-one correspondence to determine the position, shape, size and the like of each workpiece on the code spraying main body (steel plate).

A third determining module, connected with the first determining module and the second determining module, for determining the self-code spraying information of the workpiece to be sprayed according to the self-code spraying information (optionally but not limited to using coordinates (x _na ，y _na ) Representation) and placement position information (x) of the workpiece to be code-sprayed on the code-spraying main body _nb ，y _nb ) Determining initial information (x) of the code spraying of the workpiece to be code sprayed on the code spraying main body _n ，y _n ). Angle r _n An angle may be set by those skilled in the art according to the appearance or the like, and generally, for convenience of viewing, is generally set parallel to the workpiece, i.e., the angle is 0.

In this embodiment, a specific embodiment of a module formed by the first obtaining device is provided, which determines initial information of code spraying by obtaining self code spraying information of a workpiece to be code sprayed and placement position information of the workpiece to be code sprayed on a code spraying main body, namely, completes coordinate conversion from a small coordinate system of the workpiece to be code sprayed to a large coordinate system of the code spraying main body, has a simple structure, is accurate in calculation, and can ensure that the code spraying position is accurate.

Specifically, in order to accurately identify and find the self-code spraying information (as shown in fig. 7, the code spraying area (dashed line frame)/the code spraying start point/the end point/the center point (solid circle)) of the workpiece to be code sprayed and other information representing the code spraying position, the invention also provides a device for identifying the code spraying of the workpiece (used as a first determining module in a correction system), and the specific device for identifying the code spraying of the workpiece, optionally but not exclusively determines the self-code spraying information of the workpiece to be code sprayed according to the center of gravity position of the workpiece to be code sprayed. More specifically, as shown in fig. 26, the identifying means may optionally include, but is not limited to:

The center-of-gravity calculating small module is used for calculating the center-of-gravity positions of the whole workpiece and the local workpiece of the workpiece to be sprayed with the code. Specifically, the center of gravity is optionally, but not limited to, calculated according to a binary image of the workpiece to be coded. Specifically, the profile data of each workpiece is extracted from file data and picture data obtained from the production trepanning map, and then the binary map of each workpiece is obtained by filling the workpiece profile. More specifically, the center of gravity calculating small module is optionally, but not limited to, a control unit with functions of image recognition, processing, data receiving, processing, transmitting and the like, and can achieve the corresponding functions.

The solid area judging small module is connected with the gravity center calculating small module and is used for judging whether the gravity center position is the solid area of the workpiece to be sprayed with the code; specifically, taking a local workpiece including 4 quadrants as an example, it is necessary to determine whether or not 5 barycentric positions (barycentric positions of 1 entire workpiece+barycentric positions of 4 local workpieces) are solid regions, respectively. More specifically, it is optional but not limited to determine whether each calculated center of gravity position (coordinate) is a solid area of the workpiece to be coded according to the file data and the picture data obtained from the production trepanning chart.

The deleting small module is connected with the solid area judging small module and is used for deleting the gravity center position of the solid area; specifically, if the center of gravity position is not a solid area, the center of gravity position is represented not to be on the workpiece to be sprayed (for example, the center of gravity position of the circular ring is in the center circle), and the position of the hole cannot be necessarily used as a preselected position of the code spraying area and needs to be deleted. Specifically, the storage sequence is optionally but not limited to be removed from the calculated plurality of center of gravity positions. Specifically, the method is optional but not limited to removing the storage sequence from the calculated positions of the centers of gravity, and deleting the number, the name, the code spraying content and the like corresponding to the workpiece to be code sprayed in the storage unit.

The code spraying information determining small module is connected with the solid area judging small module and is used for determining the code spraying information of the workpiece to be sprayed by taking the gravity center position of the solid area as a key point. Specifically, the self code spraying information is optional but not limited to any information which characterizes the position of the code to be sprayed by taking the gravity center position of the solid area as the starting point/end point/center point of the area to be sprayed. More specifically, since in step S111, a plurality of barycentric positions are calculated, in step S114, there may be a plurality of barycentric positions in the solid area, and at this time, one of them is selected as the code spraying information as will be understood by those skilled in the art, and the following improved preferred scheme is provided.

In the embodiment, the recognition device for the code spraying of the workpiece is provided, and the gravity center positions of the whole workpiece and the local workpiece of the workpiece to be code sprayed are calculated, so that the gravity center position of a solid area is selected as a key point.

Specifically, the center of gravity calculation small module may optionally include, but is not limited to:

and the dividing unit is used for dividing the whole workpiece of the workpiece to be sprayed into a plurality of local workpieces by taking the gravity center position of the whole workpiece as a dividing point and determining a dividing line according to the dividing point. Specifically, but not limited to, the center of gravity position is used as a dividing point, and the center of gravity position (center of gravity coordinate point) is used as a horizontal axis parallel to the horizontal plane and a vertical axis perpendicular to the horizontal plane to divide the workpiece to be sprayed into four quadrants of local workpiece. It should be noted that the specific form, position, etc. of the dividing line may be optionally but not limited to any setting according to the actual shape, size, precision requirement of the code spraying position, etc. of the workpiece to be sprayed. Selecting only one dividing line, and dividing the workpiece to be sprayed into two quadrants of local workpieces; or selecting more dividing lines to divide the workpiece to be sprayed into more quadrants of local workpieces. More specifically, when the workpiece to be sprayed is in a symmetrical structure, the workpiece to be sprayed is optionally, but not limited to, an excessive center position, and the symmetrical axis of the workpiece to be sprayed is used as a dividing line to divide the workpiece to be sprayed into a plurality of local workpieces.

The second calculating unit is used for calculating the gravity center position of the local workpiece of the workpiece to be sprayed with the code respectively; specifically, taking a local workpiece including 4 quadrants as an example, 5 center-of-gravity positions (center-of-gravity positions of 1 entire workpiece+center-of-gravity positions of 4 local workpieces) are obtained up to this point. More specifically, the center of gravity position is optionally, but not limited to, the same calculated by a binary image of the workpiece to be sprayed.

In this embodiment, a specific embodiment of how to calculate the center of gravity positions of the whole workpiece and the partial workpiece of the workpiece to be sprayed is provided, the center of gravity position of the whole workpiece is used as a dividing point, the dividing line is selected to divide the workpiece to be sprayed into a plurality of partial workpieces, then the center of gravity position of the partial workpiece is determined, the center of gravity position of the solid region can be selected from the plurality of center of gravity positions to be used as the spraying information, the situation that the whole center of gravity position of some workpieces is not in the solid region (belonging to the hollow part) of the workpiece and can not be used as the sprayable region is further avoided, the phenomena of missing spraying, misspraying and the like are effectively avoided, and powerful guarantee is provided for the following processes of recognition, assembly and the like of the workpiece.

More specifically, the code spraying information determining small module optionally but not exclusively comprises:

A sprayable code area determining unit for determining sprayable code areas (pre-sprayed code areas) around the key points by taking the gravity center position of the solid area as the key point; specifically, the code sprayable region determining unit may optionally, but not only, use the center of gravity position of the solid region as the code sprayable center point, use r as radius to draw a circle or use a as side length to draw a rectangular frame, and determine the solid region around the center of gravity position as the code sprayable region. It should be noted that, the key point is selected as the code spraying center point, the surrounding solid area is selected as the code spraying area, and those skilled in the art can understand that the key point is selected as the starting point (foot drop point) of the code spraying area, and the code spraying area is calculated by the solid area at the left lower part (upper left, lower right, upper right) of the key point, which can be used as an alternative of the present invention.

More specifically, the code spraying information determining small module may also optionally but not be limited to include: and the area calculating unit is connected with the code-sprayable area determining unit and is used for calculating the area of the code-sprayable area. Specifically, the center of gravity of each solid area is taken as a key point, and the area of the code-sprayable area around the center of gravity is calculated, namely, the area of the solid area (the non-hollow area of the code-sprayable area) around the center of gravity is calculated and taken as the pre-code-spraying area.

The self-code spraying information determining unit is connected with the area calculating unit and is used for comparing the sizes of the code spraying areas and selecting key points corresponding to the maximum code spraying areas as self-code spraying information. Namely, the maximum code-spraying area (maximum solid area) is selected as the area to be sprayed, and the corresponding key point is the center point/starting point/end point of the code spraying. It is noted that the code spraying information determined herein is the code spraying information of the workpiece to be sprayed, and is irrelevant to the position of the workpiece on the code spraying main body. More specifically, but not limited to, the center of gravity is selected as the center of the code spraying area, and a rectangular frame with r as the radius and a as the side length is selected as the code spraying area.

In this embodiment, a specific embodiment of how to select the best self-code-spraying information by taking the center of gravity position of the solid area as a key point is provided, which can select the area with the largest code-sprayable area and the corresponding key point as the code-spraying information, on one hand, the condition that the code-spraying is incomplete (the code-spraying position is insufficient, so that part of code is not displayed) can be avoided to the greatest extent; on the other hand, the code printing font and the code printing content can be expanded to the greatest extent, so that the work piece to be code printed can be identified more clearly and more completely by the staff. It should be noted that, the area is only an exemplary example of the best self-spraying code information, and those skilled in the art may also select, but not limited to, the most obvious position or the like as the evaluation criterion of the best self-spraying code information, so as to make the best selection for the gravity center positions of the solid areas.

Specifically, as shown in fig. 27, for the corner positioning method, the present invention further provides a corner positioning device (serving as the second determining module) based on a neural network and a detection algorithm, which optionally but not exclusively includes:

Details of the specific second determining module structure may be selected but not limited to the corresponding method, and will not be described herein.

Specifically, the third determining module determines the code spraying correction information of the workpiece to be code sprayed on the code spraying main body by adopting formulas (1) - (3).

x _n1 ＝x _n cos r _c -y _n sin r _c +x _c (1)

y _n1 ＝y _n sin r _c +y _n cos r _c +y _c (2)

r _n1 ＝r _n +r _c (3)

Specifically, as shown in fig. 28, the present invention further provides a code spraying path planning apparatus, including:

and the code spraying information acquisition small module is used for acquiring the code spraying information of the code spraying main body. Specifically, the code spraying information is selected from but limited to code spraying information of each workpiece to be code sprayed (N workpieces to be code sprayed with the same or different types and names) on the code spraying main body, and is selected from but not limited to code spraying information of each workpiece by serial number. More specifically, in order to ensure the accuracy of the code spraying position, avoid missing spraying and misspraying, the conditions of displacement, angle deviation and the like are optionally but not limited to fully considered when the code spraying main body reaches the code spraying point, the code spraying information is determined as the code spraying information after each workpiece to be code sprayed is corrected, namely the code spraying correction information, and the code spraying correction information is optionally but not limited to the information obtained by correcting the code spraying main body by the coordinate angle (x _n1 ，y _n1 ，r _n1 ) The code spraying information is obtained by connecting the correction system, and is not described herein.

Selecting a small module as a first workpiece, selecting a workpiece to be sprayed, which is closest to the corner point, as a first workpiece to be sprayed by taking the corner point of the code spraying main body as a starting point, wherein i=1, and starting a code spraying process; specifically, the distance calculation may be selected, but is not limited to, using the Euclidean formula.

The workpiece deleting small module deletes the ith workpiece to be sprayed in the code spraying sequence when finishing the code spraying process of the ith workpiece to be sprayed; specifically, the subsequent workpiece to be sprayed with the code, which is understood by those skilled in the art, is the workpiece to be sprayed with the code which has not been finished in the code spraying process, that is, the workpiece to be sprayed with the code which has been finished in the code spraying process needs to be deleted at the position, and the subsequent distance comparison is not performed, so that the code spraying device is prevented from returning to the position where the code has been sprayed again, and the code is sprayed again, thereby reducing the time efficiency and increasing the cost on one hand; on the other hand, the code spraying device can be caused to go back and forth between certain two points, so that the code spraying is disordered and cannot be carried out.

the code spraying completion judging small module judges whether i is equal to N; n is the total number of the workpieces to be sprayed; i.e. whether the work piece to be sprayed is completely finished with the code spraying process; if not, continuing code spraying; if yes, the code spraying is finished.

In this embodiment, a specific embodiment of the code spraying path planning device of the invention is provided, which uses the nearest distance as an evaluation standard, and arranges the whole code spraying process of the code spraying device according to the optimal planning path, so that the distance that the truss needs to move is minimized, the code spraying time is shortened, the code spraying efficiency is improved, and the code spraying cost consumption is reduced.

More specifically, as shown in fig. 29, the present invention further provides a code spraying total system, which includes a truss, a code spraying system, a control system and a correction system; the invention is characterized in that the control system is connected with the correction system and used for controlling the truss to drive the code spraying system to move to the code spraying point position designated by the code spraying correction information according to the code spraying correction information, and starting the code spraying process.

In this embodiment, the inventive concept of the code spraying total system is created based on the correction system, how to obtain the code spraying correction information, and adaptively adjust the code spraying system to the correction position according to the real-time pose of the steel plate to start code spraying, so as to avoid error spraying and missing spraying.

More specifically, the code spraying total system also optionally but not exclusively comprises a code spraying path planning device, which is connected with the control system and is used for providing path planning for the control system; and the control system controls the truss to finish the code spraying process in sequence according to the path planning and the path planning. Specifically, the truss drives the code spraying device to sequentially move to the designated code spraying position according to the path planning action track, the code spraying machine is triggered to start code spraying when the truss moves to the code spraying position, the process is circulated to the next code spraying position until all the code spraying workpieces to be sprayed on the code spraying main body are completely sprayed with codes. More specifically, the code spraying system also optionally but not exclusively comprises a sensing device, and can automatically sense the height of the steel plate in the code spraying process and transmit the height to the control system so as to control the truss height, thereby maintaining proper code spraying height and ensuring that the code spraying effect is clearly visible. More specifically, the truss, the specific structure of the code spraying system, the driving mode, and the like are taken as an example in fig. 29.

The technical effects and advantages of the above system and apparatus are not repeated herein, and it should be noted that each system, apparatus, module, unit, etc. is only a functional division, and does not limit the physical structure of the system, apparatus, module, etc. it will be understood by those skilled in the art that the above technical features of the above embodiment may be combined arbitrarily, and for brevity, all of the possible combinations of the technical features of the above embodiment are not described, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the disclosure.

The above examples illustrate only a few embodiments of the invention, which are described in detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims. In addition, the method and the system for correcting the code spraying of the workpiece and the code spraying total system are the general inventive concept, and are characterized in that the method and the system for correcting the code spraying initial information are based on visual guidance and are self-adaptive code spraying devices. The step S1 serving as a correction method is another invention conception of the invention, and is characterized in that the method and the device for identifying the code spraying of the workpiece correctly determine the code spraying information of the workpiece to be sprayed, so that the condition that the code spraying position is not on the workpiece is avoided. The corner positioning method and the device thereof are the other conception of the invention as the step S2 of the correction method, and the key point is how to combine the deep learning network model algorithm with the traditional detection algorithm to determine the real-time pose of the steel plate. The three inventions are mutually matched and independent, and the inventions are used as serial application division protection for avoiding the problem of singleness in the scheme and are specifically described herein.

Claims

1. A code spraying overall system, comprising: truss, code spraying system, control system and correction system;

the correction system includes:

the control system is connected with the truss, the code spraying system and the correction system and is used for controlling the truss to drive the code spraying system to move to the code spraying point position designated by the code spraying correction information according to the code spraying correction information, and starting a code spraying procedure;

the first acquisition device includes:

the third determining module is connected with the first determining module and the second determining module and is used for determining initial code spraying information of the workpiece to be sprayed on the code spraying main body according to the self code spraying information of the workpiece to be sprayed and the placement position information of the workpiece to be sprayed on the code spraying main body;

A first determination module comprising:

2. The code spraying overall system of claim 1, wherein the center of gravity calculation small module comprises:

3. The code spraying overall system of claim 1, wherein the code spraying information determining small module comprises:

4. The code spraying overall system of claim 1, wherein the second obtaining means comprises:

the correction small module is used for correcting and fusing the second corner data with the first corner data to obtain corrected corner data as a corner positioning output result;

5. The code spraying system according to any one of claims 1 to 4, wherein the control system is further connected to the code spraying path planning device, and is configured to control the truss to drive the code spraying system to complete the code spraying process according to the code spraying path of the code spraying path planning device.

6. The code spraying overall system according to claim 5, wherein the code spraying path planning device comprises:

the code spraying completion judging small module judges whether i is equal to N; if not, continuing code spraying; if yes, the code spraying is finished.