CN114454617A

CN114454617A - Code spraying total system

Info

Publication number: CN114454617A
Application number: CN202210059528.0A
Authority: CN
Inventors: 田浩楠; 冀春锟; 刘欢
Original assignee: Hunan Shibite Robot Co Ltd
Current assignee: Hunan Shibite Robot Co Ltd
Priority date: 2022-01-19
Filing date: 2022-01-19
Publication date: 2022-05-10
Anticipated expiration: 2042-01-19
Also published as: CN114454617B

Abstract

The invention relates to a total code spraying system, which is characterized by comprising a correction system, wherein the correction system can correct initially determined code spraying initial information by detecting position information of a code spraying main body to obtain the position of a workpiece to be sprayed with a code actually, namely code spraying correction information, so that wrong spraying and missing spraying are avoided, and the accuracy of code spraying position angle and the like is ensured. The code spraying total system can fully consider the ideal situation that the position information of the code spraying main body when the code spraying main body reaches the code spraying point possibly has deviation in the aspects of position, pose, angle and the like, the non-regular square is placed under the code spraying point, the angular point is not an original point, and the angle is also nonzero.

Description

Code spraying total system

Technical Field

The invention relates to the field of control, in particular to a code spraying total system based on visual identification.

Background

The main frame of a construction machine such as a vehicle or a ship is welded by various workpieces, and the workpieces are mainly formed by cutting a steel plate. Generally, workpieces cut from steel plates are various in size and shape, and different workpieces are different in procedures and purposes, so that the workpieces subjected to steel plate cutting are marked, and the subsequent identification and tracking of the workpieces are facilitated.

At present, the main method for marking the cut workpiece in the heavy industry is to spray code marks (ink jet, laser and the like). The code spraying identification mainly comprises two types of manual code spraying and mechanical code spraying. The manual code spraying is that workpieces are manually marked one by one after being confirmed according to a nesting diagram, and the mode has low efficiency, is easy to make mistakes and causes waste of manpower and materials, and is inconvenient for code spraying, identification and tracking of the workpieces especially when the number of the workpieces is large and the types are complex; more importantly, the temperature of the cut steel plate is high, and safety accidents are easy to happen. Mechanical code spraying generally positions code spraying positions (code spraying point positions) firstly, then controls a code spraying device to start a code spraying process to the specified code spraying point positions, is faster than a manual mode in efficiency, and can cause the code spraying positions to have deviation, even have the phenomena of missing spraying and wrong spraying due to the fact that a steel plate has position angle deviation when reaching the code spraying point positions.

Therefore, how to adaptively adjust the code spraying position according to the real-time position angle equipotential attitude deviation of the steel plate is an important technical problem to be solved urgently in mechanical code spraying of workpieces.

Disclosure of Invention

In view of the above technical problems, the present invention provides a total system for code spraying, which includes: the device comprises a truss, a code spraying system, a control system and a correction system;

the correction system comprises:

the first acquisition device is used for acquiring code spraying initial information of a workpiece to be code sprayed on the code spraying main body;

the second acquisition device is used for acquiring the position information of the code spraying main body;

the correcting device is connected with the first acquiring device and the second acquiring device and used for determining code spraying correction information of the workpiece to be code sprayed on the code spraying main body according to code spraying initial information of the workpiece to be code sprayed on the code spraying main body and position information of the code spraying main body;

and the control system is connected with the truss, the code spraying system and the correction system and is used for controlling the truss to drive the code spraying system to move to a code spraying point position specified by the code spraying correction information according to the code spraying correction information and starting a code spraying process.

Further, the first obtaining device includes:

the first determining module is used for determining code spraying information of a workpiece to be code sprayed;

the second determining module is used for determining the placement position information of the workpiece to be code-sprayed on the code-spraying main body;

and the third determining module is connected with the first determining module and the second determining module and is used for determining code spraying initial information of the workpiece to be code sprayed on the code spraying main body according to the code spraying information of the workpiece to be code sprayed and the placement position information of the workpiece to be code sprayed on the code spraying main body.

Further, the first determining module includes:

the small gravity center calculation module is used for calculating the gravity center positions of the whole workpiece and the local workpiece of the workpiece to be code-sprayed;

the solid area judgment small module is connected with the gravity center calculation small module and is used for judging whether the gravity center position is a solid area of the workpiece to be code-sprayed;

the small deleting module is connected with the small solid area judging module and is used for deleting the gravity center position of the area which is not the solid area;

and the code spraying information determining small module is connected with the solid area judging small module and is used for determining the code spraying information of the workpiece to be code sprayed by taking the gravity center position of the solid area as a key point.

Further, a center of gravity calculation small module comprising:

the first calculation unit is used for calculating the gravity center position of the whole workpiece of the workpiece to be subjected to code spraying;

the dividing unit is used for determining a dividing line by taking the gravity center position of the whole workpiece as a dividing point and passing the dividing point, and dividing the whole workpiece of the workpiece to be subjected to code spraying into a plurality of local workpieces;

and the second calculating unit is used for calculating the gravity center position of the local workpiece of the workpiece to be subjected to code spraying respectively.

Further, code spraying information determining small modules comprise:

the code spraying area determining unit is used for determining the code spraying areas around the key points by taking the gravity center positions of the solid areas as the key points;

the area calculation unit is connected with the code spraying area determination unit and is used for calculating the area of the code spraying area;

and the self code spraying information determining unit is connected with the area calculating unit and is used for comparing the sizes of the code spraying areas and selecting the key point corresponding to the maximum code spraying area as the self code spraying information.

Further, the second acquiring means includes:

the small sample acquisition module is used for acquiring a sample image and marking corner data on the sample image to construct a training sample set;

the model building small module is used for building a neural network model for learning corner data;

the small model training module is used for inputting a training sample set into the neural network model to obtain a trained neural network model;

the small image acquisition module is used for acquiring a real-time image of the workpiece to be detected;

the first corner data output small module is used for inputting the real-time image into the trained neural network model and outputting first corner data of the workpiece to be detected;

the image intercepting small module is used for intercepting a regional image comprising the first corner data in the real-time image according to the first corner data;

the second corner data output small module is used for detecting the area image to obtain second corner data of the workpiece to be detected;

and the small correction module is used for correcting and fusing the second corner data with the first corner data to obtain corrected corner data serving as a corner positioning output result.

And the small determining module is used for determining the position information of the code spraying main body according to the corrected corner data.

Furthermore, the control system is also connected with the code spraying path planning device and used for controlling the truss to drive the code spraying system to complete the code spraying process according to the code spraying path of the code spraying path planning device.

Further, spout a yard route planning device includes:

the code spraying information acquisition small module is used for acquiring code spraying information of the code spraying main body;

selecting a small first workpiece module, taking an angular point of the code spraying main body as a starting point, selecting a workpiece to be code sprayed, which is closest to the angular point, as a first code spraying workpiece, and starting a code spraying process when i is equal to 1;

the workpiece deleting small module deletes the ith workpiece to be code-sprayed in the sequence to be code-sprayed when the code-spraying procedure of the ith workpiece to be code-sprayed is completed;

the next workpiece selecting small module is used for selecting the workpiece to be code-sprayed, which is closest to the ith workpiece to be code-sprayed, as the next workpiece to be code-sprayed and starts code spraying, and the number is i + 1;

a small judgment module for code spraying completion judges whether i is equal to N or not; if not, continuing to code the code; if yes, the code spraying is finished

The key of the code spraying total system provided by the invention is that the code spraying total system comprises a correction system, and the initially determined code spraying initial information can be corrected by detecting the position information of the code spraying main body to obtain the position of the workpiece to be sprayed, which actually needs to spray codes, namely the code spraying correction information, so that wrong spraying and missing spraying are avoided, and the accuracy of code spraying position angle and the like is ensured. The code spraying total system can fully consider the ideal situation that the position information of the code spraying main body when the code spraying main body reaches the code spraying point possibly has deviation in the aspects of position, pose, angle and the like, the non-regular square is placed under the code spraying point, the angular point is not an original point, and the angle is also nonzero.

Drawings

FIG. 1 is a diagram of a layout of parts on a steel plate according to an example of the invention;

FIG. 2 is a flowchart of a method for correcting a code-spraying on a workpiece according to an embodiment of the present invention;

FIG. 3 is a schematic diagram illustrating a code spraying process for a workpiece to be sprayed on a non-offset steel plate according to an exemplary embodiment of the present invention;

FIG. 4 is a schematic diagram illustrating a code spraying process for a workpiece to be sprayed on an offset steel plate according to an exemplary embodiment of the present invention;

FIG. 5 is a diagram illustrating a code-spraying dot pattern of a workpiece to be code-sprayed on a steel plate according to an exemplary embodiment of the present invention;

FIG. 6 is a flowchart illustrating an embodiment of step S1 of the method for correcting a code spraying on a workpiece according to the present invention;

FIG. 7 is a schematic diagram of code spraying information of a workpiece to be code sprayed according to the present invention;

fig. 8 is a flowchart illustrating step S11 of the method for correcting a workpiece code according to the present invention, namely, an embodiment of a method for identifying a workpiece code;

fig. 9 is a flowchart illustrating an embodiment of step S111 of the method for identifying a workpiece code spraying according to the present invention;

FIG. 10 is a flowchart illustrating an embodiment of step S114 of the method for identifying a workpiece code according to the present invention;

FIG. 11 is a flowchart of step S2 of the method for correcting a workpiece inkjet, i.e. an embodiment of a corner location method based on a neural network and a detection algorithm;

FIG. 12 is a block diagram of the architecture of one embodiment of the neural network model of the present invention;

FIGS. 13-14 are schematic diagrams of corner points marked on a rectangular frame in the corner positioning method according to the present invention;

FIGS. 15-16 are schematic diagrams of rectangular frames marked with edge points according to the corner positioning method of the present invention;

FIG. 17 is a flowchart illustrating one embodiment of the step S23 in the corner positioning method according to the present invention;

FIG. 18 is a flowchart illustrating one embodiment of step S25 in the corner locating method according to the present invention;

FIG. 19 is a flowchart illustrating one embodiment of the step S27 in the corner positioning method according to the present invention;

FIG. 20 is a flowchart illustrating one embodiment of the step S272 in the corner positioning method according to the present invention;

FIG. 21 is a flow chart of another embodiment of a corner locating method of the present invention;

FIG. 22 is a flowchart of an exemplary method for inkjet path control;

FIG. 23 is a schematic diagram of a code spraying path under the code spraying path control method of the present invention;

FIG. 24 is a block diagram of a system for correcting a code-sprayed workpiece according to an embodiment of the present invention;

fig. 25 is a block diagram illustrating a first obtaining device of the system for correcting a code-spraying of a workpiece according to an embodiment of the present invention;

fig. 26 is a block diagram showing the structure of an embodiment of the device for recognizing a code sprayed on a workpiece according to the present invention;

figure 27 is a block diagram of the construction of one embodiment of the corner locating device of the present invention;

fig. 28 is a block diagram illustrating a structure of a code-spraying path planning apparatus according to an embodiment of the present invention;

fig. 29 is a schematic structural diagram of an embodiment of a total code spraying system according to the present invention.

Detailed Description

To explain the technical idea of the invention in detail, as shown in fig. 1, a code-spraying main body is taken as a steel plate, and a workpiece to be code-sprayed is taken as an example of parts scattered on the steel plate, in the prior art, before or after the steel plate is cut into various parts, a code is sprayed according to the types of the parts to distinguish different parts, and then the parts are assembled into engineering machinery such as ships, vehicles and the like. It should be noted that the above is only an example and is not limited specifically. The code spraying main body can be selected but not limited to main materials such as steel plates and alloy pieces, and can be selected but not limited to workpieces to be sprayed with codes in a scattered distribution single or multiple types. For illustrating other application ranges of the invention, the code-spraying main body can be selected but not limited to a plastic plate, parts for assembling the children toy are scattered and distributed on the code-spraying main body, and the name numbers and the like of different parts are distinguished by code-spraying marks, so that the complete children toy can be conveniently assembled in a subsequent way; the code spraying main body can also be selected from but not limited to cloth, and the cloth is scattered and distributed on the code spraying main body and is used for assembling spare parts of a whole clothes, assembling partial pictures of the whole picture and the like. It will be appreciated by those skilled in the art that any example of code-printing to identify discrete parts to assemble into a single piece is within the scope of the invention.

As shown in fig. 2 to 4, the present invention provides a method for correcting a code-spraying of a workpiece, including:

s1: the method includes the steps of obtaining code spraying initial information of a workpiece to be subjected to code spraying on a code spraying main body (as shown in an example in fig. 3, a rectangular outer frame in the drawing is the code spraying main body, such as a steel plate and the like, the shape and the size of the rectangular outer frame can be set optionally, pentagonal patterns in the drawing are the workpiece to be subjected to code spraying, the same shape and the size of the workpiece can be set optionally, other workpieces to be subjected to code spraying are distributed on the steel plate in a scattered mode and are not shown, a dotted line frame in the drawing is a code spraying area, solid dots are code spraying centers which are required to be obtained in the example, the dotted line frame can be a code spraying center point/starting point/end point and the like, and the code spraying area can be a circle, a rectangular frame and the like made at the center of gravity position). Specifically, the code spraying initial information is optional but not limited to code spraying coordinates (a code spraying center/starting point/end point example) and a code spraying angle of the workpiece to be code sprayed on the code spraying main body, that is, when it is assumed that the corner point of the steel plate is located at the original point (0, 0) and the angle is also 0 (under the ideal condition that no deviation exists when the steel plate is placed), the position (initially determined position and angle) of the code to be sprayed on each workpiece to be code sprayed is assumed. For example, the code-spraying initial information of the nth workpiece can be selected and not limited to the coordinates (x)_n，y_n) Angle r_nIs represented by (x)_n，y_n，r_n)。

S2: and acquiring the position information of the code spraying main body (as shown in figure 4, the displacement and the angle of the steel plate have deviation). Specifically, the position information may be represented by, but not limited to, angular point coordinates, offset angles, and the like for the inkjet printing main body. Taking the code spraying body as a steel plate as an example, that is, when the steel plate is transported to the code spraying point location through the mobile tray, the angular point may not be located at the original point (0, 0), the angle is not 0, but a displacement deviation and an angle deviation exist, optionally but not limited to the angular point coordinate (x)_c，y_c) Angle r_cIs represented by (x)_c，y_c，r_c). More preferably, the corner point coordinates (x) may be passed as shown in the specific embodiment of S2 below_c，y_c) And edge point coordinates (M)_c，N_c) Calculating its offset angle r_c。

S3: according to the initial code spraying information (no deviation of the steel plate and the code spraying information determined in an ideal state) of the workpiece to be code sprayed on the code spraying main body and the position information (the real-time information, the position, the angle and the like of the steel plate have deviations) of the code spraying main body, the code spraying correction information (the corrected code spraying position angle and the like) of the workpiece to be code sprayed on the code spraying main body is determined. That is, the initially determined code-spraying information is corrected according to the position information (optionally, but not limited to, displacement deviation and angle deviation) of the code-spraying main body to obtain corrected code-spraying information (code-spraying correction information), that is, the position and angle at which the code-spraying is actually required at the current code-spraying point are represented as (x)_n1，y_n1，r_n1) (as shown in fig. 4).

In this embodiment, a specific embodiment of the method for correcting a code sprayed on a workpiece according to the present invention is provided, which fully considers an ideal situation that a non-regular square is placed under a code spraying spot, an angular point of the ideal situation is not an original point and an angle of the ideal situation is also non-zero, and corrects an initially determined code spraying position by detecting position information of a code spraying main body, so as to obtain a position where the workpiece to be code sprayed should actually be code sprayed, and ensure accuracy of the code spraying position angle and the like. It is worth noting that in the correction process, the code spraying initial information of the N workpieces to be code sprayed is sequentially corrected according to the number of the workpieces to be code sprayed on the code spraying main body, and the code spraying correction information of each workpiece to be code sprayed is determined, so as to obtain the code spraying dot map shown in fig. 5 (compared with fig. 1, the increased dots are the code spraying correction information of each workpiece to be code sprayed, that is, the actual code spraying dots).

Specifically, as shown in fig. 6, step S1 may optionally but not exclusively include:

s11: determining the self-code-spraying information (as shown in fig. 7) of the workpiece to be code-sprayed, that is, determining where the code-spraying code should mark the self-position of the workpiece to be code-sprayed according to the shape, size and the like of the workpiece to be code-sprayed, regardless of which position the workpiece to be code-sprayed is located in the code-spraying main body, optionally but not limited to coordinates (x)_na，y_na) And (4) showing.

S12: determining the information of the placement position of the workpiece to be code-sprayed on the code-spraying main body (as shown in fig. 3, optionally but not limited to the coordinate representation of key points such as a certain angular point, a central point, etc. of the workpiece to be code-sprayed), specifically, the placement position of the workpiece to be code-sprayed on the code-spraying main body is optionally but not limited to the coordinate (x) of the feature point (such as the upper, lower, left, right angle, central point, etc. of the workpiece, and the figure is exemplified as an angular point of the workpiece) of the workpiece to be code-sprayed_nb，y_nb) And (4) showing. More specifically, the placement position can be selected, but not limited to, analyzing and extracting material data (optionally, but not limited to, information data including the number, name, and the like of each workpiece) and line data (optionally, but not limited to, graphic data including the shape, size, line segment composition, and the like of each workpiece) from a production set drawing (for example, dxf, json, and the like) by a data analyzing module; then storing the material data as file data in a file form, and converting line data combination into frame data and workpiece data of steel plate cutting and storing the frame data and the workpiece data as picture data in a picture form; and finally, the file data and the picture data are in one-to-one correspondence so as to determine the position, the shape, the size and the like of each workpiece on the code spraying main body (steel plate).

S13: code spraying information (optional but not limited to coordinate (x)) is sprayed according to the self of the workpiece to be sprayed with the codes_na，y_na) Is represented by) andthe position information (x) of the workpiece to be sprayed on the code spraying main body_nb，y_nb) Determining the initial code spraying information (x) of the workpiece to be code sprayed on the code spraying main body_n，y_n). Angle r_nAn angle can be set by a person skilled in the art according to appearance, etc., and generally, for convenience of viewing, the angle is generally set to be parallel to the workpiece, i.e., the angle is 0.

In this embodiment, a specific embodiment of how to acquire the code spraying initial information in step S1 is given, where the code spraying initial information is determined by acquiring the code spraying information of the workpiece to be code sprayed and the placement position information of the workpiece to be code sprayed on the code spraying main body, that is, coordinate conversion from the small coordinate system of the workpiece to be code sprayed to the large coordinate system of the code spraying main body is completed, and the code spraying position is simple in structure, accurate in calculation, and can be guaranteed to be accurate.

Specifically, step S11 is another inventive concept of the present invention, and provides a method for identifying a code sprayed on a workpiece, in order to accurately identify and find self-code-sprayed information of the workpiece to be code-sprayed (as shown in fig. 7, any information indicating the position of the code-sprayed area (dashed frame)/code-sprayed start point/end point/central point (solid circle)), which is used as another inventive concept of the present invention. In the specific step S11, the code-spraying information of the workpiece to be code-sprayed may be determined according to the position of the center of gravity of the workpiece to be code-sprayed. More specifically, as shown in fig. 8, S11 may optionally include, but is not limited to:

s111: and calculating the gravity center positions of the whole workpiece and the local workpiece of the workpiece to be sprayed with the code. Specifically, the position of the center of gravity may be obtained by, but not limited to, calculation according to a binary image of the workpiece to be code-sprayed. Specifically, optionally, but not limited to, extracting profile data forming each workpiece from file data and picture data acquired from the production set material diagram, and then obtaining a binary diagram of each workpiece by filling the workpiece profile.

S112: judging whether the gravity center position is a solid area of a workpiece to be code-sprayed; specifically, taking a local workpiece including 4 quadrants as an example, it is necessary to determine whether 5 gravity center positions (the gravity center position of 1 overall workpiece + the gravity center position of 4 local workpieces) are solid regions. More specifically, optionally but not limited to, in the file data and the picture data acquired according to the production nesting diagram, whether each calculated gravity center position (coordinate) is a solid area of the workpiece to be code-sprayed is respectively judged.

S113: if not, deleting the gravity center position of the region which is not solid; specifically, if the center of gravity position is not a solid area, it is characterized that the center of gravity position is not on the workpiece to be code-sprayed (for example, the center of gravity position of the circular ring is in the center circle), and the place of the cavity cannot be used as a preselected position of the code-sprayed area, and needs to be deleted. In particular, optionally but not exclusively, among the calculated positions of the center of gravity, the stored sequence is culled.

S114: and if so, determining the self code spraying information of the workpiece to be code sprayed by taking the gravity center position of the solid area as a key point. Specifically, the code spraying information itself may be, but not limited to, any information representing the position to be sprayed, such as using the center of gravity of the solid area as the start point/end point/center point of the area to be sprayed. More specifically, since a plurality of barycentric positions are calculated in step S111, and a plurality of barycentric positions in a solid area may exist in step S114, those skilled in the art can understand that one of the barycentric positions is selected as code-spraying information, and there is an improved preferred scheme below.

In the embodiment, the method for identifying the code spraying of the workpiece provided by the invention is provided, the gravity center positions of the whole workpiece and the local workpiece of the workpiece to be code sprayed are calculated, and the gravity center position in the solid area is selected as the key point, so that compared with the prior art, the method can avoid the condition that the whole gravity center position of some workpieces (such as hollow workpieces such as circular rings and the like and other special-shaped workpieces and the like) is not in the solid area of the workpiece, belongs to the hollow part and cannot be used as the code spraying area, effectively avoids the phenomena of missing spraying, mistaken spraying and the like, and provides powerful guarantee for the subsequent working procedures of identifying, assembling and the like of the workpiece.

Specifically, as shown in fig. 9, step S111 may optionally but not exclusively include:

s1111: calculating the gravity center position of the whole workpiece of the workpiece to be code-sprayed;

s1112: and determining a dividing line by taking the gravity center position of the whole workpiece as a dividing point and passing the dividing point, and dividing the whole workpiece of the workpiece to be code-sprayed into a plurality of local workpieces. Specifically, the center of gravity position is taken as a dividing point, a transverse axis parallel to the horizontal plane and a longitudinal axis perpendicular to the horizontal plane are taken as an over-center position (a center of gravity coordinate point), and the workpiece to be code-sprayed is divided into four quadrants of local workpieces, namely, an upper quadrant, a lower quadrant, a left quadrant and a right quadrant. It should be noted that the specific form, position, etc. of the dividing line can be optionally but not limited to be set arbitrarily according to the actual shape, size, precision requirement of the code-spraying position, etc. of the workpiece to be code-sprayed. Optionally, but not limited to, only selecting one dividing line, and dividing the workpiece to be code-sprayed into two quadrants of local workpieces; or selecting more dividing lines and dividing the workpiece to be code-sprayed into more quadrants of local workpieces. More specifically, when the workpiece to be code-sprayed is of a symmetrical structure, the position of the center of gravity can be selected but not limited to be passed, and the workpiece to be code-sprayed is divided into a plurality of local workpieces by taking the symmetry axis of the workpiece to be code-sprayed as a dividing line.

S1113: respectively calculating the gravity center position of a local workpiece of the workpiece to be code-sprayed; specifically, taking a partial workpiece including 4 quadrants as an example, 5 gravity center positions (gravity center position of 1 entire workpiece + gravity center position of 4 partial workpieces) are obtained up to this point. More specifically, the position of the center of gravity may be obtained by, but not limited to, the same calculation through a binary image of the workpiece to be code-sprayed.

In this embodiment, a specific example of how to calculate the center of gravity positions of the whole workpiece and the local workpiece of the workpiece to be code-sprayed in step S111 is given, the center of gravity position of the whole workpiece is used as a dividing point, a dividing line is selected to divide the workpiece to be code-sprayed into a plurality of local workpieces, the center of gravity position of the local workpiece is determined, the center of gravity position in a solid region can be selected from the plurality of center of gravity positions to be used as code-spraying information, the situation that the whole center of gravity position of some workpieces is not in the solid region (belonging to the hollow part) of the workpiece and cannot be used as a code-spraying region can be further avoided, the phenomena of missing spraying, mistaken spraying and the like are effectively avoided, and powerful guarantee is provided for the procedures of identification, assembly and the like of subsequent workpieces.

More specifically, as shown in fig. 10, step S114, optionally but not limited to, includes:

s1141: determining a code spraying area (pre-code spraying area) around the key point by taking the gravity center position of the solid area as the key point; specifically, in S1141, the center of gravity position of the solid area is taken as the code spraying center point, r is taken as the radius to draw a circle or a is taken as the side length to draw a rectangular frame, and the solid area around the center of gravity position is determined as the code spraying area. It should be noted that the method of selecting the key point as the code-spraying center point and the surrounding solid area as the code-spraying area is only an exemplary example, and those skilled in the art can understand that the method of selecting the key point as the start point (foot-down point) of the code-spraying area and calculating the code-spraying area with the solid area at the left lower side (upper left, lower right, and upper right) of the key point can be used as an alternative of the present invention.

More specifically, step S114 may optionally, but not limited to, include: s1142: and calculating the area of the code spraying area. Specifically, the center of gravity position of each solid area is used as a key point, and the area of the code-spraying area around the center of gravity position is calculated, that is, the area of the solid area around the center of gravity position (non-hollow area where code can be sprayed) is calculated as the pre-code-spraying area.

S1143: and comparing the size of the code spraying areas, and selecting the key point corresponding to the maximum code spraying area as code spraying information of the key point. Namely, the maximum code spraying area (the maximum solid area) is selected as a code spraying area to be sprayed, and the corresponding key points are the code spraying central point/starting point/end point and the like. It is worth noting that the code spraying information determined here is code spraying information of the workpiece to be code sprayed, and is irrelevant to the position of the workpiece on the code spraying main body. More specifically, the center of gravity position may be used as the center of the code spraying region, and a circle with r as a radius and a rectangular frame with a side length may be selected as the region to be sprayed.

In this embodiment, a specific embodiment of how to select the best self-code-spraying information by using the gravity center position of the solid area as the key point in step S114 is given, which can select the area with the largest code-spraying area and the key point corresponding to the area as the code-spraying information, so that on one hand, the condition that the code spraying is incomplete (the code-spraying position is insufficient and part of the code spraying is not displayed) can be avoided to the greatest extent; on the other hand, the code spraying font and the code spraying content can be expanded to the greatest extent, and the work pieces to be sprayed are identified more clearly and more completely by the staff. It should be noted that, the largest area is merely an exemplary example of the best self-spraying code information, and those skilled in the art may also select, but not limited to, the most prominent position, etc. as the evaluation criterion of the best self-spraying code information, so as to make the best selection for a plurality of gravity center positions in the solid area.

Specifically, in order to obtain the position information of the code-spraying main body (optionally, but not limited to, the coordinates of corner points, the coordinates of inclination angles, and the like of the code-spraying main body representing the code-spraying main body), step S2 is another inventive concept of the present invention, and a corner positioning method based on a neural network and a detection algorithm is provided for positioning the positions of the edge points and the corner points of the steel plate, the inclination angles of the steel plate, and the like (as shown in fig. 4, the code-spraying of the steel plate is taken as an example for explanation here, the coordinates of the corner points of the steel plate are not the original point, and the coordinates of the corner points are (X is the X coordinates)_c，Y_c) The edge point coordinate is (M)_c，N_c)). It should be noted that the corner location method based on the neural network and the detection algorithm is only a preferred example of the step S2, and is not limited thereto. Specifically, as shown in fig. 11, step S2 provides a corner location method based on a neural network and a detection algorithm, which may be selected but not limited to include:

s21: collecting a sample image, and labeling the sample image with corner data (optionally, but not limited to, labeling with a rectangular frame, and subsequently returning to the rectangular frame to calculate and determine coordinates of a side point and a corner point) to construct a training sample set; specifically, a large number of steel plate images of a construction site are collected as sample images by using an image collecting device such as a camera or a video camera, and the sample images particularly include but are not limited to photographed images of edge points/corner points. More specifically, in order to comprehensively restore the field environment and the actual situation so as to improve the accuracy of subsequent model training, the sample image can be selected but not limited to cover the conditions of the light and the dust as much as possible, so that sample images under various complex conditions are obtained, the steel plate and the workpiece have more real texture details, and the neural network model obtained by the subsequent training also has better generalization.

S22: constructing a neural network model for learning corner data; specifically, the deep learning model of yolov4 (shown in fig. 12) can be selected and used in a high-precision modified version. In the Prediction region, YOLOv4 follows the change in the number of feature maps, and accordingly outputs maps with resolutions of 76 × 76/256, 38 × 38/512, and 19 × 19/1024, respectively. In the present invention, optionally but not exclusively, a small input resolution is used to speed up the network training, yielding maps of sizes 52 × 52, 26 × 26 and 13 × 13. More specifically, the method can be selected but not limited to adjust the last layer of activation function Mish, replace the last layer of activation function Mish with the Relu activation function, and make a rapid model more suitable for field deployment under the condition of ensuring precision.

S23: inputting the training sample set into a neural network model, and optimizing parameters to obtain the trained neural network model; specifically, the loss function is continuously reduced in an iterative manner by inputting a training sample set so as to optimize parameters and obtain a neural network model with better performance.

S24: collecting a real-time image of a main body to be detected; specifically, the current real-time image of the steel plate (taking code spraying as an example, that is, the code spraying main body) may be acquired by, but not limited to, an image acquisition device such as a camera or a camera.

S25: inputting the real-time image into the trained neural network model, and outputting first corner data of a main body to be detected (a code-spraying main body such as a steel plate); specifically, the neural network model, optionally but not limited to, returns the same type of output data as the manner in which the sample image is labeled in step S21, representing the corner data of the real-time image. Specifically, the center point of each rectangular frame is used as the first corner coordinate and the first edge coordinate.

S26: intercepting a region image including the first corner data in the real-time image according to the first corner data; specifically, the first corner point coordinates and the first edge point coordinates are obtained according to the first edge data (optionally, but not limited to, the position and the size of the rectangular frame represented by the center point and the height of the rectangular frame or the diagonal coordinates of the rectangular frame), and the cutting processing is performed respectively by taking the identification point as the center, so that a cutting area image with a fixed size is obtained as the area image of the real-time image.

S27: detecting the area image to obtain second corner data; specifically, the clipped region image may be optionally but not limited to be input into the edge detection algorithm model, and the second corner data is calculated. More specifically, the edge detection algorithm model is, but not limited to, canny operator.

S28: and correcting and fusing the second corner data with the first corner data to obtain corrected corner data serving as a final corner positioning output result. In particular, optionally but not exclusively byDetecting region imageThe obtained second corner data is corrected and fused byNeural network modelAnd matching and fusing the obtained first corner data and the obtained first corner data to obtain the corrected corner data.

In this embodiment, a preferred embodiment of step S2 of the present invention is given, which combines the advantages of the neural network model and the conventional edge detection algorithm, and first obtains first corner data of the real-time image through the neural network model, then intercepts a local image (corner image) in the real-time image according to the first corner data, obtains second corner data of the real-time image through the edge detection algorithm, and modifies and fuses the first corner data and the second corner data to obtain modified corner data, i.e., a corner point, a deflection angle, etc., and feeds back position information of the inkjet printing main body, where the calculation results of the corner point and the corner point are more accurate. It should be noted that the method is only a preferred example of the step S2, and is not limited thereto, and those skilled in the art will understand that any other method that can determine the position information of the code-sprayed body (steel plate) may be adopted, such as a single neural network model method, a single edge detection algorithm, and so on.

More specifically, in step S21, when marking the corner data of the sample image, it is optional but not limited to marking the corner points of the sample image with rectangular boxes as illustrated in fig. 13-14 (specifically, the solid line is a steel plate; the dotted line is a grid for placing the steel plate; and the dotted line is a rectangular box for marking); the edge points of the sample image are labeled with rectangular boxes as illustrated in fig. 15-16. More specifically, when the sample image is labeled with the corner data by using the rectangular frame, the corner data is optionally, but not limited to, represented by parameters representing the position of the rectangular frame and the size and the dimension of the rectangular frame. By way of example, the position and size parameters may be optionally, but not limited to, represented by the center point position and the box height of a rectangular box; or in coordinates of two pairs of corner points of a rectangular box, etc.

More specifically, when the edge point data of the sample image is labeled, special attention needs to be paid to the position and the size of the rectangular frame, and the two parameters jointly determine the proportion of a target object (an angular point or an edge point) in the rectangular frame, so that the method plays a vital role in training a neural network model and optimizing parameters, and the accuracy of detecting the corner data of the real-time image by using the trained neural network model can be improved.

For example, among the rectangular boxes marked by the corner points, the rectangular box selected in fig. 14 is better than the rectangular box selected in fig. 13. Specifically, as shown in fig. 13, the left image is marked by selecting a rectangular frame with a size of 100 × 100 pixels, the pixel points are located in the center of the frame body, and large areas around the corner points are marked, which means that areas near 1/3 in the frame are all from grids of the background, and in the training process, if the network can find a case where a large number of grids are in the frame body and a small number of steel plates with uniform textures are contained, it is considered that a high matching degree is achieved on the similarity of the textures, which is not in line with the actual requirements of the user; the right picture marks a whole steel plate, and the large-area frame selection mode is influenced by the placing mode and the exposed area of the steel plate in the whole picture, which can cause mismatching/missing matching with certain probability. The labeling method of fig. 13 finds the area around the target point first, and then performs the second refinement matching in the subsequent operation. The two-step calculation method brings larger calculation amount, meanwhile, the result of the secondary matching depends on the precision of the primary coarse positioning, and if the target point cannot be identified in the coarse positioning stage, the subsequent steps are blocked. In fig. 14, most of the rectangular frame marks of the left image are main bodies of the steel plate, and the rest of the frame bodies need to be ensured to clearly contain two right-angle sides forming the corner points, so that the deep learning network can distinguish the corner points of the steel plate from the corner points of the background grid, and can not be mistakenly matched, and meanwhile, the rectangular frame can accurately find the corner points no matter what kind of placing angles and exposed areas are presented in the shot image by the steel plate. The labeling mode of the right image is used for dealing with the situation that the texture in the image is clear, the light source is bright, but a plurality of 'suspected corner points' exist, the smaller rectangular frame can filter out a plurality of corner points in the background, and the small frame has the advantage that fewer pixels of the rectangular frame, which is equivalent to mismatching of each pixel value, can cause greater loss in network calculation, so that the deep learning network is prompted to examine the matched result with a stricter standard. Therefore, the present invention preferably marks the corner point data with a rectangular box of the type of FIG. 14 to account for labeling different sample images. Specifically, a rectangular frame shown in the left image of fig. 14 is adopted for a sample image with low quality, the rectangular frame comprises an angular point and two right-angle sides, the code spraying main body occupies more than 2/3 of the area of the rectangular frame, and when a neural network model is input and first side angle data is returned, the central point of the rectangular frame and the midpoint of the angular point are taken as actual angular point coordinates; for a sample image with high quality, a rectangular frame shown in the right diagram of fig. 14 is adopted, which includes a corner point and two right-angled sides, but the pixel size (rectangular frame size) is smaller, and when the neural network model is input and first side data is returned, the center point of the rectangular frame is taken as the actual corner point coordinate.

In the rectangular frame marked by the edge points, the most important technical difficulty lies in how to reduce mismatching of the network. Compared with the detection of the angular points, the edge points have more interference points in the image, such as all grids, background metal bars, scratches of the steel plate, water stains and the like, and all the interference points can be mistakenly detected as the edge points. The four labeling methods of fig. 15 were tried, but the trained neural network models all have a certain degree of mismatching. After improvement, a rectangular frame shown in fig. 16 is adopted as a new side point marking method, the edge of the steel plate is marked by a long rectangular frame, the left and right reserved positions of the marking frame are kept equal, meanwhile, the marking frame cannot be too long, and a certain space is reserved from top to bottom, so that when the edge of the steel plate is exposed to be too short in the drawing, the marking frame body and the marking frames of other long side edge samples cannot have too large volume difference.

More specifically, as shown in fig. 17, step S23 optionally but not limited to includes: s231: and performing data cleaning on the training sample set. Namely: recognizable errors in the data file are discovered and corrected, and dirty samples which affect subsequent model training are removed or repaired. More specifically, the data cleansing step optionally but not limited to includes checking data consistency, processing invalid values and missing values, and the like. S232: inputting the training sample set after data cleaning into the neural network model, continuously iterating to reduce the loss function output by the model, and then adjusting and optimizing the model parameters through back propagation to obtain the trained neural network model. The trained neural network model has better performance.

More specifically, as shown in fig. 18, step S25 may optionally, but not limited to, include:

s251: inputting the real-time image into the trained neural network model;

s252: judging whether first corner data are obtained or not;

s253: if so, outputting first corner data of the workpiece to be detected; this is the case: the currently input real-time image quality is excellent, and the neural network model can actively identify first corner data such as a corner point and a side point.

S254: if not, the real-time image is enhanced, the trained neural network model is input, and first corner data are output. This is the case: the quality of a currently input real-time image is poor, a neural network model cannot automatically identify first corner data such as a corner point and an edge point, enhancement processing needs to be carried out by using multiple synthesis algorithms of preset values according to the illumination characteristics of a current picture to obtain a group of enhanced images, and then the enhanced images are input into the neural network model to obtain the first corner data.

More specifically, in step S254, the step of enhancing the real-time image may be, but is not limited to, using: the method has the advantages of enhancing brightness, enhancing HSV color space, enhancing up-down/horizontal inversion, enhancing Gaussian Laplace, effectively enhancing calculation parameters, efficiently and pertinently enhancing positioning and border-patrolling original data, and achieving the purposes of expanding a data set and improving generalization and accuracy of a deep learning network.

In particular, as will be appreciated by those skilled in the art, there are hundreds of enhancement algorithms that process images to augment the acquired data. The inventor selects several optimized enhancing modes to combine according to the actual environment of the steel plate through continuous research and test and creative work, and performs enhancing processing on the real-time image.

More specifically, the formula for calculating the brightness enhancement is:

g(x)＝αf(x)+β

where the parameters α >0 and β are referred to as gain and offset parameters, which control the brightness, respectively. f (x) as source image pixels, g (x) as output image pixels, indicating that the input image is enhanced in brightness to obtain a brighter or darker picture than the original image.

In the HSV color space enhancement, H represents the hue, and the value range is (0, 180); s represents saturation, and the value range is (0, 255); v represents luminance, and has a value range of (0, 255). The algorithm firstly converts the real-time image from the RGB space to the HSV space, then performs numerical amplification or reduction operation on the H, S, V three components according to experience, and finally converts the real-time image back to the RGB space, so as to achieve the effect of image enhancement.

According to the up-down/horizontal inversion enhancement method, partial areas in a real-time image are cut randomly, and then n cut images are spliced together, so that one image contains concentrated objects as much as possible, the richness of samples is increased, and a more complex identification scene is provided for a network. However, it is particularly noted that, from experience, it is not suitable to perform scaling processing in the cropping process, and the cropping should be performed according to the size of the original image, so as to avoid the loss of samples due to deviation of edge points and corner points out of the image cropping area caused by scaling the image.

Laplace enhancement formula:

g (x, y) represents a Gaussian function, and delta represents a standard deviation, namely, the image is smoothed by a Gaussian kernel, so that high-frequency noise in the image can be reduced, and the subsequent Laplace operation is facilitated.

It means that the quadratic partial operation is performed to obtain the LoG function of laplacian gaussian. Since the gaussian is a derivative operator, the sharp gray level change in the image is highlighted, and the slow gray level change area is suppressed, so that the gray edge and the discontinuous image under the dark background are generated. And superposing the Laplace image and the original image to obtain an image with a sharpening effect.

More specifically, the step of enhancing the real-time image may also be, but is not limited to, adopting: contrast enhancement, zoom enhancement, Mosaic enhancement, etc. However, due to the particularity of the scene, the inappropriate enhancement processing algorithm may be improved in the human visual effect, but is not suitable for the subsequent algorithm calculation, and finally, the performance of the neural network model is reduced. Especially, under the scene of patrolling the limit in the location of steel sheet limit point, angular point, possess obvious advantage, mainly reflect: the contrast difference caused by the contrast enhancement method can increase the discrimination between the steel plate and the background, thereby reducing the generalization of the deep learning network and being not beneficial to identifying the steel plate under complex conditions; the zooming enhancement and Mosaic enhancement methods are usually applicable, but because the positions of the edge points and the corner points of the steel plate are uncertain, the edge positions of the image may appear, and a part of the labeling information is easily lost by common zooming and splicing. When too many blank unlabeled samples exist in the training data of the deep learning network, the number of training samples is actually reduced;

more specifically, the corner data, optionally but not limited to, includes corner data, edge data, optionally but not limited to, coordinates (X) of the corner_c，Y_c) And edge point coordinates (M)_c，N_c) And (4) showing. More specifically, in the neural network model, the first corner coordinate is determined by the center point coordinate of the corner point labeling rectangular frame, and the first edge coordinate is determined by the center point coordinate of the edge point labeling rectangular frame. In the detection algorithm, second corner point coordinates and second edge point coordinates are determined with the edge line segments. Specifically, as shown in fig. 19, step S27, optionally but not limited to, includes the following corner data determination:

s271: inputting the area image into an edge detection algorithm model to obtain edge line segments in the area image; it should be noted that the edge line segment, which may be a right-angle edge of the steel plate (an edge line segment that we need to detect), may also be a scratch on the steel plate, an edge of a grid on which the steel plate is placed, and the like, needs to be preferably eliminated later.

S272: determining a preselected angular point in the regional image according to the edge line segment; specifically, optionally, but not limited to, finding all corner points (i.e. preselected corner points) existing in the region image according to the intersection condition of the right-angle sides; specifically, the corner point is a corner point determined by the edge line segment, and since the edge line segment may be a right-angle side of the steel plate, the determined corner point is an actual corner point of the steel plate and can be returned as second corner data; the determined angular points are not actual angular points of the steel plates and can not be returned as second corner data; therefore, in order to fully consider the situations of steel plate scratch and the like, angular points which are not actual angular points of the steel plate need to be excluded from the pre-selected angular points so as to improve the accuracy of corner positioning; specifically, the method comprises the following steps:

s273: judging whether a preselected angular point with the position difference with the first angular point within a first set range exists in the preselected angular points; specifically, since the preselected angular points calculated in step S272 may be angular points of the steel plate (angular points that need to be found actually), and may also be angular points of scratches of the steel plate, angular points of the grid, and the like (which may be mistaken for angular points of the steel plate), the position difference between the preselected angular points and the first angular point needs to be compared, and if the position is close, one of the preselected angular points determined by the characterization detection algorithm is a real angular point of the steel plate, and the two angular points need to be fused and corrected; if the position is far, the pre-selected angular point determined by the characterization detection algorithm does not have the real angular point of the steel plate, but the angular points of other line segments such as scratches and the like need to be eliminated without fusion and correction of the two angular points.

S274: if the position difference exists, returning a preselected angular point with the position difference within the first set range as a second angular point; and when the characterization detection algorithm detects the area image, finding the actual corner point of the steel plate.

S275: and if not, not returning to the second corner point. And when the representation detection algorithm detects the area image, the actual corner point of the steel plate is not found.

More specifically, on the basis of the determination of the corner point data, the step S28 of correcting the corner point specifically includes: if the second corner point is returned, calculating the clustering center points of the first corner point and the second corner point as corrected corner point data; and if the second corner point is not returned, taking the first corner point as the corrected corner point data.

In the embodiment, the specific steps of using corner data as corner data, calculating the second corner and correcting and fusing the first corner are given, so that unreasonable outliers (the corner such as scratch is possibly mistaken for the corner of the steel plate) can be eliminated, the coordinates of the corner are further optimized, and the accuracy of corner positioning is improved.

More specifically, S27 may optionally, but not exclusively, include edge point data determination. Specifically, the edge point data determination has similar characteristics to the corner point data determination, and optionally, but not limited to, the edge point data determination includes:

s276: according to the edge line segment, determining a point which is closest to the first edge point in the edge line segment as a preselected edge point;

s277: judging whether the position difference between the preselected edge point and the first edge point is within a second set range or not;

s278: if so, returning the preselected edge point with the position difference within the second set range as a second edge point;

s279: if not, the second edge point is not returned. And when the characterization detection algorithm detects the area image, the actual edge point of the steel plate is not found.

More specifically, based on the above edge point data determination, the step of correcting the edge point in step S28 specifically includes: and calculating the clustering center points of the first edge point and the second edge point as the corrected edge point data.

In the embodiment, specific steps of calculating the second edge point and correcting and fusing the first edge point are given out, so that the edge point coordinates are further optimized, and the accuracy of edge positioning is improved.

More specifically, S28 may optionally, but not limited to, further optimize the obtained coordinate position by calculating the corner points and the edge points and then further using various parameters such as the distance, the edge slope, and the side length where the points are located.

In this embodiment, a specific embodiment of how to integrate the first edge point data determined by the neural network model and the second edge point data determined by the detection algorithm is given, so that the advantages of the two can be integrated, the positioning accuracy of the corner points and the edge points is improved, and the positioning accuracy of the code-spraying point positions is further optimized.

Specifically, as shown in fig. 21, to improve the accuracy of corner determination, S272: also optional but not limited to include: s2721: calculating the edge area wrapped by the edge points in the regional image; s2722, finding the central moment of the image corresponding to the maximum edge area; s2723: calculating the distance between the central moment of the image and the edge line segment, fusing the edges with the distance within a certain range, and eliminating discontinuous edges; s2724: and calculating the preselected corner points in the regional image according to the edge line segments for eliminating the discontinuous edges.

Specifically, in practical applications, the method for locating the corner may also optionally but not exclusively include: and converting the obtained corner data from the image coordinate system to a space coordinate system. Specifically, taking the collection of the real-time image of the workpiece to be detected as the steel plate as an example, optionally but not limited to inputting the calculated corner data into a conversion algorithm, calling the external reference matrix of the current camera, calculating the real coordinates of the corner points/edge points of the steel plate, and returning the result.

More specifically, to further obtain the current pose of the inkjet printing main body, as shown in fig. 21, the corner positioning method of the present invention may further include, but is not limited to: s29 calculating the code-spraying main body (main body to be detected) according to the corrected corner dataAnd (4) inclining the angle. In particular, optionally but not exclusively, corner data (X) are combined_c，Y_c) Edge point data (M)_c，N_c) And measuring the inclination angle r of the steel plate according to the related parameters of the current photographing position of the camera_c。

In this embodiment, the inclination angle of the main body to be detected (optionally but not limited to steel plate) can be further determined according to the corner data and the side point data, so as to provide effective guarantee for further accurate positioning of the code spraying point location and further optimization of the code spraying effect

Specifically, in step S3, the code spraying correction information of the workpiece to be code sprayed on the code spraying main body is determined by using formulas (1) to (3).

x_n1＝x_ncos r_c-y_nsin r_c+x_c (1)

y_n1＝y_nsin r_c+y_ncosr_c+y_c (2)

r_n1＝r_n+r_c (3)

Specifically, as shown in fig. 22, the present invention further provides a method for controlling a code spraying path, including:

t1: and acquiring code spraying information of the code spraying main body. Specifically, the code spraying information may be code spraying information of each workpiece to be code sprayed (N workpieces to be code sprayed with the same type and the same or different name, and may be code spraying information of each workpiece distinguished by a serial number). More specifically, in order to ensure the accuracy of the code spraying position, avoid the missing spraying and the wrong spraying, the conditions of displacement, angle deviation and the like when the code spraying main body reaches the code spraying point are optionally but not only fully considered, and the code spraying information is determined as the code spraying information corrected by each workpiece to be sprayed, namely the code spraying correction information is optionally but not only limited to the coordinate angle (x)_n1，y_n1，r_n1) It is shown that the specific manner of acquiring the code spraying information of each workpiece to be code sprayed is optional but not limited to be determined by the above method for correcting the code spraying of the workpiece, wherein the steps S1-S3 are optional but not limited to refer to the above, and are not described herein again.

T2: taking an angular point of the code spraying main body as a starting point, selecting a workpiece to be code sprayed, which is closest to the angular point, as a first code spraying workpiece, setting i as 1, and starting a code spraying process; in particular, the distance calculation is optionally, but not limited to, using Euclidean equations.

T3: when the code spraying process of the ith workpiece to be sprayed is finished, deleting the ith workpiece to be sprayed in the code spraying sequence; specifically, the subsequent workpieces to be code-sprayed, which can be understood by those skilled in the art, are workpieces to be code-sprayed, which have not completed the code-spraying process, that is, the workpieces to be code-sprayed, which have completed the code-spraying process, need to be deleted here, and subsequent distance comparison is not performed any more, so that the code-spraying device is prevented from returning to the position where the code is sprayed again, and the code is sprayed again, thereby reducing the time efficiency and increasing the cost on the one hand; on the other hand, the code spraying device may move back and forth between two points, so that the code spraying is disordered and cannot be carried out.

T4: selecting a workpiece to be code-sprayed, which is closest to the ith workpiece to be code-sprayed, as a next workpiece to be code-sprayed, and starting code spraying, wherein the number of the workpiece to be code-sprayed is i + 1;

t5: judging whether i is equal to N; n is the total number of workpieces to be code-sprayed; whether the workpieces to be code-sprayed are all subjected to code spraying procedures or not is judged;

t6: if not, returning to the step T3;

t7: and if so, ending code spraying.

In this embodiment, a specific embodiment of code spraying path planning according to the present invention is provided, in which the shortest distance is used as an evaluation criterion, and the whole code spraying process of the code spraying device is arranged according to the optimal planned path (as shown in fig. 23 line), so that the distance that the truss needs to move is minimized, the code spraying time is shortened, the code spraying efficiency is improved, and the code spraying cost consumption is reduced.

On the other hand, the invention also provides a corresponding system based on the method.

Specifically, as shown in fig. 24, corresponding to the method for correcting a workpiece code, the present invention further provides a system for correcting a workpiece code, which may be selected but not limited to include:

the first acquisition device is used for acquiring code spraying initial information of a workpiece to be code sprayed on a code spraying main body (as shown in fig. 3, a rectangular outer frame is the code spraying main body, such as a steel plate, the shape and the size of the rectangular outer frame can be set optionally, pentagonal patterns in the rectangular outer frame are the workpiece to be code sprayed, the same shape and the size of the workpiece to be code sprayed can be set optionally, other workpieces to be code sprayed are distributed on the steel plate in a scattered manner and are not shown here, a dotted line frame is a code spraying area, solid dots are code spraying centers, the code spraying initial information needs to be acquired in our example, the dotted line frame can be a code spraying center point/starting point/end point and the like, and the code spraying area can be a circle, a rectangular frame and the like made at the center of gravity position). Specifically, the code spraying initial information can be selected from, but not limited to, code spraying coordinates, code spraying angles and the like of the workpiece to be code sprayed on the code spraying main body. For example, if the angular point of the steel plate is located at the origin (0, 0) and the angle is also 0 (in an ideal case where the steel plate is placed without any deviation), the code-spraying position of each workpiece to be code-sprayed (the initially determined position and angle if the steel plate is placed without any deviation). For example, the code-spraying initial information of the nth workpiece can be selected and not limited to the coordinates (x)_n，y_n) Angle r_nIs represented by (x)_n，y_n，r_n). More specifically, the first obtaining module, optionally but not limited to be connected to the storage unit, directly obtains the initial code spraying information preset in the nesting diagram.

And a second acquiring device, configured to acquire position information of the inkjet printing main body (as shown in fig. 4). Specifically, the position information may be, but not limited to, a position coordinate, an offset angle, and the like of the inkjet printing main body. Taking the code spraying main body as a steel plate as an example, when the steel plate is conveyed to the code spraying point location through the movable tray, the angular point is not located at the original point (0, 0), the angle is not 0, and a displacement deviation and an angle deviation exist. For example, the position information of the code-spraying body can be selected, but not limited to, in angular coordinates (x)_c，y_c) Angle r_cIs represented by (x)_c，y_c，r_c). More specifically, the present invention is directed to a method for producing,the second acquisition module, which is optional but not limited to include an image acquisition device, acquires a real-time image of the code spraying main body, and determines the current position, the pose and the like of the code spraying main body through image identification.

And the correcting device is connected with the first acquiring device and the second acquiring device and is used for determining code spraying correction information (, corrected information) of the workpiece to be code sprayed on the code spraying main body according to code spraying initial information (no deviation of the steel plate, determined code spraying information in an ideal state) of the workpiece to be code sprayed on the code spraying main body and position information (deviation of the real-time information, the position, the angle and the like of the steel plate) of the code spraying main body. That is, the initially determined code-spraying information is corrected according to the position information (optionally, but not limited to, displacement deviation and angle deviation) of the code-spraying main body to obtain corrected code-spraying information (code-spraying correction information), that is, the position and angle at which the code-spraying is actually required at the current code-spraying point are represented as (x)_n1，y_n1，r_n1) (as shown in fig. 4).

In this embodiment, a specific embodiment of the system for correcting a code sprayed on a workpiece according to the present invention is provided, which fully considers an ideal situation that a code-sprayed main body (optionally, but not limited to, a steel plate) reaches a code-sprayed point (where there may be a deviation in position, pose, angle, etc. (as shown in fig. 4), a non-regular square is placed under the code-sprayed point, and an angle point is not an original point and an angle is also non-zero, and corrects an initially determined code-sprayed position by detecting the position information of the code-sprayed main body itself, so as to obtain a position where the code-sprayed workpiece actually should be sprayed, and ensure that the code-sprayed position angle is accurate, and it is worth noting that during the correction process, the code-sprayed initial information of N workpieces to be sprayed on the code-sprayed main body needs to be corrected in sequence according to the number of the workpieces to be sprayed on the code-sprayed, so as to obtain a code-sprayed point bitmap as shown in fig. 5 (compare with fig. 1, the added points are the code spraying correction information of each workpiece to be sprayed, namely the code spraying points). It should be further noted that the above-mentioned devices and the following systems and modules are not absolutely divided in a physical sense, but are defined only in a functional sense, and are not specifically limited in a physical structure, and the modules, devices and systems may be connected, interacted and combined in any form.

Specifically, as shown in fig. 25, the first acquiring device may optionally, but not exclusively, include:

the first determining module is configured to determine code spraying information of the workpiece to be code sprayed (as shown in fig. 7), that is, the position of the code spraying code in the workpiece to be code sprayed is determined according to the shape, size, and the like of the workpiece to be code sprayed, regardless of which position the workpiece to be code sprayed is located in the code spraying main body, and optionally, but not limited to, coordinates (x) where the code spraying code is located in the workpiece to be code sprayed_na，y_na) And (4) showing.

A second determining module, configured to determine placement position information of the workpiece to be code-sprayed on the code-spraying main body (as shown in fig. 3, optionally but not limited to the coordinate representation of a key point such as a certain angular point, a central point, and the like of the workpiece to be code-sprayed), specifically, the placement position of the workpiece to be code-sprayed on the code-spraying main body is optionally but not limited to the coordinate (x) of a feature point (such as upper, lower, left, right angles, a central point, and the like of the workpiece, and the figure is exemplified as an angular point of the workpiece) of the workpiece to be code-sprayed_nb，y_nb) And (4) showing. More specifically, the placement position can be selected, but not limited to, analyzing and extracting material data (optionally, but not limited to, information data including the number, name, and the like of each workpiece) and line data (optionally, but not limited to, graphic data including the shape, size, line segment composition, and the like of each workpiece) from a production set drawing (for example, dxf, json, and the like) by a data analyzing module; then storing the material data as file data in a file form, and converting line data combination into frame data and workpiece data of steel plate cutting and storing the frame data and the workpiece data as picture data in a picture form; and finally, the file data and the picture data are in one-to-one correspondence so as to determine the position, the shape, the size and the like of each workpiece on the code spraying main body (steel plate).

The third determining module is connected with the first determining module and the second determining module and is used for spraying code information (optional but not limited to coordinate (x) for spraying code) according to the self code information of the workpiece to be sprayed_na，y_na) Representation) and the placement position information (x) of the workpiece to be code-sprayed on the code-spraying main body_nb，y_nb) Determining the initial code spraying information (x) of the workpiece to be code sprayed on the code spraying main body_n，y_n). Angle r_nAn angle can be set by those skilled in the art according to appearance, etc., and is generally set to be parallel to the workpiece for easy viewing, i.e., the angle is 0.

In the embodiment, a specific embodiment of the first obtaining device forming module is provided, the code spraying initial information is determined by obtaining the code spraying information of the workpiece to be code sprayed and the placement position information of the workpiece to be code sprayed on the code spraying main body, that is, the coordinate conversion from the small coordinate system of the workpiece to be code sprayed to the large coordinate system of the code spraying main body is completed, the structure is simple, the calculation is accurate, and the code spraying position can be ensured to be accurate.

Specifically, in order to accurately identify and find the self-code-spraying information of the workpiece to be code-sprayed (as shown in fig. 7, any information indicating the position of the code-spraying position, such as a code-spraying area (dashed frame)/a code-spraying starting point/an end point/a central point (solid circle)), the invention further provides an identification device for code-spraying of the workpiece (used as a first determination module in the correction system), and the specific identification device for code-spraying of the workpiece is optionally but not limited to determining the self-code-spraying information of the workpiece to be code-sprayed according to the position of the center of gravity of the workpiece to be code-sprayed. More specifically, as shown in fig. 26, the identification means may optionally but not exclusively include:

and the small gravity center calculating module is used for calculating the gravity center positions of the whole workpiece and the local workpiece of the workpiece to be code-sprayed. Specifically, the position of the center of gravity may be obtained by, but not limited to, calculation according to a binary image of the workpiece to be code-sprayed. Specifically, optionally, but not limited to, extracting profile data forming each workpiece from file data and picture data acquired from the production set material diagram, and then obtaining a binary diagram of each workpiece by filling the workpiece profile. More specifically, the small center-of-gravity calculation module is optionally, but not limited to, a control unit having functions of image recognition, processing, data reception, processing, transmission, and the like, and performs the corresponding functions.

The solid area judgment small module is connected with the gravity center calculation small module and is used for judging whether the gravity center position is a solid area of a workpiece to be code-sprayed; specifically, taking a local workpiece including 4 quadrants as an example, it is necessary to determine whether 5 gravity center positions (the gravity center position of 1 overall workpiece + the gravity center position of 4 local workpieces) are solid regions. More specifically, optionally but not limited to, in the file data and the picture data acquired according to the production nesting diagram, whether each calculated gravity center position (coordinate) is a solid area of the workpiece to be code-sprayed is respectively judged.

The small deleting module is connected with the small solid area judging module and is used for deleting the gravity center position of the area which is not the solid area; specifically, if the center of gravity position is not a solid area, it indicates that the center of gravity position is not on the workpiece to be code-printed (for example, the center of gravity position of the circular ring is in the center circle), and the hollow area cannot be used as a preselected position of the code-printed area, and needs to be deleted. In particular, optionally but not exclusively, among the calculated positions of the center of gravity, the stored sequence is culled. Specifically, optionally, but not limited to, the stored sequence is removed from the calculated positions of the centers of gravity, and the number, the name, the code-spraying content, and the like corresponding to the workpiece to be code-sprayed are deleted in the storage unit.

And the code spraying information determining small module is connected with the solid area judging small module and is used for determining the code spraying information of the workpiece to be code sprayed by taking the gravity center position of the solid area as a key point. Specifically, the code spraying information itself may be, but not limited to, any information representing the position to be sprayed, such as using the center of gravity of the solid area as the start point/end point/center point of the area to be sprayed. More specifically, since a plurality of barycentric positions are calculated in step S111, and a plurality of barycentric positions in a solid area may exist in step S114, those skilled in the art can understand that one of the barycentric positions is selected as code-spraying information, and there is an improved preferred scheme below.

In the embodiment, the recognition device for code spraying of the workpiece is provided, the gravity center positions of the whole workpiece and the local workpiece of the workpiece to be code sprayed are calculated, and the gravity center position in the solid area is selected as the key point, so that compared with the prior art, the recognition device can prevent the situation that the whole gravity center position of some workpieces (for example, hollow workpieces such as rings, other special-shaped workpieces and the like) is not in the solid area of the workpiece, belongs to the hollow part and cannot be used as a code spraying area, effectively avoids the phenomena of missing spraying, mistaken spraying and the like, and provides powerful guarantee for the subsequent working procedures of recognition, assembly and the like of the workpiece.

In particular, the centroid calculation submodule may optionally include, but is not limited to:

and the dividing unit is used for determining a dividing line by taking the gravity center position of the whole workpiece as a dividing point and passing through the dividing point, and dividing the whole workpiece of the workpiece to be subjected to code spraying into a plurality of local workpieces. Specifically, the center of gravity position is taken as a dividing point, a transverse axis parallel to the horizontal plane and a longitudinal axis perpendicular to the horizontal plane are taken as an over-center position (a center of gravity coordinate point), and the workpiece to be code-sprayed is divided into four quadrants of local workpieces, namely, an upper quadrant, a lower quadrant, a left quadrant and a right quadrant. It should be noted that the specific form, position, etc. of the dividing line can be optionally but not limited to be set arbitrarily according to the actual shape, size, precision requirement of the code-spraying position, etc. of the workpiece to be code-sprayed. Optionally, but not limited to, only selecting one dividing line, and dividing the workpiece to be code-sprayed into two quadrants of local workpieces; or selecting more dividing lines and dividing the workpiece to be code-sprayed into more quadrants of local workpieces. More specifically, when the workpiece to be code-sprayed is of a symmetrical structure, the position of the center of gravity can be selected but not limited to be passed, and the workpiece to be code-sprayed is divided into a plurality of local workpieces by taking the symmetry axis of the workpiece to be code-sprayed as a dividing line.

The second calculation unit is used for calculating the gravity center position of the local workpiece of the workpiece to be code-sprayed; specifically, taking a partial workpiece including 4 quadrants as an example, 5 gravity center positions (gravity center position of 1 entire workpiece + gravity center position of 4 partial workpieces) are obtained up to this point. More specifically, the position of the center of gravity may be obtained by, but not limited to, the same calculation through a binary image of the workpiece to be code-sprayed.

In the embodiment, a small center-of-gravity calculation module is provided, and a specific embodiment of how to calculate the center-of-gravity positions of an overall workpiece and a local workpiece of a workpiece to be subjected to code spraying is provided.

More specifically, the code spraying information determining small module may be selected from, but is not limited to, the following modules:

the code spraying area determining unit is used for determining a code spraying area (a pre-code spraying area) around the key point by taking the gravity center position of the solid area as the key point; specifically, the code-spraying region determining unit may optionally, but not limited to, draw a circle with r as a radius or draw a rectangular frame with a side length by using the center of gravity position of the solid region as the code-spraying center point, and determine the solid region around the center of gravity position as the code-spraying region. It should be noted that the method of selecting the key point as the code-spraying center point and the surrounding solid area as the code-spraying area is only an exemplary example, and those skilled in the art can understand that the method of selecting the key point as the start point (foot-down point) of the code-spraying area and calculating the code-spraying area with the solid area at the left lower side (upper left, lower right, and upper right) of the key point can be used as an alternative of the present invention.

More specifically, the code spraying information determining small module may also optionally but not limited to include: and the area calculation unit is connected with the code spraying area determination unit and is used for calculating the area of the code spraying area. Specifically, the center of gravity position of each solid area is used as a key point, and the area of the code-spraying area around the center of gravity position is calculated, that is, the area of the solid area around the center of gravity position (non-hollow area where code can be sprayed) is calculated as the pre-code-spraying area.

And the self code spraying information determining unit is connected with the area calculating unit and is used for comparing the sizes of the code spraying areas and selecting the key point corresponding to the maximum code spraying area as the self code spraying information. Namely, the maximum code spraying area (the maximum solid area) is selected as a code spraying area to be sprayed, and the corresponding key points are the code spraying central point/starting point/end point and the like. It is worth noting that the code spraying information determined here is code spraying information of the workpiece to be code sprayed, and is irrelevant to the position of the workpiece on the code spraying main body. More specifically, the center of gravity position may be used as the center of the code spraying region, and a circle with r as a radius and a rectangular frame with a side length may be selected as the region to be sprayed.

In the embodiment, a small code spraying information determining module is provided, and a specific embodiment of selecting the best self code spraying information by taking the gravity center position of the solid area as a key point can select the area with the largest code spraying area and the corresponding key point as the code spraying information, so that the condition that the code spraying is incomplete (the part of the code spraying is not displayed due to insufficient code spraying position) can be avoided to the greatest extent; on the other hand, the code spraying font and the code spraying content can be expanded to the greatest extent, and the work pieces to be sprayed are identified more clearly and more completely by the staff. It should be noted that, the area is only an exemplary example of the best self-spraying code information, and those skilled in the art may also select, but not limited to, the most prominent position, etc. as the evaluation criterion of the best self-spraying code information, so as to make the best selection for a plurality of gravity center positions in the solid area.

Specifically, as shown in fig. 27, for the corner positioning method, the present invention further provides a corner positioning device (serving as the second determining module) based on a neural network and a detection algorithm, which may optionally but not limited to include:

The specific details of the second determining module structure are optional but not limited to correspond to the method thereof, and are not described herein again.

Specifically, the third determining module is used for determining code spraying correction information of the workpiece to be code sprayed on the code spraying main body by adopting formulas (1) - (3).

x_n1＝x_ncos r_c-y_nsin r_c+x_c (1)

y_n1＝y_nsin r_c+y_ncos r_c+y_c (2)

r_n1＝r_n+r_c (3)

Specifically, as shown in fig. 28, the present invention further provides a device for planning a code spraying path, including:

and the code spraying information acquisition small module is used for acquiring the code spraying information of the code spraying main body. Specifically, the code spraying information may be code spraying information of each workpiece to be code sprayed (N workpieces to be code sprayed with the same type and the same or different name, and may be code spraying information of each workpiece distinguished by a serial number). More specifically, in order to ensure the accuracy of the code spraying position, avoid the missing spraying and the wrong spraying, the conditions of displacement, angle deviation and the like when the code spraying main body reaches the code spraying point are optionally but not only fully considered, and the code spraying information is determined as the code spraying information corrected by each workpiece to be sprayed, namely the code spraying correction information is optionally but not only limited to the coordinate angle (x)_n1，y_n1，r_n1) And indicating that code spraying information is obtained by connecting the correction system, and the description is omitted here.

Selecting a small first workpiece module, taking an angular point of the code spraying main body as a starting point, selecting a workpiece to be code sprayed, which is closest to the angular point, as a first code spraying workpiece, and starting a code spraying process when i is equal to 1; in particular, the distance calculation is optionally, but not limited to, using Euclidean equations.

The workpiece deleting small module deletes the ith workpiece to be code-sprayed in the sequence to be code-sprayed when the code-spraying procedure of the ith workpiece to be code-sprayed is completed; specifically, the subsequent workpieces to be code-sprayed, which can be understood by those skilled in the art, are workpieces to be code-sprayed, which have not completed the code-spraying process, that is, the workpieces to be code-sprayed, which have completed the code-spraying process, need to be deleted here, and subsequent distance comparison is not performed any more, so that the code-spraying device is prevented from returning to the position where the code is sprayed again, and the code is sprayed again, thereby reducing the time efficiency and increasing the cost on the one hand; on the other hand, the code spraying device may move back and forth between two points, so that the code spraying is disordered and cannot be carried out.

a small judgment module for code spraying completion judges whether i is equal to N or not; n is the total number of workpieces to be code-sprayed; whether the workpieces to be code-sprayed are all subjected to code spraying procedures or not is judged; if not, continuing to code the code; and if so, ending code spraying.

In this embodiment, a specific embodiment of the code spraying path planning device for the code spraying path of the invention is provided, which takes the closest distance as an evaluation standard, and arranges the whole code spraying process of the code spraying device according to the optimal planned path, so that the distance that the truss needs to move is minimized, the code spraying time is shortened, the code spraying efficiency is improved, and the code spraying cost consumption is reduced.

More specifically, as shown in fig. 29, the present invention further provides a total code spraying system, which includes a truss, a code spraying system, a control system and a correction system; the control system is connected with the correction system and used for controlling the truss to drive the code spraying system to move to the code spraying point position specified by the code spraying correction information according to the code spraying correction information and starting a code spraying process.

In this embodiment, the invention concept of the total code spraying system of the present invention is created based on the above-mentioned correction system, and is to obtain code spraying correction information, to adaptively adjust the code spraying system to the correction position to start code spraying according to the real-time pose of the steel plate, so as to avoid erroneous spraying and missing spraying.

More specifically, the code spraying total system can optionally but not exclusively comprise a code spraying path planning device, which is connected with the control system and used for providing path planning for the control system; and the control system controls the truss to plan according to the path planning, and controls the truss to complete the code spraying process in sequence. Specifically, the truss drives the code spraying device to move to the appointed code spraying position in sequence according to the action track planned by the path, the code spraying machine is triggered to start code spraying when the truss moves to the code spraying position, the process is circulated to the next code spraying position until all the workpieces to be sprayed on the code spraying main body are sprayed with codes completely. More specifically, spout a yard system, still can select but not only be limited to including sensing device, at spouting a yard in-process auto-induction steel sheet height, send control system to control truss height, thereby keep suitable spouting a yard height, ensure to spout that the yard effect is clear visible. More specifically, fig. 29 is an example of specific structures, driving manners, and the like of the truss and the inkjet printing system.

The above systems and apparatuses are created based on the above method, and technical functions and advantages thereof are not described herein again, it is to be noted that each system, apparatus, module, unit, and the like are only functionally divided, and do not limit physical structures thereof at all, and those skilled in the art can understand that any combination of the technical features of the above-mentioned embodiments may be performed, and for brevity of description, all possible combinations of the technical features in the above-mentioned embodiments are not described, however, as long as there is no contradiction between combinations of the technical features, the combination should be considered as being within the scope described in the present specification.

The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims. In addition, the method and the system for correcting the workpiece code spraying and the general code spraying system are the general inventive concept of the invention, and the key points are to acquire the position information of the code spraying main body and correct the initial code spraying information, so that the method and the system are a vision-guidance-based self-adaptive code spraying device. The invention discloses a method and a device for identifying a code sprayed on a workpiece, wherein the step S1 of the method is used as a correction method, and the key point of the method is how to correctly determine the code sprayed information of the workpiece to be code sprayed, so that the condition that the code sprayed position is not on the workpiece is avoided. The corner positioning method and the device thereof, which are the step S2 of the correction method, are another concept of the invention, and the key point is how to fuse the deep learning network model algorithm and the traditional detection algorithm to determine the real-time pose of the steel plate. The three inventions are mutually matched and independent, and in order to avoid the problem of singleness, the inventions are protected as a series of applications and are specifically described herein.

Claims

1. A total system for inkjet printing, comprising: the device comprises a truss, a code spraying system, a control system and a correction system;

the correction system comprises:

2. The inkjet printing system of claim 1, wherein the first acquiring device comprises:

and the third determining module is connected with the first determining module and the second determining module and is used for determining code spraying initial information of the workpiece to be code sprayed on the code spraying main body according to the self code spraying information of the workpiece to be code sprayed and the placement position information of the workpiece to be code sprayed on the code spraying main body.

3. The total system of claim 2, wherein the first determining module comprises:

4. A total system according to claim 3, wherein the gravity center calculating module comprises:

and the second calculating unit is used for respectively calculating the gravity center position of the local workpiece of the workpiece to be sprayed with the code.

5. The total system of claim 3, wherein the code-spraying information determining module comprises:

and the code spraying information determining unit is connected with the area calculating unit and used for comparing the size of the code spraying area and selecting the key point corresponding to the largest code spraying area as the code spraying information.

6. The inkjet printing system according to claim 1, wherein the second acquiring device comprises:

7. The total system of one of claims 1 to 6, wherein the control system is further connected to a code spraying path planning device, and is configured to control the truss to drive the code spraying system to complete a code spraying process according to a code spraying path of the code spraying path planning device.

8. The inkjet printing system according to claim 7, wherein the inkjet printing path planning device comprises:

a small judgment module for code spraying completion judges whether i is equal to N or not; if not, continuing to code the code; and if so, ending code spraying.