CN111361819B - Wire rod tag hanging system and method - Google Patents

Abstract

The application discloses a wire listing system and a method, which relate to the technical field of visual detection, wherein after a sample to be listed is packed, a robot is controlled to drive a visual positioning device to move to a hollow area of the sample to be listed, and the distance between a camera detected by a distance measuring sensor and the inner wall of the sample to be listed is received; when the distance is a preset distance, starting the annular light source and the camera, and receiving a first sample image shot by the camera; determining a packaging type and a base point according to the first sample image; controlling the annular light source to be closed, starting the structural light source, and receiving a second sample image shot by the camera; calculating an optimal suspension point and a plurality of candidate suspension points sorted according to priority according to the second sample image and the base point; hanging the label clamped by the clamping jaw to the three-dimensional coordinate of the optimal hanging point; and rechecking the listing state, if the rechecking result is that the listing fails, listing at the three-dimensional coordinate of the candidate hanging point according to the priority until the listing succeeds, and finishing the rechecking. This application has improved string tablet efficiency.

Description

Wire rod tag hanging system and method

Technical Field

The application relates to the technical field of visual inspection, in particular to a wire rod tag hanging system and method.

Background

In the steel industry, after steel bar wires produced on site at present are wound into coils, the coils are firstly transferred to a packing station through a suspension assembly line, are automatically packed and bound by a plurality of steel wires or steel belts, and then are transferred to a weighing station, after weighing is finished, a printer prints a metal label in a manual operation table, and when waiting for the next finished coil to be weighed, the current coil is manually operated to hang the label. The sign and the hook are taken manually when the sign is hung, the sign is hung on the hook firstly, and then the hook is hung on different packing steel wires or steel belts with the distance of about 300mm between the inner circle of the wire rod coil and two ends, so that the sign hanging is completed.

At present, the manual signboard hanging is to take the signboard and the hook from the window of an operation room, then move to the next weighing station, stand at one end of a wire rod coil, and stretch into the wire rod coil by hands to hang the signboard. The production line is always in operation, so that the danger coefficient is large, when the wire rod is coiled to a large production amount, the label is frequently moved and hung manually, the labor intensity is high, the operation is fatigue, the label is hung manually, the uncontrollable property exists, the inconsistency exists in the hanging position (above or below) and the hanging distance, and the like, and the problems of wrong label hanging and label missing hanging are easily caused due to the manual long-time operation.

Disclosure of Invention

In order to solve the problems in the background art, the application provides a wire rod tag hanging system and a wire rod tag hanging method.

In a first aspect, the present application provides a wire tag system, comprising:

the robot comprises a robot, a visual positioning device and a clamping jaw, wherein the visual positioning device and the clamping jaw are connected with the robot; and a control system configured to execute the following program steps:

after the sample to be listed is packaged, controlling the robot to drive the visual positioning device to move into the hollow area of the sample to be listed, and receiving the distance between the camera detected by the distance measuring sensor and the inner wall of the sample to be listed;

when the distance is a preset distance, starting the annular light source and the camera, and receiving a first sample image shot by the camera;

determining a packaging type and an image coordinate system from the first sample imageuovBase point of (1: (u ₀，v ₀）；

Controlling the annular light source to be closed, starting the structural light source, and receiving a second sample image shot by the camera;

from the second sample image and the base point (u ₀，v ₀) Calculating the image coordinate systemuovThe best suspension point and a plurality of candidate suspension points in a priority order;

controlling the robot to move, and hanging the label clamped by the clamping jaws to the position where the optimal hanging point corresponds to the three-dimensional coordinates of the world coordinate system XYZ;

and rechecking the listing state, and if the rechecking result is that the listing fails, listing at the three-dimensional coordinate of the candidate hanging point according to the priority until the listing succeeds, and ending the rechecking process.

In some embodiments, the control system is further configured to computationally determine the packaging type and the base point in the image coordinate system by (u ₀，v ₀）：

Demarcating a first region of interest in the first sample image;

matching the first region of interest with a pre-acquired packaging template to determine a packaging type; the packaging type comprises a packaging line and a packaging belt;

acquiring a target area obtained by matching the first region of interest with a packaging template corresponding to the packaging type;

taking the central point of the target area as the base point: (u ₀，v ₀）。

In some embodiments, the control system is further configured to calculate the image coordinate system as followsuovBest suspension point and candidate suspension point in (1)

Defining a second region of interest in a second sample image; the second region of interest and the first region of interestThe positions of the regions correspond and the sizes of the regions are the same, and the second region of interest shares a base point with the first region of interest (u ₀，v ₀）；

In the second region of interest, at the base point: (u ₀，v ₀) Edge ofvSelecting a target structured light stripe on either positive or negative side of the axis;

the center point of each segment of the sub-stripe in the target structure light stripe is determined according tovThe coordinate values are sorted in an increasing mode, the first M central points in the sorting result are obtained, and the sorting is kept unchanged to form a pit set P (u _aj，v _aj) (ii) a M is the preset number of pixel points in the pit point set, j is the sequence number of each pixel point in the pit point set, and j is more than or equal to 1 and less than or equal to M; a is a symbol for identifying each pixel point in the pit set as a pit;

will coordinate (a)u _a1，v ₀) As a coordinate system of the imageuovThe best suspension point in (1), will coordinateu _ak，v ₀) As a coordinate system of the imageuovCandidate suspension points in (1); wherein k is more than or equal to 2 and less than or equal to M, and k is the sequence number of the candidate suspension point.

In some embodiments, the control system is further configured to select the target structured light stripe by:

traversing each pixel point in the second interested area, and obtaining a base point (a)u ₀，v ₀) And along the packing areavConstructing a plurality of subareas on the positive side and the negative side of the shaft; or, with a base point of (u ₀，v ₀) Dividing the second region of interest into a plurality of partitions by using a threshold step length as a reference;

and selecting a target partition according to the direction of the card hanging position relative to the packing position and the integrity degree of the structured light stripes in each partition, and selecting the structured light stripes included in the target partition as the target structured light stripes.

In some embodiments, the control system is further configured to perform:

according to the sequencing result of the central points of all the sub-stripes in the target structured light stripe, the target structured light stripe is subjected to the sequencingvThe central point with the maximum coordinate value is taken as the most salient point (u _t，v _t) (ii) a t is a symbol for identifying the pixel point as the most salient point;

according to the image coordinate systemuovCalculating each suspension point in the suspension point set according to the mapping relation with world coordinate system XYZ (u _aj，v ₀) Coordinate value (X) in world coordinate system XYZ_aj，Y₀) (ii) a The set of suspension points comprises an optimal suspension point (a)u _a1，v ₀) And a candidate suspension point: (u _ak，v ₀）；

According to the most salient point (u _t，v _t) And relative position parameters of the camera, the structural light source, the ranging sensor and the sample to be marked are calculated by using a triangulation method to calculate the Z coordinate of each suspension point in the suspension point set, so that the three-dimensional coordinate (X) of each suspension point in the suspension point set is obtained_aj，Y₀，Z）。

In some embodiments, the control system is further configured to review the hang-up status by:

controlling the robot to move to enable the visual positioning device to return to a shooting position corresponding to the first sample image;

controlling the structure light source to be closed, starting the annular light source, and receiving a third sample image shot by the camera;

after the third sample image is preprocessed, calculating the outline area of the third sample image;

if the contour area of the third sample image is larger than the threshold value, judging that the rechecking result is that the card hanging is successful; otherwise, judging that the rechecking result is the failure of listing;

wherein the total area of the contours other than the sign in the third sample image is less than the threshold.

In some embodiments, the system further comprises a speech device, the control system further configured to perform:

and in the rechecking process, controlling the voice device to broadcast prompt information corresponding to the listing state, wherein the listing state comprises the success and failure of listing.

In some embodiments, the system further comprises a hook maker and a printer, the control system further configured to perform:

before the robot is controlled to drive the visual positioning device to move into the hollow area of the sample to be listed, the hook making machine is controlled to make a hook, and the printer is controlled to print a label; the hook is used for hanging the label at the three-dimensional coordinate of the optimal hanging point;

in response to receiving a manufacturing success instruction sent by the hook making machine, controlling the clamping jaw to clamp a hook from the hook making machine;

and controlling the robot to drive the clamping jaw to move, so that the hook penetrates through a preset through hole on the label, and finishing the card penetrating action.

In some embodiments, when it is required to perform double-sided listing on the sample to be listed, the control system is further configured to perform:

after one side is successfully hung, controlling the robot to rotate 180 degrees to enable the camera to face the other side;

executing a hanging label and a rechecking process on the other side;

and when the other side is successfully hung, the annular light source is controlled to be turned off, and the robot is controlled to return to the initial position.

In a second aspect, the present application provides a wire rod listing method for a wire rod listing system, the wire rod listing system includes a robot, and a vision positioning device and a clamping jaw connected to the robot, the vision positioning device includes an annular light source, a structural light source, a camera and a distance measuring sensor, the method includes:

after the sample to be listed is packaged, controlling the robot to drive the visual positioning device to move into the hollow area of the sample to be listed, and receiving the distance between the camera detected by the distance measuring sensor and the inner wall of the sample to be listed;

when the distance is a preset distance, starting the annular light source and the camera, and receiving a first sample image shot by the camera;

determining a packaging type and an image coordinate system from the first sample imageuovBase point of (1: (u ₀，v ₀）；

Controlling the annular light source to be closed, starting the structural light source, and receiving a second sample image shot by the camera;

from the second sample image and the base point (u ₀，v ₀) Calculating the image coordinate systemuovThe best suspension point and a plurality of candidate suspension points in a priority order;

controlling the robot to move, and hanging the label clamped by the clamping jaw to a position where the optimal hanging point corresponds to the three-dimensional coordinates of the world coordinate system XYZ;

and rechecking the listing state, and if the rechecking result is that the listing fails, listing at the three-dimensional coordinate of the candidate hanging point according to the priority until the listing succeeds, and ending the rechecking process.

The key point of the method is that the optimal hanging point needs to be calculated by using machine vision when the label is hung every time, firstly, when the distance between a camera and the inner wall of a sample to be hung is a preset distance, an annular light source and the camera are used for shooting a first sample image, the first sample image comprises a wire and a local area of a packing device, so that whether the packing type is a packing belt or a packing line can be judged, and a base point (a) in the packing belt or the packing belt is obtained (the base point is a base point in the packing belt)u ₀，v ₀) The base point: (u ₀，v ₀) Is a reference point for subsequently determining the optimal suspension point; then, the annular light source is closed, the structural light source is started, so that the shot second sample image comprises the light band corresponding to the packing device and the structural light stripe corresponding to the wire rod, and then the characteristics and the base point (according to the characteristics and the base point of the second sample image), (b) and (d)u ₀，v ₀) To calculate the optimal suspension point and the candidate suspension point, and the three-dimensional coordinates of the optimal suspension point and the candidate suspension point in the world coordinate system XYZ, firstly, the optimal suspension point and the candidate suspension point are calculated byAnd controlling the robot to move to enable the listing carried by the clamping jaw to be hung at the three-dimensional coordinate of the optimal hanging point, and finishing listing. However, in practical application, due to factors such as errors of the optimal hanging point, the listing may be failed, at this time, the listing state needs to be rechecked to determine whether the listing is successful, if the listing is failed at the optimal hanging point, the listing process does not need to be repeated, but the listing is executed according to the candidate hanging point ranked next to the optimal hanging point, then whether the listing is successful at the candidate hanging point is determined, and so on, if the listing is still failed, the listing is performed on the subsequent candidate hanging points according to the priority order until the listing is successful, and the rechecking and listing process is finished. The automatic plate hanging device has the advantages that the optimal hanging point is automatically determined by machine vision, and the plate hanging operation is automatically performed through the robot, so that the automatic and intelligent plate hanging of the wire rod coiling is realized, the manual labor and the potential safety hazard are reduced, the plate hanging efficiency is improved, and the phenomenon that wrong plates are hung and plates are missed to hang is avoided. In addition, through setting the candidate hanging points, after the best hanging point fails to hang the cards, the complex calculation process does not need to be repeated, the card hanging efficiency is improved, and the working beat of the card hanging system is reduced.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic view illustrating an overall structure of a wire tag system;

FIG. 2 schematically illustrates a connection structure of the visual positioning apparatus;

FIG. 3 schematically illustrates a front view of the visual positioning apparatus;

FIG. 4 illustrates an electrical connection schematic of a wire listing system;

FIG. 5 is a flow chart illustrating a method of registering a wire;

fig. 6(a) schematically illustrates a strapping band template;

FIG. 6(b) is a diagram illustrating a baling line template;

fig. 7(a) exemplarily shows a calculation result of a base point when the packing type is a packing tape;

FIG. 7(b) is a view exemplarily showing a calculation result of a base point when the packing type is a packing line;

FIG. 8 is a schematic diagram illustrating a second region of interest and its regions;

FIG. 9 illustrates a schematic diagram of dynamic partitioning;

FIG. 10 illustrates a schematic diagram of static partitioning;

FIG. 11 illustrates a marker map of key points in a second region of interest;

fig. 12 exemplarily shows a third sample image when the listing is successful;

fig. 13 illustrates an electrical connection schematic of another wire listing system.

In the figure, 1-robot; 2-a scaffold; 3-visual positioning device, 31-annular light source, 32-structural light source, 33-camera, 34-distance measuring sensor; 4-clamping jaw; 5-control system, 51-PLC controller, 52-computer; 6-a voice device; 7-a hook making machine; 8-a printer; 100-sample to be listed; 101-baling means.

Detailed Description

The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The listing is carried out after waiting to list sample 100 packs in this application, and it can use packing apparatus to bind to wait to list sample 100, and packing apparatus 101 can be packing line or packing area, and the packing line is that a plurality of strands of thin steel wires constitute, and the packing area is the steel band that has certain width, adopts different packing apparatus to have different packing types promptly. In the embodiment of the present application, the sample 100 to be registered is taken as a wire coil.

As shown in fig. 1 to 4, the present application provides an embodiment of a wire tag hanging system, which includes a robot 1, a visual positioning device 3 and a clamping jaw 4 connected to the robot 1, and a control system 5. In the example of fig. 2, one end of the visual positioning device 3 is connected to the robot 1 through the bracket 2, the other end of the visual positioning device 3 is connected to the clamping jaw 4, the clamping jaw 4 and the visual positioning device 3 are adjacently arranged and face the sample 100 to be listed, in a specific implementation, the clamping jaw 4 can clamp the manufactured hook from the hook making machine 7, and the hook passes through a perforation preset on the label printed by the printer 8 (i.e. a card passing action), and then the label can be clamped to perform a hanging operation.

In the example of fig. 3, the visual positioning device 3 includes a ring light source 31, a structure light source 32, a camera 33 and a distance measuring sensor 34, the ring light source 31, the structure light source 32, the camera 33 and the distance measuring sensor 34 may be mounted on the same mounting plate, the ring light source 31 and the structure light source 32 may be distributed left and right, the camera 33 and the structure light source 32 may be arranged up and down, and the camera 33 is parallel to the inner surface of the sample 100 to be branded, but in an actual working condition, the camera 33 may be inclined to a certain degree due to a mounting error of the camera 33, in this case, the image captured by the camera 33 needs to be affine transformed to perform subsequent preprocessing, calculation of an optimal hanging point, and other steps.

For the cooperation of the ring light source 31 and the camera 33, a first sample image can be taken, which includes a partial area of the wire and the packing device, and thus can be used to determine whether the packing type is a packing tape or a packing line, and to acquire a packing line or a base point in the packing tape: (u ₀，v ₀) The base point: (u ₀，v ₀) Is the reference point for subsequent determination of the optimal suspension point. For the cooperation of the structure light source 32 and the camera 33, the structure light source 32 can generate structure light when being started, and the sample table to be listed is based on the structure lightAccording to the characteristic principle that the surface is modulated to deform, the structural light is reflected by the surface of the sample 100 to be marked and then received by the camera 33, so that the shot second sample image has a plurality of structural light stripes carrying the real deformation characteristic of the sample surface and light bands corresponding to the packing device 101, and according to the characteristic of the second sample image and the base point (a)u ₀，v ₀) And calculating an optimal hanging point and candidate hanging points to form a hanging point set, calculating three-dimensional coordinates of each hanging point in the hanging point set in a world coordinate system XYZ, generating a hanging track according to the three-dimensional coordinates of the robot and the optimal hanging point, and after the robot 1 moves according to the hanging track, hanging the signboard carried by the clamping jaw 4 at the three-dimensional coordinate position of the optimal hanging point, thereby completing the signboard hanging. The camera 33 may be an industrial camera. The candidate hanging points in the hanging point set are used in the process of hanging card reinspection, and will be described in detail below.

Referring to fig. 1, a sample 100 to be branded may be approximately in a hollow cylindrical shape, a label needs to be hung on an inner wall of a hollow area of the sample 100 to be branded, a control robot 1 drives a visual positioning device 3 to move towards the hollow area of the sample 100 to be branded, the hollow area has a certain inner diameter, and therefore a distance between a camera 33 and the inner wall of the sample 100 to be branded needs to be adjusted, and an image shooting effect is guaranteed. The distance measuring sensor 34 is used for detecting the distance between the camera 33 and the inner wall of the sample 100 to be branded, and the control system 5 determines whether the distance is a preset distance, wherein the preset distance can be set as appropriate according to the inner diameter of the sample 100 to be branded. When the distance is the preset distance, the visual positioning device 3 is considered to reach the proper shooting position, and the annular light source 31 can be started to shoot the first sample image after the shooting position is positioned; if the distance is not equal to the preset distance, the robot 1 needs to be continuously controlled to adjust the distance until the distance is equal to the preset distance. The range sensor 34 may be, for example, a laser range finder.

In the example of fig. 4, the control system 5 is electrically connected to the robot 1, the ring light source 31, the structure light source 32, the camera 33, the distance measuring sensor 34, the voice device 6, the hook maker 7, and the printer 8, respectively. The control system 5 is used for planning a motion track of the robot 1 and controlling the robot 1 to move according to the motion track, and the robot 1 can select six-axis robots and other types; the control system 5 is also used for controlling the start and stop of the annular light source 31, the structure light source 32 and the camera 33; the control system 5 is also used for receiving and positioning the shooting position of the camera 33 according to the distance between the camera 33 and the inner wall of the sample to be marked, which is detected by the distance measuring sensor 34; the control system 5 is also used for receiving and calculating the reinspection of the hanging point set and the listing state according to the sample image shot by the camera 33; the control system 5 is also used for controlling the card hanging state reported by the voice device 6 in the rechecking process, wherein the card hanging state comprises card hanging success and card hanging failure, and the voice device 6 can be arranged on the robot 1 or in other scenes needing voice prompt; the control system 5 is also used for controlling the hook making machine 7 to make a hook and controlling the printer 8 to print a label, and meanwhile, the control system 5 is also used for responding to a making success instruction sent by the hook making machine 7 and a printing success instruction sent by the printer 8 and controlling the robot 1 to drive the clamping jaw 4 to complete hook clamping and card penetrating work.

As shown in fig. 5, the present application also provides an embodiment of a wire listing method, which is used in the wire listing system described above, and is a program step executed by the control system 5, that is, the method is executed by the control system 5, and the method includes:

and step S10, after the sample to be listed is packed, controlling the robot to drive the vision positioning device to move to the hollow area of the sample to be listed, and receiving the distance between the camera detected by the distance measuring sensor and the inner wall of the sample to be listed.

The card hanging is performed after the sample to be hung is packaged, and the card hanging can be performed after the required processes such as weighing can be performed after the packaging is completed. Before the control robot 1 moves into the hollow area of the sample 100 to be branded, the control system 5 controls the hook making machine 7 to make a hook and controls the printer 8 to print the label, the hook is used for hanging the label at the three-dimensional coordinate of the optimal hanging point, in response to receiving a successful making instruction sent by the hook making machine 7, the clamping jaw 4 is controlled to clamp the hook from the hook making machine 7, the robot 1 is controlled to drive the clamping jaw 4 to move, the hook penetrates through a perforation preset on the label, then the card penetrating action is completed, the hook and the label are connected together, one end of the hook is connected with the label when the label is hung, and the other end of the hook is hung on a packing line/packing belt.

And step S20, when the distance is a preset distance, starting the annular light source and the camera, and receiving a first sample image shot by the camera.

The shooting position is positioned by the ranging sensor 34 and the preset distance, so that the visual positioning device 3 is adjusted to a position suitable for shooting the sample image. The first sample image, the second sample image, and the third sample image taken at the time of review in the present embodiment are all acquired by the camera 33 at the shooting position.

Step S30, determining a packaging type and an image coordinate system according to the first sample imageuovBase point of (1: (u ₀，v ₀）。

After receiving the first sample image, the control system 5 may first define a first Region Of Interest (ROI) in the first sample image, where the first Region Of Interest includes a packing Region and a local wire Region, and may reduce image processing and computation amount by setting the first Region Of Interest, thereby improving computation efficiency. In addition, preprocessing such as graying, filtering, corroding and binarizing can be performed on the image of the first region of interest, specifically, image preprocessing can be performed according to actual needs, and all the image preprocessing modes are the prior art, and are not specifically described in this embodiment.

In this embodiment, a template matching algorithm is used to determine the packing type and the base point (u ₀，v ₀) The method needs to acquire a packing template in advance, when the wire rod is coiled by a uniform packing line or a packing belt, the universal packing template can be manufactured in advance, images of the packing line and the packing belt are acquired in advance, and then the images of the packing line and the packing belt are preprocessed, so that the packing template is obtained. It should be noted that if the camera 33 has a problem of being installed obliquely, the envelope is covered withThe image of the strapping band also needs to be affine transformed before preprocessing. The baling templates include a baling band template as shown in fig. 6(a) and a baling line template as shown in fig. 6 (b).

And matching the first region of interest with a pre-acquired packing template, wherein the first region of interest is larger than and contains a region of the packing template, and determining that the first region of interest contains a packing belt region or a packing line region through a template matching algorithm so as to determine the packing type. Acquiring a target region obtained by matching the first region of interest with a packaging template corresponding to the packaging type, wherein fig. 7(a) and 7(b) respectively show the target region under the packaging belt and the packaging line type, and the central point of the target region is taken as the base point ((u ₀，v ₀). The first sample image is used to determine the packing type and base point (u ₀，v ₀) Base point: (u ₀，v ₀) Is the reference point for subsequent calculation of the optimal hanging point of the sign.

Step S40, controlling the annular light source to be closed, starting the structure light source, and receiving the second sample image shot by the camera

The annular light source 31 is turned off, the structured light source 32 is turned on, and under the condition that the shooting position is not changed, the shooting visual field of the same camera 33 is not changed, so that the size of the second sample image is the same as that of the first sample image, the shooting area is also the same, and the second sample image and the first sample image can be unified to the same image coordinate systemuovIn this way, the same spatial point in the shooting area has the same pixel coordinate in the second sample image and the first sample image, and generally, the upper left corner of the image is used as the origin to establish an image coordinate systemuov。

Before the calculation of the optimal suspension point is carried out, a second interested area is defined in the second sample image, the position of the second interested area corresponds to that of the first interested area, the sizes of the second interested area and the first interested area are the same, and the method is equivalent to that the second interested area and the first interested area are positioned in the same image coordinate systemuovMapping a first region of interest in the first sample image to a second sample image to form a second region of interest, the first region of interestThe coordinates of the pixel points at the same position in the interesting region and the second interesting region are the same, so that the second interesting region and the first interesting region share a base point (u ₀，v ₀). Similarly, the second region of interest may be preprocessed by graying, filtering, etching, binarization, etc.

Step S50, according to the second sample image and the base point (S)u ₀，v ₀) Calculating the image coordinate systemuovAnd a number of candidate suspension points in a prioritized order.

And step S60, controlling the robot to move, and hanging the label clamped by the clamping jaws to the position where the optimal hanging point corresponds to the three-dimensional coordinates of the world coordinate system XYZ.

Since the second sample image is taken under the structured light source 32, as shown in FIG. 8, the second region of interest includes a band of light corresponding to the packing area, base point (R) ((R))u ₀，v ₀) In the optical tape, and also on the upper side (edge) of the optical tapevAxial negative direction) and underside (edge)vAxial forward) of the plurality of structured light stripes, wherein the structured light stripes are image features of the wire rod under structured light, and can be seen from fig. 8, and each structured light stripe is composed of a plurality of sub-stripes. The characteristic that the structured light is modulated by the surface of the sample to be marked to deform is as follows: due to the fact that the surface of the sample to be branded is concave and convex, structured light irradiated to the surface of the sample to be branded can be subjected to phase modulation, the more convex parts of the sample to be branded are corresponding to light stripe pixel points, the more downward the light stripe pixel points are, and conversely, the more concave parts of the sample to be branded are corresponding to light stripe pixel points, the more upward the light stripe pixel points are.

At the base point (u ₀，v ₀) Edge ofvWith reference to the shooting angle shown in fig. 1, when the label is desired to be hung on the left side of the wrapping device 101 in actual practice, the target structured light stripe is selected on either the positive or negative side of the axis (at the base point) ((s))u ₀，v ₀) Edge ofvSelecting a target structure light stripe on one side of the negative axis direction; if it is desired to hang the tag on the right side of the wrapping device 101, at the base point（u ₀，v ₀) Edge ofvThe side of the axis in the forward direction selects the target structured light stripe. Then, based on the characteristic that the structured light is modulated by the surface of the sample to be marked to generate deformation, calculating the optimal suspension point and the candidate suspension point, specifically, calculating the center point of each segment of sub-stripe in the target structured light stripe according to the characteristicsvThe coordinate values are sorted in an increasing order (ascending order), and the first M central points in the sorting result are obtained to form a pit set P (u _aj，v _aj) Remain in the pit setvThe sorting order of the coordinate values is increased, M is the preset number of pixel points in the pit set and is also the preset number of suspension points in the suspension point set, j is the sequence number of each pixel point in the pit set, the smaller the sequence number is, the higher the priority is, and j is more than or equal to 1 and less than or equal to M. When j =1, (u _a1，v _a1) Is a global minimum point; pits in the pit set other than the global nadir(s) ((s))u _ak，v _ak) And the candidate pits are selected, wherein k is more than or equal to 2 and less than or equal to M, k is the sequence number of the candidate suspension point, and a is a symbol for identifying each pixel point in the pit set as the pit.

Due to (u _a1，v _a1) Is the global foveal point, and therefore coordinates (a), (b), (c), (du _a1，v ₀) As a coordinate system of the imageuovThe best suspension point in (1), i.e. taking the global most concave point: (u _a1，v _a) Is/are as followsuCoordinate values and base points: (u ₀，v ₀) Is/are as followsvCoordinate value combination to obtain the best suspension point in the image coordinate systemuovThe coordinate in (2) is taken as follows:v ₀one end of the hook is required to be hung on the packing wire/the packing belt at the found position of the packing wire/the packing beltu=u _aOn the straight line, because the wire rod is the most concave for the gap between the wire rod is the deepest, can avoid hanging tablet in-process sign and receive the wire rod to block, thereby be convenient for the couple to accomplish smoothly and hang the tablet. Based on the same principle, coordinate (A), (B), (C), (u _ak，v ₀) As a coordinate system of the imageuovTo constitute a suspensionSet of points P (u _aj，v ₀）={（u _a1，v ₀）；（u _a2，v ₀）；…；（u _aj，v ₀）…（u _aM，v ₀) And the lower the sequence number j, the higher the priority corresponding to the suspension point.

It should be noted that, in other possible implementations, the center point of each segment of the sub-stripe in the target structured light stripe may be determined according to the length of the target structured light stripevThe coordinate values are sorted in descending order, in this case, the last M central points in the sorting result are obtained, and then in order to express the priority order more intuitively, the M central points are arranged according to the ordervThe coordinate values are sequentially rearranged to form a pit set P (u _aj，v _aj) And the pit ordered at the head in the pit set is the global minimum.

In actual conditions, according to the needs of customers, cards may be listed on only one side (i.e. one-sided listing), or on both sides (i.e. two-sided listing). One side hanging tag is that a label is hung on the hollow inner wall of the sample, and when the side hanging tag is hung, the direction of the label relative to the packing position can be on the left side or the right side of a packing line/a packing belt; hang two signs on sample cavity inner wall during both sides are hung the tablet, two signs suspension point compare have 180 degrees corners in the axis, consequently when first side is hung the tablet and is accomplished, need control 1 rotation 180 degrees of robot, drive camera 33's shooting angle also changes 180 degrees, must lead to the reversal of image coordinate system like this, if first side is to hang the tablet in the left side of baling line/baling area, then the second side is just to hang the tablet in the right side of baling line/baling area, just can guarantee the accuracy of sign hanging position. In addition, since the camera 33 may have a problem of being installed obliquely, and the suspended wire surface is an uneven curved surface, the accuracy of the determination of the suspended position is also affected.

To this end, the embodiment performs partition processing on the second region of interest, traverses each pixel point in the second region of interest, and obtains an inclusion base point (according to the screened white pixel points), (b)u ₀，v ₀) And along the packing areavAnd constructing a plurality of subareas on the positive side and the negative side of the shaft. As shown in fig. 8, the second region of interest is composed of a plurality of white pixels and black pixels, the white region composed of the white pixels includes a packing region (i.e., a middle thicker strip light band) and a structural light stripe, and the black region composed of the black pixels is a sample background. As can be seen in fig. 8, the light bands and the structured light stripes are arranged betweenvThe optical tape and the structured light stripes can be separated by the dividing lines with a certain distance in the axial direction, and the dividing results of the 4 dividing lines in fig. 8 are as follows: the middle light band is a packing area, and 2 subareas are respectively arranged on the upper side and the lower side of the packing area.

As an example shown in FIG. 9, the white end of the intermediate band of light (i.e., on band of light)vThe pixel point with the largest coordinate value (u ₁，v ₁) And white end point: (u ₁，v ₁) Immediately below the next white starting point (i.e. in the structured light stripe below the light strip)vThe pixel point with the smallest coordinate value (u ₂，v ₂) A center position between the two pixel points is located at the dividing linelI.e. byl=（v ₁+v ₂)/2. According to the dynamic partitioning principle, the second region of interest is globally partitioned. The dynamic partitioning mode is more intelligent and has higher precision.

In other implementations, as an example shown in fig. 10, a static partition mode may be adopted, and a base point(s) may be adopted in the image coordinate systemu ₀，v ₀) By reference, by a threshold step length y _i The second region of interest is partitioned in such a way that the dividing linel=v ₀+y _i Wherein 1 is less than or equal toiN, N being the number of dividing lines, e.g. of the first dividing line in FIG. 10Threshold step size y₁= -30, threshold step size y of second segment line₂=30, threshold step size y of third dividing line₃=80, thereby dividing the second region of interest into 4 partitions, partition 1, partition 2, partition 3 and partition 4, respectively. Wherein the threshold step length y _i Can be set according to actual experience. After the static partition setting is completed, for example, the label needs to be hung to the left side of the packaging device in fig. 1, the base point (can be directly setu ₀，v ₀) The upper adjacent partition is used as a target partition, and the method has higher efficiency but relatively lower accuracy; or from the base point (u ₀，v ₀) And a partition with higher structural light stripe integrity degree is selected from the plurality of partitions on the upper side as a target partition, so that the calculated optimal suspension point has higher precision. It should be noted that the acquisition rule of the target partition is not limited to that described in this embodiment. Compared with a dynamic partitioning mode, the static partitioning mode can reduce the system beat and improve the listing efficiency, but the dividing lines can possibly separate complete structured light stripes, so the calculation precision is lower than that of the dynamic partitioning mode, and the dynamic or static partitioning mode can be selected and adopted according to actual conditions.

And after the second region of interest is partitioned, selecting a target partition according to the direction of the listing position relative to the packing position and the integrity of the structured light stripes in each partition, and selecting the structured light stripes included in the target partition as the target structured light stripes. For example, in fig. 8, the upper and lower partitions adjacent to the middle light band have complete structural light stripes, if the customer specifies that the first side needs to hang the tag on the left side of the packing device, the upper partition adjacent to the middle light band may be selected as the target partition, and after the robot 1 is rotated 180 degrees, the second side selects the lower partition adjacent to the middle light band as the target partition due to the inversion of the image coordinate system, so as to ensure the accuracy of the two-sided tag hanging. It should be noted that if one-side listing is adopted, the listing position and direction of each packaged sample can be conveniently managed in a unified manner by performing partition processing, and certainly, no partition needs to be arranged during one-side listing.

When calculating the image coordinate systemuovAfter the suspension points in (1) are collected, the suspension points in the suspension point collection need to be calculatedu _aj，v ₀) In the world coordinate system XYZ, a suspension track is generated according to the current position of the clamping jaw 4 and the three-dimensional coordinate of the optimal suspension point, the robot 1 is controlled to move according to the suspension track, and the label clamped in the clamping jaw 4 (specifically, the label is suspended to the three-dimensional coordinate of the optimal suspension point through the hook), so that the card suspension action is completed. During rechecking, if the card hanging at the optimal hanging point fails, the card hanging at the candidate hanging point is carried out based on the priority of the candidate hanging point, whether the card hanging is successful or not is judged, and the rechecking is finished when the card hanging is successful.

World coordinate system XYZ is a coordinate system established in real world space, an image coordinate systemuovAfter establishing the world coordinate system XYZ, the image coordinate system can be obtained in advance according to the camera imaging characteristics, the shooting position and other related informationuovMapping relation with world coordinate system XYZ, image coordinate systemuovAny one pixel point can find a three-dimensional coordinate point corresponding to the world coordinate system XYZ according to the mapping relation. Therefore, through the mapping relationship, each suspension point in the suspension point set can be calculated (u _aj，v ₀) Coordinate value (X) in world coordinate system XYZ_aj，Y₀) Namely, the X coordinate value and the Y coordinate value of each suspension point in the suspension point set in the world coordinate system XYZ are determined.

The Z coordinate (depth) of the optimal suspension point is the distance from the camera 33 to the packaging device, and can be determined according to the maximum salient point(s) ((u _t，v _t) And the relative position parameters of the camera 33, the structural light source 32, the ranging sensor 34 and the sample 100 to be marked are calculated by a triangulation method to obtain the three-dimensional coordinates (X) of each suspension point in the suspension point set_aj，Y₀Z) to form a set of three-dimensional coordinates P (X) of suspension points_aj，Y₀Z) and sets P (X) as three-dimensional coordinates of suspension points_aj，Y₀Z) are stored in the database of the control system 5 for subsequent review.

Wherein, according to the sequencing result of the central points of all the sub-stripes in the target structure light stripe, the central points of all the sub-stripes are sequencedvThe central point with the maximum coordinate value is taken as the most salient point (u _t，v _t) If it is pressingvThe coordinate values are sorted in an increasing (ascending) order, and the central point of the last position is used as the most salient point (u _t，v _t) (ii) a If it is pressedvSorting the coordinate values in descending order, and using the center point of the first position as the most salient point: (u _t，v _t). In this embodiment, the most salient point is selected (u _t，v _t) The purpose of participating in the calculation of the Z coordinate of the optimal suspension point is that the most salient point is the most protruded position in the wire coil, and the actual distance between the most salient point on the surface of the wire coil and the camera 33 is taken as the Z coordinate, so that the robot 1 can be prevented from colliding with the wire coil in the process of listing. Y coordinate values of all suspension points in the suspension point set are the same and are Y₀This is because Y₀The position corresponding to the packing wire/belt; the Z coordinate values of the suspension points in the set of suspension points are the same because the Z coordinate values are based on the most salient point (A)u _t，v _t) And a fixed geometric relationship calculated by a triangulation method, the most salient point: (u _t，v _t) The reason for the impossibility of changing is to prevent the robot 1 from colliding with the wire, thereby ensuring smooth listing. Therefore, the hanging points in the hanging point set only differ in X coordinate value, that is, there is a displacement between the hanging points only in the X-axis direction of the world coordinate system XYZ. Wherein t is a symbol for identifying the pixel point as the most salient point.

In order to increase the success rate of listing in actual conditions, when generating the hanging trajectory of the robot 1, three-dimensional coordinates (X) of each hanging point in the hanging point set are required_aj，Y₀Z) plus a certain deviation, i.e. (X)_aj+a，Y₀+ b, Z + c), where a, b and c are empirical values that can be obtained by debugging the listing system. In FIG. 11, the second feelingThe region of interest is marked with a base point (u ₀，v ₀) Global minimum point: (u _a1，v _a1) Most convex point: (u _t，v _t) And an optimum suspension point: (u _a1，v ₀) Candidate pits may also be marked in the second region of interest ((s))u _ak，v _ak) And a candidate suspension point: (u _ak，v ₀) (not shown in fig. 11) to learn the location distribution of these keypoints.

And step S70, rechecking the listing state, and if the rechecking result is that the listing fails, listing at the three-dimensional coordinate of the candidate hanging point according to the priority until the listing succeeds, ending the rechecking process. In a specific implementation manner of step S70, when the listing is completed, the visual positioning device 3 is located near the optimal hanging point, and it is necessary to return the visual positioning device 3 to the shooting position corresponding to the first sample image/the second sample image, that is, to the shooting position located according to the distance measuring sensor 34 and the preset distance, control the structure light source 32 to turn off, and start the ring-shaped light source 31, and shoot a third sample image by the camera 33, where the third sample image includes a local area of the packing device, the hanging listing and the wire rod, and then the third sample image may be subjected to preprocessing such as graying, filtering, corrosion, and the like, and the third sample image does not need to be subjected to affine transformation.

And then calculating the outline area of the third sample image, if the outline area of the third sample image is larger than a threshold value, judging that the card registering is successful in the rechecking result, and otherwise, judging that the card registering is failed in the rechecking result. The total area of the outlines in the third sample image except for the signs is smaller than the threshold, and the threshold can be set according to the area of the outlines of the signs, for example, the threshold can be set to be one fourth of the area of the signs which are successfully hung in the third sample image. For the outline area of the third sample image, the area ratio of the signs is high, the total area ratio of the outlines except for the signs is small and is smaller than a set threshold value, if the listing fails and no signs exist in the third sample image, the outline area of the wire and the packing device is smaller than the threshold value, and the listing can be judged to fail; if the third sample image has the label, the area of the label is obviously larger than the set threshold value, and the card hanging is judged to be successful.

If the three-dimensional coordinate (X) at the optimum suspension point is determined_a1，Y₀Z) the card hanging is successful, and the rechecking is finished; if the listing at the optimal hanging point is judged to fail, the conventional mode is to repeat the steps S10-S70, shoot the first sample image and the second sample image again to calculate the three-dimensional coordinate of the optimal hanging point, and execute the listing action again, although the listing precision is higher, the recheck needs to execute complex calculation and control flow, so that the listing efficiency is low, and the working beat of the listing system is long. In some application scenarios, the accuracy requirement for listing may be relatively low, for example, the wire coil has a large size, the gap between the wires is large, or the diameter of the packing wire/packing belt is small, in which case the system emphasizes listing efficiency more, so as to reduce the working beat of the system and ensure the production efficiency.

For this embodiment, if it is determined that the hanging of the card at the optimal hanging point is failed, the candidate hanging point 1 next to the optimal hanging point in the hanging point three-dimensional coordinate set is directly retrieved without repeating the steps S10 to S70, and the three-dimensional coordinates (X) of the candidate hanging point 1 are determined_a2，Y₀Z), generating a second hanging track, executing the hanging of the third sample image at the candidate hanging point 1, then executing the step of shooting the third sample image again, judging whether the outline area of the third sample image is larger than the threshold value, repeating the steps in the same way, and repeating the inspection according to the priority order of the candidate hanging points until the final judgment result is that the outline area of the third sample image is larger than the threshold value, namely the hanging is successful, and the repeating inspection process is finished. Fig. 12 shows a schematic view of a third sample image when the listing on the wire is successful, wherein the numerical label is the sign and the hook is connected between the sign and the packing wire.

If the single-side card hanging is adopted, the annular light source 31 is controlled to be turned off after the card hanging is successful and the rechecking process is finished, and the robot 1 is controlled to return to the initial position; if the double-side card hanging is adopted, after the card hanging at one side is successful, the robot 1 is controlled to rotate 180 degrees so that the camera 33 faces the other side, then the processes of hanging the labels and rechecking shown in the steps S10-S70 are also executed on the other side, and after the card hanging at the other side is successful, the annular light source 31 is controlled to be turned off, and the robot 1 is controlled to return to the initial position. After the single-side/double-side listing of the current sample to be listed is successful, the current sample to be listed can be moved out of the listing station, the next sample to be listed is moved to the listing station, and then the method flow is repeated.

In the rechecking process, the voice device 6 is controlled to broadcast prompt information corresponding to the listing state, wherein the listing state comprises the success and failure of listing. Whether the third sample image is hung on one side or two sides, when the third sample image is rechecked each time, the step of judging whether the outline area of the third sample image is larger than the threshold value is executed once, the voice device 6 can be controlled to broadcast corresponding prompt information according to the judged hanging state, the prompt information can be preset in the voice device 6, for example, the prompt information can be set to be that a certain sample has successfully hung a sign, or a certain sample has failed to hang the sign, and the like, so that field personnel can know the hanging state and the rechecking process, the specific content of the prompt information can be set according to the actual situation, and the embodiment is not limited.

In one implementation, as shown in fig. 13, the control system 5 may adopt a combination control structure of a PLC controller 51+ a computer 52, the camera 33, the printer 8, and the PLC controller 51 are electrically connected to the computer 52, respectively, and the robot 1, the ring light source 31, the structure light source 32, the distance measuring sensor 34, the voice device 6, and the hook making machine 7 are electrically connected to the PLC controller 51, respectively. The combined control flow of the PLC controller 51+ the computer 52 is as follows:

after the branding sample is packaged and weighed, the branding sample is sent to a branding station, the PLC 51 controls the hook making machine 7 to make a hook, and the computer 52 controls the printer 8 to print the label; when the PLC 51 receives a successful making instruction sent by the hook making machine 7, the robot 1 is controlled to move so as to drive the clamping jaw 4 to clamp the hook; when the computer 52 receives a printing success instruction fed back by the printer, the computer 52 sends a first control instruction to the PLC controller 51; the PLC 51 responds to the first control instruction, and controls the robot 1 to drive the clamping jaw to complete the card penetrating action.

After the card is put through, the PLC 51 controls the robot 1 to drive the vision positioning device to move to the hollow area of the sample 100 to be listed, and starts the distance measuring sensor 34; the PLC 51 receives the distance between the camera 33 and the inner wall of the sample to be listed, which is detected by the distance measuring sensor 34, and judges whether the distance is a preset distance, if not, the position of the visual positioning device is continuously adjusted, and if the distance is the preset distance, the PLC 51 controls the annular light source 31 and the camera 33 to be started so that the camera 33 shoots a first sample image; the computer 52 receives the first sample image taken by the camera 33 and determines the packaging type and the image coordinate systemuovBase point of (1: (u ₀，v ₀) (ii) a The computer 52 sends a second control instruction to the PLC controller 51, and the PLC controller 51 responds to the second control instruction to control the ring light source 31 to be turned off and start the structure light source 32, so that the camera 33 captures a second sample image; the computer 52 receives the second sample image captured by the camera 33 and calculates the image coordinate systemuovThe suspension point set and the three-dimensional coordinates of each suspension point in the suspension point set, that is, the suspension point three-dimensional coordinate set, and storing the suspension point three-dimensional coordinate set in the database of the computer 52; the computer 52 calculates the three-dimensional coordinates (X) of the optimal suspension point_a1，Y₀Z) and transmits the third control command to the PLC controller 51, and the PLC controller 51 responds to the third control command according to the three-dimensional coordinates (X) of the optimal suspension point_a1，Y₀And Z) generating a first hanging track, controlling the robot 1 to move according to the first hanging track, and hanging the label clamped in the clamping jaw 4 to the three-dimensional coordinate of the optimal hanging point to finish the action of hanging the label.

The robot 1 feeds back a hang tag completion instruction to the PLC controller 51, and at this time, the retest is started: the PLC 51 controls the robot 1 to move, and the visual positioning device 3 returns to the shooting position corresponding to the first sample image; the PLC controller 51 controls the structure light source 32 to be turned off, and the annular light source 31 is started, so that the camera 33 shoots a third sample image; the computer 52 receives the camera 33 shootingAfter the third sample image is obtained, preprocessing the third sample image, calculating the outline area of the third sample image, and judging whether the outline area of the third sample image is larger than a threshold value or not; if the contour area of the third sample image is larger than the threshold value, judging that the rechecking result is that the card hanging at the optimal suspension point is successful; otherwise, if the rechecking result is judged to be that the listing at the optimal hanging point fails, the computer 52 calls the three-dimensional coordinates of the candidate hanging point 1 in the hanging point three-dimensional coordinate set according to the priority, generates a fourth control instruction according to the three-dimensional coordinates of the candidate hanging point 1, sends the fourth control instruction to the PLC 51, and the PLC 51 responds to the fourth control instruction and according to the three-dimensional coordinates (X) of the candidate hanging point 1_a2，Y₀And Z) generating a second hanging track, repeating the card hanging state judgment process, and repeating the process until the card hanging is judged to be successful, and finishing the rechecking process.

If one-side listing is adopted, the computer 52 sends a fifth control instruction to the PLC 51 after the listing is successful and the rechecking process is finished; the PLC 51 responds to a fifth control instruction, controls the annular light source 31 to be closed, and controls the robot 1 to return to the initial position; if the two-side card hanging is adopted, the computer 52 sends a sixth control instruction to the PLC 51 after the one-side card hanging is successful; the PLC 51 responds to a sixth control instruction, controls the robot 1 to rotate 180 degrees so that the camera 33 faces the other side, then executes the processes of hanging the label and rechecking on the other side, and after the label is hung on the other side successfully, the computer 52 sends the seventh control instruction to the PLC 51; the PLC controller 51 controls the ring light source 31 to be turned off in response to the seventh control command, and controls the robot 1 to return to the initial position. Whether single-sided listing or double-sided listing is adopted, the computer 52 needs to send the judged listing state to the PLC 51 during the retest; the PLC controller 51 controls the voice device 6 to broadcast the corresponding prompt information according to the listing state.

After the single-side/double-side listing of the current sample to be listed is successful, the current sample to be listed can be moved out of the listing station, the next sample to be listed is moved to the listing station, and then the method flow is repeated.

According to the technical scheme, the key point of the method is that the optimal hanging point needs to be calculated by using machine vision when the label is hung every time, firstly, when the distance between the camera and the inner wall of the sample to be hung is a preset distance, the annular light source and the camera are used for shooting the first sample image, the first sample image comprises the wire and the local area of the packing device, so that whether the packing type is a packing belt or a packing line can be judged, and a base point (a base point) in the packing belt or the packing belt is obtained (the base point is the base pointu ₀，v ₀) The base point: (u ₀，v ₀) Is a reference point for subsequently determining the optimal suspension point; shooting a second sample image which comprises a light band corresponding to the packaging device and structural light stripes corresponding to the wire rods, and obtaining the most convex points (through partition processing)u _t，v _t) And a set of pits P (u _aj，v _aj) Thereby calculating a set of suspension points P (u _aj，v ₀) And the three-dimensional coordinates (X) of the suspension points included therein in the world coordinate system XYZ_aj，Y₀Z), and then the card hanging at the optimal hanging point can be finished by controlling the robot to move to enable the card hanging carried by the clamping jaw to be hung at the three-dimensional coordinate position of the optimal hanging point.

In practical application, due to factors such as errors of the optimal hanging point, the listing may be failed, at this time, the listing state needs to be rechecked to determine whether the listing is successful, if the listing is failed at the optimal hanging point, the listing process does not need to be repeated, but the listing is executed according to the candidate hanging point which is ranked next to the optimal hanging point and is generated again, then whether the listing is successful at the candidate hanging point is determined, and if the listing is still failed, the listing is performed on the subsequent candidate hanging points according to the priority order until the listing is successful, and the rechecking and listing process is finished. The automatic plate hanging device has the advantages that the optimal hanging point is automatically determined by machine vision, automatic plate hanging operation is automatically performed through the robot, automatic and intelligent plate hanging of wire rod coiling is achieved, manual work and potential safety hazards are reduced, plate hanging efficiency and the positioning accuracy of the optimal hanging point are improved, and wrong plates can be prevented from being hung and plates can be prevented from being missed. In addition, by calculating the three-dimensional coordinate set of the suspension points and storing the suspension points in a warehouse, after the listing at the best suspension point fails, the repeated and complex calculation process is not needed, the rechecking beat can be shortened, the listing can be successfully listed more quickly, the listing efficiency is improved, the working beat of the listing system is reduced, and the production efficiency of the wire rod reel is further improved. The technical scheme can be adaptively applied to the tag hanging control of other types of samples and is not limited to wire coiling.

Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims (10)

1. A wire hang tag system, comprising:

the robot comprises a robot, a visual positioning device and a clamping jaw, wherein the visual positioning device and the clamping jaw are connected with the robot; and a control system configured to execute the following program steps:

after the sample to be listed is packaged, controlling the robot to drive the visual positioning device to move into the hollow area of the sample to be listed, and receiving the distance between the camera detected by the distance measuring sensor and the inner wall of the sample to be listed;

when the distance is a preset distance, starting the annular light source and the camera, and receiving a first sample image shot by the camera;

determining a packaging type and an image coordinate system from the first sample imageuovBase point of (1: (u ₀，v ₀）；

Controlling the annular light source to be closed, starting the structural light source, and receiving a second sample image shot by the camera;

from the second sample image and the base point (u ₀，v ₀) Calculating the image coordinate systemuovThe best suspension point and a plurality of candidate suspension points in a priority order;

controlling the robot to move, and hanging the label clamped by the clamping jaws to the position where the optimal hanging point corresponds to the three-dimensional coordinates of the world coordinate system XYZ;

and rechecking the listing state, if the rechecking result is that the listing fails, listing at the three-dimensional coordinate of the candidate hanging point according to the priority, and ending the rechecking process until the rechecking result is that the listing succeeds.

2. The system of claim 1, wherein the control system is further configured to computationally determine the packaging type and the image coordinate system as followsuovBase point of (1: (u ₀，v ₀）：

Demarcating a first region of interest in the first sample image;

matching the first region of interest with a pre-acquired packaging template to determine a packaging type; the packaging type comprises a packaging line and a packaging belt;

acquiring a target area obtained by matching the first region of interest with a packaging template corresponding to the packaging type;

taking the central point of the target area as the base point: (u ₀，v ₀）。

3. The system of claim 2, wherein the control system is further configured to calculate the image coordinate system as followsuovThe most important ofBest suspension point and candidate suspension point:

defining a second region of interest in a second sample image; the second region of interest corresponds to the position of the first region of interest and has the same region size, and shares a base point with the first region of interest (c) (ii)u ₀，v ₀）；

In the second region of interest, at the base point: (u ₀，v ₀) Edge ofvSelecting a target structured light stripe on either positive or negative side of the axis;

the center point of each segment of the sub-stripe in the target structure light stripe is determined according tovThe coordinate values are sorted in an increasing mode, the first M central points in the sorting result are obtained, and the sorting is kept unchanged to form a pit set P (u _aj，v _aj) (ii) a M is the preset number of pixel points in the pit point set, j is the sequence number of each pixel point in the pit point set, and j is more than or equal to 1 and less than or equal to M; a is a symbol for identifying each pixel point in the pit set as a pit;

will coordinate (a)u _a1，v ₀) As a coordinate system of the imageuovThe best suspension point in (1), will coordinateu _ak，v ₀) As a coordinate system of the imageuovCandidate suspension points in (1); wherein k is more than or equal to 2 and less than or equal to M, and k is the sequence number of the candidate suspension point.

4. The system of claim 3, wherein the control system is further configured to select the target structured light stripe by:

traversing each pixel point in the second interested area, and obtaining a base point (a)u ₀，v ₀) And along the packing areavConstructing a plurality of subareas on the positive side and the negative side of the shaft; or, with a base point of (u ₀，v ₀) Dividing the second region of interest into a plurality of partitions by using a threshold step length as a reference;

and selecting a target partition according to the direction of the card hanging position relative to the packing position and the integrity degree of the structured light stripes in each partition, and selecting the structured light stripes included in the target partition as the target structured light stripes.

5. The system of claim 3, wherein the control system is further configured to perform:

according to the sequencing result of the central points of all the sub-stripes in the target structured light stripe, the target structured light stripe is subjected to the sequencingvThe central point with the maximum coordinate value is taken as the most salient point (u _t，v _t) (ii) a t is a symbol for identifying the pixel point as the most salient point;

according to the image coordinate systemuovCalculating each suspension point in the suspension point set according to the mapping relation with world coordinate system XYZ (u _aj，v ₀) Coordinate value (X) in world coordinate system XYZ_aj，Y₀) (ii) a The set of suspension points comprises an optimal suspension point (a)u _a1，v ₀) And a candidate suspension point: (u _ak，v ₀）；

According to the most salient point (u _t，v _t) And relative position parameters of the camera, the structural light source, the ranging sensor and the sample to be marked are calculated by using a triangulation method to calculate the Z coordinate of each suspension point in the suspension point set, so that the three-dimensional coordinate (X) of each suspension point in the suspension point set is obtained_aj，Y₀，Z）。

6. The system of claim 1, wherein the control system is further configured to review the listing status by:

controlling the robot to move to enable the visual positioning device to return to a shooting position corresponding to the first sample image;

controlling the structure light source to be closed, starting the annular light source, and receiving a third sample image shot by the camera;

after the third sample image is preprocessed, calculating the outline area of the third sample image;

if the contour area of the third sample image is larger than the threshold value, judging that the rechecking result is that the card hanging is successful; otherwise, judging that the rechecking result is the failure of listing;

wherein the total area of the contours other than the sign in the third sample image is less than the threshold.

7. The system of claim 1, further comprising a voice device, the control system further configured to perform:

and in the rechecking process, controlling the voice device to broadcast prompt information corresponding to the listing state, wherein the listing state comprises the success and failure of listing.

8. The system of claim 1, further comprising a hook maker and a printer, the control system further configured to perform:

before the robot is controlled to drive the visual positioning device to move into the hollow area of the sample to be listed, the hook making machine is controlled to make a hook, and the printer is controlled to print a label; the hook is used for hanging the label at the three-dimensional coordinate of the optimal hanging point;

in response to receiving a manufacturing success instruction sent by the hook making machine, controlling the clamping jaw to clamp a hook from the hook making machine;

and controlling the robot to drive the clamping jaw to move, so that the hook penetrates through a preset through hole on the label, and finishing the card penetrating action.

9. The system of claim 6, wherein when double-sided listing of samples to be listed is required, the control system is further configured to perform:

after one side is successfully hung, controlling the robot to rotate 180 degrees to enable the camera to face the other side;

executing a hanging label and a rechecking process on the other side;

and when the other side is successfully hung, the annular light source is controlled to be turned off, and the robot is controlled to return to the initial position.

10. The wire rod listing method is used for a wire rod listing system, the wire rod listing system comprises a robot, a visual positioning device and a clamping jaw, the visual positioning device is connected with the robot and comprises an annular light source, a structural light source, a camera and a distance measuring sensor, and the method comprises the following steps:

after the sample to be listed is packaged, controlling the robot to drive the visual positioning device to move into the hollow area of the sample to be listed, and receiving the distance between the camera detected by the distance measuring sensor and the inner wall of the sample to be listed;

when the distance is a preset distance, starting the annular light source and the camera, and receiving a first sample image shot by the camera;

determining a packaging type and an image coordinate system from the first sample imageuovBase point of (1: (u ₀，v ₀）；

Controlling the annular light source to be closed, starting the structural light source, and receiving a second sample image shot by the camera;

from the second sample image and the base point (u ₀，v ₀) Calculating the image coordinate systemuovThe best suspension point and a plurality of candidate suspension points in a priority order;

controlling the robot to move, and hanging the label clamped by the clamping jaw to a position where the optimal hanging point corresponds to the three-dimensional coordinates of the world coordinate system XYZ;

and rechecking the listing state, if the rechecking result is that the listing fails, listing at the three-dimensional coordinate of the candidate hanging point according to the priority, and ending the rechecking process until the rechecking result is that the listing succeeds.

Images