WO2021238094A1 - 一种线材的挂牌系统及方法 - Google Patents

一种线材的挂牌系统及方法 Download PDF

Info

Publication number
WO2021238094A1
WO2021238094A1 PCT/CN2020/128642 CN2020128642W WO2021238094A1 WO 2021238094 A1 WO2021238094 A1 WO 2021238094A1 CN 2020128642 W CN2020128642 W CN 2020128642W WO 2021238094 A1 WO2021238094 A1 WO 2021238094A1
Authority
WO
WIPO (PCT)
Prior art keywords
listing
point
sample
camera
sample image
Prior art date
Application number
PCT/CN2020/128642
Other languages
English (en)
French (fr)
Inventor
孙茂杰
李福存
孙敬忠
苏循亮
杨文�
朱正清
李敏
林启森
周鼎
刘彦麟
Original Assignee
江苏金恒信息科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 江苏金恒信息科技股份有限公司 filed Critical 江苏金恒信息科技股份有限公司
Publication of WO2021238094A1 publication Critical patent/WO2021238094A1/zh

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65C—LABELLING OR TAGGING MACHINES, APPARATUS, OR PROCESSES
    • B65C7/00—Affixing tags

Definitions

  • This application relates to the field of visual inspection technology, and in particular to a system and method for listing wires.
  • the steel wire rods currently produced on site are wound into coils and transferred to the packing station through the suspension assembly line. After the multiple steel wires or steel belts are automatically bundled and bundled, they are transferred to the weighing station. After the weighing is completed , The metal label is printed by the printer in the manual operating table, and the current coil is manually listed while waiting for the next roll of finished coil to be weighed. Manually take the signs and hooks when listing, first hang the signs on the hooks, and then hang the hooks on different packing steel wires or steel belts with a pitch of about 300mm at both ends of the wire coil to complete the listing.
  • the current listing requires the operator to take the signs and hooks from the operating room window, and then walk to the next weighing station, stand on one end of the wire coil, and reach into the wire coil with bare hands to hang the signs. Because the assembly line is always in operation, the risk factor is relatively large, and when the wire coil production volume is large, the operator frequently runs back and forth and hangs signs, long-term work, labor intensity is high, and work fatigue is easy to occur. Manual hanging of signs is uncontrollable, and there are inconsistencies in the hanging position (above or below) and the distance of the hanging distance, and long-time manual operation can easily cause the problems of wrong signs and missing signs.
  • the present application provides a system and method for listing wires.
  • this application provides a wire listing system, including:
  • control robot drives the visual positioning device to move to the hollow area of the sample to be listed, and receives the distance between the camera detected by the ranging sensor and the inner wall of the sample to be listed;
  • Re-check the listing status If the result of the re-check is that listing fails, perform listing at the three-dimensional coordinates of the candidate suspension point according to the priority until the listing is successful, then the re-check process ends.
  • control system is further configured to calculate and determine the packing type and the base point (u 0 , v 0 ) in the image coordinate system according to the following steps:
  • the packaging type includes a packaging line and a packaging belt;
  • the center point of the target area is used as the base point (u 0 , v 0 ).
  • control system is further configured to calculate the optimal suspension point and the candidate suspension point in the image coordinate system uov according to the following steps
  • a second region of interest is delineated in the second sample image; the second region of interest corresponds to the position of the first region of interest and the size of the region is the same, and the second region of interest is the same as the first region of interest.
  • the shared base point of the region of interest (u 0 , v 0 );
  • the center points of the sub-stripes in the target structured light stripe are sorted in ascending order according to the v coordinate value, and the first M center points in the sorting result are obtained and the sorting is kept unchanged to form a concave point set P(u aj , v aj );
  • M Is the preset number of pixels included in the pit set, j is the serial number of each pixel in the pit set, 1 ⁇ j ⁇ M; a is used to identify each pixel in the pit set as a pit symbol;
  • control system is further configured to select the target structured light stripe according to the following steps:
  • a target partition is selected, and the structured light stripes included in the target partition are selected as the target structured light stripes.
  • control system is further configured to perform:
  • the center point with the largest v coordinate value is regarded as the most convex point (u t , v t ); t is used to identify the pixel as the most convex point symbol;
  • the Z coordinate of each suspension point in the suspension point set is calculated by the triangulation distance method to obtain the suspension point
  • the three-dimensional coordinates (X aj , Y 0 , Z) of each suspension point in the set is calculated by the triangulation distance method to obtain the suspension point.
  • control system is further configured to recheck the listing status according to the following steps:
  • the light source of the control structure is turned off, the ring light source is activated, and the third sample image taken by the camera is received;
  • the contour area of the third sample image is greater than the threshold, it is determined that the re-inspection result is a successful listing; otherwise, it is determined that the re-inspection result is a listing failure;
  • the total area of the contours other than the sign in the third sample image is less than the threshold.
  • system further includes a voice device
  • control system is further configured to perform:
  • the voice device is controlled to broadcast the prompt information corresponding to the listing status, and the listing status includes successful listing and failed listing.
  • system further includes a hook making machine and a printer
  • control system is further configured to execute:
  • control robot drives the visual positioning device to move to the hollow area of the sample to be labeled, control the hook making machine to make hooks and control the printer to print the signs; the hooks are used to hang the signs on the The three-dimensional coordinates of the best suspension point;
  • the robot is controlled to drive the gripper to move, so that the hook passes through the preset perforation on the sign to complete the card threading action.
  • control system when the sample to be listed needs to be listed on both sides, the control system is further configured to execute:
  • the ring light source is controlled to turn off, and the robot is controlled to return to the initial position.
  • the present application provides a method for listing a wire for use in a listing system for a wire.
  • the listing system for a wire includes a robot, a visual positioning device and a gripper connected to the robot, and the visual positioning device includes a ring A light source, a structured light source, a camera, and a ranging sensor, the method includes:
  • control robot drives the visual positioning device to move to the hollow area of the sample to be listed, and receives the distance between the camera detected by the ranging sensor and the inner wall of the sample to be listed;
  • Re-check the listing status If the result of the re-check is that listing fails, perform listing at the three-dimensional coordinates of the candidate suspension point according to the priority until the listing is successful, then the re-check process ends.
  • the key point of this application is to use machine vision to calculate the best hanging point every time a sign is hung.
  • the first sample image includes the wire and the local area of the packing device. Therefore, it can be determined whether the packing type is a packing tape or a packing line, and the base point (u 0 , v 0 ) in the packing line or the packing belt can be obtained.
  • the base point (u 0 , v 0 ) is the reference point for later determining the best suspension point; then, turn off the ring light source and start the structured light source, so that the second sample image taken includes the light strip corresponding to the packing device and the structured light stripe corresponding to the wire, and then you can
  • the optimal suspension point and candidate suspension point are calculated, and their three-dimensional coordinates in the world coordinate system XYZ are used to control the movement of the robot to make the gripper carry
  • the listing can be completed by hanging at the three-dimensional coordinates of the best hanging point. However, in practical applications, due to factors such as the error of the optimal suspension point, the listing may fail.
  • the listing status needs to be rechecked to determine whether the listing is successful. If the listing fails at the optimal suspension point, there is no need to repeat the listing. In the above listing process, based on the candidate suspension point ranked next to the best suspension point, the suspension trajectory is generated again, the listing is performed, and then it is determined whether the listing is successful at the candidate suspension point, and so on. If the listing still fails, then The subsequent candidate suspension points are listed in the order of priority until the listing is successful, and the re-inspection and listing process ends.
  • This application uses machine vision to automatically determine the best hanging point, and automatically executes the listing operation through a robot, which realizes the automation and intelligent listing of wire coils, reduces manual labor and safety hazards, improves the efficiency of listing, and avoids misplacing and leakage. Hang signs.
  • candidate suspension points there is no need to repeat the complicated calculation process after the listing of the best suspension points fails, which improves the efficiency of listing and reduces the working rhythm of the listing system.
  • Figure 1 exemplarily shows a schematic diagram of the overall structure of a wire listing system
  • Figure 2 exemplarily shows a schematic diagram of the connection structure of the visual positioning device
  • Fig. 3 exemplarily shows a schematic diagram of the front structure of the visual positioning device
  • Figure 4 exemplarily shows a schematic diagram of the electrical connection of the wire tagging system
  • Fig. 5 exemplarily shows a flow chart of a method for listing a wire
  • Figure 6(a) exemplarily shows a schematic diagram of a strap template
  • Figure 6(b) exemplarily shows a schematic diagram of a packing line template
  • Figure 7(a) exemplarily shows the calculation result of the time base point when the packing type is packing tape
  • Fig. 7(b) exemplarily shows the calculation result of the time base point when the packing type is the packing line
  • FIG. 8 exemplarily shows a schematic diagram of the second region of interest and its division
  • Fig. 9 exemplarily shows a schematic diagram of dynamic partitioning
  • Figure 10 exemplarily shows a schematic diagram of static partitioning
  • Fig. 11 exemplarily shows a label map of key points in the second region of interest
  • Figure 12 exemplarily shows the third sample image when the listing is successful
  • Fig. 13 exemplarily shows the electrical connection diagram of another wire listing system.
  • the listing in this application is performed after the sample 100 to be listed is packaged.
  • the sample 100 to be listed can be bundled using a packing device.
  • the packing device 101 can be a packing line or a packing belt.
  • the packing line is composed of several strands of thin steel wires. Steel belts of a certain width have different types of packaging when different packaging devices are used.
  • the sample 100 to be listed is a wire coil as an example for description.
  • the tagging system includes a robot 1, a visual positioning device 3 connected with the robot 1, a gripper 4, and a control system 5.
  • one end of the visual positioning device 3 is connected to the robot 1 through the bracket 2
  • the other end of the visual positioning device 3 is connected to the gripper 4, and the gripper 4 and the visual positioning device 3 are arranged adjacently and are facing to be listed.
  • the clamping jaw 4 can clamp the finished hook from the hook making machine 7, and pass the hook through the preset perforation on the label printed by the printer 8 (ie, the card threading action). Clamp the sign to perform the hanging operation.
  • the visual positioning device 3 includes a ring light source 31, a structured light source 32, a camera 33, and a distance measuring sensor 34.
  • the ring light source 31, the structured light source 32, the camera 33 and the distance measuring sensor 34 can be installed on the same mounting board.
  • the ring light source 31 and the structured light source 32 can be distributed left and right, the camera 33 and the structured light source 32 can be arranged up and down, the camera 33 is parallel to the inner surface of the sample 100 to be listed, but in actual working conditions, due to the installation error of the camera 33 , Resulting in a certain degree of tilt of the camera 33. In this case, it is necessary to perform affine transformation on the image taken by the camera 33 before performing subsequent preprocessing and calculation of the optimal suspension point.
  • the first sample image can be taken. Since the first sample image includes the wire and the local area of the packing device, it can be used to determine whether the packing type is a packing tape or a packing line, and obtain the packing
  • the base point (u 0 , v 0 ) in the line or packing tape, the base point (u 0 , v 0 ) is the reference point for determining the best suspension point subsequently.
  • the structured light source 32 can generate structured light when it is activated. Based on the characteristic principle that the structured light is modulated by the surface of the sample to be listed, the structured light is reflected by the surface of the sample to be listed 100.
  • the camera 33 receives the second sample image so that the captured second sample image has multiple structured light stripes carrying the real deformation characteristics of the sample surface and the light band corresponding to the packing device 101, according to the characteristics and base points (u 0 , v 0 of the second sample image). ) To calculate the best suspension point and candidate suspension points to form a suspension point set, and calculate the three-dimensional coordinates of each suspension point in the suspension point set in the world coordinate system XYZ. According to the three-dimensional coordinates of the robot and the best suspension point, A suspension trajectory can be generated. After the robot 1 moves according to the suspension trajectory, the tag carried by the gripper 4 is suspended at the three-dimensional coordinate position of the optimal suspension point, thereby completing the tagging.
  • the camera 33 may be an industrial camera.
  • the candidate suspension points in the suspension point set are used during the re-examination of the listing, which will be described in detail below.
  • the sample to be listed 100 can be approximated as a hollow cylindrical shape, and the label needs to be hung on the inner wall of the hollow area of the sample to be listed 100, and the robot 1 is controlled to drive the visual positioning device 3 to move to the hollow area of the sample to be listed 100, hollow
  • the area has a certain inner diameter, so it is necessary to adjust the distance between the camera 33 and the inner wall of the sample 100 to be listed to ensure the image shooting effect.
  • the distance measuring sensor 34 is used to detect the distance between the camera 33 and the inner wall of the sample 100 to be listed, and the control system 5 determines whether the distance is a preset distance.
  • the preset distance here can be set as appropriate according to the inner diameter of the sample 100 to be listed.
  • the ring light source 31 can be started to take the first sample image; if the distance is not equal to the preset Distance, it is necessary to continue to control the robot 1 to adjust the distance until the distance is equal to the preset distance.
  • the distance measuring sensor 34 may be a laser distance meter.
  • control system 5 is electrically connected to the robot 1, the ring light source 31, the structured light source 32, the camera 33, the distance measuring sensor 34, the voice device 6, the hook making machine 7 and the printer 8 respectively.
  • the control system 5 is used to plan the motion trajectory of the robot 1 and control the robot 1 to move according to the motion trajectory.
  • the robot 1 can choose six-axis robots; the control system 5 is also used to control the ring light source 31, the structured light source 32 and the camera 33 to start and close; the control system 5 is also used to receive and locate the shooting position of the camera 33 according to the distance between the camera 33 detected by the distance measuring sensor 34 and the inner wall of the sample to be listed; the control system 5 is also used to receive and shoot according to the camera 33 To calculate the set of hanging points and the re-examination of the listing status; the control system 5 is also used to control the listing status broadcast by the voice device 6 during the re-inspection process.
  • the listing status includes successful listing and listing failure.
  • the voice device 6 can be set in On the robot 1, or in other scenes that require voice prompts; the control system 5 is also used to control the hook making machine 7 to make hooks and the printer 8 to print signs, and the control system 5 also responds to the production success instruction sent by the hook making machine 7 In response to the successful printing instruction sent by the printer 8, the robot 1 is controlled to drive the gripper 4 to complete the hook gripping and card threading work.
  • the present application also provides an embodiment of a method for listing a wire.
  • the method is used in the above-mentioned listing system for a wire. It is a program step in which the control system 5 is configured to execute, that is, the method is executed by The control system 5, the method includes:
  • Step S10 after the packaging of the sample to be listed is completed, the control robot drives the visual positioning device to move to the hollow area of the sample to be listed, and receives the distance between the camera and the inner wall of the sample to be listed detected by the distance measuring sensor.
  • the control system 5 controls the hook making machine 7 to make hooks, and controls the printer 8 to print the signs.
  • the hooks are used to hang the signs at the three-dimensional coordinates of the best hanging point.
  • the control system 5 controls the gripper 4 to grip the hook from the hook-making machine 7, and controls the robot 1 to drive the gripper 4 to move so that the hook passes through the pre-printed label. Place the perforation to complete the action of piercing the card, thereby connecting the hook and the sign.
  • the sign is listed, one end of the hook is connected with the sign, and the other end of the hook is hung on the packing line/packing belt.
  • Step S20 When the distance is the preset distance, the ring light source and the camera are activated, and the first sample image taken by the camera is received.
  • the shooting position is positioned, so that the visual positioning device 3 is adjusted to a suitable position for shooting the sample image.
  • the first sample image, the second sample image, and the third sample image taken during the re-inspection involved in this embodiment are all collected by the camera 33 at the shooting position.
  • Step S30 Determine the packing type and the base point (u 0 , v 0 ) in the image coordinate system uov according to the first sample image.
  • the control system 5 After the control system 5 receives the first sample image, it can first define the first region of interest (Region Of Interest, ROI) in the first sample image.
  • the first region of interest includes the packing area and the local wire area. Setting the first region of interest can reduce the amount of image processing and calculations, and improve calculation efficiency.
  • the image of the first region of interest can also be preprocessed such as grayscale, filtering, corrosion, and binarization. Specifically, image preprocessing can be performed according to actual needs. These image preprocessing methods are all existing technologies. This embodiment will not be described in detail.
  • a template matching algorithm is used to determine the packing type and base point (u 0 , v 0 ), which requires pre-obtaining the packing template.
  • a universal packing can be made in advance Templates, pre-acquisition the images of the packing line and the packing tape, and then preprocessing the images of the packing line and the packing tape to obtain the packing template. It should be noted that if the camera 33 has the problem of oblique installation, the images of the packing line and the packing tape need to be subjected to affine transformation before preprocessing.
  • the packing template includes the packing belt template as shown in Fig. 6(a) and the packing line template as shown in Fig. 6(b).
  • the first region of interest must be larger than and contain the area of the packaging template.
  • the template matching algorithm it can be determined that the first region of interest contains the packaging belt area or Packing line area to determine the type of packing.
  • Figure 7(a) and Figure 7(b) show the target area under the packaging tape and packaging line types, respectively.
  • the center point of is used as the base point (u 0 , v 0 ).
  • the first sample image is used to determine the packaging type and base point (u 0 , v 0 ), and the base point (u 0 , v 0 ) is the reference point for subsequent calculations of the best hanging point of the sign.
  • Step S40 control the ring light source to turn off, start the structured light source, and receive the second sample image taken by the camera
  • the second sample image has the same size as the first sample image, and the shooting area is also the same. Therefore, the second sample image and the first sample image can be unified into the same image coordinate system uov, so that the same spatial point in the shooting area has the same pixel coordinates in the second sample image and the first sample image.
  • the image The upper left corner is used as the origin to establish the image coordinate system uov.
  • the second region of interest corresponds to the position of the first region of interest and has the same size, which is equivalent to being in the same image coordinate system uov
  • the first region of interest in the first sample image is mapped to the second sample image to form a second region of interest.
  • the coordinates of the pixels at the same position in the first region of interest and the second region of interest are the same, so that the first The second region of interest shares the base point (u 0 , v 0 ) with the first region of interest.
  • preprocessing such as grayscale, filtering, corrosion, and binarization can be performed on the second region of interest.
  • Step S50 According to the second sample image and the base point (u 0 , v 0 ), calculate the best suspension point in the image coordinate system uov and a number of candidate suspension points sorted by priority.
  • step S60 the robot is controlled to move, and the sign clamped by the gripper is suspended to the position where the optimal suspension point corresponds to the three-dimensional coordinates of the world coordinate system XYZ.
  • the second region of interest includes the light band corresponding to the packing area, and the position of the base point (u 0 , v 0 ) is in the light band. 2.
  • the region of interest also includes a number of structured light fringes distributed on the upper side (along the v-axis negative direction) and the lower side (along the v-axis positive direction) of the light strip.
  • the structured light fringe is the image characteristic of the wire under structured light. It can be seen from Figure 8 that each structured light stripe is composed of several segments of sub-stripes.
  • the characteristic of structured light that is deformed by modulation on the surface of the sample to be listed is: due to the unevenness of the surface of the sample to be listed, the structured light irradiated on the surface of the sample to be listed will be phase-modulated, resulting in the more protruding part of the sample to be listed The lower the corresponding light stripe pixel point, on the contrary, the more recessed part of the sample to be listed, the higher the corresponding light stripe pixel point.
  • the target structured light stripe at the base point (u 0 , v 0 ) along either side of the positive or negative direction of the v-axis refer to the shooting angle shown in Figure 1, and in actual working conditions, you want to hang the sign on the packing device 101 the left, then needs to point (u 0, v 0) selecting a target light pattern along one side of the v-axis negative direction; to to the right side plate suspended from the packaging apparatus 101, then the point (u 0 , V 0 ) Select the target structured light stripe along the positive side of the v axis. Then, based on the characteristics of structured light being modulated by the surface of the sample to be listed and deformed, the optimal suspension point and candidate suspension point are calculated.
  • the center points of the sub-stripes in the target structured light stripe are sorted in ascending order according to the v coordinate value ( Ascending order), get the first M center points in the sorting result to form a concave point set P(u aj , v aj ), the concave point set still maintains the same sorting order of increasing v coordinate value, M is included in the concave point set
  • the preset number of pixels is also the preset number of suspension points included in the suspension point set.
  • j is the serial number of each pixel in the concave point set. The smaller the serial number, the higher the priority, 1 ⁇ j ⁇ M.
  • (u a1 , v a1 ) is the global most concave point;
  • the concave points (u ak , v ak ) in the concave point set except the global most concave point are candidate concave points, where 2 ⁇ k ⁇ M, k is the serial number of the candidate suspension point, and a is a symbol used to identify each pixel in the pit set as a pit.
  • the center points of the sub-stripes in the target structured light stripe can be sorted in descending order according to the v coordinate value (descending order).
  • the last M centers in the sorting result are obtained.
  • the M center points are reordered according to the increasing order of the coordinate value of v to form the concave point set P(u aj , v aj ), and the first concave point in the concave point set is sorted The point is the most concave point in the world.
  • One-side hanging is to hang a label on the hollow inner wall of the sample.
  • the direction of the label relative to the packing position can be on the left or right side of the packing line/packing belt; when hanging on both sides, hang on the hollow inner wall of the sample
  • the two sign suspension points have a 180-degree rotation angle compared to the central axis. Therefore, when the first side tagging is completed, the robot 1 needs to be controlled to rotate 180 degrees, driving the camera 33 to change the shooting angle of 180 degrees. This is bound to It will cause the image coordinate system to be reversed.
  • the first side is listed on the left side of the packing line/packing belt, then the second side is listed on the right side of the packing line/packing belt to ensure the accuracy of the hanging position of the sign.
  • the camera 33 since the camera 33 may be installed at an angle, and the suspended wire surface is an uneven curved surface, it will also affect the accuracy of the suspension position determination.
  • this embodiment performs partition processing on the second region of interest, traverses each pixel in the second region of interest, and obtains a packed area including the base point (u 0 , v 0) according to the filtered white pixels. And a number of partitions are constructed on both sides of the packing area along the positive and negative directions of the v-axis.
  • the second area of interest is composed of many white pixels and black pixels.
  • the white area formed by the white pixels includes the packed area (the thicker strip of light in the middle) and the structured light stripes, black
  • the black area formed by the pixels is the sample background.
  • a gray threshold can be preset, and each pixel can be distinguished by the gray threshold. , So as to separate the white area and the black area. It can be seen from Figure 8 that there is a certain distance between the light strips and each structured light stripe in the v-axis direction, and the light strips and each structured light stripe can be separated by dividing lines. The result of dividing the four dividing lines in Figure 8 It is: the light band in the middle is the packing area, and there are 2 partitions on the upper and lower sides of the packing area.
  • the white end point of the middle light band that is, the pixel point (u 1 , v 1 ) with the largest v coordinate value on the light band
  • the white starting point that is, the pixel point (u 2 , v 2 ) with the smallest v coordinate value in the structured light stripe under the light band
  • the dynamic partition method is more intelligent and has higher precision.
  • a static partitioning method can be used.
  • the base point (u 0 , v 0 ) is used as the reference, and the threshold step y i is used for the second interest.
  • the area is partitioned.
  • the dividing line l v 0 +y i , where 1 ⁇ i ⁇ N, and N is the number of dividing lines.
  • the threshold step size of the first dividing line y 1 -30
  • the threshold step size of the second segmentation line y 2 30
  • the threshold step size y i can be set according to actual experience. After the static partition setting is completed, for example, if you need to hang the sign to the left side of the packaging device in Figure 1, you can directly use the adjacent partition above the base point (u 0 , v 0 ) as the target partition. This method is more efficient. But the accuracy rate is relatively low; or select the area with higher integrity of the structured light stripes from the several areas on the upper side of the base point (u 0 , v 0) as the target area, and the optimal suspension point calculated by this method has higher accuracy . It should be noted that the acquisition rules of the target partition are not limited to those described in this embodiment.
  • the static partitioning method can reduce the system beat and improve the efficiency of listing, but because the dividing line may break the complete structured light stripe, the calculation accuracy is lower than the dynamic partitioning method, so you can choose according to the actual situation.
  • the target partition is selected according to the direction of the listing position relative to the packaging position and the completeness of the structured light stripes in each partition, and the structured light stripes included in the target partition are selected as The target structured light stripes.
  • the upper and lower partitions adjacent to the middle light strip have relatively complete structured light stripes. If the customer specifies that the first side needs to hang the sign on the left side of the packing device, you can choose the middle light
  • the adjacent upper partition is the target partition.
  • the robot 1 is rotated 180 degrees, because the image coordinate system is reversed, the second side selects the lower partition adjacent to the middle light belt as the target partition to ensure that both sides are listed. Accuracy. It should be noted that if single-sided listing is used, partitioning will facilitate the unified management of the listing position and direction of each packaged sample. Of course, there is no need to set up partitions when single-sided listing is used.
  • the three-dimensional coordinates of each suspension point (u aj , v 0 ) in the suspension point set in the world coordinate system XYZ need to be calculated, and then according to the current position and maximum of the gripper 4
  • the three-dimensional coordinates of the best suspension point are used to generate a suspension trajectory, and the robot 1 can be controlled to move according to the suspension trajectory, and the sign clamped in the gripper 4 (in particular through a hook) can be suspended to the three-dimensional coordinates of the best suspension point. So as to complete the listing action.
  • the candidate suspension point is listed to determine whether the listing is successful. When the listing is successful, the re-inspection ends.
  • the world coordinate system XYZ is a coordinate system established in the actual world space. After the image coordinate system uov and the world coordinate system XYZ are established, the image coordinate system uov and the world coordinate system can be obtained in advance according to the imaging characteristics and shooting position of the camera.
  • the mapping relationship between XYZ, any pixel in the image coordinate system uov can find the three-dimensional coordinate point corresponding to the world coordinate system XYZ according to the mapping relationship.
  • the coordinate value (X aj , Y 0 ) of each suspension point (u aj , v 0 ) in the suspension point set in the world coordinate system XYZ can be calculated, that is, it is determined that each suspension point in the suspension point set is in the world X coordinate value and Y coordinate value in the coordinate system XYZ.
  • Optimum suspension point Z coordinate (depth) is the distance from the camera 33 to the packing means may be based on the most bumps 100 in accordance with (u t, v t) and a camera 33, a light source structure 32, a distance measuring sensor 34 and the sample to be listed
  • the point three-dimensional coordinate set P (X aj , Y 0 , Z), and the suspension point three-dimensional coordinate set P (X aj , Y 0 , Z) are stored in the database of the control system 5 for subsequent re-inspection.
  • the center point with the largest v coordinate value is regarded as the most convex point (u t , v t ), if it is sorted according to the v coordinate value ascending (ascending order) , then the last bit of the sort center as most bumps (u t, v t); if the coordinate values are decremented by v (descending order), then the first sorting center point as the outermost bumps (u t , V t ).
  • the purpose of selecting the most convex point (u t , v t ) to participate in the calculation of the Z coordinate of the optimal suspension point is because the most convex point is the most protruding position in the wire coil, and the surface of the wire coil is the most convex
  • the actual distance between the point and the camera 33 is taken as the Z coordinate, which can prevent the robot 1 from colliding with the wire coil during the listing process.
  • the Y coordinate value of each suspension point in the suspension point set is the same as Y 0 , because Y 0 corresponds to the position of the packing line/packing belt; the Z coordinate value of each suspension point in the suspension point set is the same because the Z coordinate value is Based on the most convex point (u t , v t ) and a fixed geometric relationship, calculated by the triangulation method, the most convex point (u t , v t ) cannot be changed to prevent the robot 1 from colliding with the wire, thereby ensuring smooth Listed. Therefore, only the X coordinate values are different between the suspension points in the suspension point set, that is, the suspension points only have a displacement in the X axis direction of the world coordinate system XYZ. Among them, t is a symbol used to identify a pixel as the most convex point.
  • Step S70 Re-check the listing status. If the result of the re-check is that listing fails, perform listing at the three-dimensional coordinates of the candidate suspension point according to the priority until the listing is successful, then the recheck process ends.
  • the visual positioning device 3 when the listing is completed, the visual positioning device 3 is located near the optimal suspension point, and the visual positioning device 3 needs to be returned to the shooting position corresponding to the first sample image/the second sample image , That is, return to the shooting position located according to the distance measuring sensor 34 and the preset distance, control the structure light source 32 to turn off, and start the ring light source 31, the camera 33 captures the third sample image, the third sample image includes the packing device and the hook
  • the local area of the listing and the wire can then be pre-processed such as grayscale, filtering, and corrosion on the third sample image, and the third sample image does not need to be subjected to affine transformation.
  • the contour area of the third sample image is calculated. If the contour area of the third sample image is greater than the threshold, it is determined that the re-examination result is a successful listing; otherwise, it is determined that the re-inspection result is a listing failure.
  • the total area of the contours other than the sign in the third sample image is less than the threshold, and the threshold can be set according to the contour area of the sign, for example, it can be set to the area of the sign successfully hung in the third sample image. A quarter.
  • the signage area accounts for a relatively high proportion, while the total area of the contours other than the signage is relatively small and less than the set threshold. If the listing fails, there is no signage in the third sample image.
  • the contour area of the wire and the packing device is less than the threshold, it can be judged as a listing failure; if there is a sign in the third sample image, the sign area is obviously larger than the set threshold, and it can be judged as a successful listing.
  • the re-inspection is ended; if the listing fails at the determination of the best suspension point, the conventional way is to repeat steps S10 to S70 and restart Take the first sample image and the second sample image to calculate the three-dimensional coordinates of the best suspension point, and perform the listing action again. Although the listing accuracy is higher in this way, the re-inspection requires complex calculation and control procedures, resulting in listing efficiency Low, long working cycle of the listing system.
  • the accuracy requirements for listing may be relatively low, such as the size of the wire coil is larger, the gap between the wires is larger, or the diameter of the packing line/packing belt is small. In this case, the system is more Most places focus on listing efficiency to reduce the system's working rhythm and ensure production efficiency.
  • Fig. 12 shows a schematic diagram of the third sample image when the tag is successfully listed on the wire, in which the digital sign is the sign, and a hook is connected between the sign and the packing line.
  • the ring light source 31 is controlled to turn off and the robot 1 is controlled to return to the initial position; if double-sided tagging is adopted, the robot 1 is controlled to rotate after the tagging is successful on one side 180°, so that the camera 33 faces the other side, and then perform the procedures of hanging signs and re-inspection shown in steps S10 to S70 on the other side, and after the other side is also successfully listed, then control the ring light source 31 Close and control the robot 1 to return to the initial position. After the one-sided/double-sided listing of the current sample to be listed is successful, the current sample to be listed can be moved out of the listing station, and the next sample to be listed can be moved to the listing station, and then repeat the above method process.
  • the voice device 6 is controlled to broadcast the prompt information corresponding to the listing status, and the listing status includes successful listing and failed listing. Regardless of whether it is one-sided or two-sided listing, the step of determining whether the contour area of the third sample image is greater than the threshold is performed every time the re-examination is performed, and the voice device 6 can be controlled to broadcast the corresponding prompt information according to the determined listing status.
  • the prompt information can be preset in the voice device 6. As an example, the prompt information can be set to be that a certain sample has successfully hung a sign, or a certain sample has failed to hang a sign, so that on-site personnel can know the listing status and re-inspection process.
  • the specific content of the prompt information can be set according to actual conditions, which is not limited in this embodiment.
  • the control system 5 can adopt a combined control structure of a PLC controller 51 + a computer 52, the camera 33, the printer 8 and the PLC controller 51 are electrically connected to the computer 52, and the robot 1,
  • the ring light source 31, the structured light source 32, the distance measuring sensor 34, the voice device 6 and the hook making machine 7 are electrically connected to the PLC controller 51, respectively.
  • the combined control flow of PLC controller 51 + computer 52 is as follows:
  • the PLC controller 51 controls the hook making machine 7 to make hooks, and the computer 52 controls the printer 8 to print the signs; when the PLC controller 51 receives the data sent by the hook making machine 7 Make a successful instruction to control the movement of the robot 1 to drive the gripper 4 to grip the hook; when the computer 52 receives the printing success instruction fed back by the printer, the computer 52 sends the first control instruction to the PLC controller 51; the PLC controller 51 responds to the first The control instruction is to control the robot 1 to drive the gripper to complete the card threading action.
  • the PLC controller 51 controls the robot 1 to drive the visual positioning device to move to the hollow area of the sample to be listed 100, and activates the ranging sensor 34; the PLC controller 51 receives the camera 33 and the camera 33 detected by the ranging sensor 34 The distance between the inner walls of the sample to be listed, and determine whether the distance is a preset distance, if not, continue to adjust the position of the visual positioning device, if the distance is a preset distance, the PLC controller 51 controls the ring light source 31 and the camera 33 is activated, so that the camera 33 takes the first sample image; after the computer 52 receives the first sample image taken by the camera 33, it determines the packaging type and the base point (u 0 , v 0 ) in the image coordinate system uov; The PLC controller 51 sends a second control instruction, and the PLC controller 51 responds to the second control instruction, controls the ring light source 31 to turn off, and starts the structured light source 32 so that the camera 33 captures the second sample image; the computer 52
  • the PLC controller 51 responds to the third control instruction according to the optimal
  • the three-dimensional coordinates of the suspension point (X a1 , Y 0 , Z) generate the first suspension trajectory, and the robot 1 is controlled to move according to the first suspension trajectory, and the sign clamped in the gripper 4 is suspended to the three-dimensional coordinate of the best suspension point Place, complete the listing action.
  • the robot 1 feeds back the listing completion instruction to the PLC controller 51, and then starts the recheck: the PLC controller 51 controls the robot 1 to move, so that the visual positioning device 3 returns to the shooting position corresponding to the first sample image; the PLC controller 51 controls the structure
  • the light source 32 is turned off, and the ring-shaped light source 31 is activated so that the camera 33 can take the third sample image; after the computer 52 receives the third sample image taken by the camera 33, it preprocesses the third sample image to calculate the contour area of the third sample image, And judge whether the contour area of the third sample image is greater than the threshold; if the contour area of the third sample image is greater than the threshold, the re-inspection result is judged to be successful at the best suspension point; otherwise, the re-inspection result is judged to be the best suspension point
  • the computer 52 retrieves the three-dimensional coordinates of the candidate suspension points in the three-dimensional coordinate set of the suspension points according to the priority, and generates the fourth control instruction according to the three-dimensional
  • the computer 52 sends a fifth control instruction to the PLC controller 51; the PLC controller 51 responds to the fifth control instruction to control the ring light source 31 to turn off and control the robot 1 Return to the initial position; if double-sided tagging is used, after one side tagging is successful, the computer 52 sends a sixth control instruction to the PLC controller 51; the PLC controller 51 responds to the sixth control instruction and controls the robot 1 to rotate 180° to Turn the camera 33 toward the other side, and then perform the same procedures for hanging signs and re-inspection on the other side, and after the other side is also successfully listed, the computer 52 sends the seventh control instruction to the PLC controller 51; the PLC controller In response to the seventh control instruction, 51 controls the ring light source 31 to turn off, and controls the robot 1 to return to the initial position. Regardless of whether single-sided or double-sided listing is used, during the re-inspection, the computer 52 sends a fifth control instruction to the PLC controller 51; the PLC controller 51 responds to the fifth control
  • the current sample to be listed can be moved out of the listing station, and the next sample to be listed can be moved to the listing station, and then repeat the above method.
  • the key to this application is to use machine vision to calculate the optimal suspension point every time the sign is hung.
  • a sample image because the first sample image includes the wire and the local area of the packing device, it can be determined whether the packing type is a packing tape or a packing line, and the base point (u 0 , v 0 ) in the packing line or the packing tape can be obtained,
  • the base point (u 0 , v 0 ) is the reference point for subsequent determination of the optimal suspension point;
  • a second sample image is taken, and the second sample image includes the light band corresponding to the packing device and the structured light fringe corresponding to the wire.
  • the listing may fail. At this time, the listing status needs to be rechecked to determine whether the listing is successful. If the listing fails at the optimal suspension point, there is no need to repeat the above listing. The process is based on the candidate suspension point ranked next to the best suspension point, generates the suspension trajectory again, executes the listing, and then determines whether the listing is successful at the candidate suspension point, and so on. If the listing still fails, the priority will be followed The subsequent candidate suspension points are listed in order of level until the listing is successful, and the re-inspection and listing process ends.
  • This application uses machine vision to automatically determine the best suspension point, and automatically executes the listing operation through a robot, which realizes the automated and intelligent listing of wire coils, reduces manual labor and safety hazards, and improves the efficiency of listing and the positioning of the optimal suspension point. Accuracy, can avoid hanging wrong signs and missing signs.
  • this application calculates the three-dimensional coordinate collection of the suspension point and stores it in the library, so that after the listing of the optimal suspension point fails, there is no need to repeat the complicated calculation process, which can shorten the re-inspection cycle, can successfully list more quickly, and improve the efficiency of listing. , Which reduces the working beat of the listing system, thereby improving the production efficiency of wire coils.
  • the technical solution described in this application can be adaptively applied to the listing control of other types of samples, and is not limited to wire coils.

Abstract

一种线材的挂牌系统及方法，涉及视觉检测技术领域，在待挂牌样品（100）打包完成后，控制机器人（1）带动视觉定位装置（3）移动至待挂牌样品（100）的中空区域内，接收测距传感器（34）检测的相机（33）与待挂牌样品（100）内壁的间距；当间距为预设距离时，启动环形光源（31）和相机（33），接收相机（33）拍摄的第一样品图像；根据第一样品图像确定打包类型和基点（u 0，v 0）；控制环形光源（31）关闭，启动结构光源（32），接收相机（33）拍摄的第二样品图像；根据第二样品图像和基点（u 0，v 0），计算最佳悬挂点和按优先级排序的若干候选悬挂点；将夹爪（4）夹持的标牌悬挂至最佳悬挂点的三维坐标处；对挂牌状态进行复检，如果复检结果为挂牌失败，按照优先级在候选悬挂点的三维坐标处进行挂牌直至挂牌成功，复检结束。

Description

一种线材的挂牌系统及方法 技术领域
本申请涉及视觉检测技术领域，尤其涉及一种线材的挂牌系统及方法。
背景技术
在钢铁行业，目前现场生产的钢筋线材绕成盘卷后，经悬挂流水线先流转至打包工位，由多股钢线或钢带自动打包捆绑之后，再流转至称重工位，称重结束后，在人工操作台内由打印机打印金属标牌，在等待下一卷成品盘卷称重时，由人工对当前盘卷操作挂牌。挂牌时人工拿取标牌和挂钩，先将标牌挂至挂钩上，再将挂钩悬挂于线材盘卷内圈距两端约300mm的不同打包钢线或钢带上，从而完成挂牌。
目前的挂牌需要操作人员从操作室窗口拿取标牌和挂钩，再走至称重下一站，站立于线材盘卷的一端，徒手伸进线材盘卷内部进行悬挂标牌。由于流水线一直处于运行中，危险系数较大，并且在线材盘卷生产量较大时，操作人员来回频繁跑动并悬挂标牌，长时间作业，劳动强度较大，易产生作业疲劳。人工挂标牌存在不可控性，悬挂位置(上方或下方)和悬挂距离的远近等都存在不一致性，且人工长时间作业，容易产生挂错标牌和漏挂标牌问题。
发明内容
为解决上述背景技术中所述的问题，本申请提供一种线材的挂牌系统及方法。
第一方面，本申请提供一种线材的挂牌系统，包括：
机器人和与所述机器人连接的视觉定位装置和夹爪，所述视觉定位装置包括环形光源、结构光源、相机和测距传感器；以及，控制系统，被配置为执行如下程序步骤：
在待挂牌样品打包完成以后，控制机器人带动视觉定位装置移动至所述待挂牌样品的中空区域内，并接收所述测距传感器检测的相机与待挂牌样品内壁的间距；
当所述间距为预设距离时，启动环形光源和相机，接收所述相机拍摄的第一样品图像；
根据所述第一样品图像确定打包类型和图像坐标系uov中的基点(u 0，v 0)；
控制环形光源关闭，启动结构光源，接收所述相机拍摄的第二样品图像；
根据所述第二样品图像和所述基点(u 0，v 0)，计算图像坐标系uov中的最佳悬挂点和按优先级排序的若干候选悬挂点；
控制机器人移动，将所述夹爪夹持的标牌悬挂至所述最佳悬挂点对应于世界坐标系XYZ的三维坐标处；
对挂牌状态进行复检，如果复检结果为挂牌失败，按照所述优先级在所述候选悬挂点的三维坐标处进行挂牌直至挂牌成功，则复检过程结束。
在一些实施例中，所述控制系统被进一步配置为按如下步骤计算确定打包类型和图像坐标系中的基点(u 0，v 0)：
在所述第一样品图像中划定第一感兴趣区域；
将所述第一感兴趣区域与预先获取的打包模板进行匹配，确定打包类型；所述打包类型包括打包线和打包带；
获取所述第一感兴趣区域与所述打包类型对应的打包模板匹配得到的目标区域；
将所述目标区域的中心点作为所述基点(u 0，v 0)。
在一些实施例中，所述控制系统被进一步配置为按如下步骤计算图像坐标系uov中的最佳悬挂点和候选悬挂点
在第二样品图像中划定第二感兴趣区域；所述第二感兴趣区域与所述第一感兴趣区域的位置对应并且区域大小相同，所述第二感兴趣区域与所述第一感兴趣区域共享基点(u 0，v 0)；
在第二感兴趣区域中，在基点(u 0，v 0)沿v轴正向或负向任一侧选定目标结构光条纹；
对所述目标结构光条纹中各段子条纹的中心点按照v坐标值递增排序，获取排序结果中的前M个中心点并保持排序不变组成凹点集合P(u aj，v aj)；M为凹点集合中包括的像素点的预设数量，j为凹点集合中各像素点的序位号，1≤j≤M；a为用于标识凹点集合中各像素点为凹点的符号；
将坐标(u a1，v 0)作为图像坐标系uov中的最佳悬挂点，将坐标(u ak，v 0)作为图像坐标系uov中的候选悬挂点；其中，2≤k≤M，k为候选悬挂点的序位号。
在一些实施例中，所述控制系统被进一步配置为按照如下步骤选定目标结构光条纹：
遍历第二感兴趣区域中的各个像素点，根据筛选出的白色像素点，得到包括基点(u 0，v 0)的打包区域，以及在所述打包区域沿v轴正向和负向两侧构建若干分区；或者，以基点(u 0，v 0)为基准，通过阈值步长将所述第二感兴趣区域划分为若干分区；
根据挂牌位置相对于打包位置的方向，以及各分区内结构光条纹的完整程度，选取目标分区，并将所述目标分区内包括的结构光条纹选定为所述目标结构光条纹。
在一些实施例中，所述控制系统还被配置为执行：
根据所述目标结构光条纹中各段子条纹的中心点的排序结果，将v坐标值最大的中心点作为最凸点(u t，v t)；t为用于标识像素点为最凸点的符号；
根据图像坐标系uov与世界坐标系XYZ的映射关系，计算悬挂点集合中各悬挂点(u aj，v 0)在世界坐标系XYZ的坐标值(X aj，Y 0)；所述悬挂点集合包括最佳悬挂点(u a1，v 0)和候选悬挂点(u ak，v 0)；
根据最凸点(u t，v t)以及相机、结构光源、测距传感器和待挂牌样品的相对位置参数，利用三角测距法计算悬挂点集合中各悬挂点的Z坐标，从而得到悬挂点集合中各悬挂点的三维坐标(X aj，Y 0，Z)。
在一些实施例中，所述控制系统还被配置为按照如下步骤对挂牌状态进行复检：
控制机器人移动，使视觉定位装置返回至第一样品图像对应的拍摄位置；
控制结构光源关闭，启动环形光源，接收所述相机拍摄的第三样品图像；
对所述第三样品图像进行预处理后，计算所述第三样品图像的轮廓面积；
如果所述第三样品图像的轮廓面积大于阈值，则判定复检结果为挂牌成功；反之，则判定复检结果为挂牌失败；
其中，所述第三样品图像中除标牌之外其他轮廓的总面积小于所述阈值。
在一些实施例中，所述系统还包括语音装置，所述控制系统还被配置为执行：
在复检过程中，控制所述语音装置播报与所述挂牌状态对应的提示信息，所述挂牌状态包括挂牌成功和挂牌失败。
在一些实施例中，所述系统还包括制钩机和打印机，所述控制系统还被配置为执行：
在控制机器人带动视觉定位装置移动至所述待挂牌样品的中空区域内之前，控制所述制钩机制作挂钩，以及控制所述打印机打印标牌；所述挂钩用于将所述标牌悬挂于所述最佳悬挂点的三维坐标处；
响应于接收到所述制钩机发送的制作成功指令，控制所述夹爪从所述制钩机上夹取挂钩；
控制机器人带动夹爪移动，使所述挂钩穿过所述标牌上预置的穿孔，则完成穿牌动作。
在一些实施例中，当需要对待挂牌样品进行双侧挂牌时，所述控制系统还进一步配置为执行：
当一侧挂牌成功后，控制所述机器人旋转180°，以使所述相机朝向另一侧；
对另一侧执行悬挂标牌以及复检流程；
当另一侧挂牌成功后，控制环形光源关闭，并控制所述机器人回到初始位置。
第二方面，本申请提供一种线材的挂牌方法，用于线材的挂牌系统，所述线材的挂牌系统包括机器人和与所述机器人连接的视觉定位装置和夹爪，所述视觉定位装置包括环形光源、结构光源、相机和测距传感器，所述方法包括：
在待挂牌样品打包完成以后，控制机器人带动视觉定位装置移动至所述待挂牌样品的中空区域内，并接收测距传感器检测的相机与待挂牌样品内壁的间距；
当所述间距为预设距离时，启动环形光源和相机，接收所述相机拍摄的第一样品图像；
根据所述第一样品图像确定打包类型和图像坐标系uov中的基点(u 0，v 0)；
控制环形光源关闭，启动结构光源，接收所述相机拍摄的第二样品图像；
根据所述第二样品图像和所述基点(u 0，v 0)，计算图像坐标系uov中的最佳悬挂点和按优先级排序的若干候选悬挂点；
控制机器人移动，将夹爪夹持的标牌悬挂至所述最佳悬挂点对应于世界坐标系XYZ的三维坐标处；
对挂牌状态进行复检，如果复检结果为挂牌失败，按照所述优先级在所述候选悬挂点的三维坐标处进行挂牌直至挂牌成功，则复检过程结束。
本申请的关键在于每次悬挂标牌时需要利用机器视觉计算最佳悬挂点，首先在相机与待挂牌样品内壁的间距为预设距离时，利用环形光源和相机拍摄到第一样品图像，由于第一样品图像包括线材和打包装置的局部区域，因此可以判定打包类型是打包带还是打包线，并获取打包线或打包带中的基点(u 0，v 0)，该基点(u 0，v 0)是后续确定最佳悬挂点的基准点；然后，关闭环形光源，启动结构光源，使得拍摄的第二样品图像中包括打包装置对应的光带以及线材对应的结构光条纹，然后即可根据第二样品 图像的特征和基点(u 0，v 0)来计算出最佳悬挂点和候选悬挂点，及其在世界坐标系XYZ中的三维坐标，通过控制机器人移动，使夹爪携带的挂牌被悬挂在最佳悬挂点的三维坐标处，即可完成挂牌。然而，在实际应用中，由于最佳悬挂点的误差等因素，可能导致挂牌失败，这时需要对挂牌状态进行复检，以确定是否挂牌成功，如果在最佳悬挂点挂牌失败，则无需重复上述挂牌流程，而是根据排序在最佳悬挂点下一位的候选悬挂点，再次生成悬挂轨迹，执行挂牌，然后判定在该候选悬挂点是否挂牌成功，以此类推，若仍挂牌失败，则按照优先级顺序对之后的候选悬挂点进行挂牌，直至挂牌成功，复检和挂牌流程结束。本申请利用机器视觉自动确定最佳悬挂点，并通过机器人自动执行挂牌操作，实现了线材盘卷的自动化、智能化挂牌，减少人工劳作和安全隐患，提高了挂牌效率，避免挂错标牌和漏挂标牌。此外，通过设置候选悬挂点，使得在最佳悬挂点挂牌失败后，无需重复复杂的计算流程，提高了挂牌效率，降低了挂牌系统的工作节拍。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示例性示出了一种线材的挂牌系统的整体结构示意图；
图2示例性示出了视觉定位装置的连接结构示意图；
图3示例性示出了视觉定位装置的正面结构示意图；
图4示例性示出了线材的挂牌系统的电连接示意图；
图5示例性示出了一种线材的挂牌方法的流程图；
图6(a)示例性示出了打包带模板的示意图；
图6(b)示例性示出了打包线模板的示意图；
图7(a)示例性示出了打包类型为打包带时基点的计算结果；
图7(b)示例性示出了打包类型为打包线时基点的计算结果；
图8示例性示出了第二感兴趣区域及其分区示意图；
图9示例性示出了动态分区的原理图；
图10示例性示出了静态分区的原理图；
图11示例性示出了第二感兴趣区域中关键点的标记图；
图12示例性示出了挂牌成功时的第三样品图像；
图13示例性示出了另一种线材的挂牌系统的电连接示意图。
图中，1-机器人；2-支架；3-视觉定位装置，31-环形光源，32-结构光源，33-相机，34-测距传感器；4-夹爪；5-控制系统，51-PLC控制器，52-计算机；6-语音装置；7-制钩机；8-打印机；100-待挂牌样品；101-打包装置。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整的描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。 基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本申请中挂牌是在待挂牌样品100打包之后执行的，待挂牌样品100可以使用打包装置进行捆绑，打包装置101可以是打包线或打包带，打包线由若干股细钢丝组成，打包带是具有一定宽度的钢带，采用不同的打包装置即具有不同的打包类型。本申请实施例中以待挂牌样品100为线材盘卷为例进行说明。
如图1～图4所示，本申请提供一种线材的挂牌系统的实施例，挂牌系统包括机器人1、与机器人1连接的视觉定位装置3和夹爪4，以及控制系统5。在图2的示例中，视觉定位装置3的一端与机器人1通过支架2连接，视觉定位装置3的另一端与夹爪4连接，夹爪4和视觉定位装置3相邻设置并且都朝向待挂牌样品100，在具体实现中，夹爪4可以从制钩机7处夹取制作完成的挂钩，并将挂钩穿过打印机8打印的标牌上预置的穿孔(即穿牌动作)，之后即可夹持标牌执行悬挂操作。
在图3的示例中，视觉定位装置3包括环形光源31、结构光源32、相机33和测距传感器34，环形光源31、结构光源32、相机33和测距传感器34可以安装于同一个安装板上，环形光源31和结构光源32可以呈左右分布，相机33与结构光源32可以呈上下布置，相机33与待挂牌样品100的内表面平行，但在实际工况中，由于相机33存在安装误差，导致相机33呈现一定程度的倾斜，这种情况下，需要先对相机33拍摄的图像进行仿射变换，才能进行后续预处理和最佳悬挂点的计算等步骤。
对于环形光源31与相机33的配合，可以拍摄出第一样品图像，由于第一样品图像包括线材和打包装置的局部区域，因此可用于判定打包类型是打包带还是打包线，并获取打包线或打包带中的基点(u 0，v 0)，该基点(u 0，v 0)是后续确定最佳悬挂点的基准点。对于结构光源32和相机33的配合，结构光源32启动时可以产生结构光，基于结构光受待挂牌样品表面的调制而发生形变的特性原理，结构光经待挂牌样品100的表面反射后，被相机33接收，从而使拍摄到的第二样品图像具有携带样品表面真实形变特征的多条结构光条纹以及打包装置101对应的光带，根据第二样品图像的特征和基点(u 0，v 0)来计算出最佳悬挂点和候选悬挂点，组成悬挂点集合，并计算出悬挂点集合中的各个悬挂点在世界坐标系XYZ中的三维坐标，根据机器人和最佳悬挂点的三维坐标，可以生成一条悬挂轨迹，机器人1按照该悬挂轨迹移动后，使夹爪4携带的挂牌被悬挂在最佳悬挂点的三维坐标位置处，从而完成挂牌。相机33可以采用工业相机。悬挂点集合中的候选悬挂点是在挂牌复检时使用，具体将在下文详细说明。
参照图1，待挂牌样品100可以近似为中空的圆柱形，需要将标牌悬挂在待挂牌样品100中空区域的内壁上，控制机器人1带动视觉定位装置3向待挂牌样品100的中空区域移动，中空区域具有一定的内径，因此需要调节好相机33与待挂牌样品100内壁之间的距离，保证图像拍摄效果。测距传感器34用于检测相机33与待挂牌样品100内壁的间距，控制系统5判断所述间距是否为预设距离，这里所述的预设距离可以根据待挂牌样品100的内径酌情设定。当所述间距为预设距离时，认为视觉定位装置3已达合适的拍摄位置，拍摄位置定位完成，即可启动环形光源31进行第一样品图像的拍摄；如果所述间距不等于预设距离，则需要继续控制机器人1调节所述间距，直至所述间距等于预设距离为止。作为示例，测距传感器34可选择激光测距仪。
在图4的示例中，控制系统5分别与机器人1、环形光源31、结构光源32、相机33、测距传感器34、语音装置6、制钩机7和打印机8电连接。控制系统5用于规划机器人1的运动轨迹，并控制机器人1按照运动轨迹移动，作为示例，机器人1可选择六轴机器人等类型；控制系统5还用于控制环形光源31、结构光源32和相机33的 启动和关闭；控制系统5还用于接收并根据测距传感器34检测的相机33与待挂牌样品内壁的间距，定位相机33的拍摄位置；控制系统5还用于接收并根据相机33拍摄的样品图像，来计算悬挂点集合以及挂牌状态的复检；控制系统5还用于复检过程中控制语音装置6播报的挂牌状态，挂牌状态包括挂牌成功和挂牌失败，语音装置6可设置在机器人1上，或者其他需要进行语音提示的场景中；控制系统5还用于控制制钩机7制作挂钩以及控制打印机8打印标牌，同时控制系统5还响应于制钩机7发送的制作成功指令和打印机8发送的打印成功指令，控制机器人1带动夹爪4完成挂钩夹取和穿牌工作。
如图5所示，本申请还提供一种线材的挂牌方法的实施例，该方法用于如上所述线材的挂牌系统，是其中控制系统5被配置执行的程序步骤，即该方法执行主体为控制系统5，所述方法包括：
步骤S10，在待挂牌样品打包完成以后，控制机器人带动视觉定位装置移动至所述待挂牌样品的中空区域内，并接收测距传感器检测的相机与待挂牌样品内壁的间距。
挂牌必须是在待挂牌样品打包完成后续执行的，当然在打包完成之后，可以执行如称重等所需流程之后再执行挂牌，具体以实际工况需求为准。在控制机器人1移动至待挂牌样品100的中空区域内之前，控制系统5控制制钩机7制作挂钩，以及控制打印机8打印标牌，所述挂钩用于将标牌悬挂于最佳悬挂点的三维坐标处，控制系统5响应于接收到制钩机7发送的制作成功指令，控制夹爪4从制钩机7上夹取挂钩，并控制机器人1带动夹爪4移动，使挂钩穿过标牌上预置的穿孔，完成穿牌动作，从而将挂钩和标牌连接在一起，挂牌时挂钩的一端与标牌连接，挂钩的另一端悬挂在打包线/打包带上。
步骤S20，当所述间距为预设距离时，启动环形光源和相机，接收所述相机拍摄的第一样品图像。
通过测距传感器34以及预设距离，定位拍摄位置，从而将视觉定位装置3调整到合适拍摄样品图像的位置。本实施例中涉及的第一样品图像、第二样品图像以及复检时拍摄的第三样品图像都是相机33在该拍摄位置处采集得到。
步骤S30，根据所述第一样品图像确定打包类型和图像坐标系uov中的基点(u 0，v 0)。
控制系统5接收第一样品图像后，可以先在第一样品图像中划定第一感兴趣区域(Region Of Interest，ROI)，第一感兴趣区域中包括打包区域和局部线材区域，通过设定第一感兴趣区域可以降低图像处理和运算量，提高运算效率。此外，还可以对第一感兴趣区域的图像进行灰度化、滤波、腐蚀和二值化等预处理，具体可以根据实际所需进行图像预处理，这些图像预处理方式均为现有技术，本实施例不再具体说明。
本实施例中采用模板匹配算法确定打包类型和基点(u 0，v 0)，这就需要预先获取打包模板，当线材盘卷采用统一的打包线或打包带时，可以预先制作出通用的打包模板，预先采集打包线和打包带的图像，然后对打包线和打包带的图像进行预处理，从而得到打包模板。需要说明的是，如果相机33存在倾斜安装的问题，则打包线和打包带的图像在预处理之前还需要进行仿射变换。打包模板包括如图6(a)所示的打包带模板和如图6(b)所示的打包线模板。
将第一感兴趣区域与预先获取的打包模板进行匹配，第一感兴趣区域要大于且包含打包模板的区域，通过模板匹配算法，即可确定第一感兴趣区域中包含的是打包带区域或者打包线区域，从而确定打包类型。获取第一感兴趣区域与所述打包类型对应 的打包模板匹配得到的目标区域，图7(a)和图7(b)分别示出了打包带和打包线类型下的目标区域，将目标区域的中心点作为所述基点(u 0，v 0)。第一样品图像用于确定打包类型和基点(u 0，v 0)，基点(u 0，v 0)是后续计算标牌的最佳悬挂点的基准点。
步骤S40，控制环形光源关闭，启动结构光源，接收所述相机拍摄的第二样品图像
关闭环形光源31，开启结构光源32，在拍摄位置不变的情况下，由于同一个相机33的拍摄视野不变，因此第二样品图像与第一样品图像尺寸相同，拍摄的区域也相同，所以第二样品图像与第一样品图像可以统一到同一图像坐标系uov中，这样拍摄区域内同一个空间点在第二样品图像和第一样品图像中具有相同的像素坐标，一般将图像左上角作为原点，建立图像坐标系uov。
进行最佳悬挂点计算之前，在第二样品图像中划定第二感兴趣区域，第二感兴趣区域与第一感兴趣区域的位置对应并且区域大小相同，相当于在同一图像坐标系uov中将第一样品图像中的第一感兴趣区域映射到第二样品图像，形成第二感兴趣区域，第一感兴趣区域和第二感兴趣区域中同一位置像素点的坐标相同，从而使得第二感兴趣区域与第一感兴趣区域共享基点(u 0，v 0)。同样地，可以对第二感兴趣区域进行灰度化、滤波、腐蚀和二值化等预处理。
步骤S50，根据所述第二样品图像和所述基点(u 0，v 0)，计算图像坐标系uov中的最佳悬挂点和按优先级排序的若干候选悬挂点。
步骤S60，控制机器人移动，将所述夹爪夹持的标牌悬挂至所述最佳悬挂点对应于世界坐标系XYZ的三维坐标处。
由于第二样品图像是在结构光源32下拍摄的，如图8所示，第二感兴趣区域中包括打包区域对应的光带，基点(u 0，v 0)的位置在光带中，第二感兴趣区域还包括在光带上侧(沿v轴负向)和下侧(沿v轴正向)分布的若干条结构光条纹，结构光条纹是线材在结构光下呈现的图像特征，由图8可见，每条结构光条纹由若干段的子条纹组成。结构光受待挂牌样品表面的调制而发生形变的特性为：由于待挂牌样品的表面存在凹凸性，会使照射到待挂牌样品表面的结构光发生相位调制，造成待挂牌样品越凸出的部分对应的光条纹像素点越偏下，反之，待挂牌样品越凹进去的部分对应的光条纹像素点越偏上。
在基点(u 0，v 0)沿v轴正向或负向任一侧选定目标结构光条纹，参照图1所示的拍摄角度，在实际工况中想要将标牌悬挂在打包装置101的左侧，那么就需要在基点(u 0，v 0)沿v轴负向的一侧选择目标结构光条纹；若想要将标牌悬挂在打包装置101的右侧，则在基点(u 0，v 0)沿v轴正向的一侧选择目标结构光条纹。然后基于结构光受待挂牌样品表面的调制而发生形变的特性，计算最佳悬挂点和候选悬挂点，具体来说，对目标结构光条纹中各段子条纹的中心点按照v坐标值递增排序(升序)，获取排序结果中的前M个中心点组成凹点集合P(u aj，v aj)，凹点集合中仍保持v坐标值递增的排序顺序不变，M为凹点集合中包括的像素点的预设数量，也是悬挂点集合中包括的悬挂点的预设数量，j为凹点集合中各像素点的序位号，序位号越小则优先级越高，1≤j≤M。当j＝1时，(u a1，v a1)为全局最凹点；凹点集合中除全局最凹点之外的凹点(u ak，v ak)为候选凹点，其中2≤k≤M，k为候选悬挂点的序位号，a为用于标识凹点集合中各像素点为凹点的符号。
由于(u a1，v a1)为全局最凹点，因此将坐标(u a1，v 0)作为图像坐标系uov中的最佳悬挂点，即取全局最凹点(u a1，v a)的u坐标值和基点(u 0，v 0)的v坐标值组合 得到最佳悬挂点在图像坐标系uov中的坐标，这样取值的目的在于：v 0是找到的打包线/打包带的位置，挂钩的一端需要悬挂在打包线/打包带上，在u＝u a的直线上，由于线材最凹，使得线材之间的缝隙最深，能避免挂牌过程中标牌受线材阻挡，从而便于挂钩顺利完成挂牌。基于同样的原理，将坐标(u ak，v 0)作为图像坐标系uov中的候选悬挂点，从而组成悬挂点集合P(u aj，v 0)＝{(u a1，v 0)；(u a2，v 0)；…；(u aj，v 0)…(u aM，v 0)}，序位号j越小的悬挂点对应的优先级越高。
需要说明的是，在其他可能的实现方式中，可以对目标结构光条纹中各段子条纹的中心点按照v坐标值递减排序(降序)，这种情况下，获取排序结果中的后M个中心点，这时为了更直观地表达优先级顺序，将这M个中心点按照v坐标值递增的顺序重新排序组成凹点集合P(u aj，v aj)，凹点集合中排序在首位的凹点为全局最凹点。
在实际工况中，根据客户需求，可能仅对一侧进行挂牌(即单侧挂牌)，或者是对两侧进行挂牌(即双侧挂牌)。一侧挂牌就是在样品中空内壁上悬挂一个标牌，在该侧挂牌时，标牌相对于打包位置的方向可以是在打包线/打包带左侧或右侧；两侧挂牌时在样品中空内壁上悬挂两个标牌，两个标牌悬挂点相较于中轴线具有180度的转角，因此当第一侧挂牌完成时，需要控制机器人1旋转180度，带动相机33的拍摄角度也转变180度，这样势必会导致图像坐标系的颠倒，如果第一侧是在打包线/打包带的左侧挂牌，那么第二侧就是在打包线/打包带的右侧挂牌，才能保证标牌悬挂位置的准确性。此外，由于相机33可能存在倾斜安装的问题，并且悬挂的线材面是凹凸不平的曲面，也会影响悬挂位置确定的准确性。
对此，本实施例对第二感兴趣区域进行分区处理，遍历第二感兴趣区域中的各个像素点，根据筛选出的白色像素点，得到包括基点(u 0，v 0)的打包区域，以及在所述打包区域沿v轴正向和负向两侧构建若干分区。如图8所示，第二感兴趣区域由众多白色像素点和黑色像素点组成，白色像素点构成的白色区域包括打包区域(即中间的较粗长条的光带)以及结构光条纹，黑色像素点构成的黑色区域为样品背景，由于光带、结构光条纹和样品背景具有各自明显的特征，灰度不同，因此可以预设一个灰度阈值，通过灰度阈值对每个像素点进行区分，从而分割出白色区域和黑色区域。由图8可见，光带以及各结构光条纹之间在v轴方向上具有一定的间距，可以通过分割线来分隔开光带以及各结构光条纹，图8中4条分割线的划分结果为：中间的光带为打包区域，在打包区域上侧和下侧分别有2个分区。
如图9所示的示例，中间光带的白色终点(即光带上v坐标值最大的像素点(u 1，v 1))，以及白色终点(u 1，v 1)下方紧邻的下一个白色起点(即光带下方的结构光条纹中v坐标值最小的像素点(u 2，v 2))，在这两个像素点之间的中心位置坐分割线l，即l＝(v 1+v 2)/2。按照此动态分区的原理，对第二感兴趣区域完成全局分区。动态分区方式更加智能化，并且精度更高。
在其他实现方式中，如图10所示的示例，可以采用静态分区的方式，在图像坐标系中，以基点(u 0，v 0)为基准，通过阈值步长y i对第二感兴趣区域进行分区，这种方式下，分割线l＝v 0+y i，其中1≤i≤N，N为分割线的数量，比如图10中，第一条分割线的阈值步长y 1＝-30，第二条分割线的阈值步长y 2＝30，第三条分割线的阈值步长y 3＝80，从而将第二感兴趣区域划分为4个分区，分别为分区1、分区2、分区3和分区4。其中，阈值步长y i可以根据实际经验设置。静态分区设置完成后，比如需要将标牌悬挂至图1中打包装置的左侧，则可以直接将基点(u 0，v 0)上侧相邻的分区作为目标分区，这种方式效率更高，但是准确率相对较低；或者从基点(u 0，v 0)上侧的若干分区中选取结构光条纹完整程度更高的分区作为目标分区，这种方式计算出的最 佳悬挂点精度更高。需要说明的是，目标分区的获取规则不限于本实施例所述。相较于动态分区方式，静态分区能够降低系统节拍，提高挂牌效率，但由于分割线可能会将完整的结构光条纹隔断开来，因此计算精度比动态分区方式低，因此可以根据实际情况酌情选择采用动态或静态分区方式。
当第二感兴趣区域分区完成后，根据挂牌位置相对于打包位置的方向，以及各分区内结构光条纹的完整程度，选取目标分区，并将所述目标分区内包括的结构光条纹选定为所述目标结构光条纹。比如图8中，与中间光带相邻的上侧和下侧分区中具有较为完整的结构光条纹，如果客户指定第一侧需要将标牌悬挂于打包装置的左侧，则可以选取与中间光带相邻的上侧分区为目标分区，当将机器人1旋转180度后，由于图像坐标系颠倒，第二侧则选取与中间光带相邻的下侧分区为目标分区，以保证双侧挂牌的准确性。需要说明的是，如果采用单侧挂牌，进行分区处理则便于统一管理各个打包样品的挂牌位置和方向，当然单侧挂牌时也可不用设置分区。
当计算出图像坐标系uov中的悬挂点集合后，需要计算悬挂点集合中各悬挂点(u aj，v 0)在世界坐标系XYZ的三维坐标，然后先根据夹爪4的当前位置和最佳悬挂点的三维坐标来生成一条悬挂轨迹，控制机器人1按照该悬挂轨迹运动，即可将夹爪4中夹持的标牌(具体体现为通过挂钩)悬挂至最佳悬挂点的三维坐标处，从而完成挂牌动作。在复检时，如果最佳悬挂点处挂牌失败，则基于候选悬挂点的优先级，对候选悬挂点进行挂牌，判定是否挂牌成功，当挂牌成功时复检结束。
世界坐标系XYZ是在实际世界空间中建立的坐标系，图像坐标系uov和世界坐标系XYZ建立后，可以根据相机成像特性和拍摄位置等相关信息，预先获取到图像坐标系uov与世界坐标系XYZ之间的映射关系，图像坐标系uov中任意一像素点都可以根据映射关系找到在世界坐标系XYZ对应的三维坐标点。因此，通过映射关系，可以计算悬挂点集合中各悬挂点(u aj，v 0)在世界坐标系XYZ的坐标值(X aj，Y 0)，即确定了悬挂点集合中各悬挂点在世界坐标系XYZ中的X坐标值和Y坐标值。
最佳悬挂点的Z坐标(深度)即为相机33到打包装置的距离，可以根据根据最凸点(u t，v t)以及相机33、结构光源32、测距传感器34和待挂牌样品100的相对位置参数等几何关系，利用三角测距法计算悬挂点集合中各悬挂点的Z坐标，从而得到悬挂点集合中各悬挂点的三维坐标(X aj，Y 0，Z)，从而组成悬挂点三维坐标集合P(X aj，Y 0，Z)，并将悬挂点三维坐标集合P(X aj，Y 0，Z)存储在控制系统5的数据库中，以便后续复检时使用。
其中，根据述目标结构光条纹中各段子条纹的中心点的排序结果，将v坐标值最大的中心点作为最凸点(u t，v t)，如果是按v坐标值递增(升序)排序，则将排序在最末位的中心点作为最凸点(u t，v t)；如果是按v坐标值递减(降序)排序，则将排序在首位的中心点作为最凸点(u t，v t)。本实施例中，选取最凸点(u t，v t)参与计算最佳悬挂点的Z坐标的目的是因为最凸点是线材盘卷中最凸出的位置，以线材盘卷表面最凸点与相机33的实际距离作为Z坐标，可以避免挂牌过程中机器人1与线材盘卷相撞。悬挂点集合中各悬挂点的Y坐标值相同都为Y 0，这是因为Y 0对应于打包线/打包带的位置；悬挂点集合中各悬挂点的Z坐标值相同，因为Z坐标值是基于最凸点(u t，v t)以及固定的几何关系，通过三角测距法计算得到，最凸点(u t，v t)不能变动是为了防止机器人1与线材相撞，从而保证顺利挂牌。因此，悬挂点集合中各悬挂点之间仅仅是X坐标值不同，即各悬挂点之间仅仅是在世界坐标系XYZ的X轴方向上存在位移。其中，t为用于标识像素点为最凸点的符号。
在实际工况中，为了提高挂牌的成功率，在生成机器人1的悬挂轨迹时，需要在 悬挂点集合中各悬挂点的三维坐标(X aj，Y 0，Z)的基础上加上一定的偏差，即(X aj+a，Y 0+b，Z+c)，其中a、b和c为经验值，可以根据挂牌系统进行调试获得。图11中在第二感兴趣区域中标记了基点(u 0，v 0)、全局最凹点(u a1，v a1)、最凸点(u t，v t)和最佳悬挂点(u a1，v 0)，还可在第二感兴趣区域中标记候选凹点(u ak，v ak)和候选悬挂点(u ak，v 0)(图11中未示出)，从而获知这些关键点的位置分布。
步骤S70，对挂牌状态进行复检，如果复检结果为挂牌失败，按照所述优先级在所述候选悬挂点的三维坐标处进行挂牌直至挂牌成功，则复检过程结束。在步骤S70的一种具体实现方式中，当挂牌完成时，视觉定位装置3位于最佳悬挂点的附近，需要将视觉定位装置3返回至第一样品图像/第二样品图像对应的拍摄位置，即返回至根据测距传感器34和预设距离定位的拍摄位置处，控制结构光源32关闭，并启动环形光源31，由相机33拍摄第三样品图像，第三样品图像中包括打包装置、挂钩挂牌和线材的局部区域，然后可以对第三样品图像进行如灰度化、滤波、腐蚀等预处理，第三样品图像无需进行仿射变换。
然后计算第三样品图像的轮廓面积，如果第三样品图像的轮廓面积大于阈值，则判定复检结果为挂牌成功，反之，则判定复检结果为挂牌失败。其中，第三样品图像中除标牌之外其他轮廓的总面积小于所述阈值，所述阈值可以根据标牌的轮廓面积进行设定，比如可以设定为第三样品图像中悬挂成功的标牌面积的四分之一。对于第三样品图像的轮廓面积，其中标牌面积占比较高，而除标牌之外其他轮廓的总面积占比较小，并且小于设定的阈值，如果挂牌失败，第三样品图像中不具有标牌，则线材和打包装置的轮廓面积小于阈值，即可判定为挂牌失败；如果第三样品图像中具有标牌，则标牌面积显然会大于设定的阈值，即可判定为挂牌成功。
如果判定在最佳悬挂点的三维坐标(X a1，Y 0，Z)处挂牌成功，则结束复检；如果判定最佳悬挂点处挂牌失败，常规的方式是重复步骤S10～步骤S70，重新拍摄第一样品图像和第二样品图像计算最佳悬挂点的三维坐标，并且再次执行挂牌动作，这种方式虽然挂牌精度更高，但是复检需要执行复杂的计算和控制流程，导致挂牌效率低，挂牌系统工作节拍长。在某些应用场景中，可能对挂牌的精度要求相对较低，比如线材盘卷的尺寸较大、线材之间的缝隙较大或者打包线/打包带的直径较小，这种情况下系统更多地是侧重挂牌效率，以降低系统工作节拍，保证生产效率。
对此本实施例中，如果判定最佳悬挂点处挂牌失败，则无需重复步骤S10～步骤S70，而是直接调取悬挂点三维坐标集合中排在最佳悬挂点下一位的候选候选悬挂点，根据候选悬挂点的三维坐标(X a2，Y 0，Z)生成第二条悬挂轨迹，执行在候选悬挂点处的挂牌，然后再次执行拍摄第三样品图像，判定第三样品图像的轮廓面积是否大于阈值的步骤，以此类推，按照候选悬挂点的优先级顺序复检，直至最终判定结果为第三样品图像的轮廓面积大于阈值时，即挂牌成功，复检过程结束。图12示出了在线材上挂牌成功时的第三样品图像示意图，其中带数字标识的即为标牌，标牌和打包线之间连接有挂钩。
如果采用单侧挂牌，则在挂牌成功，复检过程结束后，控制环形光源31关闭，并控制机器人1回到初始位置；如果采用双侧挂牌，则在一侧挂牌成功后，控制机器人1旋转180°，以使相机33朝向另一侧，然后对另一侧同样执行步骤S10～步骤S70示出的悬挂标牌以及复检的流程，并在另一侧也挂牌成功后，再控制环形光源31关闭，并控制机器人1回到初始位置。当前待挂牌样品单侧/双侧挂牌成功后，即可将当前的待挂牌样品移出挂牌工位，并将下一个待挂牌样品移动至挂牌工位，然后重复以上方法流程。
在复检过程中，控制语音装置6播报与挂牌状态对应的提示信息，所述挂牌状态包括 挂牌成功和挂牌失败。无论是单侧还是双侧挂牌，每次复检时，执行一次判定第三样品图像的轮廓面积是否大于阈值的步骤，都可以根据判定出的挂牌状态，控制语音装置6播报对应的提示信息，所述提示信息可以预设在语音装置6中，作为示例，提示信息可设置为某某样品已成功悬挂标牌，或者某某样品悬挂标牌失败等，以便现场人员能够获知挂牌状态和复检进程，提示信息的具体内容可以根据实际情况进行设定，本实施例不作限定。
在一种实现方式中，如图13所示，控制系统5可采用PLC控制器51+计算机52的组合控制结构，相机33、打印机8和PLC控制器51分别与计算机52电连接，机器人1、环形光源31、结构光源32、测距传感器34、语音装置6和制钩机7分别与PLC控制器51电连接。PLC控制器51+计算机52的组合控制流程如下：
待挂牌样品在打包、称重完成之后，送入挂牌工位，PLC控制器51控制制钩机7制作挂钩，计算机52控制打印机8打印标牌；当PLC控制器51接收到制钩机7发送的制作成功指令，控制机器人1移动，以带动夹爪4夹取挂钩；当计算机52接收到打印机反馈的打印成功指令，计算机52向PLC控制器51发送第一控制指令；PLC控制器51响应第一控制指令，控制机器人1带动夹爪完成穿牌动作。
穿牌完成后，PLC控制器51控制机器人1带动视觉定位装置移动至所述待挂牌样品100的中空区域内，并启动测距传感器34；PLC控制器51接收测距传感器34检测的相机33与待挂牌样品内壁的间距，并判断所述间距是否为预设距离，如果否，则继续调整视觉定位装置的位置，如果所述间距为预设距离，则PLC控制器51控制环形光源31和相机33启动，以使相机33拍摄第一样品图像；计算机52接收相机33拍摄的第一样品图像后，确定打包类型和图像坐标系uov中的基点(u 0，v 0)；计算机52向PLC控制器51发送第二控制指令，PLC控制器51响应第二控制指令，控制环形光源31关闭，启动结构光源32，以使相机33拍摄第二样品图像；计算机52接收相机33拍摄的第二样品图像后，计算图像坐标系uov中的悬挂点集合，以及悬挂点集合中各悬挂点的三维坐标，即悬挂点三维坐标集合，并将悬挂点三维坐标集合存储在计算机52的数据库中；计算机52根据最佳悬挂点的三维坐标(X a1，Y 0，Z)生成第三控制指令，并将第三控制指令发送给PLC控制器51，PLC控制器51响应第三控制指令，根据最佳悬挂点的三维坐标(X a1，Y 0，Z)生成第一条悬挂轨迹，控制机器人1按照第一条悬挂轨迹移动，将夹爪4中夹持的标牌悬挂至最佳悬挂点的三维坐标处，完成挂牌动作。
机器人1向PLC控制器51反馈挂牌完成指令，这时启动复检：PLC控制器51控制机器人1移动，使视觉定位装置3返回至第一样品图像对应的拍摄位置；PLC控制器51控制结构光源32关闭，启动环形光源31，以使相机33拍摄第三样品图像；计算机52接收相机33拍摄的第三样品图像后，对第三样品图像进行预处理，计算第三样品图像的轮廓面积，并判断第三样品图像的轮廓面积是否大于阈值；如果第三样品图像的轮廓面积大于阈值，则判定复检结果为最佳悬挂点处挂牌成功；反之，则判定复检结果为最佳悬挂点处挂牌失败，计算机52根据优先级调取悬挂点三维坐标集合中候选悬挂点的三维坐标，并根据候选悬挂点的三维坐标生成第四控制指令，并将第四控制指令发送给PLC控制器51，PLC控制器51响应第四控制指令，根据候选悬挂点的三维坐标(X a2，Y 0，Z)生成第二条悬挂轨迹，重复以上挂牌状态判定流程，以此类推，直至判定挂牌成功为止，复检过程结束。
如果采用单侧挂牌，则在挂牌成功，复检过程结束后，计算机52向PLC控制器51发送第五控制指令；PLC控制器51响应第五控制指令，控制环形光源31关闭，并控制机器人1回到初始位置；如果采用双侧挂牌，则在一侧挂牌成功后，计算机52向 PLC控制器51发送第六控制指令；PLC控制器51响应第六控制指令，控制机器人1旋转180°，以使相机33朝向另一侧，然后对另一侧同样执行以上悬挂标牌以及复检的流程，并在另一侧也挂牌成功后，计算机52向PLC控制器51发送第七控制指令；PLC控制器51响应第七控制指令，控制环形光源31关闭，并控制机器人1回到初始位置。无论采用单侧挂牌还是双侧挂牌，在进行复检时，计算机52都需要向PLC控制器51发送判定出的挂牌状态；PLC控制器51根据挂牌状态，控制语音装置6播报对应的提示信息。
当前待挂牌样品单侧/双侧挂牌成功后，即可将当前的待挂牌样品移出挂牌工位，并将下一个待挂牌样品移动至挂牌工位，然后重复以上方法流程即可。
由以上技术方案可知，本申请的关键在于每次悬挂标牌时需要利用机器视觉计算最佳悬挂点，首先在相机与待挂牌样品内壁的间距为预设距离时，利用环形光源和相机拍摄到第一样品图像，由于第一样品图像包括线材和打包装置的局部区域，因此可以判定打包类型是打包带还是打包线，并获取打包线或打包带中的基点(u 0，v 0)，该基点(u 0，v 0)是后续确定最佳悬挂点的基准点；拍摄第二样品图像，第二样品图像中包括打包装置对应的光带以及线材对应的结构光条纹，通过分区处理，获取最凸点(u t，v t)和凹点集合P(u aj，v aj)，从而计算出悬挂点集合P(u aj，v 0)及其包括的各悬挂点在世界坐标系XYZ中的三维坐标(X aj，Y 0，Z)，然后通过控制机器人移动，使夹爪携带的挂牌被悬挂在最佳悬挂点的三维坐标位置处，即可完成最佳悬挂点处挂牌。
在实际应用中，由于最佳悬挂点的误差等因素，可能导致挂牌失败，这时需要对挂牌状态进行复检，以确定是否挂牌成功，如果在最佳悬挂点挂牌失败，则无需重复上述挂牌流程，而是根据排序在最佳悬挂点下一位的候选悬挂点，再次生成悬挂轨迹，执行挂牌，然后判定在该候选悬挂点是否挂牌成功，以此类推，若仍挂牌失败，则按照优先级顺序对之后的候选悬挂点进行挂牌，直至挂牌成功，复检和挂牌流程结束。本申请利用机器视觉自动确定最佳悬挂点，并通过机器人自动执行挂牌操作，实现了线材盘卷的自动化、智能化挂牌，减少人工劳作和安全隐患，提高了挂牌效率和最佳悬挂点的定位精度，能避免挂错标牌和漏挂标牌。此外，本申请通过计算悬挂点三维坐标集合并存储入库，使得在最佳悬挂点挂牌失败后，无需重复复杂的计算流程，可以缩短复检节拍，能够更快速地成功挂牌，提高了挂牌效率，降低了挂牌系统的工作节拍，进而提高了线材盘卷的生产效率。本申请所述的技术方案可以适应性应用于其他类型样品的挂牌控制，不限于线材盘卷。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本发明的其它实施方案。本申请旨在涵盖本发明的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本发明的一般性原理并包括本发明未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本发明的真正范围和精神由所附的权利要求指出。
应当理解的是，本发明并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本发明的范围仅由所附的权利要求来限制。

Claims (10)

  1. 一种线材的挂牌系统，其特征在于，包括：
    机器人、与所述机器人连接的视觉定位装置和夹爪，所述视觉定位装置包括环形光源、结构光源、相机和测距传感器；以及，控制系统，被配置为执行如下程序步骤：
    在待挂牌样品打包完成以后，控制机器人带动视觉定位装置移动至所述待挂牌样品的中空区域内，并接收所述测距传感器检测的相机与待挂牌样品内壁的间距；
    当所述间距为预设距离时，启动环形光源和相机，接收所述相机拍摄的第一样品图像；
    根据所述第一样品图像确定打包类型和图像坐标系uov中的基点(u 0，v 0)；
    控制环形光源关闭，启动结构光源，接收所述相机拍摄的第二样品图像；
    根据所述第二样品图像和所述基点(u 0，v 0)，计算图像坐标系uov中的最佳悬挂点和按优先级排序的若干候选悬挂点；
    控制机器人移动，将所述夹爪夹持的标牌悬挂至所述最佳悬挂点对应于世界坐标系XYZ的三维坐标处；
    对挂牌状态进行复检，如果复检结果为挂牌失败，按照所述优先级在所述候选悬挂点的三维坐标处进行挂牌，直至复检结果为挂牌成功，则复检过程结束。
  2. 根据权利要求1所述的系统，其特征在于，所述控制系统被进一步配置为按如下步骤计算确定打包类型和图像坐标系uov中的基点(u 0，v 0)：
    在所述第一样品图像中划定第一感兴趣区域；
    将所述第一感兴趣区域与预先获取的打包模板进行匹配，确定打包类型；所述打包类型包括打包线和打包带，所述第一感兴趣区域大于且包含所述打包模板的区域；
    获取所述第一感兴趣区域与所述打包类型对应的打包模板匹配得到的目标区域；
    将所述目标区域的中心点作为所述基点(u 0，v 0)。
  3. 根据权利要求2所述的系统，其特征在于，所述控制系统被进一步配置为按如下步骤计算图像坐标系uov中的最佳悬挂点和候选悬挂点：
    在第二样品图像中划定第二感兴趣区域；所述第二感兴趣区域与所述第一感兴趣区域的位置对应并且区域大小相同，所述第二感兴趣区域与所述第一感兴趣区域共享基点(u 0，v 0)；
    在第二感兴趣区域中，在基点(u 0，v 0)沿v轴正向或负向任一侧选定目标结构光条纹；
    对所述目标结构光条纹中各段子条纹的中心点按照v坐标值递增排序，获取排序结果中的前M个中心点并保持排序不变组成凹点集合P(u aj，v aj)；M为凹点集合中包括的像素点的预设数量，j为凹点集合中各像素点的序位号，1≤j≤M；a为用于标识凹点集合中各像素点为凹点的符号；
    将坐标(u a1，v 0)作为图像坐标系uov中的最佳悬挂点，将坐标(u ak，v 0)作为图像坐标系uov中的候选悬挂点；其中，2≤k≤M，k为候选悬挂点的序位号。
  4. 根据权利要求3所述的系统，其特征在于，所述控制系统被进一步配置为按照如下步骤选定目标结构光条纹：
    遍历第二感兴趣区域中的各个像素点，根据筛选出的白色像素点，得到包括基点(u 0，v 0)的打包区域，以及在所述打包区域沿v轴正向和负向两侧构建若干分区；或者，以基点(u 0，v 0)为基准，通过阈值步长将所述第二感兴趣区域划分为若干分区；
    根据挂牌位置相对于打包位置的方向，以及各分区内结构光条纹的完整程度，选取目标分区，并将所述目标分区内包括的结构光条纹选定为所述目标结构光条纹。
  5. 根据权利要求3所述的系统，其特征在于，所述控制系统还被配置为执行：
    根据所述目标结构光条纹中各段子条纹的中心点的排序结果，将v坐标值最大的中心点作为最凸点(u t，v t)；t为用于标识像素点为最凸点的符号；
    根据图像坐标系uov与世界坐标系XYZ的映射关系，计算悬挂点集合中各悬挂点(u aj，v 0)在世界坐标系XYZ的坐标值(X aj，Y 0)；所述悬挂点集合包括最佳悬挂点(u a1，v 0)和候选悬挂点(u ak，v 0)；
    根据最凸点(u t，v t)以及相机、结构光源、测距传感器和待挂牌样品的相对位置参数，利用三角测距法计算悬挂点集合中各悬挂点的Z坐标，从而得到悬挂点集合中各悬挂点的三维坐标(X aj，Y 0，Z)。
  6. 据权利要求1所述的系统，其特征在于，所述控制系统还被配置为按照如下步骤对挂牌状态进行复检：
    控制机器人移动，使视觉定位装置返回至第一样品图像对应的拍摄位置；
    控制结构光源关闭，启动环形光源，接收所述相机拍摄的第三样品图像；
    对所述第三样品图像进行预处理后，计算所述第三样品图像的轮廓面积；
    如果所述第三样品图像的轮廓面积大于阈值，则判定复检结果为挂牌成功；反之，则判定复检结果为挂牌失败；
    其中，所述第三样品图像中除标牌之外其他轮廓的总面积小于所述阈值。
  7. 根据权利要求1所述的系统，其特征在于，还包括语音装置，所述控制系统还被配置为执行：
    在复检过程中，控制所述语音装置播报与所述挂牌状态对应的提示信息，所述挂牌状态包括挂牌成功和挂牌失败。
  8. 根据权利要求1所述的系统，其特征在于，还包括制钩机和打印机，所述控制系统还被配置为执行：
    在控制机器人带动视觉定位装置移动至所述待挂牌样品的中空区域内之前，控制所述制钩机制作挂钩，以及控制所述打印机打印标牌；所述挂钩用于将所述标牌悬挂于所述最佳悬挂点的三维坐标处；
    响应于接收到所述制钩机发送的制作成功指令，控制所述夹爪从所述制钩机上夹取挂钩；
    控制机器人带动夹爪移动，使所述挂钩穿过所述标牌上预置的穿孔，则完成穿牌动作。
  9. 根据权利要求6所述的系统，其特征在于，当需要对待挂牌样品进行双侧挂牌时，所述控制系统还进一步配置为执行：
    当一侧挂牌成功后，控制所述机器人旋转180°，以使所述相机朝向另一侧；
    对另一侧执行悬挂标牌以及复检流程；
    当另一侧挂牌成功后，控制环形光源关闭，并控制所述机器人回到初始位置。
  10. 一种线材的挂牌方法，用于线材的挂牌系统，其特征在于，所述线材的挂牌系统包括机器人和与所述机器人连接的视觉定位装置和夹爪，所述视觉定位装置包括环形光源、结构光源、相机和测距传感器，所述方法包括：
    在待挂牌样品打包完成以后，控制机器人带动视觉定位装置移动至所述待挂牌样品的中空区域内，并接收测距传感器检测的相机与待挂牌样品内壁的间距；
    当所述间距为预设距离时，启动环形光源和相机，接收所述相机拍摄的第一样品图像；
    根据所述第一样品图像确定打包类型和图像坐标系uov中的基点(u 0，v 0)；
    控制环形光源关闭，启动结构光源，接收所述相机拍摄的第二样品图像；
    根据所述第二样品图像和所述基点(u 0，v 0)，计算图像坐标系uov中的最佳悬挂点和按优先级排序的若干候选悬挂点；
    控制机器人移动，将夹爪夹持的标牌悬挂至所述最佳悬挂点对应于世界坐标系XYZ的三维坐标处；
    对挂牌状态进行复检，如果复检结果为挂牌失败，按照所述优先级在所述候选悬挂点的三维坐标处进行挂牌，直至复检结果为挂牌成功，则复检过程结束。
PCT/CN2020/128642 2020-05-27 2020-11-13 一种线材的挂牌系统及方法 WO2021238094A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010458050.X 2020-05-27
CN202010458050.XA CN111361819B (zh) 2020-05-27 2020-05-27 一种线材的挂牌系统及方法

Publications (1)

Publication Number Publication Date
WO2021238094A1 true WO2021238094A1 (zh) 2021-12-02

Family

ID=71201849

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/128642 WO2021238094A1 (zh) 2020-05-27 2020-11-13 一种线材的挂牌系统及方法

Country Status (2)

Country Link
CN (1) CN111361819B (zh)
WO (1) WO2021238094A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023248264A1 (en) * 2022-06-21 2023-12-28 Alping Italia S.R.L. Automated tagging apparatus and method for metal products

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111361819B (zh) * 2020-05-27 2020-08-21 江苏金恒信息科技股份有限公司 一种线材的挂牌系统及方法
CN115393553B (zh) * 2022-08-19 2024-02-06 燕山大学 基于3d视觉的用于确定线材捆线位置的挂牌方法及其装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109466841A (zh) * 2018-10-31 2019-03-15 北京中远通科技有限公司 一种自动挂标牌系统
CN209834201U (zh) * 2019-05-09 2019-12-24 小羽互联智能科技(杭州)有限公司 挂牌机器人
US20200122871A1 (en) * 2018-10-22 2020-04-23 Taikisha Ltd. Vehicle part hole patch applicator arrangement
CN111056107A (zh) * 2019-12-25 2020-04-24 北京中远通科技有限公司 线材挂牌装置及方法
CN111361819A (zh) * 2020-05-27 2020-07-03 江苏金恒信息科技股份有限公司 一种线材的挂牌系统及方法
CN111618855A (zh) * 2020-05-27 2020-09-04 江苏金恒信息科技股份有限公司 一种自动挂牌系统及方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN205736484U (zh) * 2016-04-22 2016-11-30 北京中远通科技有限公司 一种自动打标机
CN208728978U (zh) * 2018-06-21 2019-04-12 北京中远通科技有限公司 机器人自动焊标牌系统
CN210587771U (zh) * 2019-08-30 2020-05-22 江苏金恒信息科技股份有限公司 标牌自动焊接机器人系统
CN110930512B (zh) * 2019-11-28 2023-04-25 江苏金恒信息科技股份有限公司 一种钢捆标牌焊接方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200122871A1 (en) * 2018-10-22 2020-04-23 Taikisha Ltd. Vehicle part hole patch applicator arrangement
CN109466841A (zh) * 2018-10-31 2019-03-15 北京中远通科技有限公司 一种自动挂标牌系统
CN209834201U (zh) * 2019-05-09 2019-12-24 小羽互联智能科技(杭州)有限公司 挂牌机器人
CN111056107A (zh) * 2019-12-25 2020-04-24 北京中远通科技有限公司 线材挂牌装置及方法
CN111361819A (zh) * 2020-05-27 2020-07-03 江苏金恒信息科技股份有限公司 一种线材的挂牌系统及方法
CN111618855A (zh) * 2020-05-27 2020-09-04 江苏金恒信息科技股份有限公司 一种自动挂牌系统及方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023248264A1 (en) * 2022-06-21 2023-12-28 Alping Italia S.R.L. Automated tagging apparatus and method for metal products

Also Published As

Publication number Publication date
CN111361819B (zh) 2020-08-21
CN111361819A (zh) 2020-07-03

Similar Documents

Publication Publication Date Title
WO2021238094A1 (zh) 一种线材的挂牌系统及方法
CN111618855B (zh) 一种自动挂牌系统及方法
CN108827154B (zh) 一种机器人无示教抓取方法、装置及计算机可读存储介质
US20190143523A1 (en) Robotic system architecture and control processes
CN112122840A (zh) 一种基于机器人焊接的视觉定位焊接系统及焊接方法
CN108496124A (zh) 表面缺陷的自动检测和机器人辅助加工
CN108399639A (zh) 基于深度学习的快速自动抓取与摆放方法
CN108898634B (zh) 基于双目相机视差对绣花机目标针眼进行精确定位的方法
CN110355754A (zh) 机器人手眼系统、控制方法、设备及存储介质
CN103846192A (zh) 自主定位智能点胶系统
CN109629122A (zh) 一种基于机器视觉的机器人缝制方法
WO2021238096A1 (zh) 一种基于多个钢捆的自动焊接标牌方法及装置
CN109127194A (zh) 汽车涂胶视觉跟随检测方法
CN109300155A (zh) 一种避障路径规划方法、装置、设备和介质
CN108127238A (zh) 非平整面自主识别机器人增材成形的方法
CN106247943A (zh) 物品三维定位方法、装置和系统
CN106514068A (zh) 一种机器人智能焊接的控制方法
CN115816471B (zh) 多视角3d视觉引导机器人的无序抓取方法、设备及介质
CN102829769A (zh) 基于结构光视觉传感器的集装箱位置姿态测量方法
CN113689509A (zh) 基于双目视觉的无序抓取方法、系统及存储介质
CN208366871U (zh) 检测系统
CN112224590A (zh) 一种基于三维点云的钢坯贴标方法及系统
CN113284178A (zh) 物体码垛方法、装置、计算设备及计算机存储介质
CN114693529B (zh) 一种图像拼接方法、装置、设备及存储介质
CN109015632A (zh) 一种机器人手爪末端定位方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20937312

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20937312

Country of ref document: EP

Kind code of ref document: A1