CN113815323A - Steel rail marking device and method based on three-dimensional visual guidance - Google Patents

Steel rail marking device and method based on three-dimensional visual guidance Download PDF

Info

Publication number
CN113815323A
CN113815323A CN202111161375.2A CN202111161375A CN113815323A CN 113815323 A CN113815323 A CN 113815323A CN 202111161375 A CN202111161375 A CN 202111161375A CN 113815323 A CN113815323 A CN 113815323A
Authority
CN
China
Prior art keywords
steel rail
camera
point
dimensional sensor
marking
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111161375.2A
Other languages
Chinese (zh)
Inventor
苗庆伟
刘贝贝
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Henan Alson Intelligent Technology Co ltd
Original Assignee
Henan Alson Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Henan Alson Intelligent Technology Co ltd filed Critical Henan Alson Intelligent Technology Co ltd
Priority to CN202111161375.2A priority Critical patent/CN113815323A/en
Publication of CN113815323A publication Critical patent/CN113815323A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B41—PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
    • B41J—TYPEWRITERS; SELECTIVE PRINTING MECHANISMS, i.e. MECHANISMS PRINTING OTHERWISE THAN FROM A FORME; CORRECTION OF TYPOGRAPHICAL ERRORS
    • B41J3/00—Typewriters or selective printing or marking mechanisms characterised by the purpose for which they are constructed
    • B41J3/407—Typewriters or selective printing or marking mechanisms characterised by the purpose for which they are constructed for marking on special material
    • B41J3/413—Typewriters or selective printing or marking mechanisms characterised by the purpose for which they are constructed for marking on special material for metal
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B41—PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
    • B41J—TYPEWRITERS; SELECTIVE PRINTING MECHANISMS, i.e. MECHANISMS PRINTING OTHERWISE THAN FROM A FORME; CORRECTION OF TYPOGRAPHICAL ERRORS
    • B41J3/00—Typewriters or selective printing or marking mechanisms characterised by the purpose for which they are constructed
    • B41J3/407—Typewriters or selective printing or marking mechanisms characterised by the purpose for which they are constructed for marking on special material
    • B41J3/4073—Printing on three-dimensional objects not being in sheet or web form, e.g. spherical or cubic objects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Manufacturing & Machinery (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Numerical Control (AREA)

Abstract

The invention relates to the field of marking of an industrial robot for a steel rail under the three-dimensional visual guidance, in particular to a steel rail marking device and a marking method based on the three-dimensional visual guidance. The invention acquires the three-dimensional data of the steel rail, analyzes the marking position of the steel rail by looking at the three-dimensional sensor, and guides the robot to mark, thereby improving the production efficiency of enterprises, reducing the cost and increasing the competitiveness of the enterprises.

Description

Steel rail marking device and method based on three-dimensional visual guidance
Technical Field
The invention relates to the field of marking of steel rails by industrial robots under the three-dimensional visual guidance, in particular to a steel rail marking device and a steel rail marking method based on the three-dimensional visual guidance.
Background
With the rapid development of industrial production in China and the rapid improvement of the automation degree, the application of the industrial manipulator in the processing of large-scale steel plant parts is more and more common, but for most steel plant part processing application scenes using the industrial manipulator, manual teaching or offline programming is needed to plan the working path of the manipulator in advance, the flexibility and the intelligence of the industrial manipulator are strictly limited by the highly structured working mode, and the requirement of flexible production cannot be met.
In the process of marking, some steel rails produced after the working procedures of molten steel pouring and cooling pressing in a steel mill are marked by a manual teaching or off-line programming mode, but the stopping positions of the steel rails are not accurate, so that the marking positions of a manipulator are different; some methods even adopt original manual marking, and for such simple operation process, the method has the disadvantages of low efficiency and high working strength, and brings negative effects that personnel repeatedly operate in the processing process, the processing beat is slow, and the work of the personnel is not exerted.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide a method for shooting three-dimensional data of a steel rail through a three-dimensional sensor, so that the marking beat of the steel rail is accelerated, the accuracy of the marking position of the steel rail is increased, the overall production efficiency of the steel rail is improved, and the labor cost is saved.
In order to achieve the purpose, the technical scheme of the invention is as follows:
rail marking device based on three-dimensional vision guide, including the industrial computer, still include the rail carriage that support, support below set up and set up the three-dimensional sensor on the support, rail carriage one side be provided with the robot, the robot on install the manipulator, the manipulator on install the clamping jaw that is used for beating the mark, three-dimensional sensor be connected with the industrial computer.
The three-dimensional sensor comprises a shell connected with the support through a transfer plate, two cameras, a laser scanner and an interface, wherein the two cameras, the laser scanner and the interface are fixedly installed in the shell, and the cameras and the laser scanner are connected with the industrial personal computer through the interface.
Two camera symmetry's the perpendicular shell of setting in both sides, two cameras be located laser scanner's both sides, the camera lens of two cameras down, the corresponding position of bottom plate of shell all offer the hole that is used for the camera to shoot and the hole that is used for laser scanner to scan, laser scanner's laser head and camera lens orientation keep unanimous.
The marking method of the steel rail marking device based on three-dimensional visual guidance comprises the following steps:
the method comprises the following steps: the three-dimensional sensor is arranged above one side of a steel rail stopping point, and the coordinate relation between the three-dimensional sensor and a manipulator tool is calibrated in advance;
step two: after the steel rail reaches a preset position, triggering a three-dimensional sensor by a manipulator, starting scanning the steel rail by a laser scanner, respectively shooting images from a left visual angle and a right visual angle by cameras on two sides of the laser scanner, and acquiring steel rail point cloud data and storing steel rail original image data by the three-dimensional sensor after scanning is finished;
step three: the three-dimensional sensor calculates the position and posture of the marking point of the steel rail according to the point cloud data of the steel rail;
step four: the three-dimensional sensor converts the pose of the steel rail marking point into the position under the base coordinate of the manipulator, and then guides the manipulator to mark the steel rail.
The first step further comprises the step of establishing a mechanical hand tool coordinate system, the establishment of the mechanical hand tool coordinate system is realized by operating a mechanical hand 5 by using an XYZ six-point method, the original point O of the established mechanical hand tool coordinate system is required to be located in the middle of a mechanical hand clamping jaw 6, the Z positive direction is the direction perpendicular to a mechanical hand flange and pointing to the center of the flange, and meanwhile the average precision of the mechanical hand tool coordinate system to be established is not more than 1mm, so that the positioning and marking precision of the steel plate is guaranteed.
The calibration of the coordinate system of the three-dimensional sensor and the manipulator tool needs to be carried out by means of a calibration plate which is black, the calibration plate has the function of enabling the three-dimensional sensor to uniquely identify the coordinates of each coding point in the calibration plate, further calculating the internal and external parameters of the three-dimensional sensor, calculating the calibration relation of the three-dimensional sensor and the manipulator tool coordinate system by combining the pose of the manipulator, the calibration plate is mainly placed below a camera, the camera is used for calibrating the coordinates of the circle center of each coding point 12 on the calibration plate (under the camera coordinate system), then the robot is enabled to walk to the center of a corresponding circular point (the robot also has one coordinate), and then the relation between the robot and the camera coordinate system is established. The encoding use principle of the encoding points adopts four reference points as identification marks of the encoding points, and the angle information of three classification points and a central encoding point is used as the unique identification characteristic of the encoding points, so that the uniqueness of encoding point identification and calculation is realized;
in the calibration process of the three-dimensional sensor 3 and the manipulator tool coordinate system, a plurality of groups of pose data of the manipulator and the code point data shot by the three-dimensional sensor need to be recorded, and the calibration relation between the three-dimensional sensor 3 and the manipulator tool coordinate system is calculated by resolving the code point coordinates and the obtained manipulator pose. The calibration method when the three-dimensional sensor is installed on the manipulator is as follows:
(1) the manipulator 5 is controlled to move from the position A to the position B, the camera is calibrated before and after the movement, and the external parameters of the camera are obtained, so that Rc1 and tc1 are obtained. The robot motion parameters Rd1, td1 are read by the controller. Thereby resulting in a first set of constraints of R, t;
(2) the robot 5 is controlled to move from position B to position C and the previous step is repeated, resulting in Rc2, tc2, Rd2, td 2. Thereby resulting in a second set of constraints of R, t;
(3) the robot 5 is controlled to move from the position C to the position N, and the step (1) is repeated, thereby obtaining Rcn, tcn, Rdn, tdn. Thus, the nth set of constraints of R, t is obtained;
(4) solving R by the column equation, and solving t according to R;
(5) is composed of
Figure BDA0003290040780000031
Obtaining a hand-eye calibration conversion matrix X, and finishing calibration;
wherein: rc1、tc1、Rc2、tc2Rcn and tcn are external parameters calibrated by the camera in n movements respectively;
Rd1、td1、Rd2、td2rdn and tdn are parameters directly read by the robot controller in n movements, R is a rotation matrix of a relationship matrix between the robot tool and the camera to be solved, t is a translation amount of a relationship between the robot tool and the camera to be solved, and X is a relationship matrix between the robot tool and the camera.
In the second step, the three-dimensional sensor triangulation principle is used for obtaining the steel rail point cloud data, and the triangulation principle is as follows:
o1-xyz and O2-xyz are two-camera spatial coordinate systems, respectively; p1, P2 are a pair of homologous points; s1, S2 is the center position of the camera lens; w is a point in real space. P1, S1 defines one straight line in space, P2, S2 defines another straight line which intersects W in space;
spatial straight line: after the camera shoots an image, a straight line can be determined by an image point on the camera CCD and the center of the camera lens, the coordinates of the two points, namely the image point and the center of the lens, are in a camera coordinate system, and a space straight line equation formed by the two points is as follows:
Figure BDA0003290040780000032
Figure BDA0003290040780000033
wherein X, Y and Z are three-dimensional coordinates of the target point and are unknown numbers;
x, y, f are coordinates of image points, which are known quantities (obtained by analyzing the image);
xs, Ys, Zs are lens center coordinates, which are known quantities (obtained during camera calibration);
ai、bi、citransform parameters for the coordinate system, for known quantities (obtained during camera calibration);
one image can be listed with one linear equation, two images can be listed with two linear equations, 4 equation sets are formed in total, and the unknowns in the equations are only three (three-dimensional point coordinates X, Y and Z), so that three unknowns can be calculated.
And after carrying out rectangle fitting on the steel rail point cloud data, finding the minimum boundary of the steel rail according to the steel rail placing position and the rectangle fitting data, determining the central point of the minimum boundary, and finally determining the marking position of the steel rail according to the preset offset.
The steel plate workpiece coordinate system is required to be established, firstly, rectangle fitting is carried out on the steel rail point cloud data, the sequence of four points of a fitting rectangle is determined, the direction of the coordinate system X, Y is determined according to the sequence of the four points, the Z direction is determined according to a right-hand spiral rule, and the steel plate workpiece coordinate system is established.
The invention has the beneficial effects that:
when the robot works, firstly, after a steel rail reaches a preset position, the robot triggers the three-dimensional sensor to scan the steel rail, after visual scanning is finished, a marking position is calculated, the positioned coordinate is sent to the manipulator, the manipulator wipes firstly, dirt on the surface of the steel rail is simply removed, marking is then carried out, and after marking is finished, the manipulator returns to the last position to wait for the next steel rail to reach the preset position to continue scanning. The system uses a laser scanner of a three-dimensional sensor to scan the steel rail, two cameras respectively shoot images from a left visual angle and a right visual angle, then point cloud data of the steel rail is calculated through a triangulation schematic diagram to calculate a marking position, and a positioned coordinate is sent to a manipulator. According to the invention, the three-dimensional data of the steel rail is obtained, the marking position of the steel rail is analyzed by the visual three-dimensional sensor, and the robot is guided to mark, so that the production efficiency of enterprises is improved, the cost is reduced, and the competitiveness of the enterprises is increased; according to the invention, the related operation of the robot is integrated into the software positioning system, so that a complex operation process is avoided, the workpiece positioning process is simplified, and an interactive interface easy to operate is provided for a client; the three-dimensional sensor can display the scanning process, the scanning result, the steel bar marking pose data and the like in real time, is convenient for operators to know the real-time running condition of the system in detail, enables the operators to master the working state of the system and improves the maintainability of the system.
Drawings
FIG. 1 is a schematic structural diagram of the present invention.
Fig. 2 is a schematic structural diagram of the three-dimensional sensor of the present invention.
Fig. 3 is a schematic structural diagram of the calibration plate of the present invention.
Fig. 4 is a flow chart of the operation of the present invention.
Fig. 5 is a schematic diagram of triangulation employed in the present invention.
Detailed Description
In the figure: the device comprises a support 1, a steel rail conveying frame 2, a three-dimensional sensor 3, a robot 4, a manipulator 5, a clamping jaw 6, a shell 7, a camera 8, a laser scanner 9, an interface 10, a calibration plate 11 and an encoding point 12.
Referring to fig. 1, 2, 3, 4 and 5, the rail marking device based on three-dimensional visual guidance of the present invention includes an industrial personal computer, and is characterized in that: still include rail carriage 2 and the three-dimensional sensor 3 of setting on support 1 that support 1, support 1 below set up, rail carriage 2 one side is provided with robot 3, installs manipulator 5 on the robot 3, installs the clamping jaw 6 that is used for beating the mark on the manipulator 5, and three-dimensional sensor 3 is connected with the industrial computer. Three-dimensional sensor 3 includes the shell 7 that is connected with support 1 through the keysets, two cameras 8 of fixed mounting in shell 7, laser scanner 9 and interface 10, camera 8 and laser scanner 9 are connected between interface 10 and the industrial computer, two camera 8 symmetrical perpendicular both sides in setting up's shell 7, two camera 8's camera lens are down, the hole that is used for the camera to shoot and the hole that is used for laser scanner to scan are all offered to the corresponding position of bottom plate of shell 7, the laser head of laser scanner 9 keeps unanimous with the camera lens orientation.
The working principle is as follows: the first step is as follows: and (3) creating a mechanical hand tool coordinate system, wherein the mechanical hand tool coordinate system is created for calibrating the relation between the tool 3 and the mechanical hand 5 on one hand, and when marking the workpiece on the other hand, the tool coordinate system of the mechanical hand 5 and the positioning workpiece are in the same coordinate system, and the workpiece coordinate system created on the workpiece needs to have consistency, so that the clamp 6 can mark the workpiece in a proper posture. The establishment of the coordinate system of the mechanical hand tool is realized by operating the mechanical hand 5 by using an XYZ six-point method, the original point O of the coordinate system of the mechanical hand tool is required to be positioned at the middle position of the clamping jaw 6 of the mechanical hand, the positive direction Z is the direction which is vertical to the flange of the mechanical hand and points to the center of the flange, and meanwhile, the average precision of the coordinate system of the mechanical hand tool to be established is not more than 1mm, so that the positioning and marking precision of the steel plate is ensured.
The second step is that: the calibration of the coordinate system of the three-dimensional sensor and the manipulator tool needs to be performed by means of the calibration plate 11 shown in fig. 3, the calibration plate 11 is black, the calibration plate 11 has the function of enabling the three-dimensional sensor 3 to uniquely identify the coordinates of each coding point 12 in the calibration plate 11, further calculating the internal and external parameters of the three-dimensional sensor 3, and calculating the calibration relationship between the three-dimensional sensor 3 and the manipulator tool coordinate system by combining the pose of the manipulator, mainly placing the calibration plate 11 below a camera, enabling the camera 8 to identify the circle center coordinates (under the camera coordinate system) of each coding point 12 on the calibration plate 11, enabling the clamp 6 to point to the center of a corresponding circular point (the robot also has a coordinate system), and then establishing the relationship between the robot and the camera coordinate system. The encoding use principle of the encoding points adopts four reference points as identification marks of the encoding points, and the angle information of the three classification points and the central encoding point is used as the unique identification characteristic of the encoding points, so that the uniqueness of encoding point identification and calculation is realized.
In the calibration process of the three-dimensional sensor 3 and the manipulator tool coordinate system, a plurality of groups of pose data of the manipulator and the code point data shot by the three-dimensional sensor need to be recorded, and the calibration relation between the three-dimensional sensor 3 and the manipulator tool coordinate system is calculated by resolving the code point coordinates and the obtained manipulator pose. The calibration method when the three-dimensional sensor is installed on the manipulator is as follows:
(6) the manipulator 5 is controlled to move from the position A to the position B, the camera is calibrated before and after the movement, and the external parameters of the camera are obtained, so that Rc1 and tc1 are obtained. The robot motion parameters Rd1, td1 are read by the controller. Thereby resulting in a first set of constraints of R, t;
(7) the robot 5 is controlled to move from position B to position C and the previous step is repeated, resulting in Rc2, tc2, Rd2, td 2. Thereby resulting in a second set of constraints of R, t;
(8) the robot 5 is controlled to move from the position C to the position N, and the step (1) is repeated, thereby obtaining Rcn, tcn, Rdn, tdn. Thus, the nth set of constraints of R, t is obtained;
(9) solving R by the column equation, and solving t according to R;
(10) is composed of
Figure BDA0003290040780000051
And obtaining a hand-eye calibration conversion matrix X, and finishing calibration.
Wherein: rc1、tc1、Rc2、tc2Rcn and tcn are external parameters calibrated by the camera in n movements respectively;
Rd1、td1、Rd2、td2rdn and tdn are parameters directly read by the robot controller in n movements, R is a rotation matrix of a relationship matrix between the robot tool and the camera to be solved, t is a translation amount of a relationship between the robot tool and the camera to be solved, and X is a relationship matrix between the robot tool and the camera.
The third step: determining a marking position based on the three-dimensional sensor 3, namely triggering the three-dimensional sensor 3 by a mechanical arm 5 to take a picture, obtaining steel rail point cloud data by a triangulation principle, extracting the three-dimensional point cloud data, deleting point clouds except for a steel rail in a vision system, performing minimum rectangle fitting on the steel rail point cloud data, determining a pose direction according to positions of 4 corners of a rectangle, and finally determining the marking position according to a short shaft of the steel rail. And transferring the marking coordinate to a mechanical arm base coordinate system according to the calibrated calibration relation between the mechanical arm tool coordinate system and the three-dimensional sensor, thereby realizing the generation of the marking pose of the steel rail.
The fourth step: calculation of the marking position of a rail and creation of a workpiece coordinate system
Calculating the marking position of the steel rail: and after carrying out rectangle fitting on the steel rail point cloud data, finding the minimum boundary of the steel rail according to the steel rail placing position and the rectangle fitting data, determining the central point of the minimum boundary, and finally determining the marking position of the steel rail according to the preset offset.
Creation of steel plate workpiece coordinate system: after rectangular fitting is carried out on the steel rail point cloud data, the sequence of four points of a fitting rectangle is determined, the direction of a coordinate system X, Y is determined according to the sequence of the four points, the Z direction is determined according to a right-hand spiral rule, and a three-dimensional coordinate system is established.
The principle of triangulation: o1-xyz and O2-xyz in FIG. 5 are the two-camera spatial coordinate systems, respectively; p1, P2 are a pair of homologous points; s1, S2 is the center position of the camera lens; w is a point in real space. P1, S1 defines one straight line in space, and P2, S2 defines another straight line which intersects W in space.
Spatial straight line: after the camera takes an image, a straight line can be defined by an image point on the camera CCD and the center of the camera lens, as shown in FIG. 2. The coordinates of the two points, namely the image point and the lens center, are in a camera coordinate system, and a space linear equation formed by the two points is as follows:
Figure BDA0003290040780000061
Figure BDA0003290040780000062
wherein X, Y and Z are three-dimensional coordinates of the target point and are unknown numbers;
x, y, f are coordinates of image points, which are known quantities (obtained by analyzing the image);
xs, Ys, Zs are lens center coordinates, which are known quantities (obtained during camera calibration);
ai、bi、citransform parameters for the coordinate system, for known quantities (obtained during camera calibration);
one image can be listed with one linear equation, two images can be listed with two linear equations, 4 equation sets are formed in total, and the unknowns in the equations are only three (three-dimensional point coordinates X, Y and Z), so that three unknowns can be calculated.
The working process of the invention is as follows: three-dimensional sensor 3 installs in one side top of rail stop point, three-dimensional sensor 3 and manipulator instrument coordinate relation are markd in advance, after the rail reachd the predetermined position, the robot triggers three-dimensional sensor 3's laser scanner 9 scanning rail, the camera 8 that is located laser scanner 9 both sides simultaneously shoots the image respectively from two visual angles about, after the visual scanning, three-dimensional sensor 3 calculates the three-dimensional space coordinate of rail every point through the triangulation principle after the scanning, acquire rail point cloud data and store rail original image data, three-dimensional sensor 3 is through calculating the rail according to rail point cloud data and beats a position appearance. And calculating a marking position according to the three-dimensional point cloud data, sending the positioned coordinate to the manipulator 5, wiping the manipulator 5 to simply remove dirt on the surface of the steel rail, marking, and returning the manipulator to the last position to wait for the next steel rail to reach a preset position to continue scanning after marking is finished.
The foregoing shows and describes the general principles, essential features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are merely illustrative of the principles of the invention, but that various changes and modifications may be made without departing from the spirit and scope of the invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims (9)

1. Steel rail marking device based on three-dimensional visual guidance, including the industrial computer, its characterized in that: the steel rail conveying device comprises a support, a steel rail conveying frame and a three-dimensional sensor, wherein the steel rail conveying frame is arranged below the support, the three-dimensional sensor is arranged on the support, a robot is arranged on one side of the steel rail conveying frame, a mechanical arm is arranged on the robot, a clamping jaw used for marking is arranged on the mechanical arm, and the three-dimensional sensor is connected with an industrial personal computer.
2. A steel rail marking device based on three-dimensional visual guidance as claimed in claim 1, characterized in that: the three-dimensional sensor comprises a shell connected with the support through a transfer plate, two cameras, a laser scanner and an interface, wherein the two cameras, the laser scanner and the interface are fixedly installed in the shell, and the cameras and the laser scanner are connected with the industrial personal computer through the interface.
3. A steel rail marking device based on three-dimensional visual guidance as claimed in claim 1, characterized in that: two camera symmetry's the perpendicular shell of setting in both sides, two cameras be located laser scanner's both sides, the camera lens of two cameras down, the corresponding position of bottom plate of shell all offer the hole that is used for the camera to shoot and the hole that is used for laser scanner to scan, laser scanner's laser head and camera lens orientation keep unanimous.
4. The marking method of the steel rail marking device based on three-dimensional visual guidance is characterized by comprising the following steps of:
the method comprises the following steps: the three-dimensional sensor is arranged above one side of a steel rail stopping point, and the coordinate relation between the three-dimensional sensor and a manipulator tool is calibrated in advance;
step two: after the steel rail reaches a preset position, triggering a three-dimensional sensor by a manipulator, starting scanning the steel rail by a laser scanner, respectively shooting images from a left visual angle and a right visual angle by cameras on two sides of the laser scanner, and acquiring steel rail point cloud data and storing steel rail original image data by the three-dimensional sensor after scanning is finished;
step three: the three-dimensional sensor calculates the position and posture of the marking point of the steel rail according to the point cloud data of the steel rail;
step four: the three-dimensional sensor converts the pose of the steel rail marking point into the position under the base coordinate of the manipulator, and then guides the manipulator to mark the steel rail.
5. The marking method of the steel rail marking device based on the three-dimensional visual guidance as claimed in claim 4, wherein the marking method comprises the following steps: the first step further comprises the step of establishing a mechanical hand tool coordinate system, the establishment of the mechanical hand tool coordinate system is realized by operating a mechanical hand 5 by using an XYZ six-point method, the original point O of the established mechanical hand tool coordinate system is required to be located in the middle of a mechanical hand clamping jaw 6, the Z positive direction is the direction perpendicular to a mechanical hand flange and pointing to the center of the flange, and meanwhile the average precision of the mechanical hand tool coordinate system to be established is not more than 1mm, so that the positioning and marking precision of the steel plate is guaranteed.
6. The marking method of the steel rail marking device based on the three-dimensional visual guidance as claimed in claim 5, wherein the marking method comprises the following steps: the calibration of the coordinate systems of the three-dimensional sensor and the manipulator tool needs to be carried out by means of a calibration plate which is black, the calibration plate has the function of enabling the three-dimensional sensor to uniquely identify the coordinates of each coding point in the calibration plate, further calculating the internal and external parameters of the three-dimensional sensor, calculating the calibration relation of the three-dimensional sensor and the manipulator tool coordinate system by combining the pose of the manipulator, the calibration plate is mainly placed below a camera, the camera is used for respectively calibrating the circle center coordinates (under the camera coordinate system) of each coding point 12 on the calibration plate, then a clamp is pointed to the center of a corresponding circular point (the robot also has a coordinate system), and then the relation between the robot and the camera coordinate system is established. The encoding use principle of the encoding points adopts four reference points as identification marks of the encoding points, and the angle information of three classification points and a central encoding point is used as the unique identification characteristic of the encoding points, so that the uniqueness of encoding point identification and calculation is realized;
in the calibration process of the three-dimensional sensor 3 and the manipulator tool coordinate system, a plurality of groups of pose data of the manipulator and the code point data shot by the three-dimensional sensor need to be recorded, and the calibration relation between the three-dimensional sensor 3 and the manipulator tool coordinate system is calculated by resolving the code point coordinates and the obtained manipulator pose. The calibration method when the three-dimensional sensor is installed on the manipulator is as follows:
(1) the manipulator 5 is controlled to move from the position A to the position B, the camera is calibrated before and after the movement, and the external parameters of the camera are obtained, so that Rc1 and tc1 are obtained. The robot motion parameters Rd1, td1 are read by the controller. Thereby resulting in a first set of constraints of R, t;
(2) the robot 5 is controlled to move from position B to position C and the previous step is repeated, resulting in Rc2, tc2, Rd2, td 2. Thereby resulting in a second set of constraints of R, t;
(3) the robot 5 is controlled to move from the position C to the position N, and the step (1) is repeated, thereby obtaining Rcn, tcn, Rdn, tdn. Thus, the nth set of constraints of R, t is obtained;
(4) solving R by the column equation, and solving t according to R;
(5) is composed of
Figure RE-FDA0003367674070000021
Obtaining a hand-eye calibration conversion matrix X, and finishing calibration;
wherein: rc1、tc1、Rc2、tc2Rcn and tcn are external parameters calibrated by the camera in n movements respectively; rd1、td1、Rd2、td2Rdn and tdn are parameters directly read by the robot controller in n movements, R is a rotation matrix of a relationship matrix between the robot tool and the camera to be solved, t is a translation amount of a relationship between the robot tool and the camera to be solved, and X is a relationship matrix between the robot tool and the camera.
7. The marking method of the steel rail marking device based on the three-dimensional visual guidance as claimed in claim 4, wherein the marking method comprises the following steps: in the second step, the three-dimensional sensor triangulation principle is used for obtaining the steel rail point cloud data, and the triangulation principle is as follows:
o1-xyz and O2-xyz are two-camera spatial coordinate systems, respectively; p1, P2 are a pair of homologous points; s1, S2 is the center position of the camera lens; w is a point in real space. P1, S1 defines one straight line in space, P2, S2 defines another straight line which intersects W in space;
spatial straight line: after the camera shoots an image, a straight line can be determined by an image point on the camera CCD and the center of the camera lens, the coordinates of the two points, namely the image point and the center of the lens, are in a camera coordinate system, and a space straight line equation formed by the two points is as follows:
Figure RE-FDA0003367674070000022
Figure RE-FDA0003367674070000023
wherein X, Y and Z are three-dimensional coordinates of the target point and are unknown numbers;
x, y, f are coordinates of image points, which are known quantities (obtained by analyzing the image);
xs, Ys, Zs are lens center coordinates, which are known quantities (obtained during camera calibration);
ai、bi、citransform parameters for the coordinate system, for known quantities (obtained during camera calibration);
one image can be listed with one linear equation, two images can be listed with two linear equations, 4 equation sets are formed in total, and the unknowns in the equations are only three (three-dimensional point coordinates X, Y and Z), so that three unknowns can be calculated.
8. The marking method of the steel rail marking device based on the three-dimensional visual guidance as claimed in claim 4, wherein the marking method comprises the following steps: and after carrying out rectangle fitting on the steel rail point cloud data, finding the minimum boundary of the steel rail according to the steel rail placing position and the rectangle fitting data, determining the central point of the minimum boundary, and finally determining the marking position of the steel rail according to the preset offset.
9. The marking method of the steel rail marking device based on the three-dimensional visual guidance as claimed in claim 7, wherein the marking method comprises the following steps: the steel plate workpiece coordinate system is required to be established, firstly, rectangle fitting is carried out on the steel rail point cloud data, the sequence of four points of a fitting rectangle is determined, the direction of the coordinate system X, Y is determined according to the sequence of the four points, the Z direction is determined according to a right-hand spiral rule, and the steel plate workpiece coordinate system is established.
CN202111161375.2A 2021-09-30 2021-09-30 Steel rail marking device and method based on three-dimensional visual guidance Pending CN113815323A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111161375.2A CN113815323A (en) 2021-09-30 2021-09-30 Steel rail marking device and method based on three-dimensional visual guidance

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111161375.2A CN113815323A (en) 2021-09-30 2021-09-30 Steel rail marking device and method based on three-dimensional visual guidance

Publications (1)

Publication Number Publication Date
CN113815323A true CN113815323A (en) 2021-12-21

Family

ID=78916006

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111161375.2A Pending CN113815323A (en) 2021-09-30 2021-09-30 Steel rail marking device and method based on three-dimensional visual guidance

Country Status (1)

Country Link
CN (1) CN113815323A (en)

Similar Documents

Publication Publication Date Title
CN110509300B (en) Steel hoop processing and feeding control system and control method based on three-dimensional visual guidance
CN111775146B (en) Visual alignment method under industrial mechanical arm multi-station operation
CN106041937B (en) A kind of control method of the manipulator crawl control system based on binocular stereo vision
CN110421562B (en) Mechanical arm calibration system and calibration method based on four-eye stereoscopic vision
CN111089569B (en) Large box body measuring method based on monocular vision
CN110717943A (en) Method and system for calibrating eyes of on-hand manipulator for two-dimensional plane
JP2005515910A (en) Method and apparatus for single camera 3D vision guide robotics
CN110142770B (en) Robot teaching system and method based on head-mounted display device
JPWO2018043525A1 (en) Robot system, robot system control apparatus, and robot system control method
CN111531407B (en) Workpiece attitude rapid measurement method based on image processing
JP6855491B2 (en) Robot system, robot system control device, and robot system control method
CN111482964A (en) Novel robot hand-eye calibration method
CN112648938A (en) Distributed precision measurement device and method for spatial attitude of large-size flat plate type mold
CN114643578A (en) Calibration device and method for improving robot vision guide precision
CN113554757A (en) Three-dimensional reconstruction method and system for workpiece track based on digital twinning
TW201538925A (en) Non-contact measurement device and method for object space information and the method thereof for computing the path from capturing the image
CN215970736U (en) Steel rail marking device based on three-dimensional visual guidance
CN115284292A (en) Mechanical arm hand-eye calibration method and device based on laser camera
CN112529856A (en) Method for determining the position of an operating object, robot and automation system
CN112958960A (en) Robot hand-eye calibration device based on optical target
CN110992416A (en) High-reflection-surface metal part pose measurement method based on binocular vision and CAD model
CN212578650U (en) Detection apparatus for position appearance error of arm
CN113815323A (en) Steel rail marking device and method based on three-dimensional visual guidance
CN116465335A (en) Automatic thickness measurement method and system based on point cloud matching
CN113015604A (en) Robot control system and robot control method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination