CN112170825A - Long nozzle replacing method, equipment, terminal and medium based on visual servo - Google Patents
Long nozzle replacing method, equipment, terminal and medium based on visual servo Download PDFInfo
- Publication number
- CN112170825A CN112170825A CN202011073838.5A CN202011073838A CN112170825A CN 112170825 A CN112170825 A CN 112170825A CN 202011073838 A CN202011073838 A CN 202011073838A CN 112170825 A CN112170825 A CN 112170825A
- Authority
- CN
- China
- Prior art keywords
- block
- posture
- long nozzle
- end effector
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B22—CASTING; POWDER METALLURGY
- B22D—CASTING OF METALS; CASTING OF OTHER SUBSTANCES BY THE SAME PROCESSES OR DEVICES
- B22D41/00—Casting melt-holding vessels, e.g. ladles, tundishes, cups or the like
- B22D41/50—Pouring-nozzles
- B22D41/56—Means for supporting, manipulating or changing a pouring-nozzle
Abstract
The invention provides a long nozzle replacing method, equipment, a terminal and a medium based on visual servo.A visual positioning mark block is assembled on a tundish to obtain a current image comprising the visual positioning mark block and determine the current mark block posture of the visual positioning mark block; the invention also provides visual servo long nozzle replacing equipment, a terminal and a medium, which realize that the corresponding current attitude of the end effector is determined according to different positions of the tundish, and the robot is guided to reach the current attitude point of the end effector to complete the replacing work of the long nozzle, so that the automation of replacing the long nozzle is realized, the manual operation with high strength and high risk is replaced, the method is simple and easy to implement, and the practicability and the applicability are higher.
Description
Technical Field
The invention relates to the field of industrial automation, in particular to a method, equipment, a terminal and a medium for replacing a long nozzle based on visual servo.
Background
In a tundish platform of the ferrous metallurgy continuous casting process, equipment for connecting a steel ladle and a tundish is called a long nozzle and is also called a protective sleeve, and the protective sleeve is arranged below the steel ladle and is connected with a lower nozzle of a sliding plate mechanism. The long nozzle has a certain service life and needs to be periodically assembled and disassembled in the operation process of the continuous casting tundish.
In the related art, aiming at a metallurgical continuous casting production site, a modern continuous casting machine adopts a long nozzle manipulator to realize the replacement of a long nozzle by an electromechanical driving device instead of manual replacement of the long nozzle, but the operation cannot realize full automation and still needs to be operated on site by workers. Because the continuous casting tundish operation field has high temperature and high danger, the labor risk of workers is higher.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, the present invention aims to provide a method, an apparatus, a terminal and a medium for replacing a long nozzle based on visual servo, which are used for solving the technical problems that the replacement of the long nozzle in a metallurgical continuous casting production site still needs to be operated by a worker on site, and the worker has high labor risk and high danger.
In view of the above problems, the present invention provides a method for replacing a long nozzle based on visual servoing, comprising:
assembling the visual positioning mark block on the tundish;
acquiring a current image of the visual positioning marker block and determining the current marker block posture of the visual positioning marker block;
determining a current end effector posture according to the current mark block posture and a relative relation, wherein the relative relation comprises mapping between the mark block posture and the end effector posture;
and controlling the end effector to reach the current end effector posture, and installing and/or disassembling the long nozzle.
Optionally, controlling the end effector to reach the current end effector posture, and installing the long nozzle includes:
the end effector acquires a long nozzle;
controlling the end effector to reach the current end effector pose;
and installing the long nozzle.
Optionally, the long nozzle of dismantlement includes:
if the long nozzle is installed, acquiring an exit point of the end effector and determining a disassembly posture;
and controlling the end effector to reach the disassembling posture, and disassembling the long nozzle.
Optionally, the relative relationship includes a relative posture transformation matrix, and the relative posture transformation matrix is determined by the following method:
acquiring an initial image of a visual positioning mark block at a preset shooting point;
determining an initial marker block pose of the visual positioning marker block according to the initial image;
the servo-teaching robot installs the long nozzle, and obtains the initial end effector posture of the end effector when the long nozzle is installed;
and determining a relative attitude transformation matrix according to the initial mark block attitude and the initial end effector attitude.
Optionally, before acquiring the current image of the visual positioning marker block and determining the current marker block posture of the visual positioning marker block, the method further includes:
and determining a reference posture transformation matrix from a camera coordinate system to a robot coordinate system according to the hand-eye calibration and the joint parameter feedback of the robot control system.
Optionally, the visual locator block includes a central sub-flag block and at least two peripheral sub-flag blocks, the peripheral sub-flag blocks are disposed around the central sub-flag block, and the determining the current flag block posture of the visual locator block includes:
determining a region of interest in the current initial image, the region of interest comprising the visual localization marker block imaging region;
acquiring pixel information of preset points in the outline of each sub-mark block in the region of interest and relative position information between the preset points in the outline of each sub-mark block;
determining the posture of a preset point in the outline of the center sub-mark block under a camera coordinate system according to the pixel information and the relative position information;
and determining the posture of the current mark block according to the standard posture transformation matrix and the posture of a preset point in the outline of the center sub-mark block under a camera coordinate system.
Optionally, at least one of the following is also included:
the visual positioning mark block is fixedly arranged on the continuous casting tundish sliding plate mechanism;
acquiring the current image at a preset shooting point through image acquisition equipment, wherein the image acquisition equipment is assembled at the joint tail end of the robot;
the visual locator block comprises at least 3 different sub-marker blocks;
the visual positioning mark block comprises at least two background colors, and the current background color of the visual positioning mark block is determined according to the background color of the scene.
The invention also provides a long nozzle replacing device based on visual servo, which comprises:
the assembly module is used for assembling the visual positioning mark block on the tundish;
a current marker block posture acquisition module, configured to acquire a current image of the visual positioning marker block and determine a current marker block posture of the visual positioning marker block;
a determination module for determining a current end effector pose according to the current marker block pose and a relative relationship, the relative relationship comprising a mapping between the marker block pose and the end effector pose;
and the control module is used for controlling the end effector to reach the current attitude of the end effector and installing and/or disassembling the long nozzle.
The invention also provides a terminal, which comprises a processor, a memory and a communication bus;
the communication bus is used for connecting the processor and the memory;
the processor is configured to execute a computer program stored in the memory to implement the method for changing a long nozzle based on visual servoing as described in one or more of the above embodiments.
The present invention also provides a computer-readable storage medium, having stored thereon a computer program,
the computer program is for causing the computer to perform the method for visual servo based long nozzle replacement as described in any of the above embodiments.
As described above, the method, the device, the terminal and the medium for replacing the long nozzle based on the visual servo provided by the invention have the following beneficial effects:
assembling a visual positioning mark block on a tundish, acquiring a current image comprising the visual positioning mark block, determining the current mark block posture of the visual positioning mark block, determining the current end effector posture according to the current mark block posture and a relative relation, wherein the relative relation comprises mapping between the mark block posture and the end effector posture, controlling an end effector to reach the current end effector posture, and installing and/or dismantling a long nozzle; the method has the advantages that the gesture of the current installation end effector required by long nozzle replacement installation is calculated according to different positions of the tundish, the robot is guided to reach the gesture point of the current installation end effector to complete replacement installation work of the long nozzle, automation of replacement installation of the long nozzle is realized, high-strength and high-risk manual operation is replaced, the method is simple and easy to implement, and the practicability and the applicability are high.
Drawings
Fig. 1 is a schematic flow chart of a method for replacing a long nozzle based on visual servoing according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a visual alignment mark block according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of a method for replacing a long nozzle based on visual servoing according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a long nozzle replacing device based on visual servoing according to a third embodiment of the present invention;
fig. 5 is a schematic structural diagram of a terminal according to a fourth embodiment of the present invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
Example one
Referring to fig. 1, an embodiment of the present invention provides a method for replacing a long nozzle based on a visual servo, including:
s101: and assembling the visual positioning mark block on the tundish.
In some embodiments, the visual alignment marker block is fixedly mounted on a slide mechanism of the continuous casting tundish.
In some embodiments, the visual locator block includes at least 3 different sub-marker blocks.
In some embodiments, the visual locator block includes at least two background colors, and the current background color of the visual locator block is determined from the live background colors. Optionally, the outline shapes of the sub-flag blocks are the same, and the sizes of the sub-flag blocks are different.
Optionally, the size of each sub-mark block may be adjusted according to an actual working condition, and is not limited herein, and the background color of each sub-mark block is adjusted according to a field environment, and is not limited herein.
Optionally, the outline shapes of the sub-flag blocks are different, and the size of each sub-flag block is not limited herein.
Optionally, the material of each sub-flag block is not limited herein.
In some embodiments, referring to fig. 2, fig. 2 is a schematic structural diagram of a visual alignment mark block, where the visual alignment mark block includes 5 sub-mark blocks with circular outlines with different radii, the sub-mark block with the smallest radius of the circular outline is a central sub-mark block a, and the remaining 4 sub-mark blocks are uniformly distributed around the central sub-mark block a. Optionally, the preset points include circle centers of respective circle profiles, a coordinate system of the mark block is established on the visual positioning mark block with the circle center of the circle profile of the center sub-mark block a as an origin of coordinates, coordinates of the circle centers of the circle profiles of the 4 sub-mark blocks distributed around the center sub-mark block a in the coordinate system of the mark block can be directly designed and obtained in a mark block design stage, and coordinates of the circle centers of the circle profiles of the 4 sub-mark blocks distributed around the center sub-mark block a in the coordinate system of the mark block can also be obtained by measurement, so that relative position information of the circle centers of the respective circle profiles can be obtained.
In some embodiments, the contrast between the imaging of the background color of each sub-flag block at the image capturing device and the imaging of the live background at the image capturing device is greater than a first preset threshold. Optionally, the larger the contrast between the imaging of the background of the ground color of each sub-mark block in the image acquisition device and the imaging of the field background in the image acquisition device is, the more convenient the replacement of the long nozzle is, and the calculated amount is smaller.
S102: the current image of the visual alignment marker block is obtained and the current marker block posture of the visual alignment marker block is determined.
In some embodiments, the current image is acquired at a preset shooting point by an image capturing device mounted at the joint end of the robot.
In some embodiments, the image capture device is mounted on an articulating end tool flange of the robot.
Optionally, the image capture device comprises an industrial camera.
In some embodiments, the image capture device comprises a monocular camera.
In some embodiments, the current marker block pose is generated based on a preset coordinate system. Optionally, the preset coordinate system comprises a robot base coordinate system.
It should be understood that the preset coordinate system may be other coordinate systems specified by those skilled in the art, and the method for replacing the long nozzle according to the present embodiment may also be used after converting the postures of the respective coordinate systems including the visual alignment mark block, the end effector, and the like into postures based on the preset coordinate system.
In some embodiments, prior to obtaining the current image of the visual landmark block and determining the current landmark block pose of the visual landmark block, further comprising:
and determining a reference posture transformation matrix from the camera coordinate system to the robot coordinate system according to the hand-eye calibration and the joint parameter feedback of the robot control system.
Optionally, the reference posture transformation matrix includes a coordinate system transformation relationship from a camera coordinate system where the image acquisition device is located to a robot base coordinate system.
Optionally, if the image capturing device for acquiring the current image is disposed at the joint end of the robot, the reference posture transformation matrix may be determined as follows:
determining a camera coordinate system where the image acquisition equipment is located and a first posture transformation matrix of a terminal tool coordinate system where a terminal tool of the robot is located through hand-eye calibration operation;
determining a second posture transformation matrix from the terminal tool coordinate system to the robot base coordinate system through real-time joint parameter feedback of the robot control system;
and determining a reference attitude transformation matrix from the camera coordinate system to the robot base coordinate system according to the first attitude transformation matrix and the second attitude transformation matrix.
It should be noted that the image capturing device may also be installed at other positions of the robot, and calibration operation is performed by a method of the related art, so as to calculate a preset reference posture transformation matrix from the camera coordinate system to the robot base coordinate system.
In some embodiments, the image capturing device may also be mounted on other fixed structures besides the robot structure, the fixed structures may be fixed, and the fixed structures may also move along with the movement of the robot joint end joint, so as to realize the relative stillness of the image capturing device and the robot joint end joint. At this time, those skilled in the art may also adopt the related art to obtain the preset reference posture transformation matrix.
Optionally, the posture of the visual positioning mark block relative to the robot base coordinate system may be obtained according to the posture of the visual positioning mark block in the camera coordinate system and the preset reference posture transformation matrix.
In some embodiments, the visual locator block includes a central sub-flag block and at least two peripheral sub-flag blocks, the peripheral sub-flag blocks being disposed around the central sub-flag block, the determining the current flag block pose of the visual locator block including:
determining a region of interest in the current image, the region of interest comprising a visual positioning marker block imaging region;
acquiring pixel information of preset points in the outline of each sub-mark block in the region of interest and relative position information between the preset points in the outline of each sub-mark block;
determining the posture of a preset point in the outline of the center sub-mark block under a camera coordinate system according to the pixel information and the relative position information;
and determining the posture of the mark block according to the standard posture transformation matrix and the posture of a preset point in the outline of the central sub-mark block under a camera coordinate system.
Optionally, the current pose of the visual positioning marker block in the camera coordinate system includes a pose of a preset point in the outline of the center sub-marker block in the camera coordinate system.
Optionally, the region of interest includes an image of each sub-marker block, the contour extraction module extracts a contour of each sub-marker block in the region of interest according to a contour shape of each sub-marker block, obtains pixel information of a preset point in the contour of each sub-marker block and relative position information between the preset points, and based on the pixel information and the relative position information, a monocular vision PnP algorithm may be used to calculate a three-dimensional pose of the center sub-marker block in the camera coordinate system, and the three-dimensional pose is regarded as a pose of the visual positioning marker block in the camera coordinate system.
In some embodiments, the visual positioning mark block includes at least 4 circular contour sub-mark blocks with different radii, wherein the sub-mark block with the smallest radius is used as a center sub-mark block, the rest sub-mark blocks are uniformly distributed around the center sub-mark block, the region of interest includes the image of each sub-mark block, the contour extraction module processes the region of interest to obtain the pixel information of the center of each circular contour, the mark block posture determination module calculates the three-dimensional posture of the circle center of the center sub-mark block under the camera coordinate system by using a monocular visual three-dimensional positioning algorithm according to the obtained pixel information of each circular contour and the coordinate relative relationship under the mark block coordinate system of the circle center of each circular contour, and regards the three-dimensional posture as the posture of the visual positioning mark block under the camera coordinate system, wherein the coordinate relative position relationship under the mark block coordinate system of the circle center of each circular contour can be obtained by pre-measurement calculation, or recording the relative position relation between the centers of the circles of the various circle profiles after the design and processing of the visual positioning mark block are finished. Optionally, the radius of the circular contour can be adjusted according to actual working conditions, so that the image acquisition equipment can accurately image the visual positioning mark block clearly.
In some embodiments, after the relative position relationship of the preset points is designed and processed by the visual positioning mark block, the relative position relationship between the preset points is recorded, or the relative position relationship between the preset points is obtained by means of pre-measurement calculation and the like, the preset point is a certain point in the outline of the sub-mark block, and the preset point can be preset by a person skilled in the art as required.
In some embodiments, acquiring a current image including a visual landmark block and determining a current landmark block pose of the visual landmark block includes:
and calculating to obtain the three-dimensional posture of the visual positioning mark block in the camera coordinate system by adopting a monocular vision PnP algorithm including but not limited to according to the current image, and taking the three-dimensional posture as the posture of the visual positioning mark block in the camera coordinate system.
S103: and determining the current attitude of the end effector according to the current attitude and the relative relation of the marker block.
Optionally, the relative relationship comprises a mapping between the marker block pose and the end effector pose.
In some embodiments, the relative relationship may be obtained by enumerating the positions of the tundish at different positions, visually locating the marker block postures of the marker block, and in the marker block postures, the end effector of the robot reaches the end effector posture of the long nozzle mounting point, and constructing a basic table based on the marker block postures and the end effector postures, and regarding the basic table as the relative relationship. When the current marker block pose is determined, the corresponding end effector pose is looked up based on the base table.
In some embodiments, the relative relationship comprises a relative pose transformation matrix.
In some embodiments, the relative pose transformation matrix is generated from an initial marker block pose and an initial end effector pose, wherein the initial marker block pose is determined from an initial image taken by the servo robot of a successful installation of the long nozzle at a predetermined capture point that includes the visual alignment marker block; the initial end effector pose comprises a pose instructing the robot to successfully install the long nozzle, the end effector of the robot.
Optionally, the relative posture transformation matrix is determined as follows:
acquiring an initial image of a visual positioning mark block at a preset shooting point;
determining an initial marker block posture of the visual positioning marker block according to the initial image;
the servo-teaching robot is used for installing the long nozzle to obtain the initial tail end executor posture of the tail end executor when the long nozzle is installed;
a relative pose transformation matrix is determined from the initial marker block pose and the initial end effector pose.
Optionally, because the visual positioning mark block is assembled on the steelmaking continuous casting tundish, the mark block posture of the visual positioning mark block can also represent the posture of the tundish, so that the long nozzle replacement method based on the visual servo provided by the embodiment can realize matching of the corresponding current end effector posture according to the current position of the tundish, and further realize automatic loading and unloading of the long nozzle.
Optionally, if the relative position between the visual positioning mark block and the drainage port of the tundish slide plate mechanism is fixed, the relative posture transformation matrix is fixed.
Optionally, if the relative position between the visual positioning mark block and the lower nozzle of the tundish slide plate mechanism changes after the servo robot successfully installs the long nozzle, the relative attitude transformation matrix needs to be determined again.
Optionally, the servo robot can successfully install the long nozzle by manually performing servo operation to complete the installation of the long nozzle.
Optionally, if the image acquisition device is fixedly installed at the tail end of the robot joint, the preset shooting point can be any point on a path through which the tail end of the robot joint passes when the servo-teaching robot successfully installs the long nozzle.
Optionally, the current image is captured by the image capturing device at a preset capturing point.
Optionally, the shooting position of the current image may be other positions except the preset shooting point, that is, the shooting location of the current image may be different from the shooting location of the initial image, at this time, a posture conversion relationship between the current image shooting location and the image shot by the initial image shooting location needs to be created, and finally, the posture of the visual positioning marker block in the current image is converted into the posture of the current marker block based on the robot base coordinate system.
If the preset imaging point is determined in advance, the execution sequence between step S102 and step S103 is not limited herein. If the preset shooting point is not set in advance, the step S103 needs to be executed first, and then the step S102 is executed according to the shooting position of the initial image in the step S103 as the preset shooting point.
In some embodiments, each pose is normalized based on data in a predetermined coordinate system. The preset coordinate system includes, but is not limited to, a robot base coordinate system, and at this time, the current sign block posture, the relative posture transformation matrix, the current installation posture and the like are all postures in the robot base coordinate system. It should be understood that the preset coordinate system may also be another coordinate system set by a person skilled in the art, and at this time, after the posture including the visual positioning mark block, the end effector, and the like is converted into the posture under the preset coordinate system, the method for replacing the long nozzle based on the visual servo provided in this embodiment may be applied, so as to implement automatic replacement of the long nozzle.
S104: and controlling the end effector to reach the current attitude of the end effector, and installing and/or disassembling the long nozzle.
In some embodiments, when the robot adjusts the end effector to reach the current end effector posture, the end effector of the robot reaches a position point where the robot can start to directly start the installation of the long nozzle, and the end effector can install the long nozzle according to a preset installation mode.
In some embodiments, the predetermined mounting pattern may be a mounting pattern of the long nozzle when the servo robot is instructed to successfully mount the long nozzle. The replacement and installation modes of the long nozzle of the same tundish are unified, so that the preset installation mode can be set again even if the visual positioning mark block moves. Similarly, the manner of removing the long nozzle can also be determined by means including, but not limited to, parameter input, manual instruction, etc.
In some embodiments, controlling the end effector to reach the current end effector pose, installing the long nozzle comprises:
the end effector acquires a long nozzle;
controlling the end effector to reach the current end effector posture;
and (4) installing a long nozzle.
In some embodiments, disassembling the elongated gate comprises:
if the long nozzle is installed, obtaining the exit point of the end effector and determining the disassembly posture;
and controlling the end effector to reach a disassembly gesture, and disassembling the long nozzle.
In some embodiments, after the long nozzle is successfully installed, the end effector has an exit mounting point, and the posture of the exit point is determined as the disassembling posture which the end effector needs to reach when the long nozzle is disassembled.
Alternatively, the disassembly attitude may be determined by joint parameter feedback of the robot control system.
Optionally, when the long nozzle needs to be disassembled, the robot is guided to move to the disassembling posture, and the disassembling operation of the long nozzle is completed.
Since the position of the drain port is not fixed, the detaching posture of the exit mounting point (exit point) of the end effector is determined again after the robot successfully mounts the long drain port each time, and the detaching posture is regarded as the detaching posture which the end effector needs to reach when the long drain port is detached. When the long nozzle needs to be disassembled, the robot is guided to move to the disassembling posture, and the disassembling operation of the long nozzle is completed.
In some embodiments, after the robot finishes mounting or dismounting the long nozzle, the robot retracts to a specified position for standby.
In some embodiments, one robot may be responsible for loading and unloading one or more tundish long nozzles, and at this time, for different tundish initial shooting positions, the same preset shooting point or different preset shooting points may be set by a person skilled in the art as needed.
In some embodiments, when the robot is responsible for loading and unloading a plurality of tundish long nozzles and the corresponding preset shooting points of each tundish are different, the visual positioning mark blocks arranged on each tundish may be the same or different. When the robot is responsible for the loading and unloading of a plurality of tundish long nozzles, the corresponding preset shooting points of each tundish are the same, and the models of the long nozzles corresponding to each tundish are different, and the visual positioning mark blocks arranged on each tundish are different. At the moment, the model of the long nozzle needing to be assembled and disassembled currently can be determined according to each visual positioning mark block, and then the long nozzle is assembled and disassembled by adopting a corresponding assembling mode or disassembling mode.
In some embodiments, if the robot is responsible for loading and unloading a plurality of tundish long nozzles, corresponding identification marks are correspondingly arranged on each tundish, and the corresponding disassembly postures of each tundish are recorded according to the identification marks so as to facilitate the disassembly of the subsequent long nozzles.
The embodiment of the invention provides a long nozzle replacing method based on visual servo, which comprises the steps of assembling a visual positioning mark block on a tundish, obtaining a current image comprising the visual positioning mark block, determining the current mark block posture of the visual positioning mark block, determining the current end effector posture according to the current mark block posture and a relative relation, wherein the relative relation comprises mapping between the mark block posture and the end effector posture, controlling an end effector to reach the current end effector posture, and installing and/or disassembling a long nozzle; the method has the advantages that the current end effector posture required by long nozzle replacement is calculated according to different positions of the tundish, the robot is guided to reach the current end effector posture to finish long nozzle replacement, long nozzle replacement automation is realized, high-strength and high-risk manual operation is replaced, the method is simple and easy to implement, and the practicability and the applicability are high.
Optionally, obtaining a current image including the visual positioning mark block and determining a current mark block posture of the visual positioning mark block, assembling the visual positioning mark block on the tundish, obtaining a relative posture transformation matrix, determining a current end effector posture according to the current mark block posture and the relative posture transformation matrix, enabling the end effector to reach the current end effector posture, and installing and/or dismantling the long nozzle; the method has the advantages that the corresponding current posture of the end effector is determined according to different positions of the tundish, the robot is guided to reach the current posture of the end effector to finish the long nozzle replacing work, the long nozzle replacing automation is realized, high-strength and high-risk manual operation is replaced, the method is simple and easy to implement, and the practicability and the applicability are high.
Example two
The method for replacing the long nozzle based on visual servo provided by the present embodiment is exemplified by a specific embodiment, referring to fig. 3, the specific method for replacing the long nozzle based on visual servo includes:
s301: and determining a visual positioning mark block, and fixedly installing the visual positioning mark block on a sliding plate mechanism of the tundish.
In some embodiments, the outline of the visual locator block is a circular outline.
Optionally, the background color background, the shape and the size of the visual positioning mark block and the radius of the circular contour can be determined according to the field environment, and after the visual positioning mark block is designed and processed, the relative position relation between the centers of the circular contour is recorded.
In some embodiments, as shown in fig. 2, the structure of each sub-marker block on the visual positioning marker block may be schematically illustrated, where 5 sub-marker blocks with different radii are provided on the visual positioning marker block, the sub-marker block with the smallest radius is the central sub-marker block a, and the remaining 4 sub-marker blocks are uniformly distributed around the central sub-marker block a. At this time, the preset points in the sub-mark block contour are set as the center of the circle, and the relative position information between the preset points in each sub-mark block contour can be determined as follows: establishing a mark block coordinate system on the visual positioning mark block by taking the circle center of the center sub-mark block A as a coordinate origin; the coordinates of the centers of the 4 circular outlines distributed around the center sub-mark block A in the mark block coordinate system can be directly designed or measured at the later stage in the mark block design stage; the relative position information is determined based on the coordinates of the centers of the 4 circular contours distributed around the center sub-mark block a.
Optionally, the shape, size and background of the visual positioning mark block are determined according to the continuous casting tundish production operation field environment by taking clear imaging of the shooting equipment and obvious imaging characteristics of the visual positioning mark block as standards.
S302: and acquiring a relative attitude transformation matrix.
Optionally, the relative posture transformation matrix includes a posture transformation matrix between the posture of the initial visual positioning mark block under the robot base coordinate system and the posture of the initial end effector of the robot when the long nozzle is successfully installed.
In some embodiments, the reference pose transformation matrix is determined as follows:
fixedly installing an industrial camera at the tail end of a robot joint, and calculating a first posture transformation matrix of a camera coordinate system and tail end tool coordinates through hand-eye calibration operation;
calculating a second attitude transformation matrix from the terminal tool coordinate system to the robot base coordinate system through real-time joint parameter feedback of the robot control system;
and determining a reference attitude transformation matrix from the camera coordinate system to the robot base coordinate system according to the first attitude transformation matrix and the second attitude transformation matrix.
In some embodiments, the relative pose transformation matrix is determined as follows:
the method comprises the following steps that a manual servo-teaching robot carries out long water gap installation operation, a preset shooting point is set on a servo-teaching path to serve as an industrial camera shooting point, an industrial camera at the preset shooting point collects an initial image of a visual positioning mark block, the posture of the visual positioning mark block under a camera coordinate system is calculated based on the initial image, and the posture of the visual positioning mark block under the camera coordinate system is converted into the initial mark block posture of the visual positioning mark block under a robot base coordinate system according to a reference posture transformation matrix;
recording the initial tail end executor posture of the tail end executor under the robot base coordinate system when the servo teaching is finished to install the long nozzle;
and generating a relative attitude transformation matrix according to the initial mark block attitude and the initial end effector attitude.
Optionally, the visual positioning marker block includes at least 4 circular contours with different radii, the circular contour with the smallest radius is used as a central circular contour, and the remaining circular contours are distributed around the central circular contour, and the initial marker block posture is obtained in the following manner:
processing an initial image shot by an industrial camera, and extracting an image interesting region containing each sub-mark block imaging region;
extracting pixel information of the centers of all circle profiles in the region of interest;
calculating the three-dimensional posture of the center circle of the circular outline under a camera coordinate system by adopting a monocular vision PnP algorithm according to the obtained pixel information of the center of the circular outline and the relative position relationship between the centers of the circles, and taking the three-dimensional posture as the posture of a visual positioning mark block under the camera coordinate system;
and determining the initial mark block posture according to the posture of the visual positioning mark block in the camera coordinate system and the reference posture transformation matrix.
The relative position relationship between the centers of circles may be determined when designing and manufacturing the visual positioning mark block, or may be obtained in advance through measurement and calculation, or may be obtained through other related technical means, which is not limited herein.
Optionally, the relative position between the visual positioning mark block and the drainage port of the sliding plate mechanism is fixed, and the relative posture transformation matrix is fixed.
S303: and acquiring the current mark block posture.
Optionally, the industrial camera shoots a current image including the visual positioning mark block at a preset shooting point, determines the posture of the current visual positioning mark block in the camera coordinate system according to the current image based on a monocular vision PnP algorithm and other manners, and converts the posture of the current visual positioning mark block in the camera coordinate system into the posture of the current mark block of the visual positioning mark block in the robot base coordinate system according to the reference posture transformation matrix.
S304: and determining the current attitude of the end effector according to the current attitude of the mark block and the relative attitude transformation matrix, and installing the long nozzle.
Optionally, when the long nozzle needs to be automatically assembled and disassembled, the current posture of the end effector is calculated to guide the end effector of the robot to move to the current posture of the end effector, and then the operation of installing the long nozzle is completed.
An exemplary installation process is as follows: and (3) the robot moves to a preset shooting point, the industrial camera acquires a current image, calculates the current mark block posture of the current visual positioning mark block under the robot base coordinate system through image processing and posture transformation, calculates the current end effector posture by combining the relative posture transformation matrix obtained in the step (S302), guides the robot to move to the current end effector posture and finishes the operation of installing the long nozzle.
S305: and obtaining the exit point of the end effector, determining the disassembly gesture, controlling the end effector to reach the disassembly gesture, and disassembling the long nozzle.
Optionally, after the long nozzle is installed, the end effector has an exit mounting point, the posture of the exit point is recorded as a disassembly posture, and the disassembly posture is regarded as the disassembly posture which the end effector needs to reach when the long nozzle is disassembled. When the long nozzle needs to be disassembled, the robot is guided to move to the disassembling posture, and the disassembling operation of the long nozzle is completed.
The embodiment of the invention provides a specific long nozzle replacement method based on visual servo, which comprises the steps of obtaining a current mark block image of a visual positioning mark block through an industrial camera, wherein the industrial camera is installed at the tail end of a joint of an industrial robot, the visual positioning mark block is fixedly installed on a continuous casting tundish sliding plate mechanism, obtaining the current mark block posture of the visual positioning mark block under a robot base coordinate system through image processing and coordinate transformation, obtaining the current tail end actuator posture which is required to be reached by a tail end actuator when a long nozzle is installed at the current moment according to the calculation of a relative transformation matrix, guiding the robot to reach the current tail end actuator posture and completing long nozzle installation operation; the automatic installation of the long nozzle is realized, the manual operation with high strength and high risk is replaced, the method is simple and easy to implement, and the practicability and the applicability are higher. Meanwhile, the current position of the tundish can be obtained based on the position of the visual positioning mark block in the current image, so that the robot can calculate the current attitude of the end effector required for replacing the long nozzle in real time according to different positions of the tundish, and the robot is guided to reach the current attitude of the end effector to complete the replacement work of the long nozzle.
Optionally, the withdrawing point of the end effector after the long nozzle is successfully installed is recorded, the disassembling gesture is determined, the robot is guided to move to the disassembling gesture when the long nozzle needs to be disassembled, the disassembling operation of the long nozzle is completed, the automation of disassembling the long nozzle is realized, the manual operation with high strength and high risk is replaced, the method is simple and easy to implement, and the practicability and the applicability are high.
EXAMPLE III
Referring to fig. 4, a long nozzle replacing apparatus 400 based on visual servo includes:
an assembling module 401, configured to assemble the visual positioning mark block on the tundish;
a current marker block posture acquiring module 402, configured to acquire a current image of the visual positioning marker block and determine a current marker block posture of the visual positioning marker block;
a determining module 403, configured to determine a current end effector pose according to the current flag block pose and a relative relationship, where the relative relationship includes mapping between the flag block pose and the end effector pose;
and the control module 404 is used for controlling the end effector to reach the current attitude of the end effector, and installing and/or disassembling the long nozzle.
In this embodiment, the long nozzle replacing apparatus based on visual servo is substantially provided with a plurality of modules for executing the long nozzle replacing method based on visual servo in the above embodiments, and specific functions and technical effects are as described in the first embodiment, and are not described herein again.
Example four
Referring to fig. 5, an embodiment of the present invention further provides a terminal 500, including a processor 501, a memory 502, and a communication bus 503;
a communication bus 503 is used to connect the processor 501 and the memory 502;
the processor 501 is configured to execute a computer program stored in the memory 502 to implement the method for changing a long nozzle based on visual servoing as described in one or more of the first embodiment above.
An embodiment of the present invention also provides a computer-readable storage medium, characterized in that, a computer program is stored thereon,
the computer program is adapted to make the computer execute the method for changing a long nozzle based on visual servoing as described in any one of the above embodiments.
Embodiments of the present application also provide a non-transitory readable storage medium, where one or more modules (programs) are stored in the storage medium, and when the one or more modules are applied to a device, the device may execute instructions (instructions) included in an embodiment of the present application.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (10)
1. A long nozzle replacing method based on visual servo is characterized by comprising the following steps:
assembling the visual positioning mark block on the tundish;
acquiring a current image of the visual positioning marker block and determining the current marker block posture of the visual positioning marker block;
determining a current end effector posture according to the current mark block posture and a relative relation, wherein the relative relation comprises mapping between the mark block posture and the end effector posture;
and controlling the end effector to reach the current end effector posture, and installing and/or disassembling the long nozzle.
2. The method of claim 1, wherein the controlling the end effector to the current end effector position, and installing the long nozzle comprises:
the end effector acquires a long nozzle;
controlling the end effector to reach the current end effector pose;
and installing the long nozzle.
3. The method for replacing a long nozzle based on visual servoing as set forth in claim 2, wherein said disassembling the long nozzle comprises:
if the long nozzle is installed, acquiring an exit point of the end effector and determining a disassembly posture;
and controlling the end effector to reach the disassembling posture, and disassembling the long nozzle.
4. A long nozzle changing method based on visual servoing according to claim 1, characterized in that said relative relationship comprises a relative attitude transformation matrix determined by:
acquiring an initial image of a visual positioning mark block at a preset shooting point;
determining an initial marker block pose of the visual positioning marker block according to the initial image;
the servo-teaching robot installs the long nozzle, and obtains the initial end effector posture of the end effector when the long nozzle is installed;
and determining a relative attitude transformation matrix according to the initial mark block attitude and the initial end effector attitude.
5. The long nozzle replacement method based on visual servoing as set forth in any one of claims 1 to 4, wherein before said obtaining a current image of a visual alignment mark block and determining a current mark block attitude of the visual alignment mark block, further comprising:
and determining a reference posture transformation matrix from a camera coordinate system to a robot coordinate system according to the hand-eye calibration and the joint parameter feedback of the robot control system.
6. The visual servo-based long nozzle replacement method of claim 5, wherein the visual alignment marker block comprises a central sub-marker block and at least two peripheral sub-marker blocks, the peripheral sub-marker blocks being disposed around the central sub-marker block, the determining the current marker block pose of the visual alignment marker block comprising:
determining a region of interest in the current image, the region of interest comprising the visual localization marker block imaging region;
acquiring pixel information of preset points in the outline of each sub-mark block in the region of interest and relative position information between the preset points in the outline of each sub-mark block;
determining the posture of a preset point in the outline of the center sub-mark block under a camera coordinate system according to the pixel information and the relative position information;
and determining the posture of the current mark block according to the standard posture transformation matrix and the posture of a preset point in the outline of the center sub-mark block under a camera coordinate system.
7. A long nozzle changing method based on visual servo according to any of claims 1-4, characterized by further comprising at least one of the following:
the visual positioning mark block is fixedly arranged on the continuous casting tundish sliding plate mechanism;
acquiring the current image at a preset shooting point through image acquisition equipment, wherein the image acquisition equipment is assembled at the joint tail end of the robot;
the visual locator block comprises at least 3 different sub-marker blocks;
the visual positioning mark block comprises at least two background colors, and the current background color of the visual positioning mark block is determined according to the background color of the scene.
8. A long nozzle replacing device based on visual servo is characterized by comprising:
the assembly module is used for assembling the visual positioning mark block on the tundish;
a current marker block posture acquisition module, configured to acquire a current image of the visual positioning marker block and determine a current marker block posture of the visual positioning marker block;
a determination module for determining a current end effector pose according to the current marker block pose and a relative relationship, the relative relationship comprising a mapping between the marker block pose and the end effector pose;
and the control module is used for controlling the end effector to reach the current attitude of the end effector and installing and/or disassembling the long nozzle.
9. A terminal comprising a processor, a memory, and a communication bus;
the communication bus is used for connecting the processor and the memory;
the processor is adapted to execute a computer program stored in the memory to implement the method of visual servo based long nozzle replacement according to one or more of claims 1-7.
10. A computer-readable storage medium, having stored thereon a computer program,
the computer program is for causing the computer to perform the method for visual servo based long nozzle replacement according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011073838.5A CN112170825B (en) | 2020-10-09 | 2020-10-09 | Long nozzle replacing method, equipment, terminal and medium based on visual servo |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011073838.5A CN112170825B (en) | 2020-10-09 | 2020-10-09 | Long nozzle replacing method, equipment, terminal and medium based on visual servo |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112170825A true CN112170825A (en) | 2021-01-05 |
| CN112170825B CN112170825B (en) | 2022-07-12 |
Family
ID=73948619
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011073838.5A Active CN112170825B (en) | 2020-10-09 | 2020-10-09 | Long nozzle replacing method, equipment, terminal and medium based on visual servo |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112170825B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0679416A (en) * | 1992-09-01 | 1994-03-22 | Sumitomo Heavy Ind Ltd | Detector for long nozzle fitting position |
| JPH07185754A (en) * | 1993-12-28 | 1995-07-25 | Nkk Corp | Method for automatically fitting long nozzle and device therefor |
| CA2166703A1 (en) * | 1994-05-06 | 1995-11-16 | Kenji Yamamoto | Replacing device for immersion nozzles |
| CN1286150A (en) * | 1999-09-01 | 2001-03-07 | 洛阳市洛华粉体工程特种耐火材料厂 | Method and manipulator for quickly replacting sprue of tundish for conticaster |
| KR20030001661A (en) * | 2001-06-26 | 2003-01-08 | 주식회사 유한정밀 | The alignment device of automation machine and alignment method |
| CN101011729A (en) * | 2007-02-08 | 2007-08-08 | 浙江大学 | Automatic controlling method and system of detecting discharge slag from ladle |
| KR20090123610A (en) * | 2008-05-28 | 2009-12-02 | 현대제철 주식회사 | Submerged nozzle centering apparatus |
| CN107241903A (en) * | 2015-01-30 | 2017-10-10 | 首要金属科技奥地利有限责任公司 | Monitor the metallurgical tank of metal industry device |
| CN109128128A (en) * | 2018-05-07 | 2019-01-04 | 杨进 | A kind of positioning device of collector nozzle of ladle and continuous-casting steel pouring long nozzle |
| CN109570483A (en) * | 2018-11-29 | 2019-04-05 | 杨进 | A kind of continuous-casting steel pouring special manipulator of robot guidance |
| CN110976839A (en) * | 2019-12-24 | 2020-04-10 | 中冶南方工程技术有限公司 | Upper structure of continuous casting machine and automatic loading and unloading method of long nozzle |
| CN111036863A (en) * | 2018-10-15 | 2020-04-21 | 宝钢工程技术集团有限公司 | Device for positioning ladle drainage port by adopting machine vision and using method thereof |
| CN111168051A (en) * | 2020-01-04 | 2020-05-19 | 鞍钢股份有限公司 | Unmanned system for ladle pouring area and operation method thereof |
-
2020
- 2020-10-09 CN CN202011073838.5A patent/CN112170825B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0679416A (en) * | 1992-09-01 | 1994-03-22 | Sumitomo Heavy Ind Ltd | Detector for long nozzle fitting position |
| JPH07185754A (en) * | 1993-12-28 | 1995-07-25 | Nkk Corp | Method for automatically fitting long nozzle and device therefor |
| CA2166703A1 (en) * | 1994-05-06 | 1995-11-16 | Kenji Yamamoto | Replacing device for immersion nozzles |
| CN1286150A (en) * | 1999-09-01 | 2001-03-07 | 洛阳市洛华粉体工程特种耐火材料厂 | Method and manipulator for quickly replacting sprue of tundish for conticaster |
| KR20030001661A (en) * | 2001-06-26 | 2003-01-08 | 주식회사 유한정밀 | The alignment device of automation machine and alignment method |
| CN101011729A (en) * | 2007-02-08 | 2007-08-08 | 浙江大学 | Automatic controlling method and system of detecting discharge slag from ladle |
| KR20090123610A (en) * | 2008-05-28 | 2009-12-02 | 현대제철 주식회사 | Submerged nozzle centering apparatus |
| CN107241903A (en) * | 2015-01-30 | 2017-10-10 | 首要金属科技奥地利有限责任公司 | Monitor the metallurgical tank of metal industry device |
| CN109128128A (en) * | 2018-05-07 | 2019-01-04 | 杨进 | A kind of positioning device of collector nozzle of ladle and continuous-casting steel pouring long nozzle |
| CN111036863A (en) * | 2018-10-15 | 2020-04-21 | 宝钢工程技术集团有限公司 | Device for positioning ladle drainage port by adopting machine vision and using method thereof |
| CN109570483A (en) * | 2018-11-29 | 2019-04-05 | 杨进 | A kind of continuous-casting steel pouring special manipulator of robot guidance |
| CN110976839A (en) * | 2019-12-24 | 2020-04-10 | 中冶南方工程技术有限公司 | Upper structure of continuous casting machine and automatic loading and unloading method of long nozzle |
| CN111168051A (en) * | 2020-01-04 | 2020-05-19 | 鞍钢股份有限公司 | Unmanned system for ladle pouring area and operation method thereof |
Non-Patent Citations (1)
| Title |
|---|
| 徐德: "《显微视觉测量与控制》", 31 December 2014 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112170825B (en) | 2022-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110133440B (en) | Electric unmanned aerial vehicle based on pole tower model matching and visual navigation and inspection method | |
| CN110125926B (en) | Automatic workpiece picking and placing method and system | |
| CN104057457A (en) | Robot system and calibration method | |
| US8942850B2 (en) | Method and system for assisting in the handling of robotized machines in a congested environment | |
| CN110276799B (en) | Coordinate calibration method, calibration system and mechanical arm | |
| CN111192189A (en) | Three-dimensional automatic detection method and system for automobile appearance | |
| CN104802186A (en) | Robot programming apparatus for creating robot program for capturing image of workpiece | |
| CN106044570A (en) | Steel coil lifting device automatic identification device and method adopting machine vision | |
| CN111932609A (en) | Cloud deck calibration method and device for valve hall equipment inspection robot and storage medium | |
| US20110046783A1 (en) | Method for training a robot or the like, and device for implementing said method | |
| CN110096802A (en) | A kind of steel-structure roof installation method based on BIM and three-dimensional Real-time modeling set technology | |
| US20220092766A1 (en) | Feature inspection system | |
| CN103019101A (en) | Hoisting control method and system for offshore wind turbine | |
| CN112170825B (en) | Long nozzle replacing method, equipment, terminal and medium based on visual servo | |
| CN110430538B (en) | Tunnel production command cockpit | |
| CN115841516A (en) | Method and device for modeling dynamic intrinsic parameters of camera | |
| CN114434456A (en) | Machine room inspection robot and inspection method thereof | |
| EP3722052A1 (en) | A method for determining camera placement within a robotic cell environment | |
| CN103043535B (en) | Stacking system and stacking method for steel plate | |
| CN106271235B (en) | Welding bead positioning method and device based on machine vision | |
| US20230106854A1 (en) | Method for automatically performing an operation on an object with a tool carried by a polyarticulated system | |
| JP2778376B2 (en) | Camera viewpoint change method | |
| CN109244935A (en) | A kind of high-altitude robot hotline maintenance system based on VR vision | |
| CN115535288A (en) | Operation system and method for complete aircraft | |
| CN101276466B (en) | Method for monitoring large-sized component construction security based on vision difference on site |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |