CN112677159A - Mechanical arm operation path planning method and device based on visual positioning - Google Patents
Mechanical arm operation path planning method and device based on visual positioning Download PDFInfo
- Publication number
- CN112677159A CN112677159A CN202110301062.6A CN202110301062A CN112677159A CN 112677159 A CN112677159 A CN 112677159A CN 202110301062 A CN202110301062 A CN 202110301062A CN 112677159 A CN112677159 A CN 112677159A
- Authority
- CN
- China
- Prior art keywords
- manipulator
- operated
- target object
- position information
- obstacle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 56
- 238000000034 method Methods 0.000 title claims abstract description 41
- 230000002093 peripheral effect Effects 0.000 claims abstract description 32
- 230000005540 biological transmission Effects 0.000 claims description 26
- 238000012544 monitoring process Methods 0.000 claims description 12
- 238000004519 manufacturing process Methods 0.000 claims description 11
- 230000015572 biosynthetic process Effects 0.000 abstract 1
- 230000000750 progressive effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 230000002542 deteriorative effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Landscapes
- Manipulator (AREA)
- Numerical Control (AREA)
Abstract
The invention discloses a manipulator operation path planning method and a manipulator operation path planning device based on visual positioning, wherein the method comprises the following steps: starting visual equipment arranged on the manipulator to perform matching identification on a target object to be operated in a workbench within the operation range of the manipulator; positioning a target object to be operated to obtain position information of the target object to be operated; obtaining relative position information of a target object to be operated and each peripheral obstacle and the size relation of each obstacle based on visual equipment; forming an initial operation planning path; when the operation is carried out based on the initial operation planning path, the target object to be operated is positioned in real time based on the visual equipment, and the initial operation planning path is updated based on the real-time position information. The invention can realize automatic formation of the operation planning path according to the position of the target to be operated on the workbench and the condition of the obstacle, and update the operation planning path in real time in the operation process, thereby improving the operation precision.
Description
Technical Field
The invention relates to the technical field of automatic control, in particular to a manipulator operation path planning method and device based on visual positioning.
Background
The manipulator is an automatic operation device which can imitate certain motion functions of a human hand and an arm and is used for grabbing and carrying objects or operating tools according to a fixed program; the robot is characterized in that various expected operations can be completed through programming, and the robot has the advantages of both human and mechanical arm machines in structure and performance; in the conventional automated production line, the robot is basically applied to automatic work, but the path of the conventional robot work is basically planned in advance, and the work path is likely to be deviated with the lapse of work time during the work, thereby deteriorating the work accuracy.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides a manipulator operation path planning method and device based on visual positioning, which can automatically form an operation planning path according to the position of a target to be operated on a workbench and the condition of an obstacle, update the operation planning path in real time in the operation process and improve the operation precision.
In order to solve the technical problem, an embodiment of the present invention provides a manipulator operation path planning method based on visual positioning, where the method includes:
starting visual equipment arranged on a manipulator to perform matching identification on a target object to be operated in a workbench within the operation range of the manipulator;
when a target object to be operated is matched, positioning the target object to be operated to obtain position information of the target object to be operated;
obtaining relative position information of the target object to be operated and each peripheral obstacle and the size relation of each obstacle based on the visual equipment;
forming an initial operation planning path based on the position information of the target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relation of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content;
and when the manipulator operates the target object to be operated based on the initial operation planned path, the target object to be operated is positioned in real time based on the visual equipment, and the initial operation planned path is updated based on real-time position information.
Optionally, the workbench is a working area controlled by the manipulator on the production line;
the vision equipment is double-camera equipment, and the vision equipment is arranged on the manipulator and forms a hand-eye system with the manipulator.
Optionally, the starting of the visual device arranged on the manipulator matches and identifies the target object to be operated in the workbench within the operation range of the manipulator, and the starting of the visual device comprises:
the control module of the manipulator receives a control file sent by the control end based on the file transmission interface;
the control module of the manipulator analyzes the control file to obtain a target object to be operated and operation contents in the control file;
starting a visual device arranged on the manipulator to perform real-time video monitoring on a workbench in the operation range of the manipulator;
matching and identifying a target object in real-time video monitoring with a target object to be operated in the control file;
the file transmission interface comprises a video window interface and a file uploading interface.
Optionally, the positioning the target object to be operated to obtain the position information of the target object to be operated includes:
and positioning the target object to be operated based on a hand-eye system formed by the vision equipment and the manipulator to obtain the coordinate position information of the target object to be operated in the hand-eye system.
Optionally, the obtaining, based on the visual device, the relative position information between the target object to be operated and each peripheral obstacle and the size relationship between the target object to be operated and each peripheral obstacle includes:
carrying out relative position positioning identification on each obstacle around the target object to be operated based on a hand-eye system formed by the vision equipment and the manipulator, and obtaining relative coordinate position information of the target object to be operated and each obstacle around the target object to be operated in the hand-eye system;
performing three-dimensional modeling based on coordinate position information occupied by each obstacle in the hand-eye system to obtain a three-dimensional model of each obstacle;
and obtaining the size relation of each obstacle based on the three-dimensional model of each obstacle, wherein the size relation of each obstacle comprises the length, width and height relation of each obstacle.
Optionally, the forming an initial operation planning path based on the position information of the target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator, and the operation content includes:
after acquiring the position information of a target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content, the control module of the manipulator forms an initial operation planning path based on the position information of the target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content;
and the moving speed of the workbench in the working range of the manipulator is the moving speed of the production line corresponding to the working area controlled by the manipulator.
Optionally, when the manipulator performs operation on the target object to be operated based on the initial operation planned path, the positioning of the target object to be operated based on the visual device in real time includes:
and when the manipulator operates the target object to be operated based on the initial operation planning path, the target object to be operated is positioned in real time through a hand-eye system formed by the vision equipment and the manipulator.
Optionally, the updating the initial job planning path based on the real-time location information includes:
after the real-time position information is obtained, the control module of the manipulator updates the path in real time on the basis of the initial operation planned path on the basis of the real-time position information.
In addition, the embodiment of the invention also provides a manipulator operation path planning device based on visual positioning, which comprises:
a matching identification module: the visual equipment is used for starting the visual equipment arranged on the manipulator to perform matching identification on a target object to be operated in the workbench within the operation range of the manipulator;
a positioning processing module: the system comprises a target object to be operated, a positioning module and a processing module, wherein the target object to be operated is matched with the target object to be operated;
an obtaining module: the vision equipment is used for obtaining the relative position information of the target object to be operated and each peripheral obstacle and the size relation of each obstacle;
a path planning module: the system comprises a manipulator, a position information acquisition module and a position information acquisition module, wherein the manipulator is used for acquiring the position information of a target to be operated;
a path update module: and the manipulator is used for positioning the target object to be operated in real time based on the visual equipment and updating the initial operation planning path based on real-time position information when the manipulator operates the target object to be operated based on the initial operation planning path.
In the embodiment of the invention, the vision equipment is arranged on the mechanical arm, the hand-eye system is formed between the vision equipment and the mechanical arm, the target object to be operated on the workbench is positioned in real time through the hand-eye system, and meanwhile, the obstacles and the like around the target object to be operated are identified and positioned, so that an initial operation planning path is formed, and in the operation process, the operation planning path is updated, so that the operation planning path can be automatically formed according to the position of the target object to be operated on the workbench and the obstacle condition, and the operation planning path is updated in real time in the operation process, so that the operation precision is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a method for planning a working path of a manipulator based on visual positioning according to an embodiment of the present invention;
fig. 2 is a schematic structural component diagram of a robot working path planning apparatus based on visual positioning according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
referring to fig. 1, fig. 1 is a schematic flow chart illustrating a method for planning a working path of a robot based on visual positioning according to an embodiment of the present invention.
As shown in fig. 1, a method for planning a manipulator working path based on visual positioning includes:
s11: starting visual equipment arranged on a manipulator to perform matching identification on a target object to be operated in a workbench within the operation range of the manipulator;
in the specific implementation process of the invention, the workbench is a working area controlled by the manipulator on the production line; the vision equipment is double-camera equipment, and the vision equipment is arranged on the manipulator and forms a hand-eye system with the manipulator.
Further, the starting of the visual device arranged on the manipulator matches and identifies the target object to be operated in the workbench within the operation range of the manipulator, and the starting of the visual device comprises the following steps: the control module of the manipulator receives a control file sent by the control end based on the file transmission interface; the control module of the manipulator analyzes the control file to obtain a target object to be operated and operation contents in the control file; starting a visual device arranged on the manipulator to perform real-time video monitoring on a workbench in the operation range of the manipulator; matching and identifying a target object in real-time video monitoring with a target object to be operated in the control file; the file transmission interface comprises a video window interface and a file uploading interface.
Specifically, the workbench is a work area controlled by the manipulator on the production line, the vision device is a dual-camera device, both the dual-camera devices are CCD devices, and the dual-camera device is disposed on the manipulator to form a hand-eye system, which can adopt image-based vision control, position-based vision control, and hybrid vision control combining the two. For the vision control based on the image, because a closed loop is formed in the image space, the calibration error of the camera can be effectively overcome, and the requirement on the precision of the camera calibration is not high.
The manipulator is provided with a control module for controlling the manipulator to operate; the control module of the manipulator receives the control file sent by the control end, and then the control module analyzes the control file, so that a target object to be operated and operation contents in the control file are obtained; after a target object to be operated and operation contents in the control file are obtained, starting visual equipment arranged on the manipulator to carry out real-time video monitoring on a workbench in the operation range of the manipulator; simultaneously, matching and identifying a target object in the real-time video monitoring with a target object to be operated in the control file; the control end sends a control file to a control module of the manipulator through a file transmission interface, and the file transmission interface comprises a video window interface and a file uploading interface.
When the file transmission interface is a video window interface, the control end transmits a control file according to the DiiVA technology in an IP streaming media mode; specifically, the control end adopts MPEG transport stream coding conforming to DiiVA transmission mode, fine scalable coding FGS (fine granular scalable coding) or progressive fine scalable coding PFGS (progressive fine granular scalable coding) technology is adopted in MPEG-4 standard, and H.264 technology is combined with network transmission protocol; real-time transmission error control is carried out on data by adopting a real-time transmission protocol or a transmission control protocol; and when the file transmission interface is a file uploading interface, the control end sends the control file according to the file uploading interface.
S12: when a target object to be operated is matched, positioning the target object to be operated to obtain position information of the target object to be operated;
in a specific implementation process of the present invention, the positioning the target object to be operated to obtain the position information of the target object to be operated includes: and positioning the target object to be operated based on a hand-eye system formed by the vision equipment and the manipulator to obtain the coordinate position information of the target object to be operated in the hand-eye system.
Specifically, when the target object to be operated is matched, a related coordinate system is constructed through a hand-eye system formed by the vision equipment and the manipulator, and it is assumed that one camera of the double-camera equipment in the vision equipment is marked as O1, and the other camera is marked as O2; one camera is taken as the coordinate origin of a coordinate system, O1 is taken as the coordinate origin of the coordinate system, the connecting line between two camera devices is taken as an X axis, the distance between the two points is b, in the double-camera device, the X axis is vertical to the Y axis, the Z axis in the depth direction is determined by adopting the right hand criterion, and the height of the effective visual field of the double-camera device is the same as that of the single camera device; when the two cameras are placed randomly, a line is connected in the Z-axis direction in the XOZ plane of the coordinate system to form an effective field of view of the binocular system, the width of the effective field of view is w, the projections of any target point P (x, y, Z) on images collected by the two cameras in the binocular camera equipment are respectively P1 (u 1, v 1) and P2 (u 2, v 2), the opening angles of the P point relative to perspective center points O1 and O2 of the binocular camera equipment are respectively a1 and a2, the models are simplified, the cameras are assumed to have the same configuration and are placed symmetrically, the value of the coordinate P (x, y, Z) of the P point can be calculated through the geometric relationship, and the coordinate position information of the target to be operated in the hand-eye system can be obtained.
S13: obtaining relative position information of the target object to be operated and each peripheral obstacle and the size relation of each obstacle based on the visual equipment;
in the specific implementation process of the invention, based on a hand-eye system formed by the vision equipment and the manipulator, the relative position of each obstacle around the target object to be operated is positioned and identified, and the relative coordinate position information of the target object to be operated and each obstacle around the target object to be operated in the hand-eye system is obtained; performing three-dimensional modeling based on coordinate position information occupied by each obstacle in the hand-eye system to obtain a three-dimensional model of each obstacle; and obtaining the size relation of each obstacle based on the three-dimensional model of each obstacle, wherein the size relation of each obstacle comprises the length, width and height relation of each obstacle.
Specifically, a relative coordinate system established by a hand-eye system formed by vision equipment and a manipulator is utilized to perform relative position positioning identification on each obstacle around a target object to be operated, so that relative coordinate position information of the target object to be operated and each peripheral obstacle in the hand-eye system is obtained, and then three-dimensional modeling is performed through the coordinate position information occupied by each obstacle in the hand-eye system, so as to obtain a three-dimensional model of each obstacle; and finally, obtaining the size relation of each obstacle according to the three-dimensional model of each obstacle, wherein the size relation of each obstacle comprises the length, width and height relation of each obstacle.
S14: forming an initial operation planning path based on the position information of the target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relation of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content;
in a specific implementation process of the present invention, the forming an initial operation planning path based on the position information of the target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator, and the operation content includes: after acquiring the position information of a target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content, the control module of the manipulator forms an initial operation planning path based on the position information of the target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content; and the moving speed of the workbench in the working range of the manipulator is the moving speed of the production line corresponding to the working area controlled by the manipulator.
Specifically, after the control module of the manipulator obtains the position information of the target to be operated, the relative position information of the target to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content, an initial operation planning path is formed according to the position information of the target to be operated, the relative position information of the target to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content according to preset operation requirements, and the moving speed of the workbench in the working range of the manipulator is the pipeline moving speed corresponding to the working area controlled by the manipulator.
S15: and when the manipulator operates the target object to be operated based on the initial operation planned path, the target object to be operated is positioned in real time based on the visual equipment, and the initial operation planned path is updated based on real-time position information.
In a specific implementation process of the present invention, when the manipulator performs an operation on the target object to be operated based on the initial operation planned path, the positioning of the target object to be operated based on the visual device in real time includes: and when the manipulator operates the target object to be operated based on the initial operation planning path, the target object to be operated is positioned in real time through a hand-eye system formed by the vision equipment and the manipulator.
Further, the updating the initial job planning path based on the real-time location information includes: after the real-time position information is obtained, the control module of the manipulator updates the path in real time on the basis of the initial operation planned path on the basis of the real-time position information.
Specifically, when the manipulator operates the target object to be operated according to the initial operation planning path, the target object to be operated is positioned in real time through a hand-eye system formed by the vision equipment and the manipulator; and after the real-time position information is obtained, the control module of the manipulator updates the path in real time on the basis of the initial operation path according to the real-time position information.
In the embodiment of the invention, the vision equipment is arranged on the mechanical arm, the hand-eye system is formed between the vision equipment and the mechanical arm, the target object to be operated on the workbench is positioned in real time through the hand-eye system, and meanwhile, the obstacles and the like around the target object to be operated are identified and positioned, so that an initial operation planning path is formed, and in the operation process, the operation planning path is updated, so that the operation planning path can be automatically formed according to the position of the target object to be operated on the workbench and the obstacle condition, and the operation planning path is updated in real time in the operation process, so that the operation precision is improved.
Example two
Referring to fig. 2, fig. 2 is a schematic structural diagram of a robot operation path planning apparatus based on visual positioning according to an embodiment of the present invention.
As shown in fig. 2, a robot working path planning apparatus based on visual positioning includes:
the matching identification module 21: the visual equipment is used for starting the visual equipment arranged on the manipulator to perform matching identification on a target object to be operated in the workbench within the operation range of the manipulator;
in the specific implementation process of the invention, the workbench is a working area controlled by the manipulator on the production line; the vision equipment is double-camera equipment, and the vision equipment is arranged on the manipulator and forms a hand-eye system with the manipulator.
Further, the starting of the visual device arranged on the manipulator matches and identifies the target object to be operated in the workbench within the operation range of the manipulator, and the starting of the visual device comprises the following steps: the control module of the manipulator receives a control file sent by the control end based on the file transmission interface; the control module of the manipulator analyzes the control file to obtain a target object to be operated and operation contents in the control file; starting a visual device arranged on the manipulator to perform real-time video monitoring on a workbench in the operation range of the manipulator; matching and identifying a target object in real-time video monitoring with a target object to be operated in the control file; the file transmission interface comprises a video window interface and a file uploading interface.
Specifically, the workbench is a work area controlled by the manipulator on the production line, the vision device is a dual-camera device, both the dual-camera devices are CCD devices, and the dual-camera device is disposed on the manipulator to form a hand-eye system, which can adopt image-based vision control, position-based vision control, and hybrid vision control combining the two. For the vision control based on the image, because a closed loop is formed in the image space, the calibration error of the camera can be effectively overcome, and the requirement on the precision of the camera calibration is not high.
The manipulator is provided with a control module for controlling the manipulator to operate; the control module of the manipulator receives the control file sent by the control end, and then the control module analyzes the control file, so that a target object to be operated and operation contents in the control file are obtained; after a target object to be operated and operation contents in the control file are obtained, starting visual equipment arranged on the manipulator to carry out real-time video monitoring on a workbench in the operation range of the manipulator; simultaneously, matching and identifying a target object in the real-time video monitoring with a target object to be operated in the control file; the control end sends a control file to a control module of the manipulator through a file transmission interface, and the file transmission interface comprises a video window interface and a file uploading interface.
When the file transmission interface is a video window interface, the control end transmits a control file according to the DiiVA technology in an IP streaming media mode; specifically, the control end adopts MPEG transport stream coding conforming to DiiVA transmission mode, fine scalable coding FGS (fine granular scalable coding) or progressive fine scalable coding PFGS (progressive fine granular scalable coding) technology is adopted in MPEG-4 standard, and H.264 technology is combined with network transmission protocol; real-time transmission error control is carried out on data by adopting a real-time transmission protocol or a transmission control protocol; and when the file transmission interface is a file uploading interface, the control end sends the control file according to the file uploading interface.
The positioning processing module 22: the system comprises a target object to be operated, a positioning module and a processing module, wherein the target object to be operated is matched with the target object to be operated;
in a specific implementation process of the present invention, the positioning the target object to be operated to obtain the position information of the target object to be operated includes: and positioning the target object to be operated based on a hand-eye system formed by the vision equipment and the manipulator to obtain the coordinate position information of the target object to be operated in the hand-eye system.
Specifically, when the target object to be operated is matched, a related coordinate system is constructed through a hand-eye system formed by the vision equipment and the manipulator, and it is assumed that one camera of the double-camera equipment in the vision equipment is marked as O1, and the other camera is marked as O2; one camera is taken as the coordinate origin of a coordinate system, O1 is taken as the coordinate origin of the coordinate system, the connecting line between two camera devices is taken as an X axis, the distance between the two points is b, in the double-camera device, the X axis is vertical to the Y axis, the Z axis in the depth direction is determined by adopting the right hand criterion, and the height of the effective visual field of the double-camera device is the same as that of the single camera device; when the two cameras are placed randomly, a line is connected in the Z-axis direction in the XOZ plane of the coordinate system to form an effective field of view of the binocular system, the width of the effective field of view is w, the projections of any target point P (x, y, Z) on images collected by the two cameras in the binocular camera equipment are respectively P1 (u 1, v 1) and P2 (u 2, v 2), the opening angles of the P point relative to perspective center points O1 and O2 of the binocular camera equipment are respectively a1 and a2, the models are simplified, the cameras are assumed to have the same configuration and are placed symmetrically, the value of the coordinate P (x, y, Z) of the P point can be calculated through the geometric relationship, and the coordinate position information of the target to be operated in the hand-eye system can be obtained.
The obtaining module 23: the vision equipment is used for obtaining the relative position information of the target object to be operated and each peripheral obstacle and the size relation of each obstacle;
in the specific implementation process of the invention, based on a hand-eye system formed by the vision equipment and the manipulator, the relative position of each obstacle around the target object to be operated is positioned and identified, and the relative coordinate position information of the target object to be operated and each obstacle around the target object to be operated in the hand-eye system is obtained; performing three-dimensional modeling based on coordinate position information occupied by each obstacle in the hand-eye system to obtain a three-dimensional model of each obstacle; and obtaining the size relation of each obstacle based on the three-dimensional model of each obstacle, wherein the size relation of each obstacle comprises the length, width and height relation of each obstacle.
Specifically, a relative coordinate system established by a hand-eye system formed by vision equipment and a manipulator is utilized to perform relative position positioning identification on each obstacle around a target object to be operated, so that relative coordinate position information of the target object to be operated and each peripheral obstacle in the hand-eye system is obtained, and then three-dimensional modeling is performed through the coordinate position information occupied by each obstacle in the hand-eye system, so as to obtain a three-dimensional model of each obstacle; and finally, obtaining the size relation of each obstacle according to the three-dimensional model of each obstacle, wherein the size relation of each obstacle comprises the length, width and height relation of each obstacle.
The path planning module 24: the system comprises a manipulator, a position information acquisition module and a position information acquisition module, wherein the manipulator is used for acquiring the position information of a target to be operated;
in a specific implementation process of the present invention, the forming an initial operation planning path based on the position information of the target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator, and the operation content includes: after acquiring the position information of a target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content, the control module of the manipulator forms an initial operation planning path based on the position information of the target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content; and the moving speed of the workbench in the working range of the manipulator is the moving speed of the production line corresponding to the working area controlled by the manipulator.
Specifically, after the control module of the manipulator obtains the position information of the target to be operated, the relative position information of the target to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content, an initial operation planning path is formed according to the position information of the target to be operated, the relative position information of the target to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content according to preset operation requirements, and the moving speed of the workbench in the working range of the manipulator is the pipeline moving speed corresponding to the working area controlled by the manipulator.
The path update module 25: and the manipulator is used for positioning the target object to be operated in real time based on the visual equipment and updating the initial operation planning path based on real-time position information when the manipulator operates the target object to be operated based on the initial operation planning path.
In a specific implementation process of the present invention, when the manipulator performs an operation on the target object to be operated based on the initial operation planned path, the positioning of the target object to be operated based on the visual device in real time includes: and when the manipulator operates the target object to be operated based on the initial operation planning path, the target object to be operated is positioned in real time through a hand-eye system formed by the vision equipment and the manipulator.
Further, the updating the initial job planning path based on the real-time location information includes: after obtaining the real-time position information, a control module of the manipulator updates a path in real time based on the real-time position information on the basis of the initial working path.
Specifically, when the manipulator operates the target object to be operated according to the initial operation planning path, the target object to be operated is positioned in real time through a hand-eye system formed by the vision equipment and the manipulator; after the real-time position information is obtained, the control module of the manipulator updates the path in real time on the basis of the initial operation planned path according to the real-time position information.
In the embodiment of the invention, the vision equipment is arranged on the mechanical arm, the hand-eye system is formed between the vision equipment and the mechanical arm, the target object to be operated on the workbench is positioned in real time through the hand-eye system, and meanwhile, the obstacles and the like around the target object to be operated are identified and positioned, so that an initial operation planning path is formed, and in the operation process, the operation planning path is updated, so that the operation planning path can be automatically formed according to the position of the target object to be operated on the workbench and the obstacle condition, and the operation planning path is updated in real time in the operation process, so that the operation precision is improved.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic or optical disk, or the like.
In addition, the method and the device for planning the operation path of the manipulator based on visual positioning according to the embodiments of the present invention are described in detail, and a specific example is used herein to explain the principle and the implementation of the present invention, and the description of the embodiments is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (9)
1. A manipulator operation path planning method based on visual positioning is characterized by comprising the following steps:
starting visual equipment arranged on a manipulator to perform matching identification on a target object to be operated in a workbench within the operation range of the manipulator;
when a target object to be operated is matched, positioning the target object to be operated to obtain position information of the target object to be operated;
obtaining relative position information of the target object to be operated and each peripheral obstacle and the size relation of each obstacle based on the visual equipment;
forming an initial operation planning path based on the position information of the target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relation of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content;
and when the manipulator operates the target object to be operated based on the initial operation planned path, the target object to be operated is positioned in real time based on the visual equipment, and the initial operation planned path is updated based on real-time position information.
2. The manipulator work path planning method according to claim 1, wherein the work table is a work area controlled on a production line by the manipulator;
the vision equipment is double-camera equipment, and the vision equipment is arranged on the manipulator and forms a hand-eye system with the manipulator.
3. The manipulator working path planning method according to claim 1, wherein the starting of the visual device provided on the manipulator performs matching identification on the target object to be worked in the workbench within the working range of the manipulator, and includes:
the control module of the manipulator receives a control file sent by the control end based on the file transmission interface;
the control module of the manipulator analyzes the control file to obtain a target object to be operated and operation contents in the control file;
starting a visual device arranged on the manipulator to perform real-time video monitoring on a workbench in the operation range of the manipulator;
matching and identifying a target object in real-time video monitoring with a target object to be operated in the control file;
the file transmission interface comprises a video window interface and a file uploading interface.
4. The manipulator operation path planning method according to claim 1, wherein the positioning the target object to be operated to obtain the position information of the target object to be operated includes:
and positioning the target object to be operated based on a hand-eye system formed by the vision equipment and the manipulator to obtain the coordinate position information of the target object to be operated in the hand-eye system.
5. The manipulator working path planning method according to claim 1, wherein the obtaining of the relative position information between the target object to be worked and each peripheral obstacle and the size relationship between the target object to be worked and each peripheral obstacle based on the vision device includes:
carrying out relative position positioning identification on each obstacle around the target object to be operated based on a hand-eye system formed by the vision equipment and the manipulator, and obtaining relative coordinate position information of the target object to be operated and each obstacle around the target object to be operated in the hand-eye system;
performing three-dimensional modeling based on coordinate position information occupied by each obstacle in the hand-eye system to obtain a three-dimensional model of each obstacle;
and obtaining the size relation of each obstacle based on the three-dimensional model of each obstacle, wherein the size relation of each obstacle comprises the length, width and height relation of each obstacle.
6. The manipulator work path planning method according to claim 1, wherein forming an initial work path based on the position information of the target object to be worked, the relative position information of the target object to be worked and the peripheral obstacles, the size relationship of the obstacles, the moving speed of the table within the work range of the manipulator, and the work content includes:
after acquiring the position information of a target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content, the control module of the manipulator forms an initial operation planning path based on the position information of the target to be operated, the relative position information of the target object to be operated and each peripheral obstacle, the size relationship of each obstacle, the moving speed of the workbench in the working range of the manipulator and the operation content;
and the moving speed of the workbench in the working range of the manipulator is the moving speed of the production line corresponding to the working area controlled by the manipulator.
7. The manipulator working path planning method according to claim 1, wherein the manipulator locates the target object to be worked in real time based on the visual device while working the target object to be worked based on the initial working planning path, and the method includes:
and when the manipulator operates the target object to be operated based on the initial operation planning path, the target object to be operated is positioned in real time through a hand-eye system formed by the vision equipment and the manipulator.
8. The manipulator work path planning method of claim 1, wherein the updating the initial work plan path based on the real-time position information comprises:
after the real-time position information is obtained, the control module of the manipulator updates the path in real time on the basis of the initial operation planned path on the basis of the real-time position information.
9. A vision positioning based manipulator work path planning apparatus, the apparatus comprising:
a matching identification module: the visual equipment is used for starting the visual equipment arranged on the manipulator to perform matching identification on a target object to be operated in the workbench within the operation range of the manipulator;
a positioning processing module: the system comprises a target object to be operated, a positioning module and a processing module, wherein the target object to be operated is matched with the target object to be operated;
an obtaining module: the vision equipment is used for obtaining the relative position information of the target object to be operated and each peripheral obstacle and the size relation of each obstacle;
a path planning module: the system comprises a manipulator, a position information acquisition module and a position information acquisition module, wherein the manipulator is used for acquiring the position information of a target to be operated;
a path update module: and the manipulator is used for positioning the target object to be operated in real time based on the visual equipment and updating the initial operation planning path based on real-time position information when the manipulator operates the target object to be operated based on the initial operation planning path.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110301062.6A CN112677159B (en) | 2021-03-22 | 2021-03-22 | Mechanical arm operation path planning method and device based on visual positioning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110301062.6A CN112677159B (en) | 2021-03-22 | 2021-03-22 | Mechanical arm operation path planning method and device based on visual positioning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112677159A true CN112677159A (en) | 2021-04-20 |
| CN112677159B CN112677159B (en) | 2023-02-28 |
Family
ID=75455722
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110301062.6A Active CN112677159B (en) | 2021-03-22 | 2021-03-22 | Mechanical arm operation path planning method and device based on visual positioning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112677159B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113180894A (en) * | 2021-04-27 | 2021-07-30 | 浙江大学 | Visual intelligence-based hand-eye coordination method and device for multiple-obstacle person |
| CN114160507A (en) * | 2021-11-24 | 2022-03-11 | 上海航翼高新技术发展研究院有限公司 | Automatic planning method for laser cleaning path based on multiple sensing detection |
| CN116214531A (en) * | 2023-05-10 | 2023-06-06 | 佛山隆深机器人有限公司 | Path planning method and device for industrial robot |
| CN118081757A (en) * | 2024-03-24 | 2024-05-28 | 广东技术师范大学 | Control system of manipulator |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104062902A (en) * | 2014-05-15 | 2014-09-24 | 江南大学 | Delta robot time optimal trajectory planning method |
| CN204866557U (en) * | 2015-05-20 | 2015-12-16 | 大连华工创新科技股份有限公司 | Express delivery of container formula and automatic sorting system |
| CN106444458A (en) * | 2016-11-11 | 2017-02-22 | 华南智能机器人创新研究院 | Method and system for remotely controlling industrial robot |
| CN109261531A (en) * | 2018-10-30 | 2019-01-25 | 福建省亿顺机械设备有限公司 | Stacking baling line based on intelligent sorting |
| CN109911549A (en) * | 2019-01-25 | 2019-06-21 | 东华大学 | A kind of the Robotic Dynamic tracking grasping system and method for fragile goods |
| CN109926342A (en) * | 2019-03-27 | 2019-06-25 | 杭州翼道智能科技有限公司 | A kind of efficient sorting system of express delivery and its method for sorting |
| CN110976306A (en) * | 2019-10-22 | 2020-04-10 | 江苏苏宁物流有限公司 | Goods sorting method, system, control server and medium |
| US10793355B1 (en) * | 2018-04-06 | 2020-10-06 | Amazon Technologies, Inc. | Inventory storage module with robotic transport system |
-
2021
- 2021-03-22 CN CN202110301062.6A patent/CN112677159B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104062902A (en) * | 2014-05-15 | 2014-09-24 | 江南大学 | Delta robot time optimal trajectory planning method |
| CN204866557U (en) * | 2015-05-20 | 2015-12-16 | 大连华工创新科技股份有限公司 | Express delivery of container formula and automatic sorting system |
| CN106444458A (en) * | 2016-11-11 | 2017-02-22 | 华南智能机器人创新研究院 | Method and system for remotely controlling industrial robot |
| US10793355B1 (en) * | 2018-04-06 | 2020-10-06 | Amazon Technologies, Inc. | Inventory storage module with robotic transport system |
| CN109261531A (en) * | 2018-10-30 | 2019-01-25 | 福建省亿顺机械设备有限公司 | Stacking baling line based on intelligent sorting |
| CN109911549A (en) * | 2019-01-25 | 2019-06-21 | 东华大学 | A kind of the Robotic Dynamic tracking grasping system and method for fragile goods |
| CN109926342A (en) * | 2019-03-27 | 2019-06-25 | 杭州翼道智能科技有限公司 | A kind of efficient sorting system of express delivery and its method for sorting |
| CN110976306A (en) * | 2019-10-22 | 2020-04-10 | 江苏苏宁物流有限公司 | Goods sorting method, system, control server and medium |
Non-Patent Citations (1)
| Title |
|---|
| 樊景博: "基于计算机视觉技术的焊接机器人定位精度分析", 《微型电脑应用》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113180894A (en) * | 2021-04-27 | 2021-07-30 | 浙江大学 | Visual intelligence-based hand-eye coordination method and device for multiple-obstacle person |
| CN113180894B (en) * | 2021-04-27 | 2022-02-11 | 浙江大学 | Visual intelligence-based hand-eye coordination method and device for multiple-obstacle person |
| CN114160507A (en) * | 2021-11-24 | 2022-03-11 | 上海航翼高新技术发展研究院有限公司 | Automatic planning method for laser cleaning path based on multiple sensing detection |
| CN114160507B (en) * | 2021-11-24 | 2023-06-06 | 上海航翼高新技术发展研究院有限公司 | Laser cleaning path automatic planning method based on multiple sensing detection |
| CN116214531A (en) * | 2023-05-10 | 2023-06-06 | 佛山隆深机器人有限公司 | Path planning method and device for industrial robot |
| CN116214531B (en) * | 2023-05-10 | 2023-06-30 | 佛山隆深机器人有限公司 | Path planning method and device for industrial robot |
| CN118081757A (en) * | 2024-03-24 | 2024-05-28 | 广东技术师范大学 | Control system of manipulator |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112677159B (en) | 2023-02-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112677159B (en) | Mechanical arm operation path planning method and device based on visual positioning | |
| CN107618030B (en) | Robot dynamic tracking grabbing method and system based on vision | |
| CN113276106B (en) | Climbing robot space positioning method and space positioning system | |
| CN110539109B (en) | Robot automatic welding system and method based on single-binocular vision | |
| CN104400279B (en) | Pipeline space weld seam based on CCD identifies the method with trajectory planning automatically | |
| CN110170995B (en) | Robot rapid teaching method based on stereoscopic vision | |
| JP6855492B2 (en) | Robot system, robot system control device, and robot system control method | |
| CN113333998A (en) | Automatic welding system and method based on cooperative robot | |
| CN110640730A (en) | Method and system for generating three-dimensional model for robot scene | |
| CN109702290B (en) | Steel plate groove cutting method based on visual identification | |
| CN110900611A (en) | Novel mechanical arm target positioning and path planning method | |
| CN107103624B (en) | Stereoscopic vision conveying system and conveying method thereof | |
| CN110163963B (en) | Mapping device and mapping method based on SLAM | |
| JP2004185123A (en) | Apparatus for generating three-dimensional model data | |
| CN111823223A (en) | Robot arm grabbing control system and method based on intelligent stereoscopic vision | |
| CN111624994A (en) | Robot inspection method based on 5G communication | |
| CN110039520B (en) | Teaching and processing system based on image contrast | |
| CN111975200A (en) | Intelligent welding method and intelligent welding system based on visual teaching technology | |
| CN110405775A (en) | A kind of robot teaching system and method based on augmented reality | |
| CN104808490A (en) | Uncalibrated visual servoing control method for estimating image Jacobian matrix based on echo state network facing mold protection | |
| CN116673963A (en) | Double mechanical arm cooperation flexible assembly system and method for unordered breaker parts | |
| CN111958593B (en) | Vision servo method and system for inspection operation robot of semantic intelligent substation | |
| CN114800524B (en) | System and method for actively preventing collision of man-machine interaction cooperative robot | |
| CN114299039A (en) | Robot and collision detection device and method thereof | |
| JP2012161850A (en) | Robotic device, device, program and method for detecting position |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A method and device for robot arm operation path planning based on visual positioning Granted publication date: 20230228 Pledgee: Beijiao sub branch of Guangdong Shunde Rural Commercial Bank Co.,Ltd. Pledgor: FOSHAN LONGSHEN ROBOT Co.,Ltd. Registration number: Y2024980007296 |