CN117283534A - Device and method for adjusting operation parallelism based on two-dimension code - Google Patents
Device and method for adjusting operation parallelism based on two-dimension code Download PDFInfo
- Publication number
- CN117283534A CN117283534A CN202311006715.3A CN202311006715A CN117283534A CN 117283534 A CN117283534 A CN 117283534A CN 202311006715 A CN202311006715 A CN 202311006715A CN 117283534 A CN117283534 A CN 117283534A
- Authority
- CN
- China
- Prior art keywords
- module
- agv
- mechanical arm
- adjusting
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 230000000007 visual effect Effects 0.000 claims abstract description 32
- 238000004891 communication Methods 0.000 claims abstract description 25
- 230000010365 information processing Effects 0.000 claims abstract description 21
- 230000033001 locomotion Effects 0.000 claims description 22
- 230000004048 modification Effects 0.000 claims description 16
- 238000012986 modification Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 9
- 238000004088 simulation Methods 0.000 claims description 6
- 230000003993 interaction Effects 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000001502 supplementing effect Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 8
- 238000013461 design Methods 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000012937 correction Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 241000899771 Arenga undulatifolia Species 0.000 description 1
- 102000008115 Signaling Lymphocytic Activation Molecule Family Member 1 Human genes 0.000 description 1
- 108010074687 Signaling Lymphocytic Activation Molecule Family Member 1 Proteins 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/08—Programme-controlled manipulators characterised by modular constructions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J5/00—Manipulators mounted on wheels or on carriages
- B25J5/007—Manipulators mounted on wheels or on carriages mounted on wheels
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1692—Calibration of manipulator
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Abstract
The invention discloses a device for adjusting operation parallelism based on a two-dimensional code, which relates to the technical field of robots and comprises an AGV module, a mechanical arm module, a tail end module, a visual information processing module, a communication module and a master control module. The invention also discloses a method for adjusting the operation parallelism based on the two-dimension code, which comprises the following steps of S100, constructing a map; s200, placing a two-dimensional code; s300, roughly adjusting the observation posture; s400, fine-adjusting the observation posture; s500, adjusting the operation parallelism. The invention has the characteristics of high stability, strong self-adaption, wide application range, strong generalization, obvious effect and the like, and can more efficiently meet the requirements of complex scenes.
Description
Technical Field
The invention relates to the technical field of robots, in particular to a device and a method for adjusting operation parallelism based on two-dimension codes.
Background
A new generation of mobile robotic systems should have the ability to interact with complex environments, with perception and motion control being two of the most important capabilities. Sensing refers to the ability of a mobile robot system to acquire external environmental information through various sensor devices, including cameras, lidars and the like. The motion control refers to processing information through feedback of external information, through modes such as deep network, model modeling, real-time image building and the like, and is used as a basis of design motion, and the motion control method controls the mobile robot to complete the motions such as moving, lifting, identifying, grabbing and the like under the condition of ensuring real-time interaction with a complex environment. In practical tasks, these actions often need to be designed and combined according to the in-field environmental information to meet given requirements, such as tasks that are most basic and core in mobile robotic systems-moving and manipulating target objects as needed. For this task, after moving to a specified position as required and interacting with the environment, the robot arm and the end tool are controlled to switch various types to a specified state, and the general flow of the mobile robot is roughly divided into: controlling an AGV (Automated Guided Vehicle, an automatic guided vehicle) to identify the environment, and performing SLAM (Simultaneous Localization And Mapping), synchronous positioning and image construction) composition; controlling the AGV to move the corresponding target approximate area; the mobile sensing equipment identifies target object information including position, type, gesture and the like; according to the existing information, planning a motion path and design operation actions of the mechanical arm, controlling the mechanical arm to move to a target object and completing related actions; the next switch is operated. Therefore, how to accurately control the robot system and provide a good information acquisition basis is a necessary condition for realizing high-precision operation of the robot on the target object. Currently, in a practical operating environment, the following problems still exist in the high-precision operation of mobile robot systems:
1. although the front edge or open source SLAM algorithm and the coherent navigation algorithm are mature and high in precision at present, errors of positions and angles are amplified when the algorithms are actually applied to the AGV due to influences of hardware conditions, system integration, communication modes and the like. The actual precision deviation of the fixed point navigation of the current domestic indoor AGV is approximately as follows: the position is 0.1-0.25m and the angle is 0.1-0.5rads. On the other hand, after the AGV is restarted each time, the map matching degree is also enabled to deviate to a certain extent through secondary calibration, and new errors are caused to the same data information.
2. From the aspect of the sensor, although various types of sensors have been developed to some extent, the method still uses a camera to provide visual information and a depth recognition network for operation target recognition under the scene of robot interaction with the outside, so that the method can meet the requirements of diversified and various recognition. In practical application, the existing recognition network recognition effect is affected by training environment, practical operation environment, brightness condition, recognition angle and the like. Therefore, when the AGV is positioned incorrectly, the environmental conditions such as the identification angle and the like can be changed greatly, the identification effect is greatly weakened, and especially under the condition of various and miscellaneous switches.
3. In a practical scenario, high precision operation of a robot is a necessary condition for meeting safety and stability. Taking substation inspection as an example, all the power transformation equipment operated in the inspection process has a very high safety coefficient, so that the equipment for executing the operation has the requirements of high precision and safe operation (such as interference prevention operation). However, due to the problem of the actual navigation accuracy of the AGV, the deviation of the position and the angle can seriously affect the operation accuracy, and also can cause the problem of interference with the switch operation, so that an additional mechanism or software design is required to complement the defect.
Accordingly, those skilled in the art have been dedicated to developing an apparatus and method for adjusting operation parallelism based on two-dimensional codes.
Disclosure of Invention
In view of the above-mentioned drawbacks of the prior art, the present invention aims to solve the problem that the visual recognition environment and the mechanical arm operation environment are unstable due to inaccurate AGV navigation.
The inventor analyzes the influence of the inaccuracy of the existing AGV navigation on the identification and the operation, so that the operation precision of the mechanical arm on the target object is greatly reduced. For the problem of inaccurate navigation, the algorithm is accurate and mature due to factors such as hardware facilities, system layout, perception sensitivity, transmission delay and the like, so that the improvement on the aspects of hardware and systems cannot be realized under the condition of the same cost and limitation; for the problem of unstable recognition angle, the factors of environmental change, angle change and the like are taken into consideration, a proper depth network is selected, and the pictures of each type of the targeted labels are trained, so that a great amount of time cost and personnel cost are required to achieve an ideal effect, and the effect is influenced along with the change of a field; for the problem of operation interference, an additional sensing information device is required to be introduced or a complex mechanical arm track planning is performed, which also requires a great deal of time cost and personnel cost.
The inventor uses related information of AGVs and SLAMs as priori information, and reads three-dimensional space relation between the tail end of the mechanical arm and the two-dimensional code in real time by combining a camera and an identification algorithm, and the two are matched with a design resolving gesture algorithm, so that the obtained information is comprehensively used for adaptively and real-time adjusting the pose of the mechanical arm, the plane of the camera in the tail end module is kept parallel to the plane of the target object, and a good operation environment and a target identification environment of a neural network are provided for subsequent visual identification and mechanical arm operation.
The inventors define the object to be the object of the operation of the mechanical arm, including but not limited to a switch, a knob, a button, a pressing plate, and typically the operation of the mechanical arm includes a plurality of objects.
In one embodiment of the present invention, there is provided a device for adjusting operation parallelism based on a two-dimensional code, including:
the AGV module is used for performing environment sensing and safety path planning;
the mechanical arm module is used for accurately controlling any angle and any position in a given radius, so that high-precision combination operation is realized;
the tail end module is integrated with the gripper, the camera and the light supplementing lamp equipment, is connected with the mechanical arm module through the link rod structure and is directly interacted with an external object;
the visual information processing module is used for acquiring the real-time image and the depth information, identifying the two-dimensional code and operating the depth network;
the communication module is used for carrying out information interaction among the modules by using a wireless communication mode;
the general control module is used for receiving the motion information sent by the AGV module, the mechanical arm module and the tail end module through the communication module, sending control instructions to the AGV module, the mechanical arm module and the tail end module through the communication module by the visual information processing module, and controlling and comprehensively dispatching the AGV module, the mechanical arm module and the tail end module;
the AGV module is a chassis, the visual information processing module is connected with the AGV module through a cable, the mechanical arm module is connected with the AGV module through a link structure, the tail end module is connected with the mechanical arm module through a link structure, and the visual information processing module is connected with a camera in the tail end module through a cable; the AGV module, the mechanical arm module, the tail end module and the visual information processing module are in wireless communication with the master control module through the communication module, the respective motion information is transmitted to the master control module, and meanwhile, the control instruction of the master control module is received.
Optionally, in the device for adjusting operation parallelism based on the two-dimensional code in the above embodiment, the AGV module integrates SLAM, obstacle avoidance, automatic recharging, path planning and navigation functions.
Optionally, in the device for adjusting operation parallelism based on the two-dimensional code in any embodiment, the AGV module provides an external operation interface, including an external operation interface of SLAM, navigation, and correction parameters.
Optionally, in the device for adjusting operation parallelism based on two-dimensional code in any embodiment, the AGV module provides a programming interface, so that the external device controls the AGV to complete SLAM, navigation and path planning functions through a programming language.
Further, in the device for adjusting the parallelism of operations based on the two-dimensional code in the above embodiment, the programming interface includes an api interface, a dynamic link library, a static link library, and a code use reference.
Optionally, in the device for adjusting operation parallelism based on the two-dimensional code in any embodiment, the AGV module further sets a power supply port, a net port, and a usb port, so as to mount other devices, and provide power supply or connection to the outside.
Optionally, in the two-dimensional code based device for adjusting parallelism in any of the embodiments described above, the AGV module uses an innovative AGV.
Optionally, in the device for adjusting operation parallelism based on two-dimensional code in any one of the embodiments, the mechanical arm module integrates modeling, forward and reverse solution, motion track planning and motion dead zone functions.
Optionally, in the device for adjusting operation parallelism based on two-dimensional code in any one of the embodiments, the mechanical arm module provides a terminal program to control the mechanical arm, so as to implement parameter modification and mechanical arm model simulation, wherein the parameter modification includes mechanical arm parameter modification, communication parameter modification and safety protection parameter modification, and the mechanical arm model simulation includes mechanical arm model display and movement control.
Further, in the device for adjusting the operation parallelism based on the two-dimensional code in the above embodiment, the terminal program includes APP or PC software.
Optionally, in the device for adjusting operation parallelism based on two-dimensional code in any one of the embodiments, the mechanical arm module provides a programming interface, and the mechanical arm is controlled by a programming language to complete the desired movement.
Further, in the device for adjusting the parallelism of operations based on the two-dimensional code in the above embodiment, the programming interface includes an api interface file, a dynamic link library, a static link library, and a code use reference.
Optionally, in the device for adjusting operation parallelism based on two-dimensional code in any one of the embodiments, the mechanical arm module is a six-axis mechanical arm.
Further, in the device for adjusting the operation parallelism based on the two-dimensional code in the above embodiment, the mechanical arm module selects jakazu 7.
Optionally, in the device for adjusting operation parallelism based on two-dimensional code in any one of the embodiments, the visual information processing module runs a program for identifying two-dimensional code and depth network, and obtains real-time image information and depth information through a camera connected to the terminal module in a wired manner.
Alternatively, in the apparatus for adjusting operation parallelism based on two-dimensional code in any of the above embodiments, the visual information processing module uses a jetson xavier nx processor.
Optionally, in the device for adjusting operation parallelism based on two-dimensional code in any one of the embodiments, the camera uses realsense L515.
Optionally, in the device for adjusting operation parallelism based on two-dimensional code in any one of the embodiments, the wireless communication mode includes zmq.
Optionally, in the device for adjusting operation parallelism based on two-dimensional code in any one of the embodiments, the master control module uses an upper computer or a notebook computer.
Based on any one of the above embodiments, in another embodiment of the present invention, a method for adjusting operation parallelism based on a two-dimensional code is provided, including the following steps:
s100, constructing a map, performing SLAM on environment information to obtain a two-dimensional map, and setting preset AGV points and basic information according to the distribution of target objects;
s200, placing a two-dimensional code, placing the two-dimensional code according to the distribution of a target object, and setting a mechanical arm observation posture according to the position of the two-dimensional code;
s300, roughly adjusting the observation gesture, namely roughly adjusting the observation gesture of the mechanical arm according to the preset AGV point position and the actual arrival AGV point position, so as to ensure that the two-dimensional code is in the visual field range;
s400, fine-adjusting the observation gesture, and fine-adjusting the observation gesture of the mechanical arm according to the two-dimensional code;
s500, adjusting the operation parallelism, calculating vector information of the center of the camera in the end module and the center of the two-dimensional code, if the vector information is smaller than or equal to a given threshold value, enabling the plane of the camera in the end module to be parallel to the plane of the target object, otherwise repeating S400.
Optionally, in the method for adjusting parallelism of operations based on two-dimensional codes in the above embodiment, step S100 includes:
s110, controlling the AGV to move, collecting environment information and establishing a two-dimensional map;
s120, setting a preset AGV point position, and controlling the AGV to reach a target object by combining the actual site situation, so as to ensure that the mechanical arm can control the target object, and setting the preset AGV point position according to the standard;
s130, measuring plane angle information, controlling the AGV to move, and measuring basic information of the target object in the current two-dimensional map, wherein the basic information comprises the plane angle information.
Optionally, in the method for adjusting parallelism of operations based on two-dimensional codes in the above embodiment, step S200 includes:
s210, placing two-dimensional codes, and placing the two-dimensional codes in a region where target objects are dense according to the distribution of the target objects;
s220, roughly adjusting errors, controlling the AGVs to sequentially navigate to reach all preset AGV points, manually compensating and adjusting the actual AGV reaching position errors and the actual AGV angle errors according to the information of the preset AGV points, and keeping the actual AGV reaching position errors and the preset AGV point information to be the same;
s230, setting an observation gesture, and setting the observation gesture of the mechanical arm according to the difference of preset AGV point positions and the position of the two-dimensional code.
Further, in the method for adjusting the parallelism of operations based on the two-dimensional code in the above embodiment, the specification of the two-dimensional code is 10cm×10cm.
Optionally, in the method for adjusting parallelism of operations based on two-dimensional codes in the above embodiment, step S300 includes:
s310, acquiring information of an actual AGV point position, controlling a corresponding preset AGV point position according to a target object, and acquiring the information of the actual AGV point position after navigation is finished;
s320, adjusting the observation gesture, combining preset AGV point positions and basic information, and information of actually reaching the AGV point positions, calculating deviation information and angle deviation of the positions by using two-dimensional geometry, and adjusting the observation gesture of the mechanical arm.
Optionally, in the method for adjusting operation parallelism based on two-dimensional code in any one of the embodiments, step S400 includes:
s410, observing the two-dimensional code, and moving the mechanical arm to the adjusted observation gesture to observe the two-dimensional code;
s420, calculating angle deviation, namely sequentially calculating the angle deviation of Euler angles in three angle directions according to the four corner information of the two-dimensional code identified by the two-dimensional code and the visual depth information of the corresponding pixel points;
and S430, compensating deviation, namely compensating the angle deviation based on the coordinate axis of the camera in the terminal module, compensating the angle deviation to the pose of the mechanical arm through matrix operation, and adjusting the observed pose of the mechanical arm.
Optionally, in the method for adjusting operation parallelism based on two-dimensional code in any one of the embodiments, step S500 includes:
s510, adjusting the mechanical arm, identifying the two-dimensional code according to the roughly adjusted observation gesture, and controlling the mechanical arm to move according to the identified Cartesian space coordinate so that the two-dimensional code is positioned in the visual center;
s520, calculating deviation, identifying the two-dimensional code again, calculating vector information of the center of the camera in the terminal module and the center of the two-dimensional code, and considering that the plane of the camera in the terminal module is parallel to the plane of the target object if the vector information is smaller than or equal to a given threshold value; step S400 is performed again.
The invention compensates the problem of inaccurate and unstable navigation based on the prior information and the simple two-dimensional code label assistance, compensates the error brought by the AGV into the pose of the mechanical arm by utilizing the structural relation between the geometric relation and the three-dimensional space and adopting a mathematical analysis settlement mode, achieves the aim of not influencing the identification and operation, has the characteristics of high stability, strong self-adaption, wide application range, strong generalization, obvious effect and the like, and can more efficiently meet the requirements of complex scenes.
The conception, specific structure, and technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, features, and effects of the present invention.
Drawings
Fig. 1 is a schematic diagram illustrating a structure of an apparatus for adjusting operation parallelism based on a two-dimensional code according to an exemplary embodiment;
fig. 2 is a flowchart illustrating a method of adjusting operational parallelism based on a two-dimensional code according to an exemplary embodiment.
Detailed Description
The following description of the preferred embodiments of the present invention refers to the accompanying drawings, which make the technical contents thereof more clear and easy to understand. The present invention may be embodied in many different forms of embodiments and the scope of the present invention is not limited to only the embodiments described herein.
In the drawings, like structural elements are referred to by like reference numerals and components having similar structure or function are referred to by like reference numerals. The dimensions and thickness of each component shown in the drawings are arbitrarily shown, and the present invention is not limited to the dimensions and thickness of each component. The thickness of the components is schematically and appropriately exaggerated in some places in the drawings for clarity of illustration.
The inventor designs a device for adjusting operation parallelism based on two-dimension codes, as shown in fig. 1, which comprises:
the AGV module is used for performing environment sensing and safety path planning; integrating SLAM, obstacle avoidance, automatic recharging, path planning and navigation functions; providing an external operation interface, wherein the external operation interface comprises SLAM, navigation and parameter correction; providing a programming interface, so that the external equipment can control the AGV to complete SLAM, navigation and path planning functions through a programming language, wherein the programming interface comprises an api interface, a dynamic link library, a static link library and a code use reference; the AGV module is also provided with a power supply port, a net port and a usb port so as to mount other equipment and provide power supply or connection to the outside; preferably, the AGV module uses a glossy AGV;
the mechanical arm module is used for accurately controlling any angle and any position in a given radius, so that high-precision combination operation is realized; integrating modeling, forward and reverse solution, motion track planning and motion dead zone functions; providing a terminal program to control the mechanical arm, and realizing parameter modification and mechanical arm model simulation, wherein the parameter modification comprises mechanical arm parameter modification, communication parameter modification and safety protection parameter modification, the mechanical arm model simulation comprises mechanical arm model display and mobile control, and the terminal program comprises APP or PC end software; providing a programming interface, wherein the programming interface comprises an api interface file, a dynamic link library, a static link library and a code use reference, and controlling a mechanical arm to complete expected movement through a programming language; the mechanical arm module is a six-axis mechanical arm, preferably jaka zu7;
the tail end module is integrated with the gripper, the camera and the light supplementing lamp equipment, is connected with the mechanical arm module through the link rod structure and is directly interacted with an external object; the camera preferably uses a realsense L515.
The visual information processing module is used for acquiring the real-time image and the depth information, identifying the two-dimensional code and operating the depth network; running a program for identifying the two-dimensional code and the depth network, and acquiring real-time image information and depth information through a camera connected to the tail end module in a wired manner; the visual information processing module preferably uses a jetson xavier nx processor;
the communication module is used for carrying out information interaction among the modules by using a wireless communication mode, wherein the wireless communication mode comprises zmq;
the general control module is used for receiving the motion information sent by the AGV module, the mechanical arm module and the tail end module through the communication module, sending control instructions to the AGV module, the mechanical arm module and the tail end module through the communication module by the visual information processing module, and controlling and comprehensively dispatching the AGV module, the mechanical arm module and the tail end module; the master control module uses an upper computer or a notebook computer.
The AGV module is a chassis, the visual information processing module is connected with the AGV module through a cable, the mechanical arm module is connected with the AGV module through a link structure, the tail end module is connected with the mechanical arm module through a link structure, and the visual information processing module is connected with a camera in the tail end module through a cable; the AGV module, the mechanical arm module, the tail end module and the visual information processing module are in wireless communication with the master control module through the communication module, the respective motion information is transmitted to the master control module, and meanwhile, the control instruction of the master control module is received.
Based on the above embodiment, the inventor provides a method for adjusting operation parallelism based on a two-dimensional code, as shown in fig. 2, including the following steps:
s100, constructing a map, performing SLAM on environment information to obtain a two-dimensional map, and setting preset AGV points and basic information according to the distribution of target objects; the method specifically comprises the following steps:
s110, controlling the AGV to move, collecting environment information and establishing a two-dimensional map;
s120, setting a preset AGV point position, and controlling the AGV to reach a target object by combining the actual site situation, so as to ensure that the mechanical arm can control the target object, and setting the preset AGV point position according to the standard;
s130, measuring plane angle information, controlling the AGV to move, and measuring basic information of the target object in the current two-dimensional map, wherein the basic information comprises the plane angle information.
S200, placing a two-dimensional code, placing the two-dimensional code according to the distribution of a target object, and setting a mechanical arm observation posture according to the position of the two-dimensional code; the method specifically comprises the following steps:
s210, placing two-dimensional codes, wherein the two-dimensional codes are placed in a region where target objects are dense according to the distribution of the target objects, and the specification of the two-dimensional codes is preferably 10cm x 10cm;
s220, roughly adjusting errors, controlling the AGVs to sequentially navigate to reach all preset AGV points, manually compensating and adjusting the actual AGV reaching position errors and the actual AGV angle errors according to the information of the preset AGV points, and keeping the actual AGV reaching position errors and the preset AGV point information to be the same;
s230, setting an observation gesture, according to the difference of AGV point positions and the position of the two-dimensional code,
and setting the observation gesture of the mechanical arm.
S300, roughly adjusting the observation gesture, namely roughly adjusting the observation gesture of the mechanical arm according to the preset AGV point position and the actual arrival AGV point position, so as to ensure that the two-dimensional code is in the visual field range; the method specifically comprises the following steps:
s310, acquiring information of an actual AGV point position, controlling a corresponding preset AGV point position according to a target object, and acquiring the information of the actual AGV point position after navigation is finished;
s320, adjusting the observation gesture, combining preset AGV point positions and basic information, and information of actually reaching the AGV point positions, calculating deviation information and angle deviation of the positions by using two-dimensional geometry, and adjusting the observation gesture of the mechanical arm.
S400, fine adjustment of the observation gesture, according to the two-dimensional code, the fine adjustment of the observation gesture of the mechanical arm specifically comprises:
s410, observing the two-dimensional code, and moving the mechanical arm to the adjusted observation gesture to observe the two-dimensional code;
s420, calculating angle deviation, namely sequentially calculating the angle deviation of Euler angles in three angle directions according to the four corner information of the two-dimensional code identified by the two-dimensional code and the visual depth information of the corresponding pixel points;
and S430, compensating deviation, namely compensating the angle deviation based on the coordinate axis of the camera in the terminal module, compensating the angle deviation to the pose of the mechanical arm through matrix operation, and adjusting the observed pose of the mechanical arm.
S500, adjusting the operation parallelism, calculating vector information of a camera center and a two-dimensional code center in the end module, if the vector information is smaller than or equal to a given threshold value, enabling a plane of the camera in the end module to be parallel to a plane of the target object, otherwise repeating S400; the method specifically comprises the following steps:
s510, adjusting the mechanical arm, identifying the two-dimensional code according to the roughly adjusted observation gesture, and controlling the mechanical arm to move according to the identified Cartesian space coordinate so that the two-dimensional code is positioned in the visual center;
s520, calculating deviation, identifying the two-dimensional code again, calculating vector information of the center of the camera in the terminal module and the center of the two-dimensional code, and considering that the plane of the camera in the terminal module is parallel to the plane of the target object if the vector information is smaller than or equal to a given threshold value; step S400 is performed again.
The foregoing describes in detail preferred embodiments of the present invention. It should be understood that numerous modifications and variations can be made in accordance with the concepts of the invention by one of ordinary skill in the art without undue burden. Therefore, all technical solutions which can be obtained by logic analysis, reasoning or limited experiments based on the prior art by the person skilled in the art according to the inventive concept shall be within the scope of protection defined by the claims.
Claims (10)
1. Device of adjustment operation depth of parallelism based on two-dimensional code, characterized in that includes:
the AGV module is used for performing environment sensing and safety path planning;
the mechanical arm module is used for accurately controlling any angle and any position in a given radius, so that high-precision combination operation is realized;
the tail end module is integrated with the gripper, the camera and the light supplementing lamp equipment, is connected with the mechanical arm module through a link structure and is directly interacted with an external object;
the visual information processing module is used for acquiring the real-time image and the depth information, identifying the two-dimensional code and operating the depth network;
the communication module is used for carrying out information interaction among the modules by using a wireless communication mode;
the general control module is used for receiving the AGV module, the mechanical arm module and the tail end module, the visual information processing module is used for sending motion information through the communication module and sending control instructions to the AGV module, the mechanical arm module and the tail end module through the communication module, and the visual information processing module is used for controlling and comprehensively dispatching;
the Automatic Guided Vehicle (AGV) module is a chassis, the visual information processing module is connected with the AGV module through a cable, the mechanical arm module is connected with the AGV module through a link structure, the tail end module is connected with the mechanical arm module through a link structure, and the visual information processing module is connected with a camera in the tail end module through a cable; the AGV module, the mechanical arm module, the tail end module and the visual information processing module are in wireless communication with the master control module through the communication module, transmit respective motion information to the master control module, and simultaneously receive control instructions of the master control module.
2. The two-dimensional code based device for adjusting operation parallelism of claim 1, wherein the AGV module integrates SLAM, obstacle avoidance, automatic recharging, path planning and navigation functions.
3. The device for adjusting the operation parallelism based on the two-dimensional code according to claim 2, wherein the mechanical arm module integrates modeling, forward and reverse solution, motion track planning and motion dead zone functions.
4. The device for adjusting operation parallelism based on two-dimension codes according to claim 3, wherein the mechanical arm module provides a terminal program to control the mechanical arm to realize parameter modification and mechanical arm model simulation, the parameter modification comprises mechanical arm parameter modification, communication parameter modification and safety protection parameter modification, and the mechanical arm model simulation comprises mechanical arm model display and movement control.
5. The device for adjusting operation parallelism based on two-dimensional codes according to claim 1, wherein the visual information processing module runs a program for identifying two-dimensional codes and depth networks, and acquires real-time image information and depth information through a camera connected to the terminal module in a wired manner.
6. A method for adjusting operation parallelism based on a two-dimensional code, which uses the device for adjusting operation parallelism based on a two-dimensional code according to any one of claims 1 to 5, and is characterized by comprising the following steps:
s100, constructing a map, performing SLAM on environment information to obtain a two-dimensional map, and setting preset AGV points and basic information according to the distribution of target objects;
s200, placing a two-dimensional code, placing the two-dimensional code according to the distribution of the target object, and setting the observation gesture of the mechanical arm according to the position of the two-dimensional code;
s300, roughly adjusting the observation gesture, namely roughly adjusting the observation gesture of the mechanical arm according to the preset AGV point position and the actual arrival AGV point position, so as to ensure that the two-dimensional code is in the visual field range;
s400, fine-adjusting the observation gesture, and fine-adjusting the observation gesture of the mechanical arm according to the two-dimensional code;
s500, adjusting the operation parallelism, calculating vector information of the center of the camera in the end module and the center of the two-dimensional code, if the vector information is smaller than or equal to a given threshold value, enabling the plane of the camera in the end module to be parallel to the plane of the target object, otherwise repeating S400.
7. The method for adjusting parallelism of operations based on two-dimensional codes according to claim 6, wherein the step S100 comprises:
s110, controlling the AGV to move, collecting environment information and establishing the two-dimensional map;
s120, setting a preset AGV point position, and controlling the AGV to the target object by combining with the actual site situation to ensure that the mechanical arm can control the target object, and setting the preset AGV point position according to the standard;
s130, measuring plane angle information, controlling the AGV to move, and measuring the basic information of the target object in the current two-dimensional map, wherein the basic information comprises plane angle information.
8. The method for adjusting parallelism of operations based on two-dimensional codes according to claim 7, wherein the step S200 comprises:
s210, placing two-dimensional codes, and placing the two-dimensional codes in a region where the target objects are dense according to the distribution of the target objects;
s220, roughly adjusting errors, controlling the AGVs to sequentially navigate to reach each preset AGV point, manually compensating and adjusting the actual arrival position errors and the actual angle errors of the AGVs according to the information of the preset AGV point, and keeping the actual arrival AGV point and the preset AGV point information to be the same;
s230, setting an observation gesture, and setting the observation gesture of the mechanical arm according to the difference of the preset AGV point positions and the position of the two-dimensional code.
9. The method for adjusting parallelism of operations based on two-dimensional codes according to claim 8, wherein the step S300 comprises:
s310, acquiring information of actually reaching the AGV point position, controlling a corresponding preset AGV point position according to the target object, and acquiring the information of actually reaching the AGV point position after navigation is finished;
s320, adjusting the observation gesture, combining the preset AGV point position and the basic information with the actual arrival AGV point position information, calculating the deviation information and the angle deviation of the position by using two-dimensional geometry, and adjusting the observation gesture of the mechanical arm.
10. The method for adjusting parallelism of operations based on two-dimensional codes according to claim 9, wherein the step S400 comprises:
s410, observing a two-dimensional code, and moving the mechanical arm to the adjusted observation gesture to observe the two-dimensional code;
s420, calculating angle deviation, namely sequentially calculating the angle deviation of Euler angles in three angle directions according to the four corner information of the two-dimensional code identified by the two-dimensional code and the visual depth information of the corresponding pixel points;
and S430, compensating deviation, namely compensating the angle deviation based on the coordinate axis of the camera in the tail end module, compensating the angle deviation to the pose of the mechanical arm through matrix operation, and adjusting the observation pose of the mechanical arm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311006715.3A CN117283534A (en) | 2023-08-10 | 2023-08-10 | Device and method for adjusting operation parallelism based on two-dimension code |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311006715.3A CN117283534A (en) | 2023-08-10 | 2023-08-10 | Device and method for adjusting operation parallelism based on two-dimension code |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117283534A true CN117283534A (en) | 2023-12-26 |
Family
ID=89243344
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311006715.3A Pending CN117283534A (en) | 2023-08-10 | 2023-08-10 | Device and method for adjusting operation parallelism based on two-dimension code |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117283534A (en) |
-
2023
- 2023-08-10 CN CN202311006715.3A patent/CN117283534A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ong et al. | Augmented reality-assisted robot programming system for industrial applications | |
| US11331803B2 (en) | Mixed reality assisted spatial programming of robotic systems | |
| Blankemeyer et al. | Intuitive robot programming using augmented reality | |
| CN111968262B (en) | Semantic intelligent substation inspection operation robot navigation system and method | |
| EP3733355A1 (en) | Robot motion optimization system and method | |
| Brito et al. | Dynamic collision avoidance system for a manipulator based on RGB-D data | |
| Lloyd et al. | Model-based telerobotics with vision | |
| Nguyen et al. | An efficient approach for the elevator button manipulation using the visual-based self-driving mobile manipulator | |
| Milgram et al. | An augmented reality based teleoperation interface for unstructured environments | |
| Wolf et al. | Towards robotic laboratory automation Plug & play: Survey and concept proposal on teaching-free robot integration with the lapp digital twin | |
| Pedersen et al. | Using human gestures and generic skills to instruct a mobile robot arm in a feeder filling scenario | |
| Hefele et al. | Robot pose correction using photogrammetric tracking | |
| CN117283534A (en) | Device and method for adjusting operation parallelism based on two-dimension code | |
| Astad et al. | Vive for robotics: Rapid robot cell calibration | |
| Gasteratos et al. | PRONTO: a system for mobile robot navigation via CAD-model guidance | |
| Cheng | Robot manipulation of 3D cylindrical objects with a robot-mounted 2D vision camera | |
| Pedrosa et al. | A skill-based architecture for pick and place manipulation tasks | |
| CN114571460A (en) | Robot control method, device and storage medium | |
| Lin et al. | Interactive control and environment modelling for a mobile robot based on multisensor perceptions | |
| Tinoco et al. | Scara self posture recognition using a monocular camera | |
| Kidiraliev et al. | Using optical sensors for industrial robot-human interactions in a Gazebo environment | |
| Shoureshi et al. | Vision-based intelligent control for automated assembly | |
| Lin et al. | Interactive environment modeling and vision-based task specification for a teleoperated mobile robot | |
| Kovács | Path planning of autonomous service robots | |
| CN116197918B (en) | Manipulator control system based on action record analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |