CN111633653A - Mechanical arm control system and method based on visual positioning - Google Patents
Mechanical arm control system and method based on visual positioning Download PDFInfo
- Publication number
- CN111633653A CN111633653A CN202010497898.3A CN202010497898A CN111633653A CN 111633653 A CN111633653 A CN 111633653A CN 202010497898 A CN202010497898 A CN 202010497898A CN 111633653 A CN111633653 A CN 111633653A
- Authority
- CN
- China
- Prior art keywords
- mechanical arm
- positioning
- visual positioning
- pose
- computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 53
- 238000000034 method Methods 0.000 title claims abstract description 24
- 239000012636 effector Substances 0.000 claims abstract description 15
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 239000003550 marker Substances 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 208000027418 Wounds and injury Diseases 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000006378 damage Effects 0.000 description 2
- 208000014674 injury Diseases 0.000 description 2
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a mechanical arm control system and method based on visual positioning. The mechanical arm control system comprises a visual positioning system, a computer system and a mechanical arm system. The visual positioning system is used for capturing the pose of the positioning handle held by an operator and sending the pose to the computer system; the manipulator system sending the manipulator and end effector states to the computer system; and the computer system plans the operation target of the mechanical arm system according to the position and the posture of the positioning handle and the current state of the mechanical arm system, and sends operation data to a tablet personal computer. The invention reduces the requirement of operators on professional skills, reduces instruction operation and improves working efficiency.
Description
Technical Field
The invention relates to a mechanical arm control system and method based on visual positioning, and belongs to the technical field of industrial robots.
Background
With the rapid development of the modern industrialization process, the level of mechanization and automation is continuously improved, and the application field of the industrial robot is continuously expanded. In order to avoid the robot injury accident of the robot in the production process, a special area is usually divided for the mechanical arm, personnel are prohibited to enter when the mechanical arm runs, and when complex operation is needed, the operator adopts a demonstrator to operate. Although the method can avoid the occurrence of robot injury accidents, on one hand, precious space resources are wasted, and on the other hand, the method is not beneficial to the field operation of operators in complex operation. Therefore, a convenient and fast robot arm man-machine interaction control system is needed.
Disclosure of Invention
The invention aims to provide a mechanical arm control system and method based on visual positioning, which are used for simply and efficiently controlling a mechanical arm.
In order to achieve the above object, the technical solution of the present invention is to provide a robot arm control system based on visual positioning, which includes a visual positioning system, a computer system, and a robot arm system; the visual positioning system and the mechanical arm system are respectively connected with the computer system.
The vision positioning system is used for capturing a real-time pose image of the positioning handle held by an operator and transmitting real-time pose data to the computer system; the positioning handle sends button state data to the computer system; the mechanical arm control cabinet collects the mechanical arm running state data (joint angle and angular speed) and sends the data to the computer system; the computer system carries out motion planning on the mechanical arm system according to the real-time pose data and the current operation state of the mechanical arm system, and sends planned operation data to the mechanical arm system; the mechanical arm control cabinet receives data sent by the computer system and controls the mechanical arm to operate; the control system includes control of the position, attitude, and end effector execution state of the robotic arm in the robotic arm system.
The visual positioning system comprises a visual positioning camera, a positioning handle and a camera bracket; the camera support is placed on the front side of an operator, and the visual positioning camera is mounted on the camera support; the positioning handle is held by an operator.
The positioning handle comprises a control device with a button function and a plurality of marking points (marker) fixed on the control device for visual positioning.
The computer system comprises an industrial personal computer and a switch; the switch is respectively connected with the visual positioning system, the mechanical arm system and the industrial personal computer; the industrial personal computer is connected with the tablet personal computer.
A mechanical arm control method based on visual positioning is applied to the system; the method comprises the following steps:
the method comprises the following steps of firstly, acquiring a real-time pose image of a positioning handle held by an operator and shot by a visual positioning system;
secondly, determining the real-time space pose of the positioning handle in the visual positioning system by adopting a multi-view visual positioning method according to a real-time image shot by the visual positioning system;
step three, acquiring the button state of the positioning handle;
acquiring the joint state of the mechanical arm, and calculating the pose of the end effector;
fifthly, determining a motion conversion relation between the dynamic capturing system and the mechanical arm system;
and sixthly, calculating the target pose of the mechanical arm according to the real-time space pose of the positioning handle.
The invention has the beneficial effects that:
1. the accurate pose and motion state of the positioning handle held by an operator are monitored in real time through a visual positioning system and a computer system, so that the real-time accurate control of the mechanical arm is realized.
2. And operating personnel avoid dangerous environment of an operation site during remote operation, so that the safety of personnel is guaranteed.
3. The corresponding mechanical arm action is generated according to the pose of the recognition positioning handle, a complex operation instruction is not needed, the requirement of an operator on the level of professional skill is reduced, the operation is simpler and more efficient, the workload of the operator is reduced, and the working efficiency is improved.
Drawings
FIG. 1 is a schematic structural diagram of a robot arm control system based on visual positioning according to an embodiment of the disclosure;
FIG. 2 is a schematic view of a positioning handle and marker point arrangement according to an embodiment of the disclosure;
FIG. 3 is a front view of the positioning handle of FIG. 2;
FIG. 4 is a schematic structural diagram of a computer system according to an embodiment of the disclosure;
FIG. 5 is a schematic structural diagram of a robot system according to an embodiment of the present disclosure;
wherein the reference numerals in figures 1-5 are: the system comprises a visual positioning system 1, a visual positioning camera 11, a positioning handle 12, a camera support 13, a computer system 2, an industrial personal computer 21, a switch 22, a tablet personal computer 23, a mechanical arm system 3, a mechanical arm control cabinet 31, a mechanical arm 32 and an end effector 33.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic structural diagram of a robot arm control system based on visual positioning according to an embodiment of the disclosure. Referring to fig. 1, the robot arm control system: comprises a visual positioning system 1, a computer system 2 and a mechanical arm system 3; the visual positioning system 1 and the mechanical arm system 3 are respectively connected with the computer system 2.
The vision positioning system 1 is used for capturing a real-time pose image of the positioning handle 12 held by an operator and transmitting real-time pose data to the computer system 2; the positioning handle 12 sends button status data to the computer system 2; the mechanical arm control cabinet 31 collects the operation state data of the mechanical arm 32, and sends the joint angle and the angular speed to the computer system 2; the computer system 2 plans the motion of the mechanical arm system 3 according to the real-time pose data and the current operation state of the mechanical arm system 3, and sends the planned operation data to the mechanical arm system 3; the mechanical arm control cabinet 31 receives data sent by the computer system 2 and controls the mechanical arm 32 to operate; the control system includes control of the position, posture, and execution state of the end effector 33 of the robot arm 32 in the robot arm system 3.
Fig. 2 is a schematic diagram of a positioning handle and a marker point arrangement disclosed in the embodiment of the invention. Referring to fig. 2 and 3, the positioning handle 12 includes a controller with a button function capable of being held by hand and N marker points fixed on the controller. The marker point is used as a mark point for the vision positioning system 1 to determine the pose of the positioning handle 13, wherein N is more than or equal to 3 in order to ensure the precision and the stability. The visual positioning camera 11 detects the spatial position of a marker point through infrared light reflected by the marker, and the real-time image of the marker point of the positioning handle 12 is continuously shot, so that the spatial pose of the positioning handle 12 is obtained through calculation and operation, and the real-time visual positioning of the positioning handle 12 is realized.
To achieve effective control of the mechanical arm, the positioning handle 12 has the following buttons and control functions:
fig. 4 is a schematic structural diagram of a computer system according to an embodiment of the disclosure. Referring to fig. 4, the computer system 2 includes an industrial personal computer 21 and a switch 22. The switch 22 is respectively connected with the visual positioning system 1, the mechanical arm system 3 and the industrial personal computer 21; the industrial personal computer 21 is connected with the tablet personal computer 23.
Fig. 5 is a schematic structural diagram of a robot arm system according to an embodiment of the present invention. Referring to fig. 5, the robot arm system 3 includes a robot arm control cabinet 31, a robot arm 32, and an end effector 33. The mechanical arm control cabinet 31 collects motion state data (joint angle and angular velocity) of the mechanical arm 33, processes the data and sends the processed data to the computer system 2; the mechanical arm control cabinet 31 receives the motion planning data sent by the computer system 2 to control the operation of the mechanical arm 32. The end effector 33 receives the control signal sent by the computer system 2 for operation; and collects the on-off state of the computer, and sends the data to the computer system 2.
The industrial personal computer 22 receives and processes the data output by the visual positioning system 1 and the mechanical arm system 3, calculates a moving target of the mechanical arm system 3 by combining the data acquired by the visual positioning system 1 and the current state of the mechanical arm system 3, and further sends target data to the mechanical arm system 3; meanwhile, the data is processed and then sent to the tablet computer 23.
In practical application, the process of operating the mechanical arm by an operator by using the mechanical arm control system and the method provided by the invention is as follows:
1. the operator selects the operation mode through the tablet computer 23 and holds the positioning handle 12.
2. The operator presses the button on the positioning handle 12 to realize corresponding operation; the translation/rotation of the end of the robotic arm 32 is controlled by translating/rotating the positioning handle 12 after the corresponding operation is selected.
3. The state of the robot arm system 33 is displayed on the tablet pc 23, and the operator determines the operation state of the robot arm 32 and the end effector 33 by looking at the tablet pc 23 and directly observing the robot arm system 3, and performs the next operation.
Based on the mechanical arm control system provided by the invention, the invention also provides a mechanical arm control method based on visual positioning, which is applied to the mechanical arm control system, and the mechanical arm control method comprises the following steps:
acquiring a real-time pose image of the positioning handle 12 held by an operator and shot by the visual positioning system 1; determining the real-time spatial pose of the positioning handle 12 in the visual positioning system 1 according to a real-time pose image shot by the visual positioning system 1 by adopting a multi-view visual positioning method;
acquiring the button state of the positioning handle 12;
acquiring a joint angle and an angular velocity of the mechanical arm 32, and calculating a pose and a motion velocity of the end effector 33;
calculating the target motion state of the mechanical arm 32 according to the button state and the pose of the positioning handle 12 and the motion state of the mechanical arm 32;
acquiring the state of the end effector 33, and generating a corresponding command according to the button state of the positioning handle 12;
calculating a coordinate conversion relation between the visual positioning system 1 and the mechanical arm system 3 according to the spatial arrangement positions of the two systems;
and calculating the target pose of the mechanical arm 32 according to the coordinate conversion relation between the vision positioning system 1 and the mechanical arm system 3 and by combining the real-time space pose of the positioning handle 12.
The display content of the tablet computer 23 may include:
1 each joint angle, three-dimensional shape of the robot arm 32;
2 the position, attitude, movement speed, and angular speed of the end effector 33;
3 the spatial position and posture of the positioning handle 12;
4, the larger the value of the ratio between the translational motion of the positioning handle 12 and the translational and rotational motion of the end of the robot arm 32, the larger the value of the ratio, the unit distance of the translational motion/the unit angle of the rotational motion of the positioning handle 12, and the larger the distance of the translational motion/the unit angle of the rotational motion of the end effector 33.
By adopting the mechanical arm control system based on visual positioning, the operation of an operator on the mechanical arm can be simply and efficiently completed, the fussy operation instruction input and the requirement on professional skills are reduced, the workload of the operator is reduced, and the working efficiency is improved.
The principles and embodiments of the present invention have been described herein using specific examples, which are presented solely to aid in the understanding of the apparatus and its core concepts; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (5)
1. A robotic arm control system based on visual positioning, the robotic arm control system comprising: the system comprises a visual positioning system (1), a computer system (2) and a mechanical arm system (3); the visual positioning system (1) and the mechanical arm system (3) are respectively connected with the computer system (2);
the visual positioning system (1) comprises a positioning handle (12), and the visual positioning system (1) is used for capturing a real-time pose image of the positioning handle (12) held by an operator and transmitting real-time pose data to the computer system (2); the positioning handle (12) sending button status data to the computer system (2);
the mechanical arm system (3) comprises a mechanical arm control cabinet (31), a mechanical arm (32) and an end effector (33), wherein the mechanical arm control cabinet (31) collects operation state data of the mechanical arm (32) and sends the operation state data to the computer system (2), and the state data comprises joint angles and angular velocities; the computer system (2) carries out motion planning on the mechanical arm system (3) according to the real-time pose data and the current operation state of the mechanical arm system (3), and sends planned operation data to the mechanical arm system (3); the mechanical arm control cabinet (31) receives data sent by the computer system (2) and controls the mechanical arm (32) to operate; the control system includes control of the position, attitude, and execution state of the end effector (33) of the robot arm (32) in the robot arm system (3).
2. A vision positioning based robotic arm control system as claimed in claim 1, characterized in that said vision positioning system (1) further comprises a vision positioning camera (11) and a camera support (13); the visual positioning camera (11) is mounted on the camera support (13); the positioning handle (12) is held by an operator.
3. A mechanical arm control system based on visual positioning as claimed in claim 1, characterized in that the positioning handle (12) comprises a control device with a button function and a plurality of marking points fixed on the control device.
4. A vision based robotic arm control system according to claim 1, characterized in that the computer system (2) comprises an industrial control computer (21), a switch (22); the switch (22) is connected with the visual positioning system (1), the mechanical arm system (3) and the industrial personal computer (21); the industrial personal computer (21) is connected with the tablet personal computer (23).
5. A mechanical arm control method based on visual positioning is characterized in that the method is applied to a mechanical arm control system based on visual positioning as claimed in any one of claims 1 to 4; the method comprises the following steps:
the method comprises the following steps of firstly, acquiring a real-time pose image of a positioning handle (12) held by an operator and shot by a visual positioning system (1);
secondly, a multi-view visual positioning method is adopted, according to the real-time image shot by the visual positioning system (1),
determining a real-time spatial pose of the positioning handle (12) in the visual positioning system (1);
thirdly, acquiring the button state of the positioning handle (12);
fourthly, acquiring the joint state of the mechanical arm (32), and calculating the pose of the end effector (33);
fifthly, determining a motion conversion relation between the dynamic capturing system (1) and the mechanical arm system (3);
and sixthly, calculating the target pose of the mechanical arm (32) according to the real-time space pose of the positioning handle (12).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010497898.3A CN111633653A (en) | 2020-06-04 | 2020-06-04 | Mechanical arm control system and method based on visual positioning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010497898.3A CN111633653A (en) | 2020-06-04 | 2020-06-04 | Mechanical arm control system and method based on visual positioning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111633653A true CN111633653A (en) | 2020-09-08 |
Family
ID=72325304
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010497898.3A Pending CN111633653A (en) | 2020-06-04 | 2020-06-04 | Mechanical arm control system and method based on visual positioning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111633653A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112627261A (en) * | 2020-11-19 | 2021-04-09 | 徐州徐工筑路机械有限公司 | Shovel blade attitude control system and method based on machine vision and land leveler |
| CN112650120A (en) * | 2020-12-22 | 2021-04-13 | 华中科技大学同济医学院附属协和医院 | Robot remote control system, method and storage medium |
| CN113653504A (en) * | 2021-08-18 | 2021-11-16 | 中铁隧道局集团有限公司 | Intermediate wall for use in tunnel, installation apparatus and installation method thereof |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107363831A (en) * | 2017-06-08 | 2017-11-21 | 中国科学院自动化研究所 | The teleoperation robot control system and method for view-based access control model |
| CN107901057A (en) * | 2017-12-15 | 2018-04-13 | 佛山科学技术学院 | A kind of hand-held human-computer interaction device for Manipulation of the machine people |
| CN109571513A (en) * | 2018-12-15 | 2019-04-05 | 华南理工大学 | A kind of mobile crawl service robot system of immersion |
| CN110119193A (en) * | 2018-02-06 | 2019-08-13 | 广东虚拟现实科技有限公司 | Visual interactive device |
| CN110216698A (en) * | 2019-03-11 | 2019-09-10 | 浙江工业大学 | A kind of mechanical arm remote control system based on ROS |
| CN110450143A (en) * | 2019-08-02 | 2019-11-15 | 珞石（北京）科技有限公司 | workpiece fatigue testing method based on cooperative robot |
-
2020
- 2020-06-04 CN CN202010497898.3A patent/CN111633653A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107363831A (en) * | 2017-06-08 | 2017-11-21 | 中国科学院自动化研究所 | The teleoperation robot control system and method for view-based access control model |
| CN107901057A (en) * | 2017-12-15 | 2018-04-13 | 佛山科学技术学院 | A kind of hand-held human-computer interaction device for Manipulation of the machine people |
| CN110119193A (en) * | 2018-02-06 | 2019-08-13 | 广东虚拟现实科技有限公司 | Visual interactive device |
| CN109571513A (en) * | 2018-12-15 | 2019-04-05 | 华南理工大学 | A kind of mobile crawl service robot system of immersion |
| CN110216698A (en) * | 2019-03-11 | 2019-09-10 | 浙江工业大学 | A kind of mechanical arm remote control system based on ROS |
| CN110450143A (en) * | 2019-08-02 | 2019-11-15 | 珞石（北京）科技有限公司 | workpiece fatigue testing method based on cooperative robot |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112627261A (en) * | 2020-11-19 | 2021-04-09 | 徐州徐工筑路机械有限公司 | Shovel blade attitude control system and method based on machine vision and land leveler |
| CN112650120A (en) * | 2020-12-22 | 2021-04-13 | 华中科技大学同济医学院附属协和医院 | Robot remote control system, method and storage medium |
| CN113653504A (en) * | 2021-08-18 | 2021-11-16 | 中铁隧道局集团有限公司 | Intermediate wall for use in tunnel, installation apparatus and installation method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111633653A (en) | Mechanical arm control system and method based on visual positioning | |
| US11197730B2 (en) | Manipulator system | |
| JP7058126B2 (en) | Robot control device and automatic assembly system | |
| US6157873A (en) | Robot programming system and method | |
| US20200055195A1 (en) | Systems and Methods for Remotely Controlling a Robotic Device | |
| US4833624A (en) | Functioning-distributed robot control system | |
| US20160346921A1 (en) | Portable apparatus for controlling robot and method thereof | |
| CA2647435A1 (en) | Intelligent interface device for grasping of an object by a manipulating robot and method of implementing this device | |
| KR101876845B1 (en) | Robot control apparatus | |
| US20170197308A1 (en) | Teaching data generating device and teaching data-generating method for work robot | |
| CN111590567B (en) | Space manipulator teleoperation planning method based on Omega handle | |
| CN112828916B (en) | Remote operation combined interaction device for redundant mechanical arm and remote operation system for redundant mechanical arm | |
| CN105511400A (en) | Control system of stamping robots | |
| CN113119077A (en) | Industrial robot handheld teaching device and teaching method | |
| CN113021357A (en) | Master-slave underwater double-arm robot convenient to move | |
| JP2011101915A (en) | Robot system | |
| CN112894827B (en) | Method, system and device for controlling motion of mechanical arm and readable storage medium | |
| US20080306630A1 (en) | System of Manipulators and Method for Controlling Such a System | |
| CN114683288B (en) | Robot display and control method and device and electronic equipment | |
| Negishi et al. | Operation assistance using visual feedback with considering human intention on master-slave systems | |
| CN214025708U (en) | Intuitive industrial robot demonstration system | |
| Schwandt et al. | Robot manipulator programming interface based on augmened reality | |
| JPH03213278A (en) | Remote control support system for robot | |
| CN112936258A (en) | Man-machine cooperation system and method for aerial work robot | |
| Matsumoto et al. | Operation assist for a teleoperated robot system controlled with a lightweight and high-operable master device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200908 |